A community for students.

Here's the question you clicked on:

55 members online
  • 0 replying
  • 0 viewing

anonymous

  • one year ago

In lecture 16, in minute 27, the professor talks about solving for the least squared error using calculus and taking partial derivatives. I can't under stand why we set for example d=0 and proceed ? What relation does this have to gradient descent ?

  • This Question is Open
  1. JoshDanziger23
    • one year ago
    Best Response
    You've already chosen the best response.
    Medals 0

    salehmamdouh1984, in effect we are considering the square of the length of the error vector ||e||^2 as a function f(C,D); Prof Strang shows on the board that in his example f(C,D) = (C+D-1)^2 + (C+2D-2)^2 + (C+3D-2)^2; the intention is to choose C and D to minimise f(C,D). The way to find the minimum is to find the two partial derivatives df/dC and df/dD; we know that there is a stationary point where df/dC=df/dD=0. Prof Strang glosses over the point, but it's not hard to show (by taking second partial derivatives) that the stationary point is a minimum. Of course the whole point is to demonstrate that calculus gets you the same answer you can get much more quickly by looking for e to be orthogonal to the columns of A ==> A'Ax = A'b. Josh.

  2. anonymous
    • one year ago
    Best Response
    You've already chosen the best response.
    Medals 0

    What I am trying to understand,what is the difference between doing the projection method VS the gradient descent algorithm ?

  3. JoshDanziger23
    • one year ago
    Best Response
    You've already chosen the best response.
    Medals 0

    salehmamdouh1984, isn't the gradient descent algorithm a numerical technique to use when you can't find a minimum any other way? I'm not sure that that's called for when you can go direct to the solution either by projection or by calculus. Josh.

  4. Not the answer you are looking for?
    Search for more explanations.

    • Attachments:

Ask your own question

Sign Up
Find more explanations on OpenStudy
Privacy Policy

Your question is ready. Sign up for free to start getting answers.

spraguer (Moderator)
5 → View Detailed Profile

is replying to Can someone tell me what button the professor is hitting...

23

  • Teamwork 19 Teammate
  • Problem Solving 19 Hero
  • You have blocked this person.
  • ✔ You're a fan Checking fan status...

Thanks for being so helpful in mathematics. If you are getting quality help, make sure you spread the word about OpenStudy.

This is the testimonial you wrote.
You haven't written a testimonial for Owlfred.