## KonradZuse 3 years ago 6.4 #2b In Exercise 2 find the least squares solution of the linear equation Ax = b. b. > A = (Matrix(3, 2, {(1, 1) = 2, (1, 2) = -2, (2, 1) = 1, (2, 2) = 1, (3, 1) = 3, (3, 2) = 1})); b = (Vector(3, {(1) = 2, (2) = -1, (3) = 1}));

$\left[\begin{matrix}2 & -2 \\ 1 & 1 \\ 3 & 1\end{matrix}\right]$ $\left(\begin{matrix}2 \\ -1 \\ 1\end{matrix}\right)$

now my book says to use the formula A^T A = a^T b

and then solve.... I'll post the pratice problem.

5. tkhunny

You need a most marvelous construct: $$\left(A^{t}\cdot A\right)^{-1}\cdot A^{t}$$ It is a beautifu thing.

how do we get that solution of x1 and x2? Maybe I'm over tihnking it...

Hmm... That's diff from the formula they say...

8. tkhunny

No, it is the same. It's just the "solved" version given the conclusion that $$\left(A^{t}\cdot A\right)$$ is NonSingular.

IUc... so how do I figure out x1 and x2 at the end? I'm just confused on that part... I understand how to do it if it was just 1 matrix, but a matrix * x = another matrix I'm not so sure about?

like how did they get the answers to the question I posted above?

11. tkhunny

Construct that delightful matrix. I get [3/7,-2/3]^T Find $$A^{t}\cdot A$$, first.

The one I posted, or their example?

13. tkhunny

I didn't look at the piosted example. Let's go with your original problem statement.

kk

so we have $\left[\begin{matrix}2 & -2 \\ 1 & 1 \\ 3 & 1\end{matrix}\right]$

16. tkhunny

That's "A".

yup now to find A^t

18. tkhunny

Which we hope is trivial, except for the type-setting. :-)

A^T should be $\left[\begin{matrix}2 & 1 & 3 \\ -2 & 1 & 1 \end{matrix}\right]$

20. tkhunny

Multiply away and you will be pleasantly surprised at the result.

$\left[\begin{matrix}14 & 0 \\ 0 & 6\end{matrix}\right]$ is what I got.

22. tkhunny

Side note: Be a little more careful with your notation. So far, you have used "A" and "a' interchangeably. Also, both "T" and "t" for transpose. More consistent behavior will eliminate errors.

23. tkhunny

Perfect. Now the inverse of that.

The book doesn't do that but okay.

25. tkhunny

Positive and Diagonal - Definitely NonSingular!

$\left[\begin{matrix}6 & 0 \\ 0 & 14 \end{matrix}\right]$

27. tkhunny

Fair enough. I took a look at the example. They are setting up a small system of equestions and asking you to solve the system using any method at your disposal. On method of solving such a system is hte Matrix Inverse. That is what we are doing.

28. tkhunny

Close! Divide by 84.

30. tkhunny

Remember that $$A\cdot A^{-1} = I$$

31. tkhunny

Yes, that's all you left off. Good work.

1/(16 * 4) - (0 * 0)

= 1/64

34. tkhunny

84

woop sI did 4 not 6 :p

and 16 not 14 lol...

yeah 1/84

so we don't have to take the inverse, do we?

39. tkhunny

ahahah Yes, $$16*4 \ne 14*6$$

40. tkhunny

You did already find the inverse. It is 1/84 times the 6,14 diagonal matrix.

yeah but since the book doesn't, do we have to?

42. tkhunny

Yes, the book does. When it introduces x1 and x2, that is exactly what it is doing. The book just isn't using matrix methods to solve the system. This strikes me as rather odd, since this is inherently a matrix problem.

hmmm... Konfused... but if you say so ;P.

44. tkhunny

You started with Ax = b We added the transpose: $$A^{t}\cdot A\cdot x = A^{t}\cdot b$$

mhm.

46. tkhunny

Now we have a system to solve. One way to solve this is to find hte inverse of $$A^{t}\cdot A$$, if it has an inverse. It does.

$\left[\begin{matrix}\frac{1}{14} & 0 \\ 0 & \frac{1}{6}\end{matrix}\right]$

48. tkhunny

This gives: $$(A^{t}\cdot A)^{-1}\cdot (A^{t}\cdot A)\cdot x = (A^{t}\cdot A)^{-1}\cdot A^{t}\cdot b$$, using the inverse to solve the system.

seems like we are just adding it in :P

shall I solve for A^T B?

and solving the first portion yeilds the identity matrix.

52. tkhunny

The inverse SOLVES the system, just like substitution or elimination or whatever other method you might use to solve the system.

53. tkhunny

Yes, that is what hte inverse does. The left-hand-side is just 'x' when we are done. This symbolizes the solved system.

I think I'm understanding where this is going... Instead of having to row reduce....

lke a billion times easier :P

I think I've been doing it the hard way :p

$\left(\begin{matrix}6 \\ -4\end{matrix}\right)$

A^Tb

59. tkhunny

Different problems may benefit from different methods. I think you're not quite done with this one. We don't need just $$A^{t}\cdot b$$, we need $$x = (A^{t}\cdot A)^{-1}\cdot A^{t}\cdot b$$

that * the inverse = $\left(\begin{matrix}\frac{3}{7} \\ \frac{-2}{3}\end{matrix}\right)$

61. tkhunny

Done! VERY good work. This was cutting-edge stuff 40 years ago. That's WAY closer to current research than your basic 300-400 year old calculus. :-)

so x1 = 3/7 and x2 = -2/3???

or am I supposed to solve for that now?!

64. tkhunny

That's it.

65. tkhunny

Just for the record, just this Friday, I did a job for my employer using exactly this technique. My "A" was (462 x 4), so I was pretty glad I didn't have to do it by hand. :-)

Damn.... Well that was easier than what the book was doing if it came out all nicely like that :) Thanks!

67. tkhunny

Like I said, it is a little odd that they would introduce this technique and just bail at the end, requiring you to resort to Algebra II. Maybe the author thinks you're not ready for the Inverse? Odd, indeed.

This book really SUCKS..... Sometimes it's okay....

I have another Q that no one really answered if you don't mind taking a look at it.. I just wanted to be sure I did it correctly.

70. tkhunny

Well, good luck with THAT (the book and author, that is)! (laughing)