## KonradZuse Group Title 6.4 #2b In Exercise 2 find the least squares solution of the linear equation Ax = b. b. > A = (Matrix(3, 2, {(1, 1) = 2, (1, 2) = -2, (2, 1) = 1, (2, 2) = 1, (3, 1) = 3, (3, 2) = 1})); b = (Vector(3, {(1) = 2, (2) = -1, (3) = 1})); one year ago one year ago

$\left[\begin{matrix}2 & -2 \\ 1 & 1 \\ 3 & 1\end{matrix}\right]$ $\left(\begin{matrix}2 \\ -1 \\ 1\end{matrix}\right)$

now my book says to use the formula A^T A = a^T b

and then solve.... I'll post the pratice problem.

5. tkhunny Group Title

You need a most marvelous construct: $$\left(A^{t}\cdot A\right)^{-1}\cdot A^{t}$$ It is a beautifu thing.

how do we get that solution of x1 and x2? Maybe I'm over tihnking it...

Hmm... That's diff from the formula they say...

8. tkhunny Group Title

No, it is the same. It's just the "solved" version given the conclusion that $$\left(A^{t}\cdot A\right)$$ is NonSingular.

IUc... so how do I figure out x1 and x2 at the end? I'm just confused on that part... I understand how to do it if it was just 1 matrix, but a matrix * x = another matrix I'm not so sure about?

like how did they get the answers to the question I posted above?

11. tkhunny Group Title

Construct that delightful matrix. I get [3/7,-2/3]^T Find $$A^{t}\cdot A$$, first.

The one I posted, or their example?

13. tkhunny Group Title

I didn't look at the piosted example. Let's go with your original problem statement.

kk

so we have $\left[\begin{matrix}2 & -2 \\ 1 & 1 \\ 3 & 1\end{matrix}\right]$

16. tkhunny Group Title

That's "A".

yup now to find A^t

18. tkhunny Group Title

Which we hope is trivial, except for the type-setting. :-)

A^T should be $\left[\begin{matrix}2 & 1 & 3 \\ -2 & 1 & 1 \end{matrix}\right]$

20. tkhunny Group Title

Multiply away and you will be pleasantly surprised at the result.

$\left[\begin{matrix}14 & 0 \\ 0 & 6\end{matrix}\right]$ is what I got.

22. tkhunny Group Title

Side note: Be a little more careful with your notation. So far, you have used "A" and "a' interchangeably. Also, both "T" and "t" for transpose. More consistent behavior will eliminate errors.

23. tkhunny Group Title

Perfect. Now the inverse of that.

The book doesn't do that but okay.

25. tkhunny Group Title

Positive and Diagonal - Definitely NonSingular!

$\left[\begin{matrix}6 & 0 \\ 0 & 14 \end{matrix}\right]$

27. tkhunny Group Title

Fair enough. I took a look at the example. They are setting up a small system of equestions and asking you to solve the system using any method at your disposal. On method of solving such a system is hte Matrix Inverse. That is what we are doing.

28. tkhunny Group Title

Close! Divide by 84.

30. tkhunny Group Title

Remember that $$A\cdot A^{-1} = I$$

31. tkhunny Group Title

Yes, that's all you left off. Good work.

1/(16 * 4) - (0 * 0)

= 1/64

34. tkhunny Group Title

84

woop sI did 4 not 6 :p

and 16 not 14 lol...

yeah 1/84

so we don't have to take the inverse, do we?

39. tkhunny Group Title

ahahah Yes, $$16*4 \ne 14*6$$

40. tkhunny Group Title

You did already find the inverse. It is 1/84 times the 6,14 diagonal matrix.

yeah but since the book doesn't, do we have to?

42. tkhunny Group Title

Yes, the book does. When it introduces x1 and x2, that is exactly what it is doing. The book just isn't using matrix methods to solve the system. This strikes me as rather odd, since this is inherently a matrix problem.

hmmm... Konfused... but if you say so ;P.

44. tkhunny Group Title

You started with Ax = b We added the transpose: $$A^{t}\cdot A\cdot x = A^{t}\cdot b$$

mhm.

46. tkhunny Group Title

Now we have a system to solve. One way to solve this is to find hte inverse of $$A^{t}\cdot A$$, if it has an inverse. It does.

$\left[\begin{matrix}\frac{1}{14} & 0 \\ 0 & \frac{1}{6}\end{matrix}\right]$

48. tkhunny Group Title

This gives: $$(A^{t}\cdot A)^{-1}\cdot (A^{t}\cdot A)\cdot x = (A^{t}\cdot A)^{-1}\cdot A^{t}\cdot b$$, using the inverse to solve the system.

seems like we are just adding it in :P

shall I solve for A^T B?

and solving the first portion yeilds the identity matrix.

52. tkhunny Group Title

The inverse SOLVES the system, just like substitution or elimination or whatever other method you might use to solve the system.

53. tkhunny Group Title

Yes, that is what hte inverse does. The left-hand-side is just 'x' when we are done. This symbolizes the solved system.

I think I'm understanding where this is going... Instead of having to row reduce....

lke a billion times easier :P

I think I've been doing it the hard way :p

$\left(\begin{matrix}6 \\ -4\end{matrix}\right)$

A^Tb

59. tkhunny Group Title

Different problems may benefit from different methods. I think you're not quite done with this one. We don't need just $$A^{t}\cdot b$$, we need $$x = (A^{t}\cdot A)^{-1}\cdot A^{t}\cdot b$$

that * the inverse = $\left(\begin{matrix}\frac{3}{7} \\ \frac{-2}{3}\end{matrix}\right)$

61. tkhunny Group Title

Done! VERY good work. This was cutting-edge stuff 40 years ago. That's WAY closer to current research than your basic 300-400 year old calculus. :-)

so x1 = 3/7 and x2 = -2/3???

or am I supposed to solve for that now?!

64. tkhunny Group Title

That's it.

65. tkhunny Group Title

Just for the record, just this Friday, I did a job for my employer using exactly this technique. My "A" was (462 x 4), so I was pretty glad I didn't have to do it by hand. :-)

Damn.... Well that was easier than what the book was doing if it came out all nicely like that :) Thanks!

67. tkhunny Group Title

Like I said, it is a little odd that they would introduce this technique and just bail at the end, requiring you to resort to Algebra II. Maybe the author thinks you're not ready for the Inverse? Odd, indeed.

This book really SUCKS..... Sometimes it's okay....

I have another Q that no one really answered if you don't mind taking a look at it.. I just wanted to be sure I did it correctly.

70. tkhunny Group Title

Well, good luck with THAT (the book and author, that is)! (laughing)