offing orthogonal i n pwjwy ftp.deu.l#ew7 Hinatnx 4 entries Maydays ftp. pvojuts ,Ep} Ip ] T.dz project tup y.tn FAR project onto Fithian duty of Rn

Transcription

1 Maydays Last time onto learned we that project to spanned by line FAR vector a calculate we Apr 5 / offing pvojuts project Tci u # of Rn " ;5 orthogonal pwjwy To onto p E & Ep} {ut basis results sum if ytn n!! Tdz projwy U is Fithian Eu is of basis + U matrix take an project onto each " W tup + ftpdeul#ew7 8 ( gtip) i n p ] ftp equal n orthonornal is f an Uty of duty are vector product Hinatnx By definition of to 4 entries # n form we UTU n orthogonal matrix represents orthogonal projection onto f J Tt ( 5 Tee ) reduces to we W of # Now W subspace a at Col UUT and U

2 Here's a typical example for a 3D subspace in R4 to give you a sense of how this looks W Col ( this matrix ) W Col ( this matrix too ) & this matrix s orthogonal! See! UTU T multiplying this matrix by any vector projects that vector onto W rank ( UUT ) 3

3 64 Grain Schmidt What was that function on previous page that turned our random matrix into an orthogonal one with same column space? # # not orthogonal orthogonal t's called Grain Schmidt & idea s pretty Leave first vector as is 2 Project second vector onto first & subtract that part out Fish to make it orthogonal 3 Project third vector onto span of first two ; this is we easy have an bk already orthogonal basis for this span

4 We " Tp iud & # $ 37 % Etc n math notation } s liu set C) By VT e) 82 it a bt is Fight iffy t f Pz ip tee tff Pm Then Span { Be ip Tbp } Spank } ) and p} fb are orthogonal can also normalize m to unit length if we want 65 least Squares We will conclude course by looking at

5 a central concept in statistics least squares regression suppose we have some data that show a trend 7 d like to figure out general slope of gnors ) ' a line that roughly follows data like this one One way to do that s to look for line of which minimizes squared error ; sum squares of all lengths of red segments like shown ones above this feels like a calculus problem because we're trying to optimize a

6 The function but will see that re's an elegant linear algebra solution define t vector of sepal lengths vector of sepal widths nee Then what we want s to find a B so that B ( at + > Be 1* ~ is as small as A possible Letting [ ] and [ ] we're looking to minimize wqglfod This we know how to do! Axe Col A so we're looking for vector in Col A that s as close to 8 as possible 8 orthogonal projection of out adt! by? tax 2 [ Note Col A is a QD subspace in R " where n is number of data points thus typically quite large ]

7 (85)0 Let b) b) Let's figure out how to solve this equation without orthogonal iging projcoab & let satisfy At D n a % each column of A will a orthogonal to let A CAT at ; n AT for all j sajtcb O for all ATCD; 8 Now this equation says Atb Atax f Ata is invertible n A CATAJATB s least squares solution! this is a very famous fouuula you're sure to see again Let's see how this works [ next page]

8 We load data calculate CAT ' Atb directly & plot results line plot line this s important CATAYATB \ looks pretty good!