Unit II-2. Orthogonal projection. Orthogonal projection. Orthogonal projection. the scalar is called the component of u along v. two vectors u,v are

Transcription

1 Orthogonal projection Unit II-2 Orthogonal projection the scalar is called the component of u along v in real ips this may be a positive or negative value in complex ips this may have any complex value two vectors u,v are orthogonal if and only if proj(u,v) = 0 linearly dependent if and only if proj(u,v) = u Unit II-2 Orthogonal projection 1 Unit II-2 Orthogonal projection 3 Orthogonal projection Example. For u=(1,-3,4), v=(3,4,7) find proj(u,v) and proj(v,u). the orthogonal projection of u on v is the vector C-S says that proj(u,v) u just evaluate it and apply C-S at the critical moment Unit II-2 Orthogonal projection 2 Unit II-2 Orthogonal projection 4

2 Orthogonal complement V is an ips, S is a subset of vectors the orthogonal complement of S [ S perp ] is S is a subspace of V: w S, u,v S, and a,b scalars then au+bv,w = a u,w + b v,w =a(0) + b(0) = 0 so au+bv S if S = {w} write w instead of S for simplicity if W is subspace of V then W W = {0} (W ) = W Unit II-2 Orthogonal projection 5 Rowspace and nullspace again The row space R and null space N of a matrix are orthogonal complements. example on slide 6 illustrates this given a basis for R you can find the nullspace N by finding the orthogonal complement R given a basis for the nullspace N you can find R by finding the orthogonal complement N Unit II-2 Orthogonal projection 7 Example. Find a basis for W if W is the subspace of R 5 spanned by u=(1,2,3,-1,2), v=(2,4,7,2,-1). A geometric view of planes the vector equation of a plane in R 3 is (r - r 0 ) n = 0 or r n = r 0 n or proj(r,n) = proj(r 0,n) n r r 0 Unit II-2 Orthogonal projection 6 Unit II-2 Orthogonal projection 8

3 Orthogonal projection onto a subspace Orthogonal projection onto a subspace have projection onto a line what about onto a plane? v is decomposed into a sum of two orthogonal vectors: v = v 1 + v 2 v 1 W and v 2 W geometrically the decomposition is obviously unique v 2 = v - proj(v,w) n v 1 = proj(v,w) v For any subspace W how do we calculate proj(v,w)? Unit II-2 Orthogonal projection 9 Unit II-2 Orthogonal projection 11 Orthogonal projection onto a subspace take any subspace W of an ips V V can be decomposed as V = W W any vector v V can be decomposed uniquely as v = v 1 + v 2 with v 1 W and v 2 W for instance for T A :V U the decomposition is V = rowspace A nullspace A Orthogonal sets a set of non-zero vectors {u 1, u 2,..., u n } is an orthogonal set if u i,u j = 0 for i j an orthogonal set is an orthonormal set if u i = 1 for all i orthogonal sets are linearly independent Unit II-2 Orthogonal projection 10 Unit II-2 Orthogonal projection 12

4 Examples: orthogonal sets the standard basis vectors are an orthonormal basis for R n the basis {(1,2), (1,-1)} for R 2 is not an orthogonal basis the vectors {1, cos t, cos 2t,..., sin t, sin 2t,...} are an orthogonal set in C [-π,π] this is the foundation for Fourier analysis Orthogonal bases the decomposition of v is or Unit II-2 Orthogonal projection 13 Unit II-2 Orthogonal projection 15 Orthogonal bases choose an orthogonal basis {u 1, u 2,..., u n } for V express a vector v in terms of this basis: Example. S={u 1,u 2,u 3,u 4 } with u 1 =(1,1,0,-1), u 2 =(1,2,1,3), u 3 =(1,1,-9,2), u 4 =(16,-13,1,3) is an orthogonal basis. Find the coordinates of a general vector (a,b,c,d) with respect to S. v = a 1 u 1 + a 2 u a n u n now calculate v,u i = a i u i,u i = a i u i,u i and solve for the scalars these are called the Fourier coefficients of v with respect to the orthogonal basis chosen Unit II-2 Orthogonal projection 14 Unit II-2 Orthogonal projection 16

5 Projection onto a subspace illustration for a plane in R 3 v proj(v,u 2 ) proj(v,u 1 ) u 2 v 1 = proj(v,w) u 1 Unit II-2 Orthogonal projection 17 Unit II-2 Orthogonal projection 19 Orthogonal projection onto a subspace now we can answer the question about how to define and calculate proj(v,w) Example. Find the proj(v,w) for v = (1,3,5,7) and W < R 4 spanned by {u 1,u 2 } where u 1 =(1,1,1,1), u 2 =(1,-3,4,-2). choose an orthogonal basis {u 1, u 2,..., u r } for W the projection of v on W is this o n l y works if the basis is orthogonal Unit II-2 Orthogonal projection 18 Unit II-2 Orthogonal projection 20

6 Orthonormal bases if is an orthonormal basis things can be written more neatly... this gives exactly the previous formula when written as projections Gramm-Schmidt algorithm this procedure converts any old basis {u 1,u 2,...,u n } of V into an orthogonal basis define w 1 = u 1 w 2 = u 2 - proj(u 2,w 1 ) w 3 = u 3 - proj(u 3,w 1 ) - proj(u 3,w 2 )...and so on then {w 1,w 2,...,w n } is an orthogonal basis of V normalize the w i to get an orthonormal basis Unit II-2 Orthogonal projection 21 Unit II-2 Orthogonal projection 23 Projection onto a subspace Why Gramm-Schmidt works: geometrically What do you do if your basis for W isn t orthogonal? w 3 u 3 proj(u 3,w 2 ) proj(u 3,w 1 ) w 2 w 1 Unit II-2 Orthogonal projection 22 Unit II-2 Orthogonal projection 24

7 Why Gramm-Schmidt works: algebraically say you have the orthogonal set {w 1,w 2,...,w r } w r+1 = u r+1 a 1 w 1 a 2 w 2... a r w r the Fourier coefficients are w r+1 is orthogonal to all w 1,..., w r vectors Unit II-2 Orthogonal projection 25 Unit II-2 Orthogonal projection 27 Example. Find an orthonormal basis for the subspace W of R 4 spanned by {u 1,u 2,u 3 } where u 1 =(1,1,1,1), u 2 =(1,2,4,5) u 3 =(1,-3,-4,-2). Gramm-Schmidt in practice G-S is not a very robust numerical algorithm, but it s valuable for important theoretical reasons e.g. construction of the Legendre polynomials in the next example you won t usually have more than three basis vectors because the calculations can be tedious to avoid messy arithmetic clear all fractions as you make your choice for each w i vector any multiple of each w i will do the job just as well normalize all the w i vectors at the very last step once you have the orthogonal basis the procedure works on any linearly independent set it gives an orthogonal basis for the subspace it spans Unit II-2 Orthogonal projection 26 Unit II-2 Orthogonal projection 28

8 Example. P 3 (t) with f,g defined on the interval [-1,1]. Apply G-S to the mononomial basis to find an orthogonal basis called Legendre polynomials. Example. Find the projection of v = (1,3,5,7) onto the subspace W spanned by {u 1,u 2 } where u 1 =(1,1,1,1), u 2 =(1,2,3,2). Unit II-2 Orthogonal projection 29 Unit II-2 Orthogonal projection 31 Unit II-2 Orthogonal projection 30 Unit II-2 Orthogonal projection 32

9 Example. Find an orthogonal basis for w with w=(1,2,3,1) without using the G-S algorithm. Projection onto a subspace {u 1,..., u p } an orthonormal basis for W<R n U = [ u 1... u p ] is n p proj (y,w) = UU T y for all y R n Unit II-2 Orthogonal projection 33 Unit II-2 Orthogonal projection 35 Orthogonal projection onto a subspace is the closest point to y in W for any v W, is the best approximation to y by vectors in W y d W proj(y,w) Unit II-2 Orthogonal projection 34 Unit II-2 Orthogonal projection 36