Schmidt Process can be applied, this means that every nonzero subspace of 8 has an orthonormal basis.

Transcription

1 8The Gram-Schmidt Process The Gram-Schmidt Process is an algorithm used to convert any basis for a subspace [ of 8 into a new orthogonal basis for [. This orthogonal basis can then be normalized, if desired, to get an orthonormal basis for [Þ Since every nonzero subspace [ has a basis to which the Gram- Schmidt Process can be applied, this means that every nonzero subspace of 8 has an orthonormal basis. It is important to think about what the algorithm is doing: that makes it much easier to remember and use the algorithm. Therefore this presentation, while doing the very same thing as in the text, tries harder than the text to separate the idea from the calculations The idea behind Gram- Schmidt Process is contained in the following Lemma. Lemma Suppose ßÞÞÞß 3 is an orthogonal set of nonzero vectors in 8 and [ œ Span ßÞÞÞß 3. Pick a vector B  [ and write B œ B s where Bs œ proj [ B [ and [ Þ Then i) Span ßÞÞÞß 3 ß B œ Span ßÞÞÞß 3ß, and ii) the enlarged set ßÞÞÞß ß is orthogonal NOTE: in the Orthogonal Decomposition Theorem, we wrote B œ Bs D where D [ Þ Here, I'm using the letter (rather than the letter D ) to denote the orthogonal component of BÞ In this context, I think that helps the discussion to read more smoothly. Informally, what does the Lemma say? The 3 's are orthogonal to each other, so ßÞÞÞß 3 is a linearly independent set and a basis for [. Then we enlarge [ to Span ßÞÞÞß 3ß B by adding to the basis a new vector B from outside [. Since B is not a linear combination of ßÞßß 8, ßÞÞÞß 3ß B is linearly independent and is a basis for the enlarged subspace Span ßÞÞÞß ß B. But the Bs part of B was already in [ before we enlarged; it is ß the orthogonal component of B Ð from [ Ñ that actually enlarges [Þ Just adding the orthogonal component of B to ßÞÞÞß 3 has the same effect as adding the whole vector B to ßÞÞÞß 3, and since [ ß the new set ßÞÞÞß 3ß is an orthogonal basis for the enlarged subspace. Proof i) To show that Span ßÞÞÞß ß B œ Span ßÞÞÞß ß, we need to show that a) every vector A in Span ßÞÞÞß ß B is also in Span ßÞÞÞß ß, b) every vector A in Span ßÞÞÞß ß is also in Span ßÞÞÞß ß B and For short, let's use the notation lc Ð ß ÞÞÞß 3 Ñ to mean a linear combination of ß ÞÞÞß 3. With this shorthand, for example, lc Ð ßÞÞÞß Ñ lc Ð ßÞÞÞß Ñ œ lc Ð ßÞÞÞß Ñ (?) why

2 Since Bs is in [, we know that B s œ lcð ßÞÞÞß Ñß For a) If A is in Span ßÞÞÞß 3ß B, then A œ lc Ð ßÞÞÞß 3Ñ -B œ lc Ð ßÞÞÞß 3Ñ -Ð B s Ñ œ lc Ð ßÞÞÞß 3Ñ -B s - œ lcð ßÞÞÞß 3 Ñ lcð ßÞÞÞß 3Ñ - œ lcð ßÞÞÞß 3 Ñ - ß so A is in Span ß ÞÞÞß ß Þ For b) If A is in Span ßÞÞÞß 3ß, then A œ lc Ð ßÞÞÞß 3Ñ - œ lc Ð ßÞÞÞß 3Ñ -ÐB Bs Ñ œ lcð ßÞÞÞß 3 Ñ - B lcð ßÞÞÞß 3Ñ œ lcð ßÞÞÞß 3 Ñ -B, so A is in Span ß ÞÞÞß ß B ii) ßÞÞÞß 3 is an orthogonal set and (each 3) because [. So ßÞÞÞß ß is an orthogonal set. ñ The Gram-Schmidt Process (GSP) If you understand the preceding lemma, the idea behind the Gram-Schmidt Process is very easy. We want to convert an basis ÖB ß ÞÞÞß B: for [ into an orthogonal basis ßÞÞÞß :. We build the orthogonal basis by replacing each vector B3 with a vector 3. We do this from the ground up ß one B3 at a time, starting with B Þ Step 1. To start, let œ B, and call [ œ SpanÖ B œ Span Ö Step 2: Now enlarge [: let [ œ SpanÖB ß B œ Span ß B Ðsince œ B ÑÞ Decompose B œ B s, where Bs œ proj[ B and [. But we don't need the whole vector B to get [. According to the Lemma, [ œ SpanÖB ß B œ Span ß B œ Span Ö ß, and, by the Lemma ß Ö ß is an orthogonal set. Of course, we already know a formula for : since [ œ SpanÖ ß œ B projw B œ B Step 3: Now enlarge [: let [ œ Span ÖB ß B, B œ Span ß,. Decompose œ B s, where Bs œ proj[ B and [ Þ But we don't need the whole vector B to get [ 3 According to the Lemma, [ œ Span ÖB ß B, B œ Span ß, B œ Span Ö ß ß ß and, by the Lemma, Ö ß ß is an orthogonal set. Of course, we already know a formula for : since [ œ Span ß, œ B proj B œ B W

3 Next: Keep repeating this process as long as you can. Suppose we have arrived at [ 3 œ SpanÖB ßÞÞÞß B 3 œ Span ßÞÞÞß 3, where ßÞÞÞß 3 is an orthogonal set. If 3 : ( that is, if there are still some B 3 's remaining), then use B 3 to enlarge [ 3 again: [ 3 œ SpanÖB ßÞÞÞßB3ß B 3 œ Span ßÞÞÞß 3ß B3 Þ Once again, we don't need the whole vector B 3 ; as before, it's enough to use 3 œ the component of B3 orthogonal to [ 3. By the Lemma, we have SpanÖB ßÞÞÞßB3ß B 3 œ Span ßÞÞÞß 3ß B 3 œ Span ßÞÞÞß 3ß 3 and ßÞÞÞß ß is an orthogonal set. 3 And, of course, we know a formula for : since [ œ Span ß ßÞÞÞ, œ B proj W3 B œ B ÞÞÞ 3 3 Finally, we run out of B3 's and arrive at [: œ SpanÖB ßÞÞÞß B : œ Span ßÞÞÞß :, and ßÞÞÞß : is an orthogonal set that is a basis for the original subspace [ œ [: Þ (For certain purposes it's useful to remember that at each step in the process, the set ÖB ßÞÞÞß B has the same span as the set ßÞÞÞß You'll feel much better about the Gram-Schmidt Process if you understand the preceding material about what's going on in the calculations. However, when you solve problems you might just use a programmed recipe for the vectors in the orthogonal basis. Suppose ÖB ß ÞÞÞß B : is a basis for the subspace [ of 8. Then ß ß ÞÞÞß : is an orthogonal basis for [ where œ B œ B proj B œ B B B B œ B proj proj œ B % œ B% proj B proj B proj B % % % B % B % B % œ B % ã : œ B: proj B proj B ÞÞÞ... proj B : : : : B : B : B : : œ B ÞÞÞ... : : : :

4 Example Use the Gram-Schmidt Process to find an orthogonal basis for Ô Ô Ô [ œ Span ßߟ and explain some of the details at each step. Õ Õ Õ Å Å Å B B You can check that B ßB ß are linearly independent and therefore form a basis for [. What would happen if we applied the Gram-Schmidt Process to a linearly dependent set of vectors ÖB ßÞÞÞß B :? Ô Let œ B œ ß and [ œ SpanÖ B Span œ Ö Þ Õ Let [ œ SpanÖB ß B œ Span ß B Þ Using the Lemma, we can replace B with œ the component of B orthogonal to [ œ B proj B proj [ B Ô Ô Ô Ô Ô œ œ œ, so Õ Õ œ Ô Ô Ö Ù Õ Õ Õ Õ Õ Ô Ô Ô œ B proj [ B œ œ Ö Ù Ö Ù Õ Õ Õ Optional: rescale to avoid working with the fractions in the remaining calculations: multiply by and instead use % Ô Ô œ œ ( explain why rescaling doesn't mess anything up. ) Õ Õ % Then SpanÖB ß B œ Span ß B œ Span ß Þ % ( cont. )

5 Let [ œ Span ÖB ß B, œ Span ß, Using the Lemma, we can replace with œ the component of orthogonal to [ œ proj [ œ B proj B proj B proj [ B œ œ Ô Ô Ô Ô Ô Õ Õ Õ Õ % Ô Ô Ô Ô Õ Ù Ö Ù Ö Õ Õ Õ % Õ % Ô Õ % Ô Ô Ô * œ œ %, Õ Õ % Õ so Ô Ô Ô œ proj [ œ œ Õ Õ Õ Then [ œ Span ÖB ß B, B œ Span Ö ß,, and we have replaced all the B 's. [ is the 3 subspace [ we were originally given, and an orthogonal basis for [ is ß ß œ Ô Ô Ô ßÖ ÙßÖ ÙŸÞ Õ Õ % Õ