Discover Linear Algebra

These statements follow from combining Proposition 37.5.1 with Lemma 18.5.7 and Corollary 20.5.6.

The first statement follows immediately from the Linearity of inner products. And the second statement follows immediately from the first using \(a_j = \frac{1}{\norm{\uvec{v}_j}} \text{.}\) (Note that dividing by each norm can be done because Proposition 37.5.1 implies that none of the vectors in the orthogonal set can be zero.)

When \(\ell = 2\text{,}\) we have the familiar Pythagorean formula, but we provide a proof based on the inner product:

where the mixed inner product terms are zero using the orthogonality assumption.

For \(\ell \gt 2\text{,}\) the proof can proceed inductively, where the \(\ell = 2\) case will be required in the induction step.

1. This is straightforward computation using Linearity of inner products, along with the orthonormal conditions:
   \begin{align*} \inprod{\uvec{e}_i}{\uvec{e}_i} \amp = 1 \text{,} \amp \inprod{\uvec{e}_i}{\uvec{e}_j} \amp = 0 \qquad (j \neq i)\text{.} \end{align*}
2. Apply Theorem 37.5.4 to the expansion for \(\uvec{v}\) relative to \(\basisfont{B}\text{,}\) first noting Proposition 37.5.3.

First, note that there \(n\) vectors in the original basis of \(\uvec{v}_j\) vectors, so that the dimension of the inner product space is \(n\text{.}\) And there are just as many of the \(\uvec{e}_j\) vectors, so if we can prove orthogonality then the \(\uvec{e}_j\) vectors will also form a basis by Statement 2 of Corollary 37.5.2.

The proof of orthogonality is by induction. First we prove that \(\uvec{e}_2\) is orthogonal to \(\uvec{e}_1\text{:}\)

Next, we inductively assume that \(S_\ell = \{\uvec{e}_1,\dotsc,\uvec{e}_\ell\}\) is an orthogonal set for some fixed \(\ell\text{,}\) with \(2 \le \ell \lt n\text{.}\) Based on this assumption, we will show that \(\uvec{e}_{\ell + 1}\) is orthogonal to each of the vectors in \(S_\ell\text{.}\)

For every index \(j\text{,}\) \(1 \le j \le \ell\text{,}\) we have

By our assumption that \(S_\ell\) is an orthogonal set, each of the inner products \(\inprod{\uvec{e}_i}{\uvec{e}_j}\) in the brackets is zero except for the \(\nth[j]\) one:

This shows that each new \(\uvec{e}_{\ell + 1}\) vector is always orthogonal to each of the previous ones.

An orthogonal set is linearly independent (Proposition 37.5.1), and every linearly independent set in a finite-dimensional space can be enlarged to a basis (Proposition 20.5.4). But can it be enlarged to an orthogonal basis?

Suppose \(\{\uvec{e}_1,\dotsc,\uvec{e}_\ell\}\) is an orthogonal set in a finite-dimensional inner product space. Enlarge the orthogonal set to a (possibly non-orthogonal) basis

We could then apply the Gram-Schmidt process to obtain an orthogonal basis. But since the \(\uvec{e}_j\) are orthogonal, the first \(\ell\) steps will just result in the original \(\{\uvec{e}_1,\dotsc,\uvec{e}_\ell\}\) (see Discovery 37.7). So the result of the Gram-Schmidt process will be an orthogonal basis that contains the \(\uvec{e}_j\) as its first \(\ell\) vectors.

If we were to “unravel” the inductive definition of each \(\uvec{e}_j\text{,}\) we would find that it is ultimately a linear combination of the vectors \(\uvec{v}_1, \dotsc, \uvec{v}_j\text{,}\) and so we can conclude that

is always a subspace of

(Statement 1 of Proposition 17.5.6).

But we also know that each spanning set above is linearly independent, so these two subspaces have the same dimension. Therefore, since one is contained in the other, they must be the same subspace (Statement 3 of Proposition 20.5.8).

Apply the Subspace Test.

The first statement is obvious because every vector \(\uvec{v}\) in \(V\) satisfies \(\inprod{\zerovec}{\uvec{v}} = 0 \text{.}\)

The second statement is also straightforward, as a vector in \(\orthogcmp{V}\) must be orthogonal to every vector in \(V\text{,}\) even itself. But Axiom RIP 4 (real case) and Axiom CIP 4 (complex case) both forbid nonzero vectors to satisfy \(\uvecinprod{x}{x} = 0\text{.}\)

Suppose \(U\) is a subspace of a finite-dimensional inner product space. Rather than use the definition of independent subspaces, we will appeal to Theorem 28.6.4, which says that it is sufficient to check that \(U\) and \(\orthogcmp{U}\) share only the zero vector in common. But this is immediate, since if a vector is in both \(U\) and \(\orthogcmp{U}\text{,}\) then it must be orthogonal to itself. However, Axiom RIP 4 (real case) and Axiom CIP 4 (complex case) imply that the zero vector is orthogonal to itself.

We have now shown that every vector in the collection \(\orthogcmp{U}\) is also in the collection \(\orthogcmp{S}\text{,}\) and vice versa, which means that they must be the same collection of vectors.

For convenience, write

From Proposition 37.5.17, we know that \(\orthogcmp{U} = \orthogcmp{S}\text{,}\) the collection of vectors that are orthogonal to every vector in \(S\text{.}\) Since the entire set

is orthogonal, we know that each vector in \(S'\) is in \(\orthogcmp{U}\text{.}\) And since \(\orthogcmp{U}\) is a subspace (Proposition 37.5.13), it must then contain all of \(\Span S' \) (Proposition 17.5.5). However, any basis for \(\orthogcmp{U}\) cannot be larger in size than \(S'\text{,}\) since otherwise it wouldn't remain independent with \(S\) (see Theorem 37.5.16). Since \(S'\) is an orthogonal set, it is a linearly independent set (Proposition 37.5.1), and it contains the right number of vectors to be a basis for \(\orthogcmp{U}\text{.}\) Therefore, \(\orthogcmp{U} = \Span S'\text{,}\) as desired.

We already know from Theorem 37.5.16 that in a finite-dimensional inner product space, a subspace and its orthogonal complement are independent subspaces. Then Corollary 37.5.10 can be combined with Proposition 37.5.18 to conclude that the dimensions of a subspace and its orthogonal complement must add to the dimension of the whole space, from which we can conclude that they form a complete set of subspaces.

Suppose \(U\) is a subspace of a finite-dimensional inner product space \(V\text{.}\) Choose a basis for \(U\text{,}\) and then extend into a basis for \(V\text{.}\) Applying the Gram-Schmidt process, we obtain an orthogonal basis for \(V\text{,}\) which can be split in two parts: the first part will be a basis for \(U\) and the second part will be a basis for \(\orthogcmp{U}\text{,}\) as in Proposition 37.5.18. (Also see Proposition 37.5.11.) But then we could apply Proposition 37.5.18 again, reversing our point of view: the second part will still be a basis for \(\orthogcmp{U}\text{,}\) but we could view the first part as a basis for \(\orthogcmp{(\orthogcmp{U})}\text{.}\) Matching the two points of view confirms that \(\orthogcmp{(\orthogcmp{U})} = U\text{.}\)