Section 28.6 Theory
In this section.
Subsection 28.6.1 Properties of block-diagonal form
First, we'll record some of the properties of block-diagonal matrices explored in the latter part of Discovery guide 28.2. As we have sufficiently explored most of these patterns in Discovery guide 28.2, we state these properties without proof.
Proposition 28.6.1.
Suppose \(A\) and \(B\) are \(n \times n\) matrices in block-diagonal form such that each has the same number of blocks and corresponding blocks are the same size. That is, assume
where \(A_1\) and \(B_1\) are both \(n_1 \times n_1\text{,}\) \(A_2\) and \(B_2\) are both \(n_2 \times n_2\text{,}\) etc..
- The product \(A B\) can be computed by taking the product of corresponding blocks, and powers (including the inverse) of \(A\) can be computed by taking powers of blocks. That is,\begin{align*} A B \amp= \begin{bmatrix} A_1 B_1 \\ \amp A_2 B_2 \\ \amp \amp \ddots \\ \amp \amp \amp A_\ell B_\ell \end{bmatrix} \text{,} \amp A^k \amp= \begin{bmatrix} A_1^k \\ \amp A_2^k \\ \amp \amp \ddots \\ \amp \amp \amp A_\ell^k \end{bmatrix} \text{,} \end{align*}where the second pattern is valid for all integer powers (assuming \(A\) is invertible in the case \(k \le 0\)).
- If \(\uvec{x}_1\) is a vector in \(\R^{n_1}\text{,}\) \(\uvec{x}_2\) is a vector in \(\R^{n_2}\text{,}\) etc., and \(\uvec{x}\) is the vector in \(\R^n\) formed by concatenating the vectors \(\uvec{x}_1,\uvec{x}_2,\dotsc,\uvec{x}_\ell\) together, then the product \(A \uvec{x}\) is similarly the vector in \(\R^n\) formed by concatenating the vectors \(A_1\uvec{x}_1,A_2\uvec{x}_2,\dotsc,A_\ell\uvec{x}_\ell\) together. That is,\begin{equation*} A \uvec{x} = \begin{bmatrix} A_1 \\ \amp A_2 \\ \amp \amp \ddots \\ \amp \amp \amp A_\ell \end{bmatrix} \begin{bmatrix} \uvec{x}_1 \\ \uvec{x}_2 \\ \vdots \\ \uvec{x}_\ell \end{bmatrix} = \begin{bmatrix} A_1 \uvec{x}_1 \\ A_2 \uvec{x}_2 \\ \vdots \\ A_\ell \uvec{x}_\ell \end{bmatrix}\text{.} \end{equation*}
- If \(\uvec{x}_j\) is a null space vector of block \(A_j\text{,}\) for each index \(j\text{,}\) then the vector \(\uvec{x}\) in \(\R^n\) formed by concatenating the vectors \(\uvec{x}_1,\uvec{x}_2,\dotsc,\uvec{x}_\ell\) together will be a null space vector for \(A\text{.}\)
- The rank \(A\) is the sum of the ranks of the blocks \(A_j\text{,}\) and similarly for the nullity of \(A\text{.}\)
- The determinant of \(A\) is the product of the determinants of the blocks in \(A\text{.}\) That is,\begin{equation*} \det A = (\det A_1) (\det A_2) \dotsm (\det A_\ell) \text{.} \end{equation*}
- The characteristic polynomial of \(A\) is the product of the characteristic polynomials of the blocks in \(A\text{.}\) That is,\begin{equation*} c_A(\lambda) = c_{A_1}(\lambda) c_{A_2}(\lambda) \dotsm c_{A_\ell}(\lambda) \text{.} \end{equation*}
- The eigenvalues of \(A\) are precisely the collection of eigenvalues of the blocks of \(A\text{.}\) If \(\uvec{x} \in \R^{n_j}\) is an eigenvector of block \(A_j\) corresponding to eigenvalue \(\lambda\text{,}\) then the vector\begin{equation*} \left[\begin{array}{l} \zerovec_{n_1} \\ \zerovec_{n_2} \\ \vdots \\ \uvec{x} \\ \vdots \\ \zerovec_{n_\ell} \end{array}\right] \end{equation*}is an eigenvector of \(A\) corresponding to \(\lambda\text{,}\) where each zero vector \(\zerovec_{n_k}\) has size corresponding to the size \(n_k\) of the block \(A_k\text{,}\) and vector \(\uvec{x}\) appears in the \(\nth[j]\) block.
Subsection 28.6.2 Invariant subspaces
As usual, to understand a subspace it is enough to understand a spanning set for the subspace, and this is true for invariant subspaces as well.
Proposition 28.6.2.
Suppose \(A\) is an \(n \times n\) matrix, and \(\{ \uvec{w}_1, \uvec{w}_2, \dotsc, \uvec{w}_\ell \}\) is a spanning set for a subspace \(W\) of \(\R^n\text{.}\) Then \(W\) is \(A\)-invariant if and only if each of the vectors \(A\uvec{w}_1, A\uvec{w}_2, \dotsc, A\uvec{w}_\ell\) is again in \(W\text{.}\)
Proof.
Recall that with an “if and only if” statement, there are two things to prove.
(\(\Rightarrow\)).
Assume that \(W\) is \(A\)-invariant. Then, by definition, for every vector \(\uvec{w}\) in \(W\) the vector \(A\uvec{w}\) is again in \(W\text{.}\) Since each of the spanning vectors \(\uvec{w}_1,\uvec{w}_2,\dotsc,\uvec{w}_\ell\) lies in \(W\text{,}\) then clearly each of the vectors \(A\uvec{w}_1,A\uvec{w}_2,\dotsc,\uvec{w}_\ell\) lies again in \(W\text{.}\)
(\(\Leftarrow\)).
Assume that each of the vectors \(A\uvec{w}_1,A\uvec{w}_2,\dotsc,\uvec{w}_\ell\) lies in \(W\text{.}\) We must prove that for every vector \(\uvec{w}\) in \(W\text{,}\) the vector \(A\uvec{w}\) will again lie in \(W\text{.}\) Since \(\{\uvec{w}_1,\dotsc,\uvec{w}_\ell\}\) is a spanning set for \(W\text{,}\) a vector \(\uvec{w}\) in \(W\) can be expressed as a linear combination of these spanning vectors, say
Then,
Above we have expressed \(A \uvec{w}\) as a linear combination of the vectors \(A \uvec{w}_1, A \uvec{w}_2, \dotsc, A \uvec{w}_\ell\text{,}\) each of which, by assumption, lies in \(W\text{.}\) Therefore, since \(W\) is a subspace, it is closed under vector operations, and so we also have \(A\uvec{w}\) in \(W\text{.}\)
Here is a special example of an invariant subspace that will become important in our further study of matrix forms. It also helps explain diagonal form as a special case of block-diagonal form.
Proposition 28.6.3.
Suppose \(\lambda\) is an eigenvalue of the \(n \times n\) matrix \(A\text{.}\) Then the eigenspace \(E_\lambda (A)\) is an \(A\)-invariant subspace of \(\R^n\text{.}\)
Proof idea.
The point that must be verified is that if \(\uvec{w}\) is a vector in the eigenspace \(E_\lambda (A)\text{,}\) then the transformed vector \(\uvec{u} = A \uvec{w}\) is also in that eigenspace. Use the definition of eigenvector (i.e. \(A \uvec{x} = \lambda \uvec{x}\)), both applied to \(\uvec{x} = \uvec{w}\text{,}\) and as the condition to be verified in the case \(\uvec{x} = \uvec{u}\text{.}\)
Subsection 28.6.3 Independent subspaces
Here is a special case of testing independence of subspaces, in the case of two subspaces.
Theorem 28.6.4.
Subspaces \(W_1,W_2\) of a finite dimensional vector space are independent if and only if \(W_1 \cap W_2 = \{\zerovec\}\text{.}\)
Proof.
Again, with an “if and only if” statement, there are two things to prove. In both cases, begin with bases
for \(W_1\) and \(W_2\text{,}\) respectively, where \(d_1 = \dim W_1\) and \(d_2 = \dim W_2\text{.}\)
(⇒)
Assume that \(W_1\) and \(W_2\) are independent. By definition, this means that the combined set
is linearly independent.
We must prove that \(W_1 \cap W_2 = \{\zerovec\}\text{.}\) So suppose \(\uvec{v}\) lies in the intersection \(W_1 \cap W_2\text{.}\) If we are to prove that \(W_1\cap W_2\) consists of only the zero vector, then we must prove that \(\uvec{v} = \zerovec\text{.}\)
Since \(\uvec{v}\) is in \(W_1\text{,}\) there exist scalars \(a_1, a_2, \dotsc,a_{d_1}\) such that
But \(\uvec{v}\) also lies in \(W_2\text{,}\) and so there exist scalars \(b_1, b_2, \dotsc, b_{d_2}\) such that
Now, from the vector space identity \(\uvec{v} - \uvec{v} = \zerovec\text{,}\) we can subtract these two linear combination expressions for \(\uvec{v}\) to get
Linear independence of the combined set of basis vectors says that all of the scalars above are zero, so that
as desired.
(⇐)
Assume that \(W_1 \cap W_2 = \{\zerovec\}\text{.}\) We must prove that \(W_1,W_2\) are independent. That is, we need to verify that the combined collection \(\{ \uvec{u}_1, \dotsc, \uvec{u}_{d_1}, \uvec{w}_1, \dotsc, \uvec{w}_{d_2} \}\) remains linearly independent.
Setting up the test for linear independence, we assume that
Set
Then \(\uvec{v}\) is in \(W_1\text{.}\) But by moving all the \(\uvec{w}_j\) terms to the other side of (\(\star\)), we obtain a second expression for \(\uvec{v}\text{,}\)
so that \(\uvec{v}\) is also in \(W_2\text{.}\) Therefore, \(\uvec{v} \in W_1 \cap W_2\text{.}\) By assumption, this intersection contains only the zero vector, so we must have \(\uvec{v} = \zerovec\text{.}\) Our two different expressions for \(\uvec{v}\) as linear combinations lead to
These expressions are essentially the test for linear independence, one for basis \(\basisfont{B}_1\) and one for basis \(\basisfont{B}_2\text{.}\) As each of these bases must be an independent set, these two tests for independence lead to the conclusions
So, in fact, all of the coefficients in (\(\star\)) must be zero, and the test for independence has been confirmed.
Warning 28.6.5.
The condition \(W_1 \cap W_2 = \{\zerovec\}\) is not saying that the intersection of \(W_1\) and \(W_2\) is empty! In fact, it is impossible for two subspaces of any vector space to be disjoint (i.e. have empty intersection), since every subspace of a vector space contains the zero vector. The condition \(W_1 \cap W_2 = \{\zerovec\}\) is saying that the intersection of \(W_1\) and \(W_2\) contains only the zero vector.
Can independent subsets always be “completed” to a complete set of independent subspaces? In the finite-dimensional case, the answer is yes, for essentially the same reason that Proposition 20.5.4 is true.
Proposition 28.6.6. Complement spaces.
For every subspace \(W\) of a vector space \(V\text{,}\) there exists a complement subspace \(W'\) of \(V\) so that \(W,W'\) form a complete set of independent subsets.
Proof idea.
Choose a basis for \(W\text{,}\) and use Proposition 20.5.4 to enlarge it to a basis for all of \(V\text{.}\) Take \(W'\) to be the span of the new vectors that were used to enlarge the initial basis for \(W\text{,}\) and verify that the pair \(W,W'\) satisfies the definition of complete set of independent subspaces. It might be convenient to use Theorem 28.6.4 to check independence of the pair.
Corollary 28.6.7.
If \(W_1,W_2,\dotsc,W_m\) form a collection of independent subspaces of a vector space \(V\text{,}\) then there exists a subspace \(W'\) of \(V\) so that the enlarged collection \(W_1,W_2,\dotsc,W_m,W'\) is a complete set of independent subspaces.
Proof idea.
Apply Proposition 28.6.6, taking \(W\) to be the span of the initial collection of subspaces \(W_1,W_2,\dotsc,W_m\) all together.
Similar to how a matrix can be broken into blocks, a complete set of independent subspaces lets us break up a vector in the space into constituent parts.
Proposition 28.6.8.
Suppose \(W_1,W_2,\dotsc,W_m\) form a complete set of independent subspaces of a vector space \(V\text{.}\) Then for every vector \(\uvec{w}\) in \(V\) there exists a unique collection of vectors \(\uvec{w}_1,\uvec{w}_2,\dotsc,\uvec{w}_m\text{,}\) with each \(\uvec{w}_j\) in the corresponding space \(W_j\text{,}\) so that
Proof idea.
Choose a collection of bases \(\basisfont{B}_1,\basisfont{B}_2,\dotsc,\basisfont{B}_m\) for the subspaces \(W_1,W_2,\dotsc,W_m\text{,}\) and express \(\uvec{w}\) as a linear combination of these basis vectors taken all together.
As mention in Subsubsection 28.4.4.1 the concept of independent subspaces is in fact a generalization of the concept of linearly independent set of vectors. We'll formally state this fact now.
Proposition 28.6.9.
A collection of vectors \(\uvec{w}_1,\uvec{w}_2,\dotsc,\uvec{w}_m\) is linearly independent if and only if the collection of subspaces
is independent.
Proof.
We leave this proof to you, the reader.
Finally, here is a special example of collection of independent subspaces that will become important in our further study of matrix forms. Along with Proposition 28.6.3, it helps explain diagonal form as a special case of block-diagonal form.
Theorem 28.6.10.
The various eigenspaces of an \(n \times n\) matrix always form an independent set of subspaces of \(\R^n\text{.}\)
Proof idea.
Combine Proposition 28.6.9 with Proposition 25.6.6.