Section 40.4 Concepts
Subsection 40.4.1 Diagonalization of Hermitian and symmetric matrices
Our first test case, explored in Discovery 40.3, was of a Hermitian matrix. In that discovery activity, with a full slate of linearly independent eigenvectors provided, we were easily able to produce a transition matrix \(P\) so that \(\inv{P} H P\) was diagonal. And, since the provided eigenvectors were already orthogonal, to convert \(P\) into a unitary matrix \(U\) so that \(\adjoint{U} H U\) was diagonal, all that was required was to normalize the eigenvectors to unit vectors.
Aside: Note.
As considered in Discovery 40.3.c, the crucial fact to make this work was that the eigenvectors were orthogonal, since unit vectors are then easy to produce by normalization.
Now, it may seem that lack of orthogonality is also easy to “fix” using the Gram-Schmidt process, and indeed we could use that process to convert an eigenspace basis into an orthogonal one. But as we found in the non-Hermitian example of Discovery 40.4, it won’t work to fix eigenvectors from different eigenspaces that are not orthogonal to each other. In trying to correct a lack of one crucial property (orthogonality of basis vectors), Gram-Schmidt may affect the other crucial property for diagonalization: basis vectors must be eigenvectors.
A diagram in three-dimensions illustrating that when Gram-Schmidt is applied to a set of eigenvectors from different eigenspaces, the resulting vectors are orthogonal but not necessarily eigenvectors. A parallelogram with a shaded-in interior is drawn. The interior of this parallelogram should be imagined as if it is a two-dimensional, solid, rectangular surface suspended within a three-dimensional space (similar to a tabletop “suspended” above the floor in a room), but viewed at an angle from above. Embedded within this two-dimensional surface is a point representing the zero vector, and two directed line segments parallel to the shaded surface, representing the vectors \(\uvec{e}_1\) and \(\uvec{e}_2\text{,}\) emanate from this point, so that they appear to lie along the shaded surface. Within the shaded surface, these two vectors are at a right angle, and the shaded surface represents a portion of the plane spanned by them. This plane is labelled as representing a two-dimensional eigenspace \(E_{\lambda_1}\) for some matrix.
A directed line segment representing a third vector \(\uvec{v}\) is drawn with initial point at point zero vector, but rising up out of the shaded surface at an acute angle. A line is drawn “through” and parallel to this vector and labelled as representing a one-dimensional eigenspace \(E_{\lambda_2}\) for the matrix. A final directed line segment representing vector \(\uvec{e}_3\) is drawn with initial point at point zero vector, rising up out of the shaded surface at a right angle so that its terminal point is at the same height above the shaded surface as the terminal point of \(\uvec{v}\text{.}\) A rectangle is drawn in dashed lines sitting vertically atop the shaded surface, with vertices at the point zero vector, the terminal points of \(\uvec{e}_3\) and \(\uvec{v}\text{,}\) and the point on the plane where the projection of \(\uvec{v}\) onto the plane would land.
Just as the fact that eigenvectors from different eigenspaces are automatically linearly independent (Proposition 25.6.5) was crucial to the process for diagonalization, eigenvectors from different eigenspaces being orthogonal is crucial to the prospect of being able to orthogonally/unitarily diagonalize a given matrix. Not every matrix has this property, but in Discovery 40.5 we discovered that self-adjoint matrices do have this property. And, in fact, we will prove in Subsection 40.6.2 that every self-adjoint matrix is orthogonally/unitarily diagonalizable.
It turns out that this is the end of the story in the real case — a real matrix is orthogonally diagonalizable if and only if it is symmetric (Theorem 40.6.13). So we will give a procedure for the real case now. (And essentially the same procedure will work for Hermitian matrices in the complex case.)
Procedure 40.4.2. Orthogonal diagonalization of a symmetric matrix.
Given a real, \(n \times n\text{,}\) symmetric matrix \(S\text{,}\) we can construct an orthogonal transition matrix \(P\) so that \(\inv{P} S P = \utrans{P} S P\) is diagonal follows.
-
Compute the eigenvalues of \(S\text{.}\) For each eigenvalue, compute a basis for the corresponding eigenspace.
-
To each eigenspace basis from the first step, apply the Gram-Schmidt orthogonalization process to produce an orthogonal basis for that eigenspace.
-
Normalize each eigenspace basis vector from the previous step to produce an orthonormal basis for each eigenspace.
-
Form the \(n\times n\) matrix \(P\) whose columns are the orthonormal eigenspace vectors from the third step collected all together.
Since the columns of \(P\) are constructed to be an orthonormal basis of \(\R^n\text{,}\) this transition matrix will be orthogonal (Statement 4 of Theorem 39.5.6). And since the columns of \(P\) also form a basis of \(\R^n\) consisting of eigenvectors for \(S\text{,}\) we will have \(\inv{P} S P = \utrans{P} S P = D\text{,}\) a diagonal matrix with the eigenvalues of \(S\) down the diagonal (Theorem 25.6.2).
See Example 40.5.1 for an example of carrying out this procedure on a real symmetric matrix, and see Example 40.5.2 for an example of carrying out this procedure on a complex Hermitian matrix.
Subsection 40.4.2 Diagonalization of normal matrices
As Discovery 40.6 demonstrated, not every unitarily diagonalizable complex matrix is Hermitian. One property that unitarily diagonalizable matrices possess that does not seem to be exclusive to Hermitian matrices is commutativity with its own adjoint (see Discovery 40.7). A complex matrix with this property is called a normal matrix. And we will prove that normal matrices have the crucial property of orthogonal eigenspaces (see Statement 2 of Item 2), so that every normal matrix is unitarily diagonalizable (see Theorem 40.6.14).
Since each self-adjoint matrix obviously commutes with itself, every Hermitian matrix is normal, and it seems like we have now completely characterized the class of unitarily diagonalizable matrices. And the procedure to unitarily diagonalize a complex normal matrix is essentially identical to the procedure to orthogonally diagonalize a real symmetric matrix.
Procedure 40.4.3. Unitary diagonalization of a normal matrix.
Given a complex, \(n \times n\text{,}\) normal matrix \(N\text{,}\) we can construct a unitary transition matrix \(U\) so that \(\inv{U} N U = \utrans{U} N U\) is diagonal follows.
-
Compute the eigenvalues of \(N\text{.}\) For each eigenvalue, compute a basis for the corresponding eigenspace.
-
To each eigenspace basis from the first step, apply the Gram-Schmidt orthogonalization process to produce an orthogonal basis for that eigenspace.
-
Normalize each eigenspace basis vector from the previous step to produce an orthonormal basis for each eigenspace.
-
Form the \(n\times n\) matrix \(U\) whose columns are the orthonormal eigenspace vectors from the third step collected all together.
Since the columns of \(U\) are constructed to be an orthonormal basis of \(\C^n\text{,}\) this transition matrix will be unitary (Statement 4 of Theorem 39.5.6). And since the columns of \(U\) also form a basis for \(\C^n\) consisting of eigenvectors for \(N\text{,}\) we will have \(\inv{U} N U = \utrans{U} N U = D\text{,}\) a diagonal matrix with the eigenvalues of \(N\) down the diagonal (using the complex version of Theorem 25.6.2).
See Example 40.5.4 for an example of carrying out this procedure.
