DLA Theory

Section 22.6 Theory

In this section.

🔗

Subsection 22.6.1 Similar matrices

First, we’ll record just a few of the facts about general similar matrices from Section 22.3.

🔗

Proposition 22.6.1. Properties of similar matrices.

Similar matrices have the same determinant.
Similar matrices have the same characteristic polynomial.
Similar matrices have the same eigenvalues, with the same algebraic multiplicities.

🔗

Proof of Statement 1.

Suppose square matrices

A

and

B

are similar, and

P

is a transition matrix that realizes the similarity, so that

B = P^{- 1} A P .

We know from Proposition 10.5.6 that the determinant of a product is the product of the determinants. And we also know from Proposition 10.5.8 that the determinant of an inverse is the inverse of the determinant. So we can compute

det B

\begin{aligned} det B & = det (P^{- 1} A P) \\ = (det P^{- 1}) (det A) (det P) \\ = {(det P)}^{- 1} (det A) (det P) \\ = \frac{(det A) (det P)}{det P} \\ = det A . \end{aligned}

Thus, the similar matrices

A

and

B

have the same determinant.

Careful: In this proof, it would have been incorrect to cancel the

P^{- 1}

with the

P

immediately, because order of matrix multiplication matters! It was only after we split the determinant into a product of determinants that we could cancel

det (P^{- 1})

with

det P

because all three of the determinants are numbers, and order of number multiplication does not matter.

Proof of Statement 2.

Suppose square matrices

A

and

B

are similar, and

P

is a transition matrix that realizes the similarity, so that

B = P^{- 1} A P .

The characteristic polynomials of these two matrices are computed as

\begin{aligned} c_{A} (λ) & = det (λ I - A), & c_{B} (λ) & = det (λ I - B) . \end{aligned}

Using our assumption

B = P^{- 1} A P,

along with

I = P^{- 1} I P,

we can express the matrix involved in the characteristic polynomial for

B

λ I - B = λ P^{- 1} I P - P^{- 1} A P = P^{- 1} (λ I - A) P,

where in the last step we have factored the common

P^{- 1}

and

P

factors out of the difference (making sure to factor each to the correct side, because order of matrix multiplication matters). We have now shown that matrices

λ I - A

and

λ I - B

are also similar, via the same transition matrix

P,

and so by Statement 1 they have the same determinant. That is,

c_{B} (λ) = det (λ I - B) = det (λ I - A) = c_{A} (λ),

and thus the similar matrices

A

and

B

have the same characteristic polynomial.

Proof of Statement 3.

Statement 3 follows immediately from Statement 2, as the eigenvalues of a matrix are precisely the roots of the characteristic polynomial of the matrix, and the algebraic multiplicity of an eigenvalue is the number of times that value is repeated as a root of the characteristic polynomial.

🔗

Subsection 22.6.2 Diagonalizable matrices

We start with the justification that a transition matrix made up of linearly independent eigenvectors will diagonalize a matrix.

🔗

Theorem 22.6.2. Characterization of diagonalizability.

n \times n

matrix

A

is diagonalizable if and only if there exists a set of

n

linearly independent vectors in

R^{n},

each of which is an eigenvector of

A .

P

is an

n \times n

matrix whose columns are linearly independent eigenvectors of

A,

then

P

diagonalizes

A .

🔗

Proof.

This fact follows from our analysis of the transition matrix

P

and the diagonal form matrix

P^{- 1} A P

in Subsection 22.4.1.

We will refine this theorem using our more sophisticated notions of algebraic and geometric multiplicity in the next subsection. But first, here is a surprising result that demonstrates how central eigenvalues are in matrix theory.

🔗

Proposition 22.6.3. Determinant versus eigenvalues.

If a square matrix is diagonlizable, then its determinant is equal to the product of its eigenvalues (including multiplicities).

🔗

Proof.

Suppose

A

is a diagonalizable matrix. Then it is similar to some diagonal matrix

D .

The eigenvalues of a diagonal matrix are precisely the diagonal entries, and the algebraic multiplicity of each of these eigenvalues is the number of times that eigenvalue is repeated down the diagonal. So if

λ_{1}, λ_{2}, \dots, λ_{ℓ}

are all of the distinct eigenvalues of

D

(i.e. there are no repeats in this list of eigenvalues), and

m_{1}, m_{2}, \dots, m_{ℓ}

are the corresponding algebraic multiplicities of these eigenvalues (i.e. each

m_{j}

is equal to the number of times

λ_{j}

appears on the main diagonal of

D

), then

det D = λ_{1}^{m_{1}} λ_{2}^{m_{2}} \dots λ_{ℓ}^{m_{ℓ}},

because the determinant of a diagonal matrix is just the product of its diagonal entries (Statement 1 of Proposition 8.5.2). But from Statement 1 of Proposition 22.6.1 we know that the similar matrices

A

and

D

have the same determinant, and have all the same eigenvalues with the same corresponding algebraic multiplicities. Thus, the expression

det A = det D = λ_{1}^{m_{1}} λ_{2}^{m_{2}} \dots λ_{ℓ}^{m_{ℓ}}

can be viewed as an expression for

det A

as a product of the eigenvalues of

A,

including multiplicities.

Remark 22.6.4.

The above fact is actually true about all square matrices, if you allow complex eigenvalues. In a second linear algebra course, you may learn that diagonalizable matrices are a special case of a more general theory, in which every matrix can be triangularized. That is, every square matrix is similar to a special form of triangular matrix (either upper or lower), though for many matrices both the transition matrix and the triangular form matrix might need to contain complex numbers in its entries. In this more general theory, it is again the case that the diagonal entries of the triangular form matrix will be precisely the eigenvalues of the original matrix, with each eigenvalue repeated down the diagonal according to its algebraic multiplicity, so the proof provided for the fact above can be adapted to work in this slightly more general setting.

🔗

Subsection 22.6.3 The geometry of eigenvectors

We require that the columns of a transition matrix

P

be linearly independent, so that

P

is invertible. Basis vectors for a particular eigenspace are linearly independent by definition of basis. But when we lump basis vectors from different eigenspaces together, will they all remain linearly independent together? The next fact answers this question with a more general version of what we explored in Discovery 22.6.

🔗

Proposition 22.6.5. Eigenvectors from different eigenvalues are independent.

Suppose

A

is an

n \times n

matrix, and

S

is a linearly independent set of vectors in

R^{n},

each of which is an eigenvector for

A .

Further suppose that

v

is another eigenvector for

A

that is linearly independent from those vectors in

S

that are from the same eigenspace as

v .

Then the enlarged collection

S^{'}

of eigenvectors consisting of all vectors in

S

along with

v

is also linearly independent.

🔗

Proof.

Let’s write

S = {v_{1}, v_{2}, \dots, v_{ℓ}, w_{1}, w_{2}, \dots, w_{m}},

where the

v_{j}

are those eigenvectors in

S

that are in the same eigenspace as

v,

and the

w_{j}

are those that are not. Write

λ

for the eigenvalue of

A

corresponding to

v

(hence also to each

v_{j}

), and write

λ_{j}

for the eigenvalue corresponding to

w_{j} .

We have assumed that the full set

S

is linearly independent, and therefore so are the subsets

{v_{1}, v_{2}, \dots, v_{ℓ}}

and

{w_{1}, w_{2}, \dots, w_{m}}

(Statement 2 of Statement 17.5.3). In addition, we have assumed that the set

{v, v_{1}, v_{2}, \dots, v_{ℓ}}

remains linearly independent.

The strategy in this proof is essentially the same as explored in Discovery 22.6. To prove independence, we must prove that the assumption

\begin{matrix} (✶) & \begin{aligned} k v + & a_{1} v_{1} + a_{2} v_{2} + \dots + a_{ℓ} v_{ℓ} \\ + b_{1} w_{1} + b_{2} w_{2} + \dots + b_{m} w_{m} = 0 \end{aligned} \end{matrix}

leads to the conclusion that each of the scalars

k, a_{1}, a_{2}, \dots, a_{ℓ}, b_{1}, b_{2}, \dots, b_{m}

0 .

Since each of the vectors in the combination above is an eigenvector for

A,

if we multiply both sides of equation (✶) by the matrix

A,

we may substitute

A v = λ v,

A v_{j} = λ v_{j},

and

A w_{j} = λ_{j} w_{j} .

Making these substitutions, we obtain

\begin{aligned} k λ v + & a_{1} λ v_{1} + a_{2} λ v_{2} + \dots + a_{ℓ} λ v_{ℓ} \\ + b_{1} λ_{1} w_{1} + b_{2} λ_{2} w_{2} + \dots + b_{m} λ_{m} w_{m} = 0 . \end{aligned}

Compare this “

A

times (✶)” equation with the result of multiplying (✶) through by the scalar

λ :

\begin{aligned} k λ v + & a_{1} λ v_{1} + a_{2} λ v_{2} + \dots + a_{ℓ} λ v_{ℓ} \\ + b_{1} λ w_{1} + b_{2} λ w_{2} + \dots + b_{m} λ w_{m} = 0 . \end{aligned}

Notice that the

v

and

v_{j}

terms of both the “

A

times (✶)” equation and the “

λ

times (✶)” equation are identical, so if we subtract these equations and collect like

w_{j}

-terms, we obtain

b_{1} (λ_{1} - λ) w_{1} + b_{2} (λ_{2} - λ) w_{2} + \dots + b_{m} (λ_{m} - λ) w_{m} = 0 .

Since the collection of

w_{j}

vectors are linearly independent, the scalar coefficient expressions in this new linear combination must all be zero. That is, each scalar expression

b_{j} (λ_{j} - λ)

must be zero. However, none of the

w_{j}

is from the same eigenspace as

v,

so each

λ_{j} - λ

is nonzero, which forces each of the

b_{j}

to be zero.

Substituting this new information into equation (✶), we have

k v + a_{1} v_{1} + a_{2} v_{2} + \dots + a_{ℓ} v_{ℓ} = 0 .

But the collection

{v, v_{1}, v_{2}, \dots, v_{ℓ}}

is assumed independent, so each of the scalars in the remaining combination on the left above is also zero.

We have now successfully shown that the only way equation (✶) can be true is if each of the scalars involved is

0,

as required.

The proposition above is somewhat similar in effect to Proposition 17.5.6, in that it lets us build up a linearly independent set of eigenvectors one-by-one. But the above fact is a little stronger, in that when we look to add a new eigenvector to our collection, we only need to worry about it being linearly independent from the eigenvectors we already have from that eigenspace. This leads to the following corollary.

🔗

Corollary 22.6.6. Eigenspaces are independent.

Given a collection of bases for the different eigenspaces of a matrix, the collection of all these eigenspace basis vectors together will still be linearly independent.

🔗

Proof.

Let

A

be a square matrix, and write

λ_{1}, λ_{2}, \dots, λ_{ℓ}

for its eigenvalues. Suppose we have a basis

B_{1}

for eigenspace

E_{λ_{1}} (A),

and a basis

B_{2}

for eigenspace

E_{λ_{2}} (A),

and so on. Begin with

B_{1},

which is linearly independent because it is a basis for a subspace. Enlarge

B_{1}

with vectors from

B_{2},

one at a time. At each step we may apply Proposition 22.6.5, because each new vector from

B_{2}

is both

from a different eigenspace than the vectors in $B_{1},$ and
linearly independent from the previous vectors from $B_{2}$ already included in the new enlarged collection.

Proposition 22.6.5 tells us that at each step of enlarging our collection by one, the new, larger collection will remain linearly independent. Once we run out of vectors in

B_{2},

we begin enlarging our collection with vectors from

B_{3},

one at a time. Again, Proposition 22.6.5 applies at each enlargement step, so that each collection of eigenvectors along the way remains linearly independent. Carry this process through to the end, until finally all vectors from

B_{ℓ}

are also included, and Proposition 22.6.5 will still apply at the last step to tell us that the complete set of basis eigenvectors is linearly independent.

In the next subsection, we will use this corollary to refine our initial characterization of diagonalizability stated in Theorem 22.6.2. In the meantime, we will formally state the relationship between geometric and algebraic multiplicities that we discussed in Subsection 22.4.2.

🔗

Theorem 22.6.7. Geometric versus algebraic multiplicity.

The geometric multiplicity of an eigenvalue is always less than or equal to its algebraic multiplicity.

🔗

Proof.

We will not include the proof of this statement here — you may encounter it in further study of matrix forms, perhaps in a second course in linear algebra.

Remark 22.6.8.

As we’ve noted already, the geometric multiplicity of an eigenvalue is always at least one, since otherwise it wouldn’t have any corresponding nonzero eigenvectors!

🔗

Subsection 22.6.4 More about diagonalizable matrices

Corollary 22.6.6 tells us that when collecting eigenvectors to make up the transition matrix

P,

we only have to worry about linear independence inside eigenspaces; linear independence between eigenspaces is automatic. But linear independence inside an eigenspace

E_{λ_{j}} (A)

is taken care of for us when we row reduce

λ_{j} I - A .

So our initial characterization of diagonalization in Theorem 22.6.2 can be refined so that we don’t actually have to worry about linear independence of eigenvectors at all — we just have to worry about having enough eigenspace basis vectors. It turns out that the algebraic multiplicity of each eigenvector is exactly the necessary number of basis vectors for the corresponding eigenspace, and the next statements record this thinking.

🔗

Corollary 22.6.9. More characterizations of diagonalizability.

A matrix with real eigenvalues is diagonalizable if and only if each eigenvalue has geometric multiplicity equal to its algebraic multiplicity.
An $n \times n$ matrix that has $n$ different real eigenvalues must be diagonalizable.

🔗

Proof of Statement 1.

We need

n

linearly independent eigenvectors to make up the columns of the

n \times n

transition matrix

P .

The maximum number of linearly independent eigenvectors we can get from a single eigenspace is

\dim (E_{λ} (A)),

the geometric multiplicity of the eigenvalue

λ .

So the maximum number of linearly independent eigenvectors we can get in total is the sum of the geometric multiplicities of the eigenvalues. But the characteristic polynomial

c_{A} (λ)

has degree

n,

and

n

is the sum of the algebraic multiplicities of the eigenvalues, because if

A

has all real eigenvalues, then

c_{A} (λ)

factors as

c_{A} (λ) = (λ - λ_{1})^{m_{1}} (λ - λ_{2})^{m_{2}} \dots (λ - λ_{ℓ})^{m_{ℓ}} .

So if even one eigenvalue is deficient in the sense that its geometric multiplicity is strictly less than its algebraic multiplicity, we won’t obtain enough linearly independent eigenvectors from that eigenspace to contribute to the

n

linearly eigenvectors we need in total.

On the other hand, if each eigenvalue has geometric multiplicity equal to its algebraic multiplicity, then forming eigenspace bases and collecting them all together will provide us with exactly

n

eigenvectors, and Proposition 22.6.5 tells us that these

n

eigenvectors will be linearly independent.

Proof of Statement 2.

In the case that a square matrix has

n

different real eigenvalues, then each of these eigenvalues must have algebraic multiplicity

1,

since otherwise these

n

algebraic multiplicities would add up to more than

n,

the degree of the characteristic polynomial. So each geometric multiplicity is no greater than

1 .

But also, as in noted in Remark 22.6.8, each geometric multiplicity must be at least

1 .

Thus, each geometric multiplicity for this matrix is exactly

1,

and so is equal to the corresponding algebraic multiplicity.

The result now follows from the first statement of this corollary.

🔗

Prev Top Next