Discover Linear Algebra: A first course in linear algebra

A homogeneous system with square coefficient matrix has nontrivial solutions precisely when that coefficient matrix is not invertible, which is the case precisely when the determinant of that coefficient matrix is equal to zero (Theorem 10.5.3). So there will exist eigenvectors of $A$ corresponding to a particular scalar $\lambda$ precisely when $\lambda$ is a root of the characteristic equation $det(\lambda I-A)=0$.

Once we know all possible eigenvalues of a square matrix $A\text{,}$ we can substitute those values into the matrix equation $A\mathbf{x}=\lambda \mathbf{x}$ one at a time. With a value for $\lambda$ substituted in, this matrix equation is no longer nonlinear and can be solved for all corresponding eigenvectors $\mathbf{x}\text{.}$ But the homogeneous version $(\lambda I-A)\mathbf{x}=\mathbf{0}$ is more convenient to work with, since to solve this system we just need to row reduce the coefficient matrix $\lambda I-A\text{.}$

Determining eigenvectors is the same as solving the homogeneous system $(\lambda I-A)\mathbf{x}=\mathbf{0}\text{,}$ so the eigenvectors of $A$ corresponding to a specific eigenvalue $\lambda$ are precisely the nonzero vectors in the null space of $\lambda I-A\text{.}$ In particular, since a null space is a subspace of ${\mathbb{R}}^n\text{,}$ we see that the collection of all eigenvectors of $A$ that correspond to a specific eigenvalue $\lambda$ creates a subspace of ${\mathbb{R}}^n\text{,}$ once we also include the zero vector in the collection. This subspace is called the eigenspace of $A$ for eigenvalue $\lambda \text{,}$ and we write $E_{\lambda }(A)$ for it.

Recall that we do not call the zero vector an eigenvector of a square matrix $A\text{,}$ because it would not correspond to one specific eigenvalue — the equality $A\mathbf{0}=\lambda \mathbf{0}$ is true for all scalars $\lambda \text{.}$ However, the scalar $\lambda =0$ can (possibly) be an eigenvalue for a matrix $A\text{,}$ and we explored this possibility in Discovery 21.7.

In the case of $\lambda =0\text{,}$ the matrix equation $A\mathbf{x}=\lambda \mathbf{x}$ turns into the homogeneous system $A\mathbf{x}=\mathbf{0}\text{.}$ And for $\lambda =0$ to actually be an eigenvalue of $A\text{,}$ there needs to be nontrivial solutions to this equation — which we know will occur precisely when $A$ is not invertible (Theorem 6.5.2).

Multiplication of column vectors by a particular matrix can be thought of as a sort of function, i.e. an input-output process. But unlike the types of functions you are probably used to encountering, where the input is a number $x$ and the output is a number $y\text{,}$ this matrix-multiplication sort of function has a column vector $\mathbf{x}$ as input and a column vector $\mathbf{y}$ as output.

When the particular matrix used to form such a function is square, then the input and output vectors live in the same space (i.e. ${\mathbb{R}}^n\text{,}$ where $n$ is the size of the matrix), so we can think of the matrix transforming an input vector into its corresponding output vector geometrically. See Figure 21.4.5 for an example of this geometric transformation point of view.

When the input vector $\mathbf{x}$ is an eigenvector of the transformation matrix $A\text{,}$ then the output vector $A\mathbf{x}$ is a scalar multiple of $\mathbf{x}$ (where the scale factor is the corresponding eigenvalue). See Figure 21.4.6 for a geometric example of this view of eigenvectors.

Geometrically, one vector is a scalar multiple of another if and only if the two vectors are parallel. So we can say that a vector is an eigenvector of a matrix precisely when it is transformed to a parallel vector when multiplied by the matrix.