Discover Linear Algebra: A first course in linear algebra

In Discovery 13.1, we compared the graph of the cosine function on the domain $0\le \theta \le \pi$ with the formula

where $\theta$ is the angle between nonzero vectors $\mathbf{u}$ and $\mathbf{v}\text{.}$ On the right of equation (✶), the denominator is always positive, so whether the whole fraction is positve, negative, or zero depends entirely on the dot product in the numerator. On the left, the cosine function is positive, negative, or zero precisely when the angle $\theta$ is acute, obtuse, or right. So we come to the following conclusions.

As mentioned, since $P$ is the closest point, the directed line segment $\overrightarrow{PQ}$ must be perpendicular to $\ell \text{.}$ On the diagram above, we have used the vector $\mathbf{n}$ to represent this direct line segment. As in Discovery 13.3.b, we know that $\mathbf{n}\mathbf{\cdot }\mathbf{a}$ must be zero — this is the perpendicular condition. However, the vector $\mathbf{n}$ is unknown as well, since we don’t know its initial point. But we can also use the triangle formed by $\mathbf{u}\text{,}$ $\mathbf{n}\text{,}$ and $\mathbf{p}$ to replace $\mathbf{n}\text{:}$

Replacing $\mathbf{n}$ by this expression in the condition $\mathbf{n}\mathbf{\cdot }\mathbf{a}=0$ gives us an equation of numbers that we can solve for the unknown scale factor $k\text{,}$ as we did in Discovery 13.3.d:

This vector $\mathbf{p}=k\mathbf{a}$ pointing from the origin to the desired closest point $P$ is called the projection of $\mathbf{u}$ onto $\mathbf{a}$ or sometimes the vector component of $\mathbf{u}$ parallel to $\mathbf{a}$, and we write ${\mathrm{proj}}_{\mathbf{a}}\mathbf{u}$ to represent it.

The normal vector $\mathbf{n}$ in the diagram above is sometimes called the vector component of $\mathbf{u}$ orthogonal to $\mathbf{a}$. Together, the projection vector and corresponding normal vector are called components of $\mathbf{u}$ (relative to $\mathbf{a}$) because they represent an orthogonal decomposition of $\mathbf{u}$:

where $\mathbf{p}$ is parallel to $\mathbf{a}$ and $\mathbf{n}$ is orthogonal to $\mathbf{a}\text{.}$ While this decomposition is relative to $\mathbf{a}\text{,}$ it is really only the direction of $\mathbf{a}$ that matters — if ${\mathbf{a}}^{\prime }$ is parallel to $\mathbf{a}$ (even possibly opposite to $\mathbf{a}$), then both

will be true.

Consider the line $2x+3y=0$ that we investigated in Discovery 13.6. The point $(3,-2)$ is on this line, since

The left-hand side of this calculation looks a lot like a dot product — we could reinterpret equation (✶✶) as

So verifying that the point $(3,-2)$ is on the line is equivalent to checking that the corresponding vector $\mathbf{v}=(3,-2)$ (with its tail at the origin) is orthogonal to the vector $\mathbf{n}=(2,3)$ whose components are the coefficients from our line equation.

Every other point $\mathbf{x}=(x,y)$ on the line satisfies the same relationship, as the equation for the line could be rewritten in a vector form as

The vector $\mathbf{n}$ is called a normal vector for the line. Note that normal vectors for a line are not unique — every nonzero scalar multiple of $\mathbf{n}$ will also be normal to the line, and this is equivalent to noting that we could multiply the equation $2x+3y=0$ by any nonzero factor to obtain a different equation that represents the same line in the plane.

In the homogeneous case, vectors from the origin determined by a point on the line were also parallel to the line. Since things have shifted away from the origin in the nonhomogeneous case, to get a vector parallel to the line we need to consider two vectors from the origin to points on the line. Two convenient points for this the line are $Q(1,2)$ and $R(4,0)\text{,}$ with corresponding vectors ${\mathbf{x}}_0=(1,2)$ and ${\mathbf{x}}_1=(4,0)\text{.}$ Then the difference vector

is parallel to the line, as in the diagram above. In fact, this vector $\mathbf{v}$ is the same as previous vector $\mathbf{v}$ that appears parallel to the line through the origin in the diagram for the homogeneous case above, so we know it satisfies $\mathbf{n}\mathbf{\cdot }\mathbf{v}=0\text{.}$

Every such difference vector $\mathbf{x}-{\mathbf{x}}_0$ is parallel to the line and hence orthogonal to the normal vector $\mathbf{n}\text{,}$ so that we can describe the line as all points where the corresponding vector $\mathbf{x}$ satisfies

This is called the point-normal form for the line, referring to the point on the line at the terminal point of ${\mathbf{x}}_0$ and the normal vector $\mathbf{n}\text{.}$

So if $\mathbf{u}=(u_1,u_2,u_3)$ and $\mathbf{v}=(v_1,v_2,v_3)$ are our starting vectors, we would like to simultaneously solve the equations

for the unknown vector $\mathbf{x}=(x,y,z)\text{.}$ Expanding out the dot products, we get (surprise!) a system of linear equations:

Specifically, we get a homogeneous system of two equations in the three unknown coordinates $x,y,z\text{.}$ Now, since this system is homogeneous, it is consistent. But its general solution will also require at least one parameter, since its rank is at most $2\text{,}$ while we have three variables. In the diagram above, we can see what the “freedom” of a parameter corresponds to — we can make $\mathbf{x}$ longer or shorter, or turn it around to be opposite of the way it is pictured, and it will remain orthogonal to $\mathbf{u}$ and $\mathbf{v}\text{.}$ Our end goal is a calculation formula and procedure that will compute one particular solution to this problem, so let’s introduce a somewhat arbitrary additional equation to eliminate the need for a parameter in the solution.

In matrix form, this system can be expressed as $A\mathbf{x}=\mathbf{b}\text{,}$ with

Assuming that $detA\ne 0\text{,}$ Cramer’s rule tells us the solution to this system.

Now, each of $x,y,z$ has a common factor of $1/detA\text{,}$ and all this common factor does is scale the length of our solution vector $\mathbf{x}$ without affecting orthogonality with $\mathbf{u}$ and $\mathbf{v}\text{.}$ Even worse, $detA$ depends on that extra equation we threw in, and we would like our solution to depend only on $\mathbf{u}$ and $\mathbf{v}\text{.}$ So let’s remove it and use solution

We call this the cross product of $\mathbf{u}$ and $\mathbf{v}$, and write $\mathbf{u}\times \mathbf{v}$ instead of $\mathbf{x}\text{.}$ There is a trick to remembering how to compute the cross product: if we replace the top row of $A$ by the standard basis vectors $\mathbf{i},\mathbf{j},\mathbf{k}$ in ${\mathbb{R}}^3\text{,}$ then the cross product will be equal to its determinant expanded by cofactors along the first row. That is, setting

and expanding the determinant along the first row yields

as desired. See Example 13.4.4 in Subsection 13.4.3 for an example of using formula (†††) to compute cross products.

Computing $\mathbf{v}\times \mathbf{u}$ instead of $\mathbf{u}\times \mathbf{v}$ should still produce a vector that is orthogonal to both $\mathbf{u}$ and $\mathbf{v}\text{,}$ but the right-hand rule tells that the two should be opposite to each other. From equation (†††) we can be even more specific. Computing $\mathbf{v}\times \mathbf{u}$ would swap the second and third rows of the special matrix in equation (†††), and we know that the resulting determinant would be the negative of that for the matrix for computing $\mathbf{u}\times \mathbf{v}\text{,}$ and so

See Proposition 13.5.5 in Subsection 13.5.3 for more properties of the cross product.