Section 12.5 Theory
Subsection 12.5.1 Norm and dot product
We’ll begin with algebraic properties of norm and dot product.
Proposition 12.5.1. Properties of the norm.
The following are true for all vectors \(\uvec{u}\) and \(\uvec{v}\) in \(\R^n\) and all scalars \(k\text{.}\)
- \(\unorm{v} \ge 0\text{,}\) and \(\unorm{v} = 0\) only for \(\uvec{v} = 0\text{.}\)
- \(\norm{-\uvec{v}} = \unorm{v}\text{.}\)
- \(\norm{k\uvec{v}} = \abs{k}\unorm{v}\text{.}\)
- \(\norm{\uvec{v}-\uvec{u}} = \norm{\uvec{u}-\uvec{v}}\text{.}\)
Warning 12.5.2.
The norm is not additive; that is, it is not true in general that \(\norm{\uvec{u}+\uvec{v}}\) is equal to \(\unorm{u}+\unorm{v}\text{.}\) Sometimes the two quantities are equal, as you are asked to consider below, but the best we can say about the norm of a sum is contained in Theorem 12.5.6 below.
Proposition 12.5.3. Algebra rules of the dot product.
The following are true for all vectors \(\uvec{u}\text{,}\) \(\uvec{v}\text{,}\) and \(\uvec{w}\) in \(\R^n\text{,}\) and all scalars \(k\text{.}\)
- \(\udotprod{v}{u} = \udotprod{u}{v}\text{.}\)
- \(\dotprod{\uvec{u}}{(\uvec{v}+\uvec{w})} = \udotprod{u}{v} + \udotprod{u}{w}\text{.}\)
- \(\dotprod{\uvec{u}}{(\uvec{v}-\uvec{w})} = \udotprod{u}{v} - \udotprod{u}{w}\text{.}\)
- \(\dotprod{(\uvec{u}+\uvec{v})}{\uvec{w}} = \udotprod{u}{w} + \udotprod{v}{w}\text{.}\)
- \(\dotprod{(\uvec{u}-\uvec{v})}{\uvec{w}} = \udotprod{u}{w} - \udotprod{v}{w}\text{.}\)
- \(\dotprod{(k\uvec{u})}{\uvec{v}} = k(\udotprod{u}{v})\text{.}\)
- \(\dotprod{\uvec{u}}{(k\uvec{v})} = k(\udotprod{u}{v})\text{.}\)
- \(\udotprod{v}{v} = \unorm{v}^2\text{.}\)
- Both \(\dotprod{\uvec{v}}{\zerovec} = 0\) and \(\dotprod{\zerovec}{\uvec{v}} = 0\text{.}\)
Subsection 12.5.2 Vector geometry inequalities and uniqueness of vector angles
Subsubsection 12.5.2.1 The Cauchy-Schwarz inequality
Here we will state the Cauchy-Schwarz inequality in its usual form. Note that this version applies to every pair of vectors, even if one is (or both are) the zero vector.
Theorem 12.5.4. The Cauchy-Schwarz inequality.
For every pair of vectors \(\uvec{u}\) and \(\uvec{v}\) in \(\R^n\text{,}\) we have \(\abs{\udotprod{u}{v}} \le \unorm{u}\unorm{v}\text{.}\)
Proof.
We will show that \((\udotprod{u}{v})^2 \le (\unorm{u}\unorm{v})^2\text{.}\) Once this is established, then for \(\udotprod{u}{v}\) to have a smaller square than \(\unorm{u}\unorm{v}\text{,}\) it must be smaller in magnitude. That is, \((\udotprod{u}{v})^2 \le (\unorm{u}\unorm{v})^2\) can only be true if \(\abs{\udotprod{u}{v}} \le \abs{\unorm{u}\unorm{v}}\) is true. But since neither \(\unorm{u}\) nor \(\unorm{v}\) can be negative, we have \(\abs{\unorm{u}\unorm{v}} = \unorm{u}\unorm{v}\text{,}\) and so
\begin{equation*}
\abs{\udotprod{u}{v}} \le \unorm{u}\unorm{v}
\end{equation*}
will be established.
So, we will try to prove that \((\udotprod{u}{v})^2 \le (\unorm{u}\unorm{v})^2\) is always true for every pair of vectors \(\uvec{u}\) and \(\uvec{v}\) in \(\R^n\text{.}\) We might as well assume that \(\uvec{v}\) is nonzero, since if it is zero then both \((\udotprod{u}{v})^2\) and \((\unorm{u}\unorm{v})^2\) are \(0\text{,}\) and the required inequality is true. In the case that \(\uvec{v}\) is nonzero, then also \(\unorm{v}\neq 0\) (Statement 1 of Proposition 12.5.1), and we can form the vector
\begin{align*}
\uvec{w} \amp= \uvec{u}-a\uvec{v}, \amp
\text{where } a \amp= \frac{\udotprod{u}{v}}{\unorm{v}^2}
\end{align*}
without worry that we’ve accidentally divided by zero. We will find that \(\unorm{w}^2\) is related to the inequality we are trying to prove, so compute
\begin{align*}
\unorm{w}^2
\amp= \udotprod{w}{w}
\amp \amp\text{(i)}\\
\amp= \dotprod{(\uvec{u}-a\uvec{v})}{(\uvec{u}-a\uvec{v})}
\amp \amp\text{(ii)}\\
\amp= \dotprod{(\uvec{u}-a\uvec{v})}{\uvec{u}}
- \dotprod{(\uvec{u}-a\uvec{v})}{(a\uvec{v})}
\amp \amp\text{(iii)}\\
\amp= \udotprod{u}{u} - \dotprod{(a\uvec{v})}{\uvec{u}}
- \bbrac{\dotprod{\uvec{u}}{(a\uvec{v})} - \dotprod{(a\uvec{v})}{(a\uvec{v})}}
\amp \amp\text{(iv)}\\
\amp= \udotprod{u}{u} - a(\udotprod{v}{u}) - a(\udotprod{u}{v}) + a\bbrac{a(\udotprod{v}{v})}
\amp \amp\text{(v)}\\
\amp= \udotprod{u}{u} - a(\udotprod{u}{v}) - a(\udotprod{u}{v}) + a^2(\udotprod{v}{v})
\amp \amp\text{(vi)}\\
\amp= \unorm{u}^2 - 2a(\udotprod{u}{v}) + a^2\unorm{v}^2
\amp \amp\text{(vii)}\\
\amp= \unorm{u}^2
- 2\left(\frac{\udotprod{u}{v}}{\unorm{v}^2}\right)(\udotprod{u}{v})
+ \left(\frac{\udotprod{u}{v}}{\unorm{v}^2}\right)^2\unorm{v}^2
\amp \amp\text{(viii)}\\
\amp= \unorm{u}^2
- 2\frac{(\udotprod{u}{v})^2}{\unorm{v}^2}
+ \frac{(\udotprod{u}{v})^2}{\unorm{v}^2},\\
\amp= \unorm{u}^2 - \frac{(\udotprod{u}{v})^2}{\unorm{v}^2}\text{,}
\end{align*}
with justifications
- using the definition of \(\uvec{w}\) above;
- Rule 8 of Proposition 12.5.3; and
- using the definition of \(a\) above.
Now, \(\unorm{w}^2\) cannot be negative, so we have
\begin{align*}
0 \amp\le \unorm{u}^2 - \frac{(\udotprod{u}{v})^2}{\unorm{v}^2}\\
\frac{(\udotprod{u}{v})^2}{\unorm{v}^2} \amp\le \unorm{u}^2\\
(\udotprod{u}{v})^2 \amp\le \unorm{u}^2\unorm{v}^2\text{,}
\end{align*}
where multiplying both sides of the second inequality by the nonnegative quantity \(\unorm{v}^2\) does not change the direction of the inequality.
Because \(\udotprod{u}{v}\) could be negative, we will change our last inequality above to
\begin{equation*}
\abs{\udotprod{u}{v}}^2 \le (\unorm{u}\unorm{v})^2\text{.}
\end{equation*}
In words, this inequality says that the square of one number is less than or equal to the square of another number. But when we square two numbers, the bigger number will always result in the bigger square (as long as neither number is negative). Since neither \(\abs{\udotprod{u}{v}}\) nor \(\unorm{u}\unorm{v}\) can be negative, the bigger number must be \(\unorm{u}\unorm{v}\) to result in a bigger square (or the two numbers could be equal). That is,
\begin{equation*}
\abs{\udotprod{u}{v}} \le \unorm{u}\unorm{v}\text{.}
\end{equation*}
Corollary 12.5.5. Uniqueness of angle measures.
For every pair of nonzero vectors \(\uvec{u}\) and \(\uvec{v}\) in \(\R^n\text{,}\) there is one unique angle value \(\theta\) in the domain \(0 \le \theta \le \pi\) so that
\begin{equation*}
\cos\theta = \frac{\udotprod{u}{v}}{\unorm{u}\unorm{v}} \text{.}
\end{equation*}
Subsubsection 12.5.2.2 The triangle inequality
Here is another commonly used inequality. Remembering our view of sums of vectors as a chain of changes in position, it basically says that the shortest path between two points in \(\R^n\) is the direct path.
Theorem 12.5.6. Triangle inequality.
For every pair of vectors \(\uvec{u}\) and \(\uvec{v}\) in \(\R^n\text{,}\) we have \(\norm{\uvec{u}+\uvec{v}} \le \unorm{u} + \unorm{v}\text{.}\)
Diagram of the triangle inequality.
Proof.
As mentioned in this chapter, working with square roots algebraically is inconvenient, so we will work with the square of the norm \(\norm{\uvec{u}+\uvec{v}}\text{,}\) and use Proposition 12.5.3 to avoid working directly with the components of our vectors.
We have
\begin{align*}
\norm{\uvec{u}+\uvec{v}}^2
\amp= \dotprod{(\uvec{u}+\uvec{v})}{(\uvec{u}+\uvec{v})}
\amp \amp\text{(i)}\\
\amp= \dotprod{(\uvec{u}+\uvec{v})}{\uvec{u}} + \dotprod{(\uvec{u}+\uvec{v})}{\uvec{v}}
\amp \amp\text{(ii)}\\
\amp= \udotprod{u}{u} + \udotprod{v}{u} + \udotprod{u}{v} + \udotprod{v}{v}
\amp \amp\text{(iii)}\\
\amp= \udotprod{u}{u} + 2\udotprod{u}{v} + \udotprod{v}{v}
\amp \amp\text{(iv)}\\
\amp= \unorm{u}^2 + 2\udotprod{u}{v} + \unorm{v}^2
\amp \amp\text{(v)}\text{,}
\end{align*}
with justifications
- Rule 1 of Proposition 12.5.3; and
Now, keep in mind that \(\udotprod{u}{v}\) is a number, and it may be positive, negative, or zero. But every number \(x\) satisfies
\begin{gather}
x \le \abs{x} \text{,}\tag{✶}
\end{gather}
since if \(x\) is positive or zero then the two sides are equal, and if \(x\) is negative then obviously the negative number \(x\) must be less than the positive number \(\abs{x}\text{.}\) Applying this for \(x = \udotprod{u}{v}\text{,}\) we have \(\udotprod{u}{v} \le \abs{\udotprod{u}{v}}\text{,}\) and so
\begin{align*}
\norm{\uvec{u}+\uvec{v}}^2
\amp= \unorm{u}^2 + 2\udotprod{u}{v} + \unorm{v}^2
\amp \amp\text{(i)}\\
\amp\le \unorm{u}^2 + 2\abs{\udotprod{u}{v}} + \unorm{v}^2
\amp \amp\text{(ii)}\\
\amp\le \unorm{u}^2 + 2\unorm{u}\unorm{v} + \unorm{v}^2
\amp \amp\text{(iii)}\\
\amp= (\unorm{u} + \unorm{v})^2
\amp \amp\text{(iv)}\text{,}
\end{align*}
with justifications
- continued from above;
- rule (✶);
- FOIL in reverse.
Following the chain of equalities and inequalities from beginning to end, we now have
\begin{equation*}
\norm{\uvec{u}+\uvec{v}}^2 \le (\unorm{u} + \unorm{v})^2 \text{.}
\end{equation*}
In words, this says that the square of one number is less than or equal to the square of another number. But when we square two numbers, the bigger number will always result in the bigger square (as long as neither number is negative). Since neither \(\norm{\uvec{u}+\uvec{v}}\) nor \(\unorm{u} + \unorm{v}\) can be negative, the bigger number must be \(\unorm{u} + \unorm{v}\) to result in a bigger square (or the two numbers could be equal). That is,
\begin{equation*}
\norm{\uvec{u}+\uvec{v}} \le \unorm{u} + \unorm{v} \text{.}
\end{equation*}
