Skip to main content

Section 13.5 Theory

Subsection 13.5.1 Norm and dot product

We'll begin with algebraic properties of norm and dot product.

Warning 13.5.2.

The norm is not additive; that is, it is not true in general that \(\norm{\uvec{u}+\uvec{v}}\) is equal to \(\unorm{u}+\unorm{v}\text{.}\) Sometimes the two quantities are equal, as you are asked to consider below, but the best we can say about the norm of a sum is contained in Theorem 13.5.6 below.

Subsection 13.5.2 Vector geometry inequalities and uniqueness of vector angles

The Cauchy-Schwarz inequality.

Here we will state the Cauchy-Schwarz inequality in its usual form. Note that this version applies to every pair of vectors, even if one is (or both are) the zero vector.

We will show that \((\udotprod{u}{v})^2 \le (\unorm{u}\unorm{v})^2\text{.}\) Once this is established, then for \(\udotprod{u}{v}\) to have a smaller square than \(\unorm{u}\unorm{v}\text{,}\) it must be smaller in magnitude. That is, \((\udotprod{u}{v})^2 \le (\unorm{u}\unorm{v})^2\) can only be true if \(\abs{\udotprod{u}{v}} \le \abs{\unorm{u}\unorm{v}}\) is true. But since neither \(\unorm{u}\) nor \(\unorm{v}\) can be negative, we have \(\abs{\unorm{u}\unorm{v}} = \unorm{u}\unorm{v}\text{,}\) and so

\begin{equation*} \abs{\udotprod{u}{v}} \le \unorm{u}\unorm{v} \end{equation*}

will be established.

So, we will try to prove that \((\udotprod{u}{v})^2 \le (\unorm{u}\unorm{v})^2\) is always true for every pair of vectors \(\uvec{u}\) and \(\uvec{v}\) in \(\R^n\text{.}\) We might as well assume that \(\uvec{v}\) is nonzero, since if it is zero then both \((\udotprod{u}{v})^2\) and \((\unorm{u}\unorm{v})^2\) are \(0\text{,}\) and the required inequality is true. In the case that \(\uvec{v}\) is nonzero, then also \(\unorm{v}\neq 0\) (Statement 1 of Proposition 13.5.1), and we can form the vector

\begin{align*} \uvec{w} \amp= \uvec{u}-a\uvec{v}, \amp \text{where } a \amp= \frac{\udotprod{u}{v}}{\unorm{v}^2} \end{align*}

without worry that we've accidentally divided by zero. We will find that \(\unorm{w}^2\) is related to the inequality we are trying to prove, so compute

\begin{align*} \unorm{w}^2 \amp= \udotprod{w}{w} \amp \amp\text{(i)}\\ \amp= \dotprod{(\uvec{u}-a\uvec{v})}{(\uvec{u}-a\uvec{v})} \amp \amp\text{(ii)}\\ \amp= \dotprod{(\uvec{u}-a\uvec{v})}{\uvec{u}} - \dotprod{(\uvec{u}-a\uvec{v})}{(a\uvec{v})} \amp \amp\text{(iii)}\\ \amp= \udotprod{u}{u} - \dotprod{(a\uvec{v})}{\uvec{u}} - \bbrac{\dotprod{\uvec{u}}{(a\uvec{v})} - \dotprod{(a\uvec{v})}{(a\uvec{v})}} \amp \amp\text{(iv)}\\ \amp= \udotprod{u}{u} - a(\udotprod{v}{u}) - a(\udotprod{u}{v}) + a\bbrac{a(\udotprod{v}{v})} \amp \amp\text{(v)}\\ \amp= \udotprod{u}{u} - a(\udotprod{u}{v}) - a(\udotprod{u}{v}) + a^2(\udotprod{v}{v}) \amp \amp\text{(vi)}\\ \amp= \unorm{u}^2 - 2a(\udotprod{u}{v}) + a^2\unorm{v}^2 \amp \amp\text{(vii)}\\ \amp= \unorm{u}^2 - 2\left(\frac{\udotprod{u}{v}}{\unorm{v}^2}\right)(\udotprod{u}{v}) + \left(\frac{\udotprod{u}{v}}{\unorm{v}^2}\right)^2\unorm{v}^2 \amp \amp\text{(viii)}\\ \amp= \unorm{u}^2 - 2\frac{(\udotprod{u}{v})^2}{\unorm{v}^2} + \frac{(\udotprod{u}{v})^2}{\unorm{v}^2},\\ \amp= \unorm{u}^2 - \frac{(\udotprod{u}{v})^2}{\unorm{v}^2}\text{,} \end{align*}

with justifications

  1. Rule 8 of Proposition 13.5.3;
  2. using the definition of \(\uvec{w}\) above;
  3. Rule 2 of Proposition 13.5.3;
  4. Rule 4 of Proposition 13.5.3;
  5. Rule 6 and Rule 7 of Proposition 13.5.3;
  6. Rule 1 of Proposition 13.5.3;
  7. Rule 8 of Proposition 13.5.3; and
  8. using the definition of \(a\) above.

Now, \(\unorm{w}^2\) cannot be negative, so we have

\begin{align*} 0 \amp\le \unorm{u}^2 - \frac{(\udotprod{u}{v})^2}{\unorm{v}^2}\\ \frac{(\udotprod{u}{v})^2}{\unorm{v}^2} \amp\le \unorm{u}^2\\ (\udotprod{u}{v})^2 \amp\le \unorm{u}^2\unorm{v}^2\text{,} \end{align*}

where multiplying both sides of the second inequality by the non-negative quantity \(\unorm{v}^2\) does not change the direction of the inequality.

Because \(\udotprod{u}{v}\) could be negative, we will change our last inequality above to

\begin{equation*} \abs{\udotprod{u}{v}}^2 \le (\unorm{u}\unorm{v})^2\text{.} \end{equation*}

In words, this inequality says that the square of one number is less than or equal to the square of another number. But when we square two numbers, the bigger number will always result in the bigger square (as long as neither number is negative). Since neither \(\abs{\udotprod{u}{v}}\) nor \(\unorm{u}\unorm{v}\) can be negative, the bigger number must be \(\unorm{u}\unorm{v}\) to result in a bigger square (or the two numbers could be equal). That is,

\begin{equation*} \abs{\udotprod{u}{v}} \le \unorm{u}\unorm{v}\text{.} \end{equation*}
The triangle inequality.

Here is another commonly used inequality. Remembering our view of sums of vectors as a chain of changes in position, it basically says that the shortest path between two points in \(\R^n\) is the direct path.

As mentioned in this chapter, working with square roots algebraically is inconvenient, so we will work with the square of the norm \(\norm{\uvec{u}+\uvec{v}}\text{,}\) and use Proposition 13.5.3 to avoid working directly with the components of our vectors.

We have

\begin{align*} \norm{\uvec{u}+\uvec{v}}^2 \amp= \dotprod{(\uvec{u}+\uvec{v})}{(\uvec{u}+\uvec{v})} \amp \amp\text{(i)}\\ \amp= \dotprod{(\uvec{u}+\uvec{v})}{\uvec{u}} + \dotprod{(\uvec{u}+\uvec{v})}{\uvec{v}} \amp \amp\text{(ii)}\\ \amp= \udotprod{u}{u} + \udotprod{v}{u} + \udotprod{u}{v} + \udotprod{v}{v} \amp \amp\text{(iii)}\\ \amp= \udotprod{u}{u} + 2\udotprod{u}{v} + \udotprod{v}{v} \amp \amp\text{(iv)}\\ \amp= \unorm{u}^2 + 2\udotprod{u}{v} + \unorm{v}^2 \amp \amp\text{(v)}\text{,} \end{align*}

with justifications

  1. Rule 8 of Proposition 13.5.3;
  2. Rule 2 of Proposition 13.5.3;
  3. Rule 4 of Proposition 13.5.3;
  4. Rule 1 of Proposition 13.5.3; and
  5. Rule 8 of Proposition 13.5.3.

Now, keep in mind that \(\udotprod{u}{v}\) is a number, and it may be positive, negative, or zero. But every number \(x\) satisfies

\begin{gather} x \le \abs{x} \text{,}\label{equation-vector-geom-theory-abs-x-ineq}\tag{\(\star\)} \end{gather}

since if \(x\) is positive or zero then the two sides are equal, and if \(x\) is negative then obviously the negative number \(x\) must be less than the positive number \(\abs{x}\text{.}\) Applying this for \(x = \udotprod{u}{v}\text{,}\) we have \(\udotprod{u}{v} \le \abs{\udotprod{u}{v}}\text{,}\) and so

\begin{align*} \norm{\uvec{u}+\uvec{v}}^2 \amp= \unorm{u}^2 + 2\udotprod{u}{v} + \unorm{v}^2 \amp \amp\text{(i)}\\ \amp\le \unorm{u}^2 + 2\abs{\udotprod{u}{v}} + \unorm{v}^2 \amp \amp\text{(ii)}\\ \amp\le \unorm{u}^2 + 2\unorm{u}\unorm{v} + \unorm{v}^2 \amp \amp\text{(iii)}\\ \amp= (\unorm{u} + \unorm{v})^2 \amp \amp\text{(iv)}\text{,} \end{align*}

with justifications

  1. continued from above;
  2. rule (\(\star\));
  3. The Cauchy-Schwarz inequality; and
  4. FOIL in reverse.

Following the chain of equalities and inequalities from beginning to end, we now have

\begin{equation*} \norm{\uvec{u}+\uvec{v}}^2 \le (\unorm{u} + \unorm{v})^2 \text{.} \end{equation*}

In words, this says that the square of one number is less than or equal to the square of another number. But when we square two numbers, the bigger number will always result in the bigger square (as long as neither number is negative). Since neither \(\norm{\uvec{u}+\uvec{v}}\) nor \(\unorm{u} + \unorm{v}\) can be negative, the bigger number must be \(\unorm{u} + \unorm{v}\) to result in a bigger square (or the two numbers could be equal). That is,

\begin{equation*} \norm{\uvec{u}+\uvec{v}} \le \unorm{u} + \unorm{v} \text{.} \end{equation*}