Appendix

1 Linear Al gebra

1.1 R

a s a vector space

As a set, R

is the collection of all n-tuple (x

; :::; x

), x

2R. Equivalently, R

can be deﬁned

as the Cartesian product of n copies of R as R

= R ×···×R. A tuple u = (x

; :::; x

) has

a double life, as a point in the Cartesian space with coordinates x

, and as a vector u~ with

the tail at the origin and head at u. For this reason, R

can be considered as a set of points

or a set of vectors .

We ne ed two more structures on R

to make it a vector space, i .e. , the scalar multipli-

cation, and the vector addition. Let c 2R be a scalar, and u~ = (x

; :::x

); v~ = (y

; :::; y

) two

arbitrary vectors. We deﬁne

c u~ = (cx

; :: :; cx

); u~ + v~ = (x

+ y

; :::; x

+ y

The geometry of these two o perations are shown in Fig.1.

cu~ ; c < 0

cu~ ; c > 0

u~ + v~

u~ ¡v~

Figure 1.

It is seen that R

is closed under the deﬁned operations, that is, for any two vectors u~ ;

v~ in R

, and c

; c

2R, vector c

u~ + c

v~ is in R

. Moreover, the following properties hold for

any vectors u~ ; v~ ; w~ 2R

and constant c 2R:

i. u~ + v~ = v~ + u~

ii. c (u~ + v~) = c u~ + c v~

iii. (c

+ c

)u~ = c

u~ + c

iv. (u~ + v~) + w~ = u~ + (v~ + w~ )

has n standard unit vectors e^

; :::; e^

deﬁned below

; e^

; :::; e^

The set fe^

k=1

is a basis for R

in the sense that any vector u~ =(x

; :::; x

) can be uniquely

represented as

u~ = x

+ ···+ x

Remark. The magnitude or norm of a vector u~ = (x

; :: :; x

) is deﬁned as follows

ku~ k=

k=1

A vector is called unit or a direction vector if its no rm is equal 1. We usually denote unit

vector by notation ^.

1.2 Dot and Cross products

The dot product of two arbitrary vectors u~ = (x

; :::; x

); v~ = (y

; :: :; y

) is deﬁned as

u~ ·v~ =

k=1

In particular, e^

·e^

= δ

i;j

where δ

i;j



1 i = j

0 i =/ j

Problem 1. Show that the dot product enjoys the following properti es for any vectors u~ ; v~ ; w~ and for

arbitrary constant c

i. (u~ + v~) ·w~ = u~ ·w~ + v~ ·w~ ,

ii. u~ ·v~ = v~ ·u~ ,

iii. (c u~ ) ·v~ = c (u~ ·v~)

Problem 2. Show the following relations

a) ku~ k

= u~ ·u~

b) ku~ + v~ k≤ku~ k+ kv~ k

c) ku~ + v~ k

¡ku~ ¡v~ k

= 4 u~ ·v~

Problem 3. The Cauchy inequality is as follows

u~ ·v~ ≤ku~ kkv~ k :

2 Appendix

Try to prove the inequality by the following method: ku~ + t v~ k

≥0 for all t 2R. Expand ku~ + t v~ k

terms of t and conclude the inequality. By this inequality, one can deﬁne the angle between two nonzero

vectors u~ ; v~ as follows

cos(θ) =

u~ ·v~

ku~ kk v~ k

By t he above equality, one can write (u~ ; v~) = ku~ kkv~ kcos(θ). Note that if u~ ·v~ = 0 for non-zero vectors

u~ ; v~, then cos(θ) = 0.

Problem 4. Show that if u~

; :::; u~

are mutually orthogonal, that is, if u~

·u~

= 0 for i =/ j then

ku~

+ · ··+ u~

= ku~

k+ ···+ ku~

There is a standard product in R

called cross or external product. For u~ = (x

; y

; z

);

v~ = (x

; y

; z

), the cross product is deﬁned as follows

u~ ×v~ =



= (y

¡z

)e^

+ (z

¡x

)e^

+ (x

¡y

)e^

Note that u~ ×v~ is a vector, while their dot p roduct is a scalar.

Problem 5. Show the relation u~ ×v~ = ¡v~ ×u~ .

Problem 6. Show that u~ ×v~ is perpendicular to u~ and v~, that is,

(u~ ×v~) ·u~ = (u~ ×v~) ·v~ = 0:

Problem 7. Show the identity

ku~ ×v~ k

= ku~ k

ku~ k

¡ju~ ·v~ j

;

and conclude the relation

ku~ ×v~ k= ku~ kkv~ ksin(θ);

where θ is the angle between u~ ; v~ in [0; π].

By the above problem, we can write

u~ ×v~ = ku~ kkv~ ksin(θ) n^;

where n^ is the unit vector perpendicular to the plane containing u~ ; v~, that is, n^ =

u~ ×v~

ku~ ×v~ k

A = jju~ ×v~ jj

u~ ×v~

v~ ×u~

Problem 8. Show the following relation for any three vectors u~ ; v~ ; w~

u~ ·(v~ ×w~ ) = v~ ·(w~ ×u~ ):

Problem 9. If v~ ; w~ are orthogonal, show the following relation

u~ ×(v~ ×w~ ) = (u~ ·w~ )v~ ¡(u~ ·v~)w~ :

Use this result and relax the condition v~ ; w~ to be orthogonal. Use the formula and determine the

conditions the following rela tion holds

u~ ×(v~ ×w~ ) = (u~ ×v~) ×w~ :

1 Linear Algebra 3

1.3 Subspaces and direct sum

Deﬁnition 1. Tow vectors u~ ; v~ in R

are called linearly dependent if there is a scalar c

such that u~ = c v~ or v~ = c u~ . A vector u~ is linearly dependent on vectors v~

; :::; v~

if there are

scalars c

; :: :; c

such that

u~ = c

~ + ···+ c

Vectors v~

; :: :; v~

in R

are linearly independent if the linear combination

+ ···+ c

= 0;

implies c

= ···= c

= 0.

Problem 10. Vectors e^

; :::; e^

are linearly independent in R

. Show that any n + 1 vectors of R

are

linearly dependent.

Let fv~

; ::: ; v~

g fo r d ≤ n be a set of linearly independent vectors i n R

. The span of

vectors in the given set is the set of all possible linear combinations of v~

; :::; v~

, i.e.,

spanfv~

; :::; v~

g= fc

+ ·· ·+ c

; c

2Rg:

Note that R

is itself equal to spanfe^

; :::; e^

Proposition 1. V

:= spanfv~

; :::; v~

g is closed under the vector addition and scalar multi-

plication of R

. For this reason, V

is called a linear subspace of R

Deﬁnition 2. Let V be a linear subspace of R

. The dimension of V is the maximum

number of linearly independent vectors in V.

Example 1. Technically speaking, R

is not a subspace of R

for m < n, however, if we

interpret R

as spanfe^

; :::; e^

g where each e^

is a vector in R

, then R

is a linear subspace

of R

If V is a linear subspaces of R

, then its orthogonal subspace V

is deﬁned as follows

= fw~ 2R

; w~ ·v~ = 0; v~ 2Vg:

Obviously, V

is a linear subspace of R

equipped with the vector addition and scalar

multiplication operations.

Problem 11. Show that if V =spanf(1; 1;0); (0; 1;1)g, then V

is the one dimensional subspace spanned

by (1; ¡1 ; 1).

Problem 12. Find the orthogonal subspace of V = f(1; 0; 1)g in R

Suppose U;V are two subspaces of R

and U \V = f0g. The direct sum U ⊕V is deﬁned

as follows

U ⊕V = fc

u~ + c

v~; u~ 2U; v~ 2Vg:

Problem 13. If V is a subspace of R

, show that R

= V ⊕V

Problem 14. Let V be an arbitrary subspace of R

. Show that every vector u~ in R

can be represented

uniquely as u~ = c

v~ + c

w~ fo r v~ 2V and w~ 2V

4 Appendix

1.4 Matrices and line ar mappings

Deﬁnition 3. A mapping f : R

! R

is called linear if for any constants c

; c

and any

vectors u~ ; v~ 2R

, the following relation holds

f(c

u~ + c

v~) = c

f(u~ ) + c

f(v~):

Problem 15. If f : R

is linear then f (0) = 0.

Proposition 2. A linear mapping f: R

can be represented by a m ×n matrix.

Proof. Remember that a matrix A= [a

]

m×n

is a structure of n-columns of vectors belonging

, i.e., A = [A

j···jA

], where A

2 R

. The acti on of A to e^

is deﬁned by the re lation

A (e^

) = A

. Now deﬁne A

= [f(e^

)jf(e^

)j···jf(e^

)]:

It is simply seen that for arbitrary vector u~ 2R

, the following relation holds f(u~ ) = A

(u~ ) . 

Problem 16. Let T : R

has the matrix representation A

2×2



1 1

0 1



in the standard basis. Find

the matrix representation of T in the basis v~





; v~



¡1



Problem 17. Prove that a matrix 2 ×2 maps any parallelogram to a parallelogram.

Problem 18. Verify that the the matrix R



cos(θ) ¡sin(θ)

sin(θ) cos(θ)



rotates vectors in the plane by θ degree

counter-clockwise. Verify that R

= R

+θ

and conclude that R

¡θ

is the identity matrix



1 0

0 1



Deﬁnition 4. Let f:R

be a linear mapping. The kernel (or null space) of f, denoted

by ker(f) (or just N

) is a set of all vectors n~ of R

such that f(n~ ) = 0 2 R

. The image

of f denoted by I m(f) is the set of all vectors w~ 2R

such that w~ = f(v~) for some v~ 2R

Proposition 3. The kernel of a linear mapping f : R

! R

is a vector subspace of R

The image of f is a vector subspace of R

Problem 19. Prove the proposition.

Theorem 1. Assume that f : R

is a linear mapping. The following relation holds

n = dim ker(f) + dim Im(f): (1)

Problem 20. Let S denote the orthogonal subspace of ker(f). Show that dim S = dim Im(f) and

conclude f (S) = Im(f ).

1.5 Linear m appings from R

to R

1.5.1 Determinant

Let f: R

be a linear mapping, and let C be a unit cube constructed on e^

; k = 1; :::;

n. The image of C under f, that is f(C), is a parallelogram. In fact, every vector u~ 2C is

represented by the linear combination

u~ = c

+ ·· ·+ c

;

1 Linear Algebra 5

for 0 ≤c

≤1, and thus

f(u~ ) = c

+ ···+ c

;

where f

= f (e^

). The set fc

+ ···+ c

g for 0 ≤c

≤1 is a parallelogram constructed on

; :: :; f

; see Fig. 2 .

= f (e^

)

= f (e^

)

Figure 2.

Deﬁnition 5. Let f : R

be a linear mapping, and let C be the unit cube constructed

on fe

^ g

k=1

. The determinant of f denoted by det(f) is the algebraic volume of parallelogram

f(C). The algebraic volume is t he signed volume with positive or negative signs.

Example 2. In R

, the determinant of A =





is deﬁned by the following formul a

det





= a

¡a

: (2)

It is simply veriﬁed that jdet(A)j= jjA(e^

)jjjjA(e^

)jjsin(θ), where θ is the angel between two

columns of A.

If det(f) = 0, then the volume degenerates, that means vectors f

; :::; f

are linearly

dependent. If det(f ) < 0, then f changes the standard orientation of the basis fe^

k=1

(remember the standard rotations in R

and R

), for example f (x; y)=(y; x) with the matrix

representation

A = [f(e^

)jf (e^

)] =



0 1

1 0



;

changes the standard rotation. Let A

= [f

j···jf

] be the representation of f: R

! R

the standard basis fe^

k=1

. The determinant det(A

) satisﬁes th e following properties:

i. If f

; ·· ·; f

are linearly dependent then det[A] = 0

ii. det[f

j···jf

] = ¡det[A

]. In general any switch b etween column i a nd j multiple

the determinant by the factor (¡1)

i+j

iii. det[c f

j···jf

] = c det(A

)

iv. det[c

+ c

j···jf

] = c

det[A

] for any k = 2; :::; n.

By the above properties, it is seen that if f ; g: R

are two linear mappings then

det(AB) = det(A) det(B)

6 Appendix

Problem 21. Verify directly the above claim for 2 ×2 matrices.

1.5.2 Injective and surjective ma pping s

Deﬁnition 6. A linear mapping f : R

! R

is called one-to-one or injective if equality

f(u~ ) = f(v~) implies u~ = v~. A linear mapping f: R

! R

is called onto or surjective, if

= f(R

Problem 22. A linear mapping f : R

is one to one if and only if ker(f) = ;, and if and only if

it is onto.

Problem 23. Let f: R

! R

be a linear mapping. Show tha t if m > n, then f can not be onto, if

m < n then f can not be one-to-one.

If f: R

! R

is one-to-one (and then onto), the mapping f

¡1

: R

! R

is called the

inverse mapping of f if the following relation holds

¡1

= f

¡1

f = Id;

where Id is the identity mapping on R

. The identity mapping has the matrix representation

diag(1; :::; 1), where diag(1; :::; 1) has 1 on the main diagonal and zero everywhere else. Note

that Id(u~ ) = u~ for any vector u~ .

Problem 24. If f: R

is a one to on e linear map, show that f

¡1

is also a one to one linear map.

1.5.3 Eigenvalues and Eigenvectors

A vector v~ 2 R

¡ f0g is called a n eigenvector of a linear mapping f : R

! R

if there is

a scalar λ such that f(v~) = λv~. It is seen that if v~ is an eigenvector, then vector w~ = tv~ for

arbitrary scalar t is also an eigenvector. Accordingly, one can deﬁne an eigendirection of f

that is spanfv~ g:= ft v~ ; t 2Rg; see Fig. 3.

λv

= f (v

)

Figure 3.

Example 3. The vector v~ = (1; 1) is an eigenvector of the matrix A =



1 1

¡1 3



with the

eigenvalue λ = 2, because



1 1

¡1 3





= 2





. Matrix A =



1 1

¡1 3



has only one eigenvector

and matrix A =



2 3

¡4 ¡5



has two eigenvectors v~

= (1; ¡1) and v

~ = (3; ¡4) with eigenvalues

= ¡1 and λ

= ¡2 respectively. The rotation matri x R



cos(θ) ¡sin(θ)

sin(θ) cos(θ)



has no (real)

eigenvector for θ =/ 0; 2π. Recall that R

rotates vectors counterclockwise by θ-angle. The

identity matrix I

2×2



1 0

0 1



has inﬁnitely many eigenvectors. In fact, every vector in R

an eigenvector of Id

2×2

with eigenvalue λ = 1.

1 Linear Algebra 7

Proposition 4. If f: R

! R

has n distinct eigenvalues λ

; :::; λ

, then their associated

eigenvectors v~

; :::; v~

are linearly independent.

Problem 25. Prove the proposition.

If v~ is an eigenvector of a linear mapping f with eigenvalue λ, then (f ¡λId)v~ = 0, and

since v~ is nonzero, v~ must belong to the kernel of f ¡λId. Let A

is a matrix representation

of f, then the following relation holds

det(A

¡λId) = 0:

The above equation, which is an algebraic equation of λ, is called the characteristic equation

of f. If A =





, the characteristic equation is as follows

¡tr(A) λ + det(A) = 0; (3)

where tr(A) (read trace A) is equal to a

+ a

Problem 26. Show that if A

2×2

has a r epeated eigenva lue λ with two linearly independent eigenvectors

then all vectors of R

is an eigenvector of A.

Problem 27. Let A be a 2 ×2 matrix. P rove that the f ollowing statements are equivalent

i. A is invertible.

ii. Two columns of A are linearly independent.

iii. The determinant of A is non-zero.

iv. No eigenvalue of A is zero.

Problem 28. If λ

; λ

are two eigenvalues of A

2×2

, show that det(A) = λ

and tr(A) = λ

+ λ

Problem 29. If Q

2×2

is an invertible matrix, show the following relations

tr(Q

¡1

AQ) = tr(A), det(Q

¡1

AQ) = det(A):

1.5.4 Symmetric mappings and Jordan forms

Deﬁnition 7. A linear mapping f : R

! R

is called symmetric if the following equality

holds for arbitrary vectors u~ ; v~ 2R

f(u~ ) ·v~ = u~ · f (v~):

Theorem 2. If the linear mapping f : R

! R

is sy mmetric, then there are n mutually

orthogonal eigenvectors v~

; :: :; v~

for f. Moreover, all eigenvalues of f are real.

Problem 30. Assume v~

; v~

are two eigenvectors of a symmetric mapping f. Show t ha t hv~

; v~

i= 0.

If f: R

! R

has n linearly independent eigenvectors v~

; :::; v~

, then R

can be

decomposed by the direct sum R

= V

⊕···⊕V

where V

= spanfv~

g. The restriction of f

to each V

is a linear mapping f

: V

, and thus we can decompose f as the direct s um

f = f

⊕···⊕ f

. With this interpretation, every vector v~ 2R

has a unique representation

v~ = c

+ ···+ c

, and thus f(v~) is

f(v~) = f

) + ···+ f

) = c

+ ·· ·+ c

8 Appendix

Deﬁnition 8. A linear mapping f: R

! R

is called positive deﬁnite if for any nonzero

vector v~, the following inequality holds

f(v~) ·v~ > 0:

A negative deﬁnite linear mapping is deﬁned similarly .

It is simply seen that if A

= [a

] is the matrix representation of the positive deﬁnite

mapping f in the standard basis, then a

> 0 for i = 1; :::; n. Moreover, all real eigenvalues

of A

must be positive.

Problem 31. Let f: R

be a symmetric mapping. A necessary and suﬃcient condition that f be

po sitive deﬁnite is that its all eigenvalues are positive.

If Q

n×n

is an invertible matrix, then two matrices B = Q

¡1

AQ and A are called similar.

It is seen that A;B have same characteristic polynomial as the following argument justiﬁes it:

det(Q

¡1

AQ ¡λId) = det Q

¡1

(A ¡λId)Q = det(Q

¡1

) det(A ¡λId) detQ = det(A ¡λId):

Proposition 5. Suppose A

n×n

has n linearly independent eigenvectors v~

; :::; v~

. Then the

following relation holds

¡1

AQ = diag(λ

; :: :; λ

);

where for Q = [v~

j·· ·jv~

], and λ

; :: :; λ

are associated eigenvalues (not necessarily distinct).

Matrix diag(λ

; :::; λ

) is called the Jordan form of A.

Problem 32. Prove the ab ove proposition.

Fig. 4 shows the relation between eigenvectors of A and Q

¡1

AQ .

Eigenvectors ofEigenvectors of A

¡1

Figure 4.

Problem 33. Let A =



¡4 ¡3

3 2



. Find the matrix Q

¡1

AQ.

If a matrix A

n×n

has n repeated eigenvalues λ with only one eigenvector, then the

Jordan form of A

n×n

is a diagonal matrix with λ on the main diagonal and 1 on the upper

diagonal. For example, for a 3 × 3 matrix with repeated eigenvalue λ, the Jordan form

λ 1 0

0 λ 1

0 0 λ

. If a matrix 2 × 2 has two complex eigenvalues λ = σ ± i!, its Jordan form is



σ ¡!

! σ



Problem 34. For a n ×n matrix A show det(A) =

k=1

, and conclude that A is one-to-one if and

only if A do es not have a zero eigenvalue.

1 Linear Algebra 9

1.6 Linear equations

Let f : R

! R

be a linear mapping, and let b

2 R

be an arbitrary vector. The linear

equation f(u~ ) = b

is solvable if and only if b

2Im(f). If f: R

is a linear one-to-one

mapping, then equation f(u~ ) = b

is simply solved for u~ = f

¡1

). If dim ker(f) > 0 and u~ is a

solution to the equation, then for any vector n~ 2ker(f), v~ = n~ + u~ is also a solution. In this

context, vectors in ker(f) are called the homogeneous solutions of f (u~ ) = 0.

Problem 35. Suppose n ≥ m and f: R

is a linear mapping. Show that if dim ker(f) = n ¡m,

then the linear equation f (u~ ) = b

is solvable for any b

. What if dim ker(f) > n ¡m? If n > m and

dim ker(f) = n ¡m, show that the equation f (u~ ) = b

has inﬁnitely many solutions.

Problem 36. Let A =



2 6

1 3



. For what values of b

, the equation Au~ = b

is solvable? Verify that

the solutions of the equation Au~ =





has the form u~ = t



¡3







for t 2(¡1; 1).

Problem 37. Let f: R

be a linear mapping and let u

~ ; u~

be two solutions to equation f(u~ ) = b

Show that u~

¡ u~

2 ker(f), and conclude every solution to the equation can be represented by n~ + u~

where n~ 2ker(f ).

Problem 38. Let f : R

be a linear mapping and suppose u~

is a solution to f(u~ ) = b

and u~

a solution to f(u~ ) = b

. Show that u~

+ u~

is a solution to f(u~ ) = b

+ b

As we saw above, equation f(u~ ) = b

is solvable if b

2Im(f). The following problem a nswer

the solvability of a linear equation f(u~ ) = b

by the aid of the transpose of f. Remember that

: R

is the transpose of f : R

if the following equality holds for any u~ 2R

and v~ 2R

f(u~ ) ·v~ = u~ ·f

(v~):

Problem 39. Let f: R

be a linear mapping. Show ker(f

) = [Im(f )]

. Co nclude that the linear

equation f (u~ ) = b

is solvable if hb

; n~ i= 0 for all n~ 2ker(f

). Also show

dim ker(f

) ¡dim ker(f) = m ¡n:

Problem 40. Find ker(f

) of the matrix A =



2 6

1 3



and verify that b





is orthogonal to ker(f

2 Fun ctions o f se veral variables

2.1 Topology of R

Fo r p = (x

; ·· ·; x

) 2R

, the Euclidean norm kpk is deﬁned as

kpk= x

+ ·· ·+ x

;

and if q = (y

; :: :; y

), the Euclidean distance is deﬁn ed as follows

kp ¡ qk= (x

¡y

)

+ ···+ (x

¡y

)

An immediate result of the above deﬁnitions is t he convergence of sequences in R

10 Appendix

Deﬁnition 9. A sequence (p

)

m=1

is called convergent to a if

lim

m!1

¡ak= 0:

Proposition 6. A sequence p

converges to a if and only if each coordinate of p

converges

to its associate coordinate of a.

An open ball of radius r centered at a 2R

is deﬁned as

(a) = fp 2R

; kp ¡ak< rg:

Problem 41. If p

!a then any ball B

(a) contains inﬁnitely many points of the sequence.

A set D ⊂R

is called open if for any point a 2D, there is r > 0 such that B

(a) ⊂D.

A set D ⊂R

is called bounded if there is r > 0 such that D ⊂B

. If D is an open set then

its complement D

is closed. The complement set D

is deﬁned as

= fp 2R

; p 2Dg:

If D is a set, its closure, cl(D) is the smallest closed set containing D, and bnd(D) denotes

the boundary set of D. A point a is called a boundary point of a set D if for any r > 0, the

following relation holds

(a) \D =/ ;; B

(a) \D

=/ ;:

The above statements means that any ball centered at a crosses both D and D

. We have

bnd(B

) = fp; kpk= rg, and cl(B

) = fp; kpk≤rg, and B

= fq; kqk> rg.

Problem 42. A set D ⊂R

is closed if and only if any convergent sequence (p

); p

2D converges in D.

Problem 43. Consider the set A =



; n = 1; 2; 3; ···



. Determine if A is open or closed.

Problem 44. Let D ⊂R

be any set, show that bnd(D) is closed.

Problem 45. Show that the set A =



y; 0 < y <

; x > 0



is open. Find bnd(A).

Problem 46. If D

; D

are open sets show that D

and D

are open. Repeat the argument

if D

; D

are closed.

2 Functions of several variables 11

2.2 Straight lines and planes in R

The parametric e quation of a straight line passi ng through a point p

= (x

; y

; z

) and

parallel to a vector r~ = (a; b; c) is p

~ + t r~ , or equivalently

x(t) = x

+ at; y(t) = y

+ bt; z(t) = z

+ ct:

If a =/ 0; b =/ 0; c =/ 0, then we can rewrite the equation as follows

x ¡x

y ¡y

z ¡z

(p~ ¡p

~ )jjr~

p~ ¡ p

Similarly, the equation of a straight plane in R

passing through a given point p

= (x

;

; z

), and perpendicular to a given vector n~ = (a; b; c) is n~ ·(p~ ¡ p

~ ) = 0, or equivalently

a(x ¡x

) + b(y ¡y

) + c(z ¡z

) = 0;

or equivalently ax + by + cz = d, for some constant d.

p~ ¡p~

(p~ ¡p

~ )?r~

The intersection of two planes in R

can be empty or a line depending on their position

to each other. For example, two planes P

: a

x + b

y + c

z = d

and P

: a

x + b

y + c

z = d

intersect if their normal vectors n~

= (a

; b

; c

) and n~

= (a

; b

; c

) are not parallel to each

other. In this case, the intersection line will be parallel to n~

×n~



= (b

¡c

; c

¡a

; a

¡b

);

12 Appendix

Hence, the equation of the intersection line is

x ¡x

¡c

y ¡y

¡a

z ¡z

¡b

;

where (x

; y

; z

) is a point on the intersection of P

; P

Example 4. Find the inte rsection of two plane P

: 2x + y = 1, P

: y ¡z = ¡1.

Solution. The associated normal vectors of two plains are n

= (2; 1; 0), n

= (0; 1; ¡1), and they are

not parallel. The intersection line of two plains are in the direction of

×n~

= (¡ 1; 2; 2):

Obviously, the point p

= (0; 1; 2) lies on both plains, and thus the intersection line equation is

¡1

y ¡1

z ¡2

2.3 Scalar functions

A function f w ith the domain D

⊂R

is called a scalar function if Im

⊂R. For example,

a mapping that measures the temperature of ea ch point of a room is a s calar function.

The graph of a scalar function y = f(x

; :::x

) is the s etf(x

; :::x

; y)g ⊂ R

n+1

where

; :::; x

) 2 D

, the domain of f. The graph of a function z = f(x; y) is the surface

f(x; y; f(x; y))g. Fo r example, function z = x

+ y

is a paraboloid in the (x; y; z)-space

The set ff(x; y) = cg for a ﬁxed c is called the level set of f with valu e c. For example,

the level sets of f(x; y) = x

+ y

, is the set of circles of radius c

centered at the origin. A

level set is also called an implicit function, for example x

+ y

+ z

= c

, that is a sphere of

radius c in R

2 Functions of several variables 13

Deﬁnition 10. A scalar function f : D !R has a limit L at a 2D if for any sequence p

2D,

=/ a, the convergence p

a implies f(p

)

L, that is, for any " > 0, there is δ > 0

such that if 0 < kp ¡ak< δ then jf (p) ¡Lj< ".

It is simply seen that function f (x; y) =

+ y

, does not have a limit at (0; 0). In fact, for

sequence (x

; y

) =

; 0



, the limit is 0. For s equence (x

; y

) =



, the limit is again

0, but the limit is

for sequence

;



Problem 47. Determine if the following functions have a limit at (0; 0)

a) f =

+ y

b) f =

sin(x) + sin(y )

x + y

Deﬁnition 11. A scalar function f: D !R is called continuous at a 2D if for any sequence

), p

! a, the sequence f (p

) converges to f(a). The statement is equivalent t o the

following: for any " > 0, there is δ > 0 such that jf(p) ¡f(a)j< " for all p 2B

(a).

Problem 48. Sup pose f: R

!R is co ntinuous. Show that for any open interval J, the set f

¡1

(J) = fp;

f(p) 2J g is open.

Problem 49. Show that the following function is not continuous at (0; 0).

f(x; y) =

+ y

(x; y) =/ (0; 0)

0 (x; y) = (0; 0)

2.4 Vecto r functions

Let I ⊂R be an interval. A mapping f: I !R

is called a vector valued function. A vector

valued function f is usually denoted by f(t) = (f

(t); :::; f

(t)) for parameter t in I.

14 Appendix

Example 5. The image of mapping f(t) = (cos(t); sin(t)) is a unit circle in the plane (x; y)

as it satisﬁes the relation [x(t)]

+ [y(t)]

= 1. The image of mapping f(t) = (cos t; sin t; t) is

a helix in R

as shown below.

A vector function f(t) has a limit at t

if and only if all co o rdinate functions f

(t) has a

limit at t

. Similarly, f is continuous if its all coordinate functions are continuous. We write

lim

t!t

f(t) = L

, if lim

t!t

(t) = l

for all k = 1; :: :; n, and L = (l

; :::; l

). A vector function is also

called a one dimensional parametric function.

The derivative of vector function is deﬁned coordinate-wise, i.e.,

(t) = (f

(t); :::; f

(t)):

Fo r a ﬁxed t

2I, f

) is the tangent vector on the curve f(I) at t

as long as f

) exists.

If f

) does not exist, we say f is singular at t

. For example, function f(t) =



; t



singular at t = 0.

2 Functions of several variables 15

If f (t) denotes the trajectory of a particle in R

, f

) is called the velocity vector of

that pa rtic le, and kf(t

)k is equal to its speed. The parametric representation of a curve

provides us with more information than its image. For example, the image of mapping

f(t) = (cos(!t); sin(!t)) is a unit circle for all values of ! =/ 0, however, if f(t) represents the

trajectory of a particle, the speed of the particle would be a function o f !, as the relation

(t)k = ! justiﬁes it. The following ﬁgure shows the velocity vectors for ! = 0.5; 1; 2

respectively from left to right.

2.5 Parametric mappings

A mapping f : D

⊂R

is called a parametric mapping if m > 1. For example if n = 2

and m = 3, mapping f(t; s) = (f

(t; s); f

(t; s)) deﬁnes a parametric surface in R

16 Appendix

Parametric mappings are ve ry common fo r representing complex surfaces. For example,

a torus is represented by the following parametric mapping

f(θ; φ) = ((c + a cos(θ))cos(φ); (c + a cos(θ)) sin(φ); a sin(θ))

where c > a are some constants. The shape is shown below.

Figure 5.

The restriction of f (t; s) to a curve in the (t; s) plane is mapped to a curve in the surface

f(t; s). For example, γ: φ = θ/6 is a straight line in (φ; θ) plane, and it is mapped under f

to the following space curve

¡(θ) = ((c + a co s(θ))cos(θ/6); (c + a cos(θ)) sin(θ/6); a sin(θ)):

The following parametric ma ppings are respectively representations of a sphere and a cylinder

S: (sin(θ) cos(φ); sin(θ) sin(φ); cos(θ)); C: (cos(θ); sin(θ); z)

The concept of limit and continuity is coordinate-wise as well, that is, f has a limit at p 2D

if each coordinate function has a limit at p 2D

3 Derivatives

3.1 Derivatives of sc alar functions

3.1.1 Partial derivatives.

Let D

be a n open set in R

. For a scalar function f: D

!R, the partial derivative @

f at

p 2D

is deﬁned by the following limit

f(p) = lim

t!0

f(p + te^

) ¡ f(p)

; (4)

as long as the limit exists. Fo r a two variable function f(x; y), the partial derivatives @

f ;

f at p = (a; b) are deﬁned respectively as follows

f(a; b) = lim

t!0

f(a + t; b) ¡ f (a; b)

;

f(a; b) = lim

t!0

f(a; b + t) ¡f(a; b)

;

3 Derivatives 17

as long as the limits exist.

Remark 1. For the sake of simplicity, we use the ﬂat notations @

; @

in this book instead

of the standard ones

;

. Another notation for the partial derivative is f

; f

for

;

Remark 2. Similarly, we can d eﬁ ne partial derivative functions @

f ; @

f in the open set

, the domain of f as

f(x; y) = lim

t!0

f(x + t; y) ¡ f(x; y)

;

f(x; y) = lim

t!0

f(x; y + t) ¡f(x; y)

;

for (x; y) 2D

Remark 3. The existence of partial derivatives of a function at a point doe s not guarantees

the continuity of the function at that point. Consider the following function

f(x; y) =

(

+ y

(x; y) =/ (0; 0)

0 (x; y) = (0; 0)

Even though, @

f(0; 0); @

f(0; 0) exist and are equal zero, the function is not continuous at

the origin. However, f(x; y) must be continuous and diﬀerentiable with respect to x at (a; b)

in order that @

f(a; b) exists. Similarly, f(x; y) must be continuous and diﬀerentiable with

respect to y a t (a; b) in order that @

f(a; b) exists.

3.1.2 Interpretations of partial derivatives

Like a single variable function, there are two related interpretations of partial derivatives.

Consider a two variable function f (x; y) deﬁned on an open set D

. Fix a point (a; b) 2

, and consider the horizontal line parallel to x-axis passing through (a; b). The partial

derivative @

f(a; b ) measures the rate of change of f at (a; b) along the horizontal line, and

similarly, @

f(a; b) measures the rate of change of f at (a; b) along the vertical line passing

through (a; b). For exam ple, the rate of change of function f(x; y) = x

+ y

at (1; 1) along

x-axis is

f(1; 1) =

+ y



(1;1)

The slope of tangent lines to the surface of f(x; y) are expressed in terms of partial deriva-

tives. The projection of line (a + t; b), for t 2(¡c; c) for some c > 0 on the graph of z = f(x; y)

is a curve of the following form

(t) = (a + t; b; f(a + t; b)):

This space curve passes through (a; b; f(a; b)) at t = 0. It is simply seen that @

f(a; b) is

equal to the slope of tangent line to ¡

(t) in the (x; z)-plane at t = 0:

d¡



t=0

= (1; 0; @

f(a; b))

18 Appendix

Similarly, @

f(a; b) is equal to the slope of tangent line to the curve ¡

(t) = (a; b + t;

f(a; b + t)) at t = 0 in the (y; z)-plane. The following ﬁgure shows the g raph of function

f(x; y) = x

+ y

, and the projection of γ

(t) = (1 + t; 1) on it:

(t) =

1 + t; 1; 2 + t

+ 2t



The black line is the tangent to the space curve ¡

at t = 0. The slope of the tangent line, is

the tangent of the ang le the line makes with the horizontal line γ

, that is,

m =

2 + t

+ 2t

t=0

Note that

d¡

(0) is th e tangent vector to the curve ¡

at time 0

d¡

(0) =



1; 0;



A similar argument holds for @

f, that is, if ¡

is the projection of γ

(t) = (1; 1 + t) on the

graph of f , then

d¡

(0) =



0; 1;



Vectors

d¡

(0);

d¡

(0) are both tangent to the g raph of f at (1; 1; 2

) and thus the plane

span



d¡

(0);

d¡

(0)



is the tangent plane to the surface of f at (1; 1; 2

). The algebraic

equation o f the tangent plane is derived by the air of n~

n~ = v~

×v~



; ¡

; 1



;

3 Derivatives 19

and thus the algebraic of the tangent plane is derived as

(x ¡1) ¡

(y ¡1) + z ¡ 2

= 0:

Remark 4. For a general function z = f(x; y), two principa l tangent lines at p = (a; b) are

= (1; 0; @

f(a; b)); v~

= (0; 1; @

f(a; b));

and thus n~ , the normal vector to the graph of f at p is

n~ = (¡@

f(p); ¡ @

f(p); 1):

Accordingly, the algebraic equation on the tangent plane at p is

¡@

f(a; b) (x ¡a) ¡@

f(a; b) (y ¡b) + z ¡f(a; b) = 0:

3.1.3 Chain rule

Let f be a scalar function on D

⊂R

, and a ssume that x; y are functions of another variable,

say t, i.e., x = x(t); y = y(t). In the ﬁnal analysis, f(x; y) is a function of t and thus the

ordinary derivative

can exist.

(t) = lim

h!0

f(x(t + h ); y(t + h)) ¡ f (x(t); y(t))

Proposition 7. Assume that f is diﬀerentiable with respect to x and y , and moreover the

partial derivatives are continuous. If x(t); y(t) are diﬀerentiable functions of t, then the

following equality that is called chain rule holds

= @

f(x(t); y(t))

+ @

f(x(t); y(t))

: (5)

There is an important interpretation for the above fo rmula. First, note that γ:(x(t); y(t))

deﬁnes a parametric curve in the (x; y)-plane, and thus f (x(t); y(t)) can be considered as the

restriction of f to γ. Also, we can consider γ as the path of a particle moving in the (x; y)-

plane. Therefore, relation (5) states the rate of change of the value of that particle along γ.

Fo r example, if f is the density distribution function in the plane, then relation (5) states

how fast or slow the density of a particle changes when it moves along path γ.

f(γ(0))

f(γ(t

))

γ(t)

f(γ(t

))

20 Appendix

On the other hand, since the graph of f is a surface in R

, f(γ(t)) is the projection of

γ(t) in the surface as shown in the following ﬁgure. With this interpretation, relation (5)

deﬁnes the slope of tangent to the curve at any instance of time.

Let us consider again equality (5) and rewrite it as follows

+ @





Note that

d x

is just the tangent vector of the curve γ = (x(t); y(t)), i.e., γ

(t). Vector

is called the gradient of f and is denoted by grad(f) or rf. Therefore, equality (5)

can be rewritten as

= rf · γ

(t);

and for this reason,

df(γ(t))

is called also the derivative of f along γ(t). The chain rule can

be extended to higher dimensions. For example, if f(x; y) is a diﬀerentiable function with

respect to x; y and x = (x(t; s)); y = y(t; s) are diﬀerentiable functions then

f = @

x + @

f = @

x + @

Problem 50. If u = u(t; x), a nd x = x(t), ﬁnd

Problem 51. If u = f(x ¡2t) ﬁnd @

u and @

3.1.4 Directional derivative

Partial derivatives are just spe cial cases of a mo re general derivative called the directional

derivative. Assume that a direction vector v^ = (v

; v

) is given (a direction vector is a unit

vector), and f: D

! R is a given continuous function. T he directional derivative of f at

(a; b) 2D

along v^ is deﬁned by the following limit

f(a; b) = lim

t!0

f(a + tv

; b + tv

) ¡ f(a; b)

;

3 Derivatives 21

as long as the limit exists. If so, then @

f(a; b) measures the rate of change of f a t (a; b)

along v^. Obviously if v^= (1; 0) then @

f(a; b) = @

f(a; b) and if v^= (0; 1), it would be equal

to @

f(a; b).

Proposition 8. If @

f ; @

f are continuous at (a; b), that is,

lim

(x;y)!(a;b)

f(x; y) = @

f(a; b); lim

(x;y)!(a;b)

f(x; y) = @

f(a; b);

then

f(a; b) = rf(a; b) ·v^

The continuity in the above proposition is crucial, for example, consider the following

function

f(x; y) =

+ y

(x; y) =/ (0; 0)

0 (x; y) = (0; 0)

If v^=



;



, then

(0; 0) = lim

t!0

2 2

;

however, rf(0; 0) =





, and thus rf (0; 0) · v^ = 0. The reason is that @

f ; @

f are not

continuous at (0; 0). To see this, let us ﬁnd @

f for (x; y) =/ 0 as

f(x; y) =

¡y

)

+ y

)

;

and observe that @

f does not have even a limit at (0; 0). Note a lso that the directional

derivative of a function f is a special case of the chain rule rf ·γ

(t).

Problem 52. For the following function

f(x; y) =

+ y

(x; y) =/ (0; 0)

0 (x; y) = (0; 0)

show that @

f(0; 0) exists for any direction r^ but partial derivatives are not continuous at (0; 0).

3.1.5 Gradient

Consider the le vel curve deﬁned by γ: f (x; y) = c. If γ( t) = (x(t); y(t)) is a parametrization

of this curve then rf ·γ

(t) = 0 for any t as long as rf is a continuous vector function. This

relation means that rf(γ(t)) is always perpendicular to γ(t). For example, the level curve

+ 2x

y + x

+ y

= 1;

22 Appendix

has the gradient

rf =

+ 4xy + 2x

+ 2y

The following ﬁgure shows a few of gradient vectors ∆f on the level curve

As it is observed, gradient vectors are perpendicular to level curves. On the other hand, if

n^ = (n

; n

) is the direction vector at a point on the level curve, the the directional derivative

f of f is rf ·n^, and since n^ =

krf k

(as long as the rf =/ 0), we obtain

f = krf k:

Therefore, the magnitude of ∆f at a point measures the rate of change of f along the normal

direction on the level curve. This result is extremely useful to maximize (or minimize) a

scalar function. The procedure is as follows. To maximize f (x; y), we ﬁx an initial point

= (x

; y

). The next point p

is obtained by the following relation

= p

+ λ

rf(p

)

krf(p

;

where λ > 0 is a small value. Geometrically, that mean we take a step of length λ along the

direction rf (p

). Iterating this procedure, that is,

n+1

= p

+ λ

rf(p

)

krf(p

;

converges the maximum point of f a s long such a local or global maximum exists, and if f

satisﬁes some other veriﬁable conditions.

In above, we used frequently the operator nabla r=

. It is applied to diﬀerentiable

function as rf =

. We study this operator in more detail later in this app endix.

3 Derivatives 23

Problem 53. Show the following relations

a) r(f + g) = rf + rg.

b) r(kf ) = k rf , k 2R.

c) r(fg) = f rg + g rf .

3.1.6 Derivative and diﬀerential

Let f: D

⊂R

!R be a s calar function deﬁned on an open set D

. T he derivative of f at

is the linear mapping D

f: R

!R such that the following relation holds for any h

lim

f(p

+ h

) ¡ f(p

) ¡D

f(h

)

= 0 (6)

Obviously if f is diﬀe rentiable at p

then it is continuous at that point. Moreover, such

a linear mapping must be unique . Note that if a function f is diﬀerentiable at a point a

then directional derivatives along any direction at a exist. The reverse holds only if partial

derivatives are continuous at a.

Problem 54. Verify that the above deﬁnition is compatible with the usual deﬁnition for sin gle variable

functions.

Problem 55. Show that if f is diﬀerentiable at p

it must be continuous at that point. Show also that

its derivative (the linear mapping) is unique.

Proposition 9. Assume that f has continuous partial derivatives at p

, that is,

lim

p!p

f(p) = @

f(p

);

for k = 1; :::; n, then D

f exists and has the matrix representation

f = [@

f(p

); ···; @

f(p

)]:

According to the above proposition, if f has continuous partial derivatives at p

then for

any h

, the following relation holds.

f(h

) = rf (p

) ·h

Fo r this reason, some texts write D

f = rf (p

) that we should keep in mind that D

f is

a 1 ×n matrix, while rf (p

) is a n ×1 vector.

24 Appendix

Example 6. For example, function

f(x; y) =

+ y

(x; y) =/ (0; 0)

0 (x; y) = (0; 0)

has directio nal derivatives in all directions at the origin, howeve r, the function is not dif-

ferentiable at this point since it is not even continuous at the origin (why?). On the other

hand, functionf(x; y) = x

+ y

has continuous partial derivatives @

f = 2x; @

f = 2y and for

any p

= (x

; y

), we have

f = 2[x

; y

Let us verify deﬁnition (6) for f at p

= (1; ¡1). For arbitrary h

= (h

; h

), we have

lim

(1 + h

)

+ (¡1 + h

)

¡2 ¡2(h

¡h

)

+ h

= lim

+ h

= lim

k= 0:

An immediate result of deﬁnition (6) is the linear approximation formula. If f: D

is continuously diﬀerentiable at p

, that is

lim

p!p

f = D

f ;

then

f(p) ≈f(p

) + D

f(p~ ¡p~

);

or equivalently

f(p) ≈f(p

) + rf (p

) ·(p~ ¡ p~

Fo r functions of two variables, the above formula reads

f(x; y) = f(x

; y

) + @

f (x ¡x

) + @

f (y ¡y

Note that the right hand side is the equation of tangent plane at (x

; y

; f (x

; y

)):

T (x; y) = (x

; y

) + @

f (x ¡x

) + @

f (y ¡y

Theorem 3. (Mean Value Theorem) Assume that f: D

! R is continuously dif-

ferentiable everywhere in D

. Fix a point p

2 D

. Then for any point p 2 D

, there is

ξ 2tp + (1 ¡t)p

, t 2(0; 1) such that

f(p) = f(p) + rf (ξ) ·(p~ ¡ p~

Problem 56. Prove the theorem. H int: deﬁne g(t)= f(tp + (1 ¡t)p

) and apply the mean value theorem

for the single variable function g(t); t 2[0; 1].

3 Derivatives 25

Deﬁnition 12. The total diﬀerential of f: D

!R at p

is deﬁned by the following formula

df(p

) = @

f(p

) dx

+ ···+ @

f(p

) dx

Remember that for a single variable function y = y(t), the diﬀerential dy is deﬁned by

the relation dy(t

) = y

) dt. Fig.(6) b elow shows this relation geometrically.

dy(t

)

y = y(t)

y(t

)

dy = y

) dt

T (x )

Figure 6.

Similarly for a 2-variable function z = f (x; y), dz is deﬁned by the f o llowing relation and

its geometry is represented in Fig. 7.

dz = @

fdx + @

fdy:

; y

)

dz = @

fdx + @

fdy

f(x

; y

)

Figure 7.

3.1.7 Critical points and local max and min

Let D

⊂R

be a n open set. A point a 2D

is called a local min (or max) of f: D

!R if

there is a ball B

(a) such that f(p) ≤ f(a) (alternatively f(p) ≥ f(a )) for all p 2 B

(a). If

f is diﬀerentiable at a, a nd if a is a local min or max, then D

f is a zero mapping. To see

26 Appendix

this, suppose a is a local min, choose an arbitrary direction vector h

, and write

0 = lim

t!0

f(a + th

) ¡ f(a) ¡D

f(th

)

kth

= lim

t!0

f(a + th

) ¡ f (a) ¡tD

f(h

)

jtj

Fo r t > 0, we have

f(h

) = lim

t!0

f(a + t h

) ¡ f(a)

≥0

Fo r t < 0, we have

f(h

) = lim

t!0

f(a + t h

) ¡ f(a)

≤0;

and thus D

f(h

) = 0 for arbitrary h

, and thus D

f is a zero mapping (meaning rf(a) is a

zero vector).

Deﬁnition 13. (Critical point) A point a is called a critical point of a function f if either

f does not exist or D

f is a zero mapping. If D

f is a zero mapping, a can be a local min,

local max, a saddle point or non of them.

In order to determine the type of a critical point in terms of min, ma x or saddle, we

need the notion o f second derivatives. Second order partial derivatives @

f are deﬁned as

f = @

f). We have the following theorem.

Theorem 4. Let D

⊂R

be an open set, and f: D

!R . Furthermore assu me that @

f is

continuous, then @

f = @

Theorem 5. Assume that f : D

!R is continuously diﬀerentiable of order 2 on open set D

Fix a 2D

, then there is ξ 2tp + (1 ¡t)a for some t 2(0; 1) such that the following relation

holds for any for any p 2D

f(p) = f(a) + rf(a) ·(p~ ¡a~ ) + (H

(ξ)(p~ ¡a~)) ·(p~ ¡a~ );

where H

is the Hessian matrix of f deﬁned as H

= [@

i;j

Corollary 1. Assume that f is second order continuously diﬀerentiable function and rf(a)=

0, then a is a local min if H

(a) is a positive deﬁnite matrix, a is a local max if H

(a)

is a negative deﬁnite matrix, and a saddle point if H

(a) has eigenvalues with opposite signs.

The standard example of above three cases is f = x

+ y

; f = ¡ x

¡y

, and f = x

¡ y

as shown below

3 Derivatives 27

Note that H

(a) is a symmetric matrix, and thus it has n orthogonal eigenvectors, and

the Jordan form diag(λ

; :::; λ

). Therefore if all eigenvalues of H

(a) is positive then a is a

local min, if its all eigenvalues are negative then a is local max, a nd if there are some po sitive

and some negative eigenvalues, it is a saddle point. If there is at least one zero eigenvalue,

or equivalently det(H

(a)) = 0, then a may not be any of these types.

Problem 57. Find all critical point s of the function f(x; y)= x

+ y

+6x and classify them.

3.2 Derivative of non-s calar mappings

3.2.1 Jacobi matrix

Let D

⊂R

be an open set and f: D

a continuous map. f is diﬀere ntiable at a 2D

if there is a linear mapping D

f: R

that satisﬁes the following relation

lim

kf(a + h

) ¡ f (a) ¡D

f(h

= 0:

Equivalently, f = (f

; :: :; f

) is diﬀerentiable at a if and only if each coordinate function f

is diﬀerentiable at a. On the other hand, the derivative of f

at a is

= [@

(a); :::; @

(a)];

and since each f

is deﬁned on R

, we obtain

f =

(a) @

(a) ··· @

(a)

(a) @

(a) ··· @

(a)

(a) @

(a) ::: @

(a)

The above matrix is called the Jacobi matrix of f at a, denoted also by J

(a). Let γ(t) be

a smooth curve passing through a at t = 0. This curve is mappe d into R

by f as f(γ(t)).

In the ﬁnal analysis, f(γ(t)) is a vector valued function and therefore, we have

df(γ(0))

= D

γ(0)

f(γ~

(0)):

28 Appendix

Since γ

(0) is the tangent vector on γ(t) at t = 0, D

γ(0)

f(γ~

(0)) is the tangent vector on

f(γ( t)) at t = 0. For example, for f(x; y) = (x

¡y

; x

+ y

) we have

(1;1)

f =



2 ¡2

2 2



If γ(t) = (e

¡t

; e

) (passing through (0; 0) at t = 0), we have γ

(0) =



¡1



and accordingly,

γ(0)

f(γ

(0)) =



¡4



. Note that

f(γ(t))j

t= 0

¡2t

¡e

; e

¡2t

+ e

¡2t

t= 0



¡4



3 Derivatives 29

Theorem 6. Assume that the Jacobi matrix of a mapping f: D

⊂R

is invertible at

a. Then there is a neighborhood B

(a) such that f is one to one on B

(a).

Note that det(J

(a)) measures the volume of a parallelogram constructed on columns of

(a). In other word, if C is a unit cube made at a, then det(J

(a)) is equal to the volume

of parallelogram J

(a)(C). Hence, if det(J

(a)) =/ 0, there is a neighborhood B

(a) such that

f is one to one on B

(a).

3.2.2 Smooth surfaces

Remember that a smooth space curve is represented by a curve map γ(t) such that γ

(t) is

nonzero. Geometrically, this condition means that γ(t) always admits a tangent vector that

varies continuously along γ.

Deﬁnition 14. A surface S in R

is called smooth if S has a nonzero normal (perpendicular)

vector at all points on S.

Let f (t; s) 2 R

be parametric surfa ce. Consider an arbitrary point f (t

; s

) on S.

Coordinate line γ

(t) = (t + t

; s

) is mapped on S as ¡

(t) = f(t + t

; s

). The tangent vector

to this space curve is just ¡

) = @

f(t

; s

). Similarly, the coordi nate line γ

(s) = (t

; s + s

)

is mapped as ¡

(s)= f (t

; s +s

) and the tangent vector is ¡

(s)=@

f(t

; s

). Notice that both

; ¡

are tangent to S and therefore n~ := ¡

×¡

is perp endicular to S a s long as it is nonzero

n~ =



x (t

; s

) @

y(t

; s

) @

z(t

; s

)

x (t

; s

) @

y(t

; s

) @

z(t

; s

)



=/ 0:

Let us see the result for a smooth f unction z = f (x; y). The graph of function is

φ(x; y) = (x; y; f(x; y));

and thus @

φ = (1; 0; @

f) and @

φ = (0; 1; @

f). The normal vector n~ is

n~ = @

φ ×@

φ = (¡@

f ; ¡@

f ; 1):

Example 7. For p

= (0; 1; 1) on the surface of f (x; y) = x

+ y

, consider coordinate lines

(x) = (x; 1); γ

(y) = (0; y). We have f (γ

) = (x; 1; 1 + x

), f(γ

) = (0; y; y

), see the ﬁgure

shown below. Respectively, the tangent vectors are as T

= (1; 0; 0), T

= (0; 1; 2). The normal

vector n~ to the surface at p

n~ = T

×T

= (0; ¡2; 1);

and therefore the equation of tangent plane is ¡ 2(y ¡1) + z ¡1 = 0.

30 Appendix

Fo r a surface represented by the implicit function S: f (x; y; z) = 0, the normal vector n~

is derived by the f o llowing procedure. Consider an arbitrary space curve γ = (x(t); y(t); z(t))

on S, that is, f(x(t); y(t); z(t)) = 0. The chain rule s tates

(γ) = rf (γ(t)) ·γ

(t) = 0:

Since γ

(t) is tangent to S, then rf is perpendicular to γ(t) if rf is nonzero. On the

other hand, since γ is arbitrary, then γ

belongs to the tangent plane on S, and thus rf is

perpendicular t o S. Therefore, we obtain n~ as

n~ = (@

f ; @

f):

Problem 58. Write the equation of curve formed by the intersection of the unit sphere x

+ y

+ z

= 4

and the plane x + y + z = 1.

3.3 Implicit function theorem

An implicit function f(x; y) = 0 deﬁnes generally a planar curve in the (x; y) p lane. If

y = y(x), then by the chain rule, we can write

= ¡

However, there is no guarante e in general that y could be solved in terms of x or x could

be solved in t erms of y. Question is this: is there any function y = g(x) deﬁned on an open

interval I such that f(x; g(x)) = 0 for all x 2I. The following theorem answers the question.

Theorem 7. (implicit function theorem) Suppose implicit function f(x; y) = 0 satisﬁes

the following conditions

i. there is a point p

= (x

; y

) such that f(x

; y

) = 0,

ii. there is an open ball B

) such that f has continuous partial derivat ives on it,

3 Derivatives 31

iii. and that @

f(x

; y

) =/ 0,

then there is an open interval I = (x

¡δ; x

+δ), and a function y = g(x) such that y

= g(x

and f(x; g(x)) = 0 for all x 2I.

Example 8. Consider the following function

+ x + sin(y) = 1:

The function deﬁnes a planar curve which is shown below in Fig.8.

-2 0 2

-4

-2

−

= 1

Figure 8.

The slopes at x = ¡0.49 and x = 1.96 are inﬁnity, which implies @

f = 0 at those points

according to the formula y

= ¡

. Now ﬁx the point p

= (0; 0) on the curve. As it is seen

from the ﬁgure, there is an explicit function y = g(x) for x 2(¡0.49; 1.96) such that

x g(x)

+ x + sin(g(x)) = 1: (7)

The result can be generali zed for functions f: R

!R as follow s.

Theorem 8. Assume t hat implicit function f (x

; x

; :::; x

) = 0 satisﬁes the following

conditions

i. there is a point a = (a

; a

; :::; a

) such that f(a

; ·· ·; a

) = 0,

ii. There is an open ball B

(a) such that f has continuous partial derivatives on it,

iii. and that @

f(a) =/ 0,

then there exists a ball B

at a

= (a

; ···; a

n¡1

) and a function g: B

) ! R such that

= g(a

; ·· ·; a

n¡1

) and f(x

; x

; :::; g(x

; :: :; x

n¡1

)) = 0 for all (x

; :::; x

n¡1

) 2B

32 Appendix

4 Integrals of mutivariable functi ons

4.1 Line integrals

Let γ: (a; b) !R

be a smooth curve map, that is γ

(t) =/ 0. The length of γ(a; b) is deﬁned

by the following integral

L =

jγ

(t)jdt:

This deﬁnition coincides the intuitive notion of the length. For example, if the image of

γ(t) is a curved metal wire, and if we straightened the wired, we get the same length if we

calculate the length by the above integration. In the above deﬁnition, γ

(t) is the tangent

vector on γ at t. The quantity dl = jγ

(t)jdt is ca lle d the diﬀerential arc length.

Now let D ⊂R

be an open set and let γ be a smooth curve in D. Assume that f: D !R

is a continuous function. The integral of f along γ is deﬁned by the follow ing integral

I =

f(γ(t))dl =

f(γ(t)) jγ

(t)jdt:

Geometrically this integral is equal to the surface area constructed on the base γ(t) an the

height f (γ(t)). In parti cular if f = 1, the above integral gives the arc length of γ. If f denote

the density function of a metal wire represented by γ, the integral denote the total mass of

the wire.

4 Integrals of mutivariable functions 33

Problem 59. Consider the metal wire in the shape of the semi-circle x

+ y

= 1, y ≥0. If the density

of the wire is given by ρ = k(1 ¡ y) for a constant k, ﬁnd the center of mass of the wire.

4.2 Inte grals over a bounded do main of R

Now, let R: [a; b] ×[c; d] be a rectangle and f: R ⊂R

!R be a continuous function. Similar

to the integrals of single variable function, we can deﬁne the following double integral

I =

Z Z

f(x; y )dA;

by the aid of a n inﬁnite sum called the Riemann sum as

I = lim

n; m!1

j=1

i=1

f(x

; y

) j∆

Here ∆

is a partition of R, j∆

j is the area of the rectangle ij and (x

; y

) is an arbitrary

point in ∆

. We have the following theorem. Geometrically, I denote the volume constructed

on the base R, and height f .

Theorem 9. (Fubini) Assume that the f is continuous (or piecewise continuous) in R = [a;

b] ×[c; d]. Then we have

f(x; y) dA =



f(x; y) dy



dx =



f(x; y) d x



dy:

Now, assume that D ⊂ R

is a closed bounded domain and assume that f: D ! R is

continuous. We can inscrib e D inside a rectangle R and extend f on R as follows

(x; y) =



f(x; y) (x; y) 2D

0 (x; y) 2D

Even though f

may be discontinuous, we have the following fact

Z Z

f(x; y) dA =

Z Z

(x; y) dA:

Problem 60. Let I = [a; b] and assume that f (s; x) and

(s; x) are continu ou s in I × I. Use the

fundamental theorem of calculus and prove the following formula

f(s; x) ds = f(x; x) +

f(s; x) ds:

In the pro of yo u may need to pass the limit inside the integral.

4.3 Change of variables in multiple integrals

Of most important techniques to calculate a double integral (and also triple integrals) is the

change of variable technique. Let D ⊂R

be a bounded domain, and assume f: D !R is a

continuous function. The goal is to calculate the integral

I =

Z Z

f(x; y) dA:

34 Appendix

Now assume that there is a one to one mapping ': R ⊂ R

! D, where R is rectangle [a;

b] ×[c; d]. If so, then we can calculate I in terms of an integral over R. The advantage is that

the integration over rectangular domains are much more simpler than general domains. The

procedure is as follows. Let '(u; v) = (x; y), where (u; v) 2R. See the following ﬁgure. The

diﬀerential area dA in (x; y)-plane in terms of the diﬀerential area dS in the (u; v)-plane is

dA = jdet(J

)jdS;

where J

is the Jac o bi matrix of the one to one transformation '.

v y

Accordingly, we have the fo llowing formula called the change of variable technique

Z Z

f(x; y) dA =

Z Z

f('(u; v)) jJ

(u; v)jdu dv:

Problem 61. Find the domain D formed by the transforming rectangle [1; 2] × [1; 2 ] under the

transformation



x = u/v

y = uv

Calculate the following integral

Z Z

y/x

dA:

4.4 Surface integrals over a surface in R

Let S be a smooth surface in R

with the representation

'(u; v) = (x(u; v); y(u; v); z(u; v));

where (u; v) 2D ⊂R

, and assume that f is a continuous functions deﬁned on S: f: S !R.

We want to calculate the following integral

I =

Z Z

f(x; y; z) dA;

where dA is the diﬀerential area of the surface S. Here we again transform the integral over

S as an integral over D as follows. Note that

dA = jj@

' ×@

'jjdS ;

4 Integrals of mutivariable functions 35

where dS is a diﬀerential area in D. Re member that @

' and @

' are tangent vectors on

S and k@

' × @

'k is equal to the area of parallelogram constructed on vectors @

'; @

Therefore, we can write

Z Z

f(x; y; z)dA =

Z Z

f('(u; v)) jj@

' ×@

'jjdS:

Problem 62. If f = f (x; y) is a smooth function deﬁned on the bounded set D ⊂ R

, show that the

area of the surface associated to u is

A =

1 + j@

f j

+ j@

f j

dx dy:

Problem 63. We show that the change of variable formula is independent of the transformation.

a) Assume ': D !S and : D

!S are two one to one transformations with image S. Deﬁne the

map '~ := '

¡1

◦ : D

!D. Verify

jj@

×@

jj= jj@

' ×@

'jjjj@

'~ ×@

'~jj:

b) Now show

f('(u; v)) jj@

' ×@

'jjdu dv =

f( (s; t))j@

×@

jdsdt:

We frequently use the polar and spherical coordinates f o r integ rals. In polar coordinate,

the transformation is deﬁn ed by the relations '(r; θ) = (r cosθ; r sinθ). The area diﬀerential

dS in this case is

dS =



cosθ sinθ

¡r sinθ r cosθ



drdθ = rdrdθ:

If f(x; y) is a function deﬁned in disk B

, a disk of radius a cente red at the origin, its double

integral in the polar coordinate is

f(x; y)dA =

2π

f(r cosθ; r sinθ) rdrdθ:

Problem 64. Show the following inequality

(1 ¡e

¡a

) ≤

¡x

¡y

ddxdy ≤

(1 ¡e

¡2a

);

and conclude

lim

a!1

¡x

¡y

dA =

Use the above result and ﬁnd

I =

¡x

dx:

In the spherical coordinate, the transformation is

'(ρ; φ; θ) = (ρ cosφ s inθ; ρ sinφ sinθ; ρ cosθ):

The volume diﬀerential in this coordinate is

dV =



sinθ cosφ sinθ sinφ cosθ

ρ c o sθ cosφ ρ cosθ sinφ ¡ρ sinθ

¡ρ sinθ sinφ ρ sinθ cosφ 0



dρdφdθ = ρ

sinθdρdφdθ:

36 Appendix

If f(x; y; z) is deﬁned, for example, in a sphere of radius a, its integral on this sphere is equal

f(x; y; z)dS = a

2π

f(a sinθ cosφ; a sinθ sinφ; a c o sθ) sinθdθdφ:

5 Calcul us of vector ﬁelds

5.1 Vecto r ﬁeld

Let D ⊂R

be an open set. A vector ﬁeld is a mapping f: D !R

. For e a ch p 2D, f(p) is

a vector in R

, and thus we can interpret this mapping as an assignment a vector f(p) to

every point p 2D, that is, p 7! f (p). This assignment is called a vector ﬁeld.

A vector ﬁeld p 7!f(p) is continuous if the association varies continuously with respect to

p, that is, the any change from p to an adjacent point q, the vector f (p) continuously varies

to f (q). Mathematically speaking, this is equivalent to the continuity of f as a mapping

on D. Remember that f = (f

; :::; f

) is a continuous mapping if and only if ea ch scalar

function f

is a continuous function. Similarly, a vector ﬁeld p 7!f (p) is called c o ntinuously

diﬀerentiable if and only if all its coordi nate functions are continuously diﬀerentiable.

A vector ﬁelds models many physical phenomena. For example, an electrical charge q

located at the origin, generates an electrical ﬁeld in the space as

E(r) =

4π"

kr~ k

;

where "

is the permittivity constant of the space. This ﬁeld is a force ﬁeld an is completely

similar to the gravitational ﬁeld generated by a mass M:

g(r) = GM

kr~ k

;

where G is the universal constant.

5.2 Vecto r ﬁelds and diﬀerential equations

The theory of ordinary diﬀerential equations can be f o rmulated in terms of vector ﬁelds. In

fact, the system

= f

; :::; x

)

= f

; :: :; x

)

deﬁnes a vector ﬁeld f = (f

; :::; f

) on some domain D, and a parametric curve γ(t) =

(t); :::; x

(t)) such that the tange nt vector on γ at each instant of time t

that is γ

)

coincides with the vector assigned to point γ(t

). In other word, f(γ(t)) = γ

(t) for all t in

an open interval. For example, the following system



= ¡y

= x

;

5 Calculus of vector fie lds 37

deﬁnes vector ﬁeld f = (¡y; x). As we know, the trajectory of a point p

: (x

; y

) according

to the above system is the line

γ(t) = (x

cost ¡y

sint; x

sint + y

cost):

It is simply seen that

(t) = (¡x

sint ¡ y

cost; x

cost ¡ y

sint);

that is equal to f (γ(t)). Notice that γ(t) is just the rotation mapping applied to p

γ(t) =



cost ¡sint

sint cost





;

that coincides with the geometry of vector ﬁeld f

5.3 Gradient ﬁeld

A vector ﬁend p 7! f(p) is called a potential, conservative or just gradient ﬁeld if there is

a scalar function φ such that f = ¡rφ. The negative sign is just for historic reason. For

example, the electric ﬁeld E(r) or the gravitational ﬁeld g(r) are po tential. It is simply seen

that E(r) = ¡rφ, where φ is the following scalar function:

φ(r) =

4π"

kr~ k

Potential ﬁelds satisﬁes very nice properties that we study in sequel. In the following ﬁgure,

three vector ﬁelds are shown: f

= (x; y), f

= (¡x; ¡y), and f

= (¡y; x).

−

5 0

0 0

5 1

−

5 0

0 0

5 1

−

5 0

0 0

5 1

−

It is seen that f

; f

are potential ﬁeld with potentials φ

= ¡

, φ

. The

ﬁled f

is not potential, i.e., there is no potential function φ such that f

= rφ.

Problem 65. Prove the ab ove claim, that is, show there is no scalar function φ such that f

= rφ.

The fo rce ﬁeld generated by a potential is also called conservative. To see the reason, let

us write the second Newton's law for a un it mass as

= p(x; y)

= q(x; y)

;

38 Appendix

where the ﬁeld f = (p; q) is potential generated by a potential function φ(x; y). Let γ(t) be

the trajectory of a particle initially located at (x

; y

). The n we have

(t) = f(γ(t)) = ¡rφ(γ(t)):

Let us deﬁne the energy along γ(t) as follows

E(t) =

kγ

(t)k

+ φ(γ(t )):

The derivative of E along γ(t) is then

= γ

(t) · γ

(t) +

φ(γ(t)):

We have

φ(γ(t)) = rφ(γ(t)) · γ

(t);

and thus

= γ

(t) · γ

(t) + rφ · γ

(t) = (γ

+ rφ) · γ

= 0:

Therefore, the derivative of the energy function along a trajectory γ(t) is zero, in other world,

the energy is conserved along the trajectory of the particle.

5.4 Divergence, curl and Laplacian

Two important operations on smooth vector ﬁelds are divergence and curl. The divergence

of a ﬁeld f = (f

; :::; f

) at a point p is de ﬁned by the following relation

div f(p) =@

(p) + ···+ @

(p)

It is simply seen that the divergence of a vector ﬁeld f is equal to the dot product of

nabla r and ﬁeld f as div f = r:f. It is also equal to the trace of matrix D

f. Intuitively

speaking, div f(p) measure the ﬂow of net ﬂux passing th rough p. If div f(p) > 0, point p

acts like a source that emits or generate ﬂow. All points in ﬁled f

= (x; y) are source points

since div f (p) = 2. If div f(p) < 0, point p acts like a sink that absorbs or attracts ﬂow to

itself. All point s in ﬁeld f

= (¡x; ¡y) are sink since div f (p) = ¡2. If div f(p) = 0, then the

net ﬂow passing through p is zero, that the net amount o f incoming ﬂow is equal to outgoing

ﬂow. All points in ﬁeld f

= (¡y; x) are of th is type.

Problem 66. If φ is a smooth scalar function and let f be a smooth vector ﬁeld, show the following

formula

div (φf) = φ div f + f ·rφ (8)

Another important operation regarding a vector ﬁeld in R

is the curl of the ﬁeld at a

point. The curl of a vector ﬁeld f = (f

; f

) at p is de ﬁned by the following relation

cur(f)(p) =

(p) = (@

¡@

; @

¡@

; @

¡@

)(p):

5 Calculus of vector fie lds 39

Symbolically, we can write the curl in terms of r as r×f. Since curl(f) is a vector at every

point, it also deﬁnes a ne w vector ﬁeld in its domain. At each point p, curl(f )(p) measures the

rotation of vector ﬁeld f in three directions: 1) compo nent @

(p) ¡@

(p) that measures

the rotation of f at p around x-axis, 2) component @

(p) ¡ @

(p) that measures the

rotation of f at p around y-axis, 3) and @

(p) ¡@

(p) that measure the rotation around

z-axis . For example, curl (¡y; x) = 2 k

at all points, that means the ﬁled rotates around z-

axis with constant speed at a ll points. This rotation is e vident from the ﬁgure of the ﬁeld.

Assume that φ is a smooth scalar function deﬁned on an open subset D of R

. The

Laplacian of φ is deﬁned by the following relation

∆φ = @

φ + ···+ +@

φ:

It is simply seen that ∆φ =div (grad φ).

Problem 67. Consider ﬁeld f = (x

¡y

; y

¡z

; z

¡x

a) Find r× f at the point (1; 2; 3).

b) Verify that (r× f): i

is equal to r×(0; y

¡z

; z

¡1).

Problem 68. Assume that φ is a smooth scalar function deﬁned in R

. Show r×rφ = 0.

Problem 69. Show the following relations for smooth ﬁelds f ; g in R

and smooth function φ

a) r×(φf) = φr× f ¡f ×(rφ).

b) r:(f × g) = g:(r× f) ¡ f(r× g).

Problem 70. If f is a smooth vector ﬁeld, what is r: (r× f)?

Problem 71. Show the following relation

r×(r× f ) = r(r:f) ¡∆ f ;

where ∆f = (∆f

; ∆f

) for a smooth vector ﬁeld f = (f

; f

5.5 Line integrals i n vector ﬁelds

Assume that the f: D ⊂R

is a continuous ﬁeld, and C is a smooth curve in D. We

deﬁne a method for the integral of f along curve C. For this, we take a parametrization of

C as γ: (a; b) !D, where γ(a; b) = C. The desired integral is deﬁned as fo llows

fdc =

f(γ(t)) · γ

(t) dt: (9)

It is seen that the integral is independent of parametrization, that is, if γ

: (c; d) ! D is

another parametrization of C, then

I =

f(γ

(t)) ·γ

(t) dt:

Problem 72. Assume that γ: (a; b) ! D, γ

: (c; d) !D are two smooth parametrizations of C. Show

the relation

f(γ(t)) · γ

(t)dt =

f(γ

(t)) · γ

(t) dt:

40 Appendix

If f is a force ﬁeld, then the integral of f along a curve is called the work done by f . For

example, let γ(a) = p

, γ(b) = p

, then I measure the total work done by f to move a particle

from point p

to point p

along path γ. It is is interesting to note that if f is gradient, i.e.,

f = rφ for some potential φ, then the integral is independent of path along a particle moves

from p

to p

. This important fact is shown below

rφ(γ(t)) ·γ

(t) dt =

dφ(γ(t)) = φ(γ(b)) ¡ φ(γ(a)) = φ(p

) ¡ φ(p

The path independence property of a ﬁeld implies that the integral of ﬁeld over any closed

curve is zero, that is, if γ is a clo sed curve then

f(γ(t)) · γ

(t) dt = 0: (10)

We have the following theorem.

Theorem 10. Assume f: D ⊂R

is a smooth vector ﬁeld. If the integral of f over all

closed curves in D is zero, then f is a conservative ﬁeld on D.

Problem 73. Let the condition of the above theorem holds. Fix p

2D. For any p 2D deﬁne φ as

φ(p) =

f(γ(t)) · γ

(t) dt;

where γ(t) is an arbitrary smooth curve in D such that γ(0) = p

, and γ(t) = p.

a) Verify that φ is independent of path γ

b) Show that φ is a potential for f. For example, for two dimensional ﬁeld f = (p(x; y); q(x; y) ),

ver ify the relation

lim

h!0

φ(x + h; y) ¡ φ(x; y)

= p(x; y); lim

h!0

φ(x; y + h) ¡ φ(x; y)

= q(x; y):

Problem 74. A smooth ﬁeld f = (p; q) in R

is called exact if for all (x; y) 2R

the following relation

holds

f(x; y) = @

q(x; y):

a) Show that f is conservative.

b) Consider ﬁeld f =



¡y

+ y

;

+ y



deﬁned everywhere in R

except t he origin. Show the following

relation



¡y

+ y



= @



+ y



Now, consider closed curve γ(t) = (cos t; sin t), t 2[0; 2π], calculate the line integral of f along γ.

The result is nonzero. Does it contradict the fact claimed in (a)?

5.6 Surface integrals in ve ctor ﬁelds

Let S be a surface in R

parameterized by a smooth map ¡(t; s) where (t; s) 2D for some

open domain D. Note that ¡ is smooth if @

¡ ×@

¡ is nonzero for all points in D. Let smoo th

ﬁeld f = (f

; f

) given in R

. The integral of f on the surface S is deﬁned by the followin g

integral

I =

f ·n^ dA ;

5 Calculus of vector fie lds 41

where n^ is the unit normal vector on S. This i ntegral measures the total ﬂux passing outward

through surface S. As it is shown in the following ﬁgure, the tangential component of f

never leaves the surface (due to the fact f · n^ = 0), and only the normal component of f

contributes in the total ﬂux. For example, the ﬂow of water through a window measured in

sec

is a physical model for the ﬂux.

normal component

of f

tangent component

of f

Remark 5. In some branches of mathematics, the term ﬂux is considered as a vector and

not scalar. We will see this notion in the book for the mathematical modeling of heat ﬂow

through a conductive media.

Fo r the parametrization ¡(t; s), we have

n^ dA = (@

¡ ×@

¡) dt ds;

and therefore, the total ﬂux is expressed in terms of double integral

I =

f(¡(t; s)) ·(@

¡ ×@

¡) dt ds:

Theorem 11. (divergence) Assume D ⊂ R

is an open set with smooth (piecewise)

boundary. If f is a smooth vector ﬁeld in cl(D), the following relation holds

ZZZ

div f dv =



bnd(D)

f ·n^ dA: (11)

Note that If div f = 0 inside D, then the net amount of ﬂow passing throu g h bnd(D)

is zero. Accordingly, the divergence of a ﬁeld f in R

at a point p can be deﬁned by the

following formula

div (f)(p) = lim

r!0

Vol(B

(p))



bnd(B

(p))

f ·n^ dA : (12)

The above formula coincides with our previous statement about the physical interpretation

of divergence operator at a point, that div (f )(p) measures the net ﬂux of f passing through

point p.

Problem 75. Let f = (x; y; z). Verify the formula (12) at the origin.

Example 9 . Consider the identity ﬁeld f (x; y; z) = (x; y; z), and let B be the closed unit

ball in R

centered at the origin. The left hand side of formula (11) reads

ZZZ

div fdV = 3

ZZZ

dV = 4π: (13)

42 Appendix

The unit normal to bnd(B) is n^= (x; y; z), and then the right hand side of f ormula (11 ) reads



bnd(B)

+ y

+ z

) dA =



bnd(B)

dA = 4π: (14)

Problem 76. Assume f: D ⊂R

a smooth ﬁeld. Use the divergence th eorem and show

ZZZ

∆f dV =



bnd(D)

f ds; (15)

where @

f is the directional derivative of f in direction n^.

Problem 77. If D is a ball of radius R in R

, use the divergence theorem and show

Vol(D) =

A(D);

where Vol(D) is the volume of D and A(D) is the surface area of D. (Hint: consider the function

φ =

Problem 78. Let f be a smooth ﬁeld in R

such that

jf(r)j≤

(1 + jjrjj)

n+1

Show that

div (f) = 0 :

Proposition 10. (Integration by parts) Let D ⊂R

be an open set and f ; g: D !R are

smooth functions. We have

ZZZ

f) g dV =



bnd(D)

fgn

dA ¡

f (@

g) dV ; (16)

where n

is the ﬁrst component of n^ = (n

; n

i at bnd(D). Similar relations hold for the

derivatives with respect to other components y; z.

The following proposition generalizes the ab ove result.

Proposition 11. Let D be a domain in R

and bnd(D) is smooth. If f is a smooth vector

ﬁeld in D and g is a smooth function on D then

ZZZ

g div (f ) dV =



bnd(D)

g (f ·n^) dA ¡

ZZZ

f ·rg dV : (17)

Problem 79. Prove the ab ove proposition by the aid of divergence theorem.

Problem 80. By the above proposition show

ZZZ

g∆fdV =



bnd(D)

g @

fdA ¡

ZZZ

rf ·rg dV ;

and conclude the following relation called the Green's formula:

ZZZ

(g ∆f ¡f ∆g) dV =



bnd(D)

[g @

f ¡ f@

g] dA:

5 Calculus of vector fie lds 43

Let S be a smooth surface in R

with smooth boundary bnd(S). If f is a smooth ﬁeld

in R

, the following relation is called th e Stoke's theorem:

(r× f) ·n^ dA =

bnd(S)

f ·T

dl;

where T

is the unit tangent vector on curve bnd(S) and d` is the diﬀerential length of that

curve.

Problem 81. Use the Stoke's theorem and show

(r× f (p)) ·i

= lim

r!0

πr

2π

f(γ(t)) · γ

(t) dt;

where γ(t) = p + (r cost; r sint; 0). Derive similar formula for the second and third components, i.e.,

(r× f (p)) · j

, and (r× f(p)) ·k

. Notice that f ·T

measures the rotation of f along curve bnd(S).

Problem 82. Let f = (¡y + x; ¡x+z;¡z + y). Verify the relations of the previous problem at the origin.

Problem 83. Show the following relations

φ curl(f ) ·n^ dA =

(f ×rφ) ·n^ dA +

bnd(S)

φf ·T

dl:

ZZZ

φ div (f ) dV =



φf ·n^ dA ¡

ZZZ

f ·rφ dV

ZZZ

g ·curl(f ) dV =



(f × g) ·n^ dA +

ZZZ

f ·curl(g) dV

We have the following theorem

Theorem 12. If r×f = 0 everywhere, then f is gradient, i.e., there is a potential function

φ such that f = ¡rφ. If div (f ) = 0 everywhere, then there is a vector ﬁeld g such that

f = curl(g).

6 Orthogonal curvilinear coordin at es

In many applications, it is convenient to use an orthogonal coo rdinate rather than the Carte-

sian one. The form of diﬀerential operators in a general orthogonal curvilinear coordinate is

discussed below.

6.1 Unit vectors

Let (q

; q

) be a curvilinear coordinate system, and assume (q

; q

)

(x; y; z) is a one

to one transformation to the Cartesian s pace. We ﬁrst deﬁne the unit vectors q^

; q

^ and q

in the directions o f q

; q

and q

, and by the aid of that deﬁne the orthogonality notion. If

we consi der the restriction of T to q

as a one parameter map γ(q

), then the unit vector q^

can deﬁned as

: =

)

kγ

T k

;

44 Appendix

and since

T (q

; q

) = (x(q

; q

); y(q

; q

); z(q

; q

));

we have

T = (@

x; @

y; @

Similarly, we can deﬁne q^

; q^

T k

; q^

T k

A coordinate system (q

; q

) is called orthogonal if q^

; q^

and q^

are mutually orthogonal,

i.e., hq^

; q^

i= δ

6.1.1 Polar, cylindrical and spherical coordinates

The polar coordinate (r; θ) is deﬁned by the transformation T (r; θ) = (r cosθ; r sinθ) for

r = [0; 1) and θ 2[0; 2π). The transformation is one to one everywhere in the domain except

at the origin. Since @

T = cosθ i

+ sin θ j

, we obtain

r^ = cosθ i

+ sin θ j

Similarly, @

T = ¡r sin θ i

+ r cos θ j

, and thus

= ¡ sin θ i

+ cos θ j

The cylindrical transformation is deﬁned by transformation

T (r; θ; z) = (r cosθ:r sinθ; z)

The unit vectors are derived as

r^ = cos θ i

+ sin θ j

; θ

= ¡ sin θ i

+ cos θ j

; z^= k

The spherical coordinate is deﬁned by the transformations

T (ρ; φ; θ) = (ρ cosφ sinθ; ρ sinφ sinθ; ρ cosθ);

for ρ 2[0; 1); φ 2[0; 2π); θ = [0; π]. The mapping is not one to one at ρ = 0; θ = 0; π. The unit

vectors are

ρ^= sinθ cosφ i

+ sinθ sinφ j

+ cosθ k

; (18)

= ¡sinφ i

+ cosφ j

; (19)

= cosθ cosφ i

+ cos θ sinφ j

¡sin θ k

: (20)

6 Orthogonal curvilinear coordinates 45

6.2 Nabla r in an orthogonal coordinate

Let f be a smooth scalar function given in an orthogonal coordinate system (q

; q

). We

can write the gradient of f in this system as

rf = f

+ f

;

that implies f

= hrf ; q^

i, f

= hrf ; q^

i, and f

= hrf ; q^

i. Let us calculate f

for example.

rf is coordinate f ree, and thus we can replace it s f o rm in the Cartesian coordinate, i.e.,

rf = @

f i

+ @

f j

+ @

f k

in the a ssociated dot product and derive

hrf ; q^

T k

f i

+ @

f j

+ @

f k

; @

x i

+ @

y j

+ @

z k

By the orthogonality, we obtain

hrf ; q^

T k

x + @

y + @

z) =

T k

We obtain similar forms for f

; f

, that are,

hrf ; q^

T k

f ; hrf ; q^

T k

and therefore the operator r in (q

; q

) is

T k

6.2.1 Polar, cylindrical and spherical coordinate

Applying the formula obtained above for polar, cylindrical and spherical sy stems gives respe c-

tively

Polar: r= r^ @

Cylindrical: r= r^ @

+ k

Spherical: r= ρ^@

ρ sin θ

To calculate the Laplacian operator ∆ := r:r in (q

; q

) system, we do as follows:

∆: =r:r=



T k

;

T k



Here we need @

(q^

) for i; j = 1; 2; 3. It turns out that the derivatives of q^

always lies in the

normal plane to it, for example,

(q^

) = α q^

+ β q^

The reason is for the relation hq^

; q^

i= 1 and thus h@

(q^

); q^

i= 0.

46 Appendix

Problem 84. In polar coordinate, s how the following relations

r^ = θ

; and @

= ¡r^; (21)

and conclude

∆f = @

f +

θθ

f: (22)

Problem 85. In cylindrical coordinate show the relation

∆f = @

f +

θθ

f + @

Problem 86. In spherical coordi nate show the following relations

ρ^= sin(θ) φ

and @

ρ^= θ

; (23)

= ¡sin(θ) ρ^¡cos( θ) θ

and @

= 0; (24)

= cos(θ) φ

and @

= ¡ρ^: (25)

and conclude

∆f =

(ρ

f) +

sin

φφ

f +

sin θ

(sinθ@

f):

7 Fun ction Series

Working in a function vector space where functions play the role of well-known vectors in

, requires to study sequences whose elements are functions. These type of sequences are

a natural generalization of numeric sequences that we supp o se the reader is familiar with.

7.1 The diﬀerent notion s of convergence

We assume that the reader is familiar with numeric sequences. Here we consider sequences

and series w hos e elements are functions. Let f

: [a; b] !R for n = 1; 2; ::: be a sequence of

continuous functions. We say f

converges pointwise in [a; b] to f, and write f

!f if

lim

n!1

(x) ¡f(x)j= 0;

for all a 2[a; b]. In other word, if a

= f

(x) for a ﬁxed x, then (a

) as a numeric sequence

converges to value b = f(x). This n o tion of convergence is equivalent to the following: for

every x 2[a; b] and eve ry " > 0, there is an integer N

= N

(x; ") > 0 such that

8n ≥N

)jf

(x) ¡f(x)j< ":

A function sequence f

converges uniformly to f in [a; b] if for for any " > 0, th ere is and

integer N = N (") such that

8n ≥N

) max

x2[a;b]

(x) ¡f(x)j< ":

The norm of a continuous function in a closed interval [a; b] is deﬁned as

kf k= max

x2[a;b]

jf (x)j:

7 Function Series 47

Fo r this reason, the uniform convergence is usually written as

lim

n!1

¡f k= 0:

Remark 6. The pointwise a nd uniform convergence are two diﬀerent types of convergence

and they may should not be considered as equivalent. In fact, if f

! f uniformly, th en

!f pointwise, while the converse is not generally true. For example, consider the s equence

of functions f

(x) = x

in [0; 1]. The sequence converges pointwise to f(x) =



1 x = 1

0 otherwise

However, it is seen

max

x2[0;1]

¡f(x)j>

;

and thus f

does not converges uniformly to f . In fact, f

does not converge uniformly to

any function.

Problem 87. If a funct ion se quence (f

) converges pointwise or uniformly, then its limit function is

unique.

Problem 88. If a function sequence (f

) converges uniformly to f in [a; b] then it converges pointwise

to f as well.

Problem 89. Assume tha t f

: [a; b] !R is a sequence of continuous functions converges uniformly to

f(x). Show that f(x) is continuous.

There are other notions of convergence that we study in the book, for example, the

convergence in norm as

lim

n!1

(x) ¡f(x)jdx = 0:

Fo r example, f

(x) = x

in [0; 1] co nverges to f (x) =



1 x = 1

0 otherwise

or f(x) ≡ 0 or f (x) =

(

1 x =

0 otherwise

. As we see, the limit function is not unique in the usual sense, and we have to

do something to re medy this situation.

Problem 90. If (f

) converges uniformly to f in [a; b], it converges to f in the following sense

(x) ¡ f(x)jdx !0:

Problem 91. Assume that a function sequence (f

) of continuous functions on [a; b] converges in norm

to f as

(x) ¡ f (x)jdx !0

Show that

lim

n!1

(x) dx =

f(x) dx:

7.2 δ-sequence

A function sequence f

(x) is called a Dirac δ-sequence fu nc tion at x = 0 if fo r any continuous

bounded function g(x) at x = 0, we have

lim

n!1

g(x) f

(x) dx = g(0):

48 Appendix

Fo r example, the sequence

(x) =

(

< x <

0 otherwise

;

is a δ-sequence. If g(x) is a ny bounded continuous f unction at x = 0, then

min



;



g(x) ≤

g(x)

dx ≤ max



;



g(x);

and thus

lim

n!1

g(x)

dx = g(0):

7.3 Diﬀerentiatio n and integrati on of function sequences

Let (f

) be a function sequence in (a; b) or [a; b]. If f

! f pointw ise or uniformly, can we

claim

(x)dx !

f(x)dx

when n !1? Let us see an important example:

(x) =

x x 2





(2 ¡2

x) x 2



;

n¡ 1



0 otherwise

The graph for some n are shown in Fig.9. Each function f

is a triangle with the base



n¡ 1



and the height 2

such that the area under each triangle is equal 1 independent of n.

Figure 9.

Therefore, we have for all n:

(x)dx = 1:

7 Function Series 49

On the other hand, it is seen f

converges to f ≡0 on [0; 1] (why?) and thus

(x)dx 9

f(x)dx:

In other word,

lim

n!1

(x) dx =/

lim

n!1

(x)

}

f(x)

dx:

Problem 92. Assume that (f

), f

: I !R is a sequence of continuous functions converging uniformly

to f . Show

lim

n!1

(x) dx =

f(x) dx:

The following theorem gives a suﬃcient condition for passing the limit inside an integral.

Theorem 13. (dominant convergence) Assume that (f

) is a sequence of continuous

functions converging pointwise to a function f in (a; b)(the interval may be ﬁnite or inﬁnite).

If there is a function g(x) such that

(x)j≤g(x); 8x 2(a; b);

and

g(x) dx < 1;

then

lim

n!1

(x) dx =

f(x) dx:

We also n eed the following form of the above theorem.

Corollary 2. Assume that f (t; x) is a continuous function deﬁned on (c; d) × (a; b), and

furthermore

f(t; x) ≤ g(x); 8t 2(c; d)

such that

g(x) dx < 1

Then function

F (t) =

f(t; x) dx;

is continuous in (c; d).

Problem 93. Prove the corollary by the aid of Theorem (13).

The same story of diﬀerentiation is the same. If a sequence of diﬀerentiable functions

) converges pointwise or even uniformly to f , then there is no guarantee that f

. As

a simple example, sequence f

(x) =

sin(nx) deﬁned on (0; π) converges uniformly to the

constant function f (x) = 0. However, f

(x) = cos(nx) which is not a convergent sequence.

50 Appendix