Daily Schedule and Homework Assignments for Math 2250-1, Spring 1999 (43 lectures)

MWF 9:00-9:50am, Shagi-Di Shih, Ross Hall 215, 766-4361

updated periodically

Textbooks:

Introduction to Linear Algebra, by Gilbert Strang, 2nd edition, Wellesley-Cambridge Press, 1998.

Student Edition of Matlab Version 5 User's Guide, 1997

7 lectures for Solving Linear Equations (Chapters 2).
6 lectures for Vector Spaces and Subspaces (Chapter 3).
4 lectures for Orthogonality (Chapter 4).
3 lectures for Determinants (Chapter 5).
7 lectures for Eigenvalues and Eigenvectors (Chapter 6).
6 lectures for exams.
? lectures for other selected sections.
? lectures for Matlab held at EN 1039.

Week 1

1. Jan 11, Introduction.
• Visit author's class homepage at to see many useful information including exams.
• Home page of Matlab maker. Here is a good Matlab tutorial.
• As a byproduct of taking my class, you may be interested in taking exam 100 offered by Society of Actuaries in May or November 1999.
• The key problems in this course are to
  • solve the linear system (in matrix notation) Ax = b, i.e., to find an n by 1 vector x in Ax = b for a given n by n matrix A and given n by 1 vector b.
  • solve the eigenproblem A x = \lambda x, i.e., to find a number (real or complex) \lambda and a nonzero n by 1 vector x in A x = \lambda x for a given n by n matrix A. Here \lambda is known as eigenvalue of the matrix A along with its eigenvector x. Thus we call (\lambda, x) as an eigenpair. This can be considered as a special case of the preceding problem: (A - \lambda I) x = 0, where I is the n by n identity matrix. Moreover, we'll study sisngular value decomposition, which is very important in applications. One example is to see the movie: No Way Out.
• In addition to learning mathematical theory, it is important to use computer as a tool when the dimension of A becomes large. One possible software is Matlab or Maple.
• There are three cases in solving the single equation with one unknown ax = b for given numbers a and b:
  • If a is not zero, then only one solution is x = b/a.
  • If a is zero and b is zero, then there are infinitely many solutions: x can be any number.
  • If a is zero and b is not zero, then there is no solution.
• The same situation can be held for solving n equations with n unknowns which will be the main problems for Chapters 2 and 3.
• For the linear system x - 2y = 1, 3x + 2y = 11,
  • each equation represents a line in the xy-plane; and the number of solutions of this system depends on their relation: they intesect each other at one point in xy-plane and thus there is only one solution.
  • it can be expressed as (in the matrix notation) A z = b, where A = [1 -2; 3, 2], b = [1; 11], and z = [x; y], where we follow matrix and vector notations used in Matlab.
• Read Sections 2.1 and 2.2.
2. Jan 13, Solving linear system Ax = b: forward elimination and back substitution.
• A is a 2 by 2 matrix:
  • The system x - 2y = 1, 3x + 2y = 11 has one solution x = 3, y = 1.
  • The system x - 2y = 1, 3x - 6y = 11 has no solution.
  • The system x - 2y = 1, 3x - 6y = 3 has infinitely many solutions x = 2t + 1, y = t for any number t.
• Illustration of forward elimination for Ax = b where A is a 5 by 5 matrix.
• Section 2.1 (p. 30): 9, 13. Section 2.2 (p.41): 11, 12, 13, 18, 25.
3. Jan 15, Solving linear system Ax = b for a 3 by 3 matrix A.
• The linear system 2x + 4y - 2z = 2, 4x + 9y - 3z = 8, -2x -3y + 7z = 10 can be expressed as A*x = b where A is a 3 by 3 matrix [2 4 -2;4 9 -3;-2 -3 7] and b = [2 8 10]'.
• Carrying out forward elimination for matrix [A b] and then back substition to get x = -1, y = 2, z = 2.

Week 2

4. Jan 20, Elementary matrices and A = LU
• Matrix A = [2 4 -2;4 9 -3;-2 -3 7] is given. The multiplier in the forward elimination can be applied to the identity matrix I = [1 0 0;0 1 0;0 0 1] to obtain an elementary matrix E as shown below:
  • Elementary matrix E21 = [1 0 0;-2 1 0;0 0 1] gives E21*A = A1, where A1 = [2 4 -2;0 1 1;-2 -3 7].
  • Elementary matrix E31 = [1 0 0;0 1 0;1 0 1] gives E31*A1 = A2, where A2 = [2 4 -2;0 1 1;0 1 5].
  • Elementary matrix E32 = [1 0 0;0 1 0;0 -1 1] gives E32*A2 = U, where U = [2 4 -2;0 1 1;0 0 4] is an upper triangular matrix.
  Thus we have obtained E32*E31*E21*A = U
• A = L*U, where L = inverse of E32*E31*E21 = (E32*E31*E21)^{-1} = E21^{-1}*E31^{-1}*E32^{-1}, which is a lower triangular matrix [1 0 0;2 1 0;-1 1 1].
• Section 2.3, #3, #11, #13, #17, #18, #24, p. 50.
5. Jan 22, Matrix operations.
• Quiz 1 (solve A*x = b when A is a 3x3 matrix).
• Section 2.4, #1, #3, #17, #34, #35 (p. 59).
• Matrix addition, scalar multiplication, matrix multiplication, block multiplication. Addition of two matrices are possible if they have the same dimension. Multiplication A*B is valid only if number of columns of A = number of rows of B. More precisely, if A = [a_{ij}] is in R^{m x n}, B = [b_{ij}] is in R^{n x p}, then A*B = [c_{ij}] has dimension m x p with c_{ij} = \sum_{k=1}^{n} a_{ik} b_{kj}. For example, [1 2 3] * [a b c]' = [a + 2*b + 3*c]; while [a b c]' * [1 2 3] = [a 2*a 3*a; b 2*b 3*b; c 2*c 3*c].

Week 3

6. Jan 25, More on Matrix Operations.
• A + B, 2*A, 3*A - 4*B, A*B, block multiplication, inverse matrix A^{-1} of an n by n matrix A based on Gauss Jordan method [A | I] ---> [I | A^{-1}].
7. Jan 27, Inverse matrix.
• Section 2.5, 1, 2, 4, 9, 15, 19, 21, 22, 26, 29, 31 (p. 72).
• The matrix A = [2 -1 0; -1 2 -1; 0 -1 2] has the inverse A^{-1} = [3/4 1/2 1/4; 1/2 1 1/2; 1/4 1/2 3/4] by following the procedure: [A | I] --> [-1 2 -1 0 1 0; 2 -1 0 1 0 0; 0 -1 2 0 0 1] --> [-1 2 -1 0 1 0; 0 3 -2 1 2 0; 0 -1 2 0 0 1] --> [-1 2 -1 0 1 0; 0 -1 2 0 0 1; 0 3 -2 1 2 0] --> [-1 2 -1 0 1 0; 0 -1 2 0 0 1; 0 0 4 1 2 3] --> [1 -2 1 0 -1 0; 0 1 -2 0 0 -1; 0 0 1 1/4 1/2 3/4] --> [1 -2 0 -1/4 -3/2 -3/4; 0 1 0 1/2 1 1/2; 0 0 1 1/4 1/2 3/4] --> [1 0 0 3/4 1/2 1/4; 0 1 0 1/2 1 1/2; 0 0 1 1/4 1/2 3/4]. It is VERY easy to make mistakes and thus please compute if A*A^{-1} gives I.
• A not-so-practical formula for the inverse matrix is given in page 232.
8. Jan 29, More on inverse matrix and factorization.
• The reason why Gauss-Jordan method works for finding the inverse matrix of A (if it does exist) by using [A | I] ---> [I | A^{-1}] is based on a combination of two steps
  • The definition A*A^{-1} = I (for example 3 by 3 matrix A) can be expressed as three equations: A*x_1 = [1 0 0]', A*x_2 = [0 1 0]', A*x_3 = [0 0 1]', where x_1, x_2, x_3 are three columns of A^{-1}.
  • In solving A*x = b, one can follow from [A | b] to [I | x].
• The matrix A is singular if A can not be reduced to I via forward/back eliminations.
• (A*B)^{-1} = B^{-1}*A^{-1}. (A*B*C)^{-1} = C^{-1}*B^{-1}*A^{-1}.
• Given the factorization A = L*U, then solving A*x = b becomes two steps:
  • solve L*y = b by forward substitution
  • solve U*x = y by back substitution.
• Section 2.6, 1, 2, 3, 5, 6, 7, 8, 11, 12 (p. 84).

Week 4

9. Feb 1, Transpose and Permutation.
• Quiz #2 (elementary matrices in converting a matrix to be upper triangular in Section 2.3).
• Given the factorization A = L*U, one can have A = L*D*U_1, where D is a diagonal matrix and U_1 is an upper triagular matrix with diagonal elements 1.
• P*A = L*U happens when an exchange between rows of A takes place.
• A^T = [b_{ij}] is the transpose of A = [a_{ij}] so that b_{ij} = a_{ji}.
• (A + B)^T = A^T + B^T. (A*B)^T = B^T*A^T.
• A*x combines columns of A while (A*x)^T = x^T*A^T combines rows of A^T.
• An n by n matrix A is symmetric if A^T = A.
• For a symmetric matrix A, A = L*D*L^T.
• A permutation matrix is obtained from an identity matrix by interchanging any number of rows. For example, P = [0 1 0; 1 0 0; 0 0 1]. P^{-1} = P = P^T.
• Section 2.7, HW: 1, 2, 5, 7, 11, 19, 21, 23 (p. 95).
10. Feb 3, Answered Questions.
• Returned Quiz #2 along with its discussion. Matrices E_{21} = [1 0 0; -4 1 0; 0 0 1], E_{31} =[1 0 0;0 1 0;2 0 1], E_{32} = [1 0 0; 0 1 0; 0 -2 1] converted A = [1 1 0; 4 6 1; -2 2 0] into E_{32}*E_{31}*E_{21}*A = [1 1 0; 0 2 1; 0 0 -2], an upper triangular matrix.
• P = [0 1 0; 0 0 1; 1 0 0]. Then P^{-1} = P^T.
11. Feb 5, Exam I covering Chapter 2.

Week 5

12. Feb 8, Vector Spaces and Subspaces.
• Section 3.1, HW 1-28, p. 107.
• A vector space is a set along with two operations: addition and scalar multiplication so that it is closed under these operations and 8 rules given in p. 107 are satisfied. These properties come from R^2, R^3. More abstract vector spaces are M = space of 2 by 2 matrices, F = space of real functions, Z = space of zero vector.
• A subspace S is a subset of a vector space V such that a*u + b*v is in S for u, v in S and real numbers a, b.
• Recall that A*x is a linear combination of columns of an m by n matrix A, thus A*x = b has a solution if b is in CS(A), column space of A being a subspace of R^m containing all linear combinations of columns of A. On the other hand, NS(A), called null space of, is a subspace of R^n containing x so that A*x = 0. RS(A), called row space of A, is a subspace of R^n containing all linear combinations of rows of A.
• Did Problems #1, #4.
13. Feb 10, More on spaces and subspaces.
• Visit the website Stardust equations, which mentioned a 1995 paper # 13: Targeting Chaotic Orbits to the Moon Through Recurrence by Eric Bollt and James Meiss, University of Colorado at Boulder. This paper is of PostScript form and can be downloaded and viewed with a freeware called GhostView, which is available in UNIX, PC, or Mac, and can be used for viewing Adobe pdf as well.
• Returned Exam I along with its discussion.
• Did Problem #5.
14. Feb 12, Null space of A.
• Answered question: A = L*U can be carried out with Matlab command slu; while P*A = L*U is obtained by Matlab command lu. The key idea in the latter is based on a series of equalities such as P*E = E*P_1 for an elementary matrix E, and permutation matrices P, P_1.
• Section 3.2, HW: 1-4, 9, 13-17 (p. 118).
• The equation x + 2*y + 3*z = 0 can be considered as A*u = 0 with A = [1 2 3] and u = [x y z]'. Thus the solution is u = [-2*y - 3*z y z]' = y*[-2 1 0]' + z*[-3 0 1]'. In other words,the null space of A contains all linear combinations of [-2 1 0]' and [-3 0 1]. A basis for NS(A) is {[-2 1 0]', [-3 0 1]'}. Dim NS(A) = 2.
• The null space of A = [1 2; 3 6] contains multiples of [-2 1]'. Thus a basis for NS(A) is {[-2 1]'} and dim NS(A) = 1.
• Reduced row echelon matrix is obtained via forward elimination and back elimination with pivots to be 1. Such expression is very important for finding rank of a matrix and determining linear dependent set of vectors later on. Its Matlab command is rref. More examples will be illustrated later.

Week 6

15. Feb 15, Rank, Row-reduced Form, and A*x = b.
• Quiz #3 (Section 3.1).
• Forward elimination reduces the matrix A = [1 1 2 3; 2 2 8 10; 3 3 10 13] to the upper triangular matrix U = [1 1 2 3; 0 0 4 4; 0 0 0 0], which becomes the row reduced echelon form R = [1 1 0 1; 0 0 1 1; 0 0 0 0] by using backward elimination with 1's to be pivots. Either U or R gives rank (A) = 2 since (1,1) and (2,3) elements of U or R are non-zero.
• The set {[1 2 3]', [2 8 10]', [3 10 13]'} is linearly independent (see definition in p. 142) since [1 2 3]' + [2 8 10]' =[3 10 13]' from the columns 1, 3, 4 of R.
• The solution A*x = b with [A | b] = [1 1 2 3; 2 2 8 10; 3 3 10 13] is solvable since rank (A) = rank ([A | b]). Moreover, there are infinitely many solutions since 2 = rank (A) < number of columns of A = 3. Thus there is one free variable for x. More precisely, x = [1 0 1]' + x₂ [-1 1 0]' for any real number x₂. The first part [1 0 1]' is a solution of A*x = b; while the second part x₂ [-1 1 0]' is the general solution of A*x = 0.
• Discussed Problem 1, p. 128.
• Section 3.3, HW: 3, 8, p. 128. Section 3.4, HW: 1-6, p. 136.
16. Feb 17, Linear independence, basis, dimension, and four subspaces.
• Did Problem #8, p. 129. The solution of R*x = 0 with R = [1 0 2 3; 0 1 4 5; 0 0 0 0] is x = x₃ [-2 -4 1 0]' + x₄ [-3 -5 0 1]'. Thus two special solutions are [-2 -4 1 0]' and [-3 -5 0 1]'. A basis for NS(R) is {[-2 -4 1 0]', [-3 -5 0 1]'} since these vectors are linearly independent and they spans the subspace NS(R) of R⁴. Moreover, the dimension of NS(R) is 2.
• HW: Section 3.5, 1, 2, 5, 11, 13, 21, 22, 24 (p. 150). Section 3.6, 1, 2, 3, 6, 17 (p. 161).
• For an m by n matrix A, we four subspaces: RS(A) = CS(A^T), CS(A), NS(A), NS(A^T).
• The matrix A = [2 4; 3 6] can be reduced to R = [1 2; 0 0]. Thus CS(A) has a basis {[2 3]'} while CS(R) has a basis {[1 0]'}. RS(A) has a basis {[1 2]'} while RS(R) has a basis {[1 2]'}.
• Forward/back elimination does not change row space but does change column space.
17. Feb 19, More on linear independence, basis, dimension, and four subspaces.
• The matrix A = [2 4; 3 6] can be reduced to R = [1 2; 0 0]. Thus CS(A) has a basis {[2 3]'}. RS(A) has a basis {[1 2]'}. NS(A) has a basis {[-2 1]'}. NS(A^T) has a basis {[-3/2 1]'}.
• For an m by n matrix A with rank r, we have the following:
  • RS(A) = CS(A^T) is a subspace of Rⁿ with dimension r.
  • CS(A) is a subspace of R^m with dimension r.
  • NS(A) is a subspace of Rⁿ with dimension n - r.
  • NS(A^T) is a subspace of R^m with dimension m - r.
  This is called Fundamental Theorem of Linear Algebra, Part 1.
• Fundamental Theorem of Linear Algebra, Part 2 includes
  • NS(A) is orthogonal complement of RS(A) in Rⁿ. This is illustrated by the fact that every row of A is orthogonal to every vector x in NS(A) by virtue of A*x = 0 and (n - r) + r = n because of linear independence between every row of A and every vector in NS(A).
  • NS(A^T) is orthogonal complement of CS(A) in R^m.
  • All of these properties are shown in the matrix A = [2 4; 3 6].
• Returned Quiz #3.

Week 7

18. Feb 22, More on independence.
• Discussed Quiz #3.
• Problems #1, #2 of p. 150 are the same. As an example, Problem #2 gives by virtue of the definition of linear independence, c₁ v₁ + c₂ v₂ + c₃ v₃ + c₄ v₄ + c₅ v₅ + c₆ v₆ = 0, which is written as (in matrix form) V c = 0, where V is a 4 by 6 matrix [v₁ v₂ v₃ v₄ v₅ v₆] and c is a 6 by 1 vector [c₁ c₂ c₃ c₄ c₅ c₆]'. Forward/back elimination reduces A to R = [1 0 0 -1 -1 0; 0 1 0 1 0 -1; 0 0 1 0 1 1; 0 0 0 0 0 0]. Pivot columns 1, 2, 3 of R imply that corresponding columns of A are linearly independent, i.e., vectors v₁, v₂, v₃ are linearly independent; columns 4, 5, 6 of R imply that (-1) v₁ + (1) v₂ = v₄, (-1) v₁ + (1) v₃ = v₅, (-1) v₂ + (1) v₃ = v₆. Thus an advantage of obtaining an RREF of a matrix A gives how column vectors of A are related in terms of linear dependence or independence.
19. Feb 24, Projection.
• Sectin 4.1, 5, 13, 14, 15, 18, p. 171.
• Review four subpaces of a matrix A.
• Two vectors u and v are orthogonal if u^T v = 0.
• u^T v = ||u||*||v||*\cos(\theta), where \theta is an angle between these vectors. Next, u^T v = \sum_k=1ⁿ u_k v_k. Finally, ||u||² = u^T u.
• Section 4.2, 1, 3, 11, 12, 16, p. 181.
• Projection of a vector b in the direction of a vector a was derived to be p = P*b with the projection matrix P = (a a^T)/(a^T a).
20. Feb 26, Answered a question.
• Quiz #4 (Sections 3.2, 3.3, and 3.4).
• Did Problem #3, p. 161.

Week 8: Spring Break (March 1 - 5)

Week 9

21. March 8, More on projection.
• Illustration of projection of vector b = [1 1 1]' onto the direction of a = [1 2 2]'.
• Derivation of projection of a vector b into a subspace spanned by a linearly independent set of vectors a₁, a₂.
• Computation of projection of a vector b = [6 0 0]' into a subspace spanned by vectors a₁ = [1 1 1]', a₂ = [0 1 2]'.
22. March 10, Review.
• Returned Quiz #4 and discussed it.
• Reviewed some key ideas for Exam II.
23. March 12, Exam II covering Sections 3.1 ~ 4.2.

Week 10

24. March 15, Matlab in EN 1039.
• Returned Exam II and discussed it.
• Here is the website MATLAB Teaching Codes consisting of 37 short, text files containing MATLAB commands for performing basic linear algebra computations.
• Mathlab Demos from the creator of Mathlab.
• Gauss-Jordan Demo for pages 69-70.
• Illustration of LU (see pages 77-83).
• More Demos from MIT.
25. March 17, Least sqquares approximations.
• For 3 points (0, 6), (1, 0), (2, 0) in the tb-plane,
  • there is one and only one parabola b = 6 - 9t + 3t² by solving 3 linear equations with 3 unknowns.
  • there is no line passing through these points since b is not in the column space of A, where b = C + Dt, A = [1 0; 1 1; 1 2], x = [C D]', and b = [6 0 0]'. Thus we seek a linear least squares solution x so that ||Ax - b|| as small as possible. Moreover, from geometry, such solution can be found by solving the the normal equations A^TAx = A^Tb. The error e = b - Ax is the smallest.
• Tips on curve fitting with Matlab can be found in technical support solution 10652.
26. March 19, Matlab in EN 1039.
• Section 4.3, 1-4, 17-20, p. 192.
• Illustration of linear least squares in Matlab. There are three ways for least squares problems in Matlab: (1) solving the normal equations, using (2) lsq or (3) linefit of Matlab teaching codes.

Week 11

27. March 22, Matlab in EN 1039: LU and QR.
• Illustration of LU and QR decompositions in Matlab.
• More Matlab commands: rref, lu, slu, grams, det,eig, format long, format short, and sym.
• Quiz #5: Turn in one-page essay on your experience of using Matlab. It is due March 29.
28. March 24, Gram-Schmidt orthogonalization process.
• Section 4.4: 1, 2, 5, 10, 13, 14, 15, 17, 20, 21, 22, 23.
• Convert an independent set of vectors a = [1 -1 0]', b = [2 0 -2]', c = [3 -3 3]' to an othonormal set of vectors q₁ = [1 -1 0]'/2^1/2, q₂ = [1 1 -2]'/6^1/2, q₃ = [1 1 1]'/3^1/2.
29. March 26, Consequences of Gram-Schmidt orthogonalization process.
• For an othonormal basis {q₁, q₂, q₃} in R³, any vector v in R³ is derived to be expressed as v = (v^T q₁) q₁ + (v^T q₂) q₂ + (v^T q₃) q₃.
• An independent set of vectors {v₁, v₂, v₃} can be converted to an orthonormal set of vectors {q₁, q₂, q₃}. It then follows that A = Q R where A = [v₁ v₂ v₃], Q = [q₁, q₂, q₃], and R = [v₁^T q₁ v₂^Tq₁ v₃^Tq₁; 0 v₂^Tq₂ v₃^Tq₃; 0 0 v₃^Tq₃].
• Application of A = QR, where A, Q, R has dimensions m by n, m by n, n by n, respectively, to least squares problems: The normal equations A^TAx = A^Tb become Rx = Q^Tb. Example: A = [1 0;1 1;1 2], b = [6 0 0]'.

Week 12

30. March 29, Plane rotation.
• Rotation of axes in xy-plane through an angle \theta in the counterclockwise direction is derived: x = x' cos(\theta) - y' sin(\theta), y = x' sin(\theta) + y' cos(\theta); between old coordinates (x, y) and new coordinates (x', y').
• In the notation of matrix, we then have [x; y] = Q [x'; y'] with an orthogonal matrix Q = [cos(\theta) -sin(\theta); sin(\theta) cos(\theta)].
31. March 31, Givens rotation matrix and QR factorization. Determinant.
• Quiz #6: Linear least squares.
• Section 9.1, #11, #15 (turning a robot hand), p. 403.
• Q = [3/5 4/5; -4/5 3/5]. Then Q [3; 4] = [5; 0]. Thus [3; 4] = Q^T [5; 0].
• There are three kinds of Givens rotation matrices in 3-D: Q₂₁ = [cos(\theta) -sin(\theta) 0; sin(\theta) cos(\theta) 0; 0 0 1], Q₃₁ = [cos(\theta) 0 -sin(\theta); 0 1 0;sin(\theta) 0 cos(\theta)], Q₃₂ = [1 0 0; 0 cos(\theta) -sin(\theta); 0 sin(\theta) cos(\theta)].
• Q₂₁ = [3/5 4/5 0; -4/5 3/5 0; 0 0 1]. Then Q₂₁ [3; 4; 5] = [5; 0; 5].
• Section 5.2, #1, #2, #11, p. 225.
• det[a b;c d] = ad - bc.
• det[a b; c d] = |a b; c d|.
• det[a₁₁ a₁₂ a₁₃; a₂₁ a₂₂ a₂₃; a₃₁ a₃₂ a₃₃] = a₁₁ a₂₂ a₃₃ + a₂₁ a₃₂ a₁₃ + a₃₁ a₁₂ a₂₃ - a₃₁ a₂₂ a₁₃ - a₁₁ a₃₂ a₂₃ - a₂₁ a₁₂ a₃₃.
• Cofactor's formula, p. 223, can be used for finding the determinant of a matrix of any size. It is illustrated for a 3 by 3 matrix.

Week 13

32. April 5, Eigenvalues, eigenvectors, and diagonalization.
• Section 6.1, 3, 4, 24, 27, p. 253. Section 6.2, 1-3, p. 266.
• For a given square matrix A, its eigenvalue \lambda and corresponding eigenvector y (nonzero vector) satisfy the relation Ay = \lambda y. Thus all eigenvalues of A are found by solving the algebraic equation \det(A-\lambda I) = 0; while an eigenvector of A associated with the eigenvalue \lambda is found by solving the linear system (A-\lambda I) y = 0. The German name "eigenvalue" was used in 1904 by Hilbert.
• Eigenpairs of a 2x2 matrix A = [1 2; 2 4] are found to be (0, [2; -1]), (5, [1;2]).
33. April 7, Review.
• Answered Problems #15(a), #20, p. 204-205. In addition, reviewed the underlying concept: projection of a vector onto a vector, projection of a vector onto a subspace.
• Returned essay, and Quiz #6.
34. April 9, Exam III covering 4.3, 4.4, Givens rotation matrix, 5.2, 6.1, 6.2.

Week 14

35. April 12, Application to differential equations.
• Section 6.3, 1-6, p. 279.
• The solution of y'(x) + 2 y(x) = 0 is y(x) = c exp(-2x).
• A system of first-order DE can be obtained from a single higher-order DE.
• The matrix A = [5 4; 4 5] has eigenpairs (1, [-1 1]'), (9, [1 1]'). Thus AS = SD where S = [-1 1;1 1] and D = [1 0;0 9].
• Using eigenpairs and diagonalization, one derived that the solution of x'(t) = A x(t) is x(t) = c₁ \exp(t) [-1 1]' + c₂ \exp(9t) [1 1]'.
• The success of the above result is due to the fact that
  • there are 2 linearly independent eigenvectors for the 2 by 2 matrix A.
  • all eigenvalues of A are real.
36. April 14, Symmetric matrices.
• Returned Exam III and discussed it.
• Section 6.4, 4-6. p.290.
• A matrix A is called to be symmetric if A^T = A. Some of its important properties are listed below:
  • All eigenvalues are real. The matrix [\cos(\theta) -\sin(\theta); \sin(\theta) \cos(\theta)] has complex eigenvalues \cos(\theta)+ i \sin(\theta), \cos(\theta)- i \sin(\theta).
  • It can be diagonalizable.The matrix [2 1; 0 2] has eigenvalues 2, 2 along with eigenvector [1 0]'. Thus AS = SD with S = [1 5; 0 0] and D = [2 0; 0 2] can not be converted to A = SDS^-1 since S is a singular matrix having two linearly dependent columns. The matrix A = [5 4;4 5] has eigenpairs (1, [1 -1]'), (9, [1 1]'). Thus AS = SD with S = [1 1;-1 1] and D = [1 0;0 9] becomes A = SDS^-1 or S^-1AS = D because S is a non-singular matrix with two linearly independent columns.
  • Eigenvectors can be chosen to be orthogonal. Eigenvectors associated with different eigenvalues are orthogonal. The matrix A = [5 4;4 5] has eigenpairs (1, [1 -1]'), (9, [1 1]'), where two eigenvectors are orthogonal. Note that eigenvectors associated with the same eigenvalues can be chosen to be orthogonal via Gram-Schmidt process.
  • AQ = QD with orthogonal matrix Q and diagonal matrix D. (or, it is orthogonally similar to a diagonal matrix in terms of Section 6.6). Choose q₁ = [1 -1]'/2^1/2 and q₂ = [1 1]'/2^1/2 to form an orthogonal matrix Q = [q₁ q₂]. Then AQ = QD with A = [5 4; 4 5] and D = [1 0; 0 9]. Note that Q^TQ = I. Thus A = QDQ^T or Q^TAQ = D
  • Spectral decomposition: From A = QDQ^T, it follows that A = \lambda₁q₁q₁^T + \lambda₂q₂q₂^T via block matrix multiplication.
37. April 16, Positive definite matrices.
• Quiz #7: find eigenpairs of a 2x2 matrix.
• Teaching evaluation conducted by Junping Wang.
• A symmetric matrix A is said to be positive definite if x^T A x > 0 for all non-zero vectors x. Thus we have the following properties:
  • All eigenvalues of a positive definite matrix are positive. For an eigenpair (\lambda, x) of A, it follows that 0 < x^T A x = \lambda x^T x = \lambda ||x||².
  • For a 2 by 2 matrix A = [a b; b c] and x = [x y]', we have x^T A x = a x² + 2bxy + cy². Thus a x² + 2bxy + cy² is positive unless x = 0 and y = 0. Next, a > 0 by selecting x = [1 0]' and c > 0 by picking x = [0 1]'. Moreover, completing the square gives x^T A x = a (x + by/a)² + ((ac - b²)/a)y². Thus (ac - b²)/a > 0 or ac - b² > 0.
• Illustration of A = [1 2;2 7] for pivots, LU as well as Cholesky factorizations.
• Section 6.5, HW: 1, 2, 3, 6, p. 302.

Week 15

38. April 19, Similar matrices, SVD.
Section 6.6, HW: 1, 2, 5, p. 310. Section 6.7, HW: 1, 2, 4, 7, p. 318.
• Two matrices A, B are said to be similar if there exists an invertible matrix M so that B = M^-1AM. If A has an eigenpair (\lambda, x), then B has an eigenpair (\lambda, M^-1x). To see it, Ax = \lambda x gives B(M^-1 x) = \lambda(M^-1 x).
• Singular value decomposition of a m by n matrix A having rank (A) = r: A = U \Sigma V^T, where U is an m by m orthogonal matrix, V is an n by n orthogonal matrix, and \Sigma is an m by n matrix containing r non-zero singular values of A in its diagonal together with zeros for the remaining positions.
  • A^TA is an n by n matrix, which is symmetric, and positive semi-definite.
  • Find eigenpairs of A^TA. These n eigenvectors, which are orthogonal, can be chosen to be orthnomornal. They are v₁, v₂,....,v_n with the corrsponding eigenvalues in the decreasing order: \lambda₁ \geq \lambda₂ \geq ...\lambda_r \geq \lambda_r+1 = ... =\lambda_n = 0. These eigenvectors form columns of the matrix V. The square roots of these eigenvalues of A^TA are called singular values \sigma_k of A, which are diagonal elements of \Sigma.
  • Compute u_k = (A v_k)/\sigma_k for k = 1, 2,..., r. If r is less than m, then u_r+1,..., u_m are orthonormal basis (via Gram-Schmidt process) of the null subspace of [u₁, ..., u_r]'. Thus we have an othogonal matrix U with u_k as its columns.
• Examples (in Matlab file):
  • 1 (2 by 3 matrix having rank 2),
  • 2 (3 by 2 matrix having rank 1),
  • 3 (2 by 2 matrix having rank 2),
  • 4 (2 by 2 matrix having rank 1).
39. April 21, More on SVD
• Discussed Example 2.
40. April 23,
• Quiz #8 (expand (det(A - \lambda*I) for a 3 by 3 matrix A).
• Finding eigenvalues and eigenvectors of the matrix A = [0 1 1;1 0 1;1 1 0].
• Some theoretical issues of SVD of a m by n matrix A. From A = U \Sigma V^T, we have A^T A = V \Sigma^T \Sigma V^T, because of U^T U = I. Thus columns of V form an orthonormal set of eigenvectors of a symmetric matrix A^T A, whose eigenvalues are non-negative by virtue of being positive semi-definiteness. Note that \Sigma^T \Sigma corresponds to a diagonal matrix having eigenvalues of A^T A as its diagonal. Thus singular values of A are defined to be square root of eigenvalues of A^T A. Moreover, A V = U \Sigma gives one way of finding columns of U (another way is to think about A A^T = U \Sigma \Sigma^T U^T so that columns of U form an orthonormal set of eigenvectos of A A^T).

Week 16

41. April 26, Matlab in EN 1039.
• James Joseph Sylvester (1814-1897) was given the credit for being the first to study SVD in 1889.
• Illustration of using Matlab for singular value decomposition of matrices with (or without) full rank.
42. April 28,
• Quiz # 9 (find eigenvalues of a 3 by 3 matrix).
• Returned Quiz # 8.
43. April 30, Matlab in EN 1039.
• All make-up exams at 1pm.

May 3, 3:30pm-5:30pm, CR206, Final exam