Note | Linear Algebra

Linear System

Matrix-Vector Product

Two perspective:

• Inner product with row

• Weighted sum of columns

• The matrix A represents the system:

• the size of matrix vector should be matched

Properties of Matrix-vector Product

• is the zero vector

• is also the zero vector

A=B?

Has Solution or Not?

Only unique and infinite?

Linear Combination

1. Definition: Given a vector set
the linear combination of the vectors in the set
• are scalars (coefficients of linear combination)
• or "weighted sum" 😊

2. Parallel no solution

3. Nonparallel and are nonzero vectors, and
• If and are in :
• Nonparallel has solution
• Nonparallel has solution
• If and are in :
• Nonparallel has solution

Span

1. Definition:
• Given a vector set
• Span S is the vector set of all linear combinations of
• Span
• Vector set
• " is a generating set for " or " generates "
• One way to describe a vector set with infinite elements

2. has solution or not？
Is the linear combination of the columns of ？
is in the span of the columns of ？

3. Redundant vector (in my way, not the teacher's 😊)
• Given a vector set and

How many solutions?

Dependent and independent

1. Dependent: A set of n vectors , is linear dependent
If there exist scalars , not all zero, such that

2. Independent: A set of n vectors , is linear independent
Only if

3. Zero vector is the linear combination of any other vectors

4. Any set contains zero vector would be linear dependent

5. If dependent, once we have solution, we have infinite

6. Homogeneous linear equations: always has as solution

Rank and Nullity

1. The rank of a matrix is defined as the maximum number of linearly independent columns in the matrix

2. Nullity = Numbers of columns - rank
• If is a matrix, columns of are independent

❓ Conclusion

Solving System of Linear Equations

Equivalent

1. Equivalent: two systems of linear equations are equivalent if they have exactly the same solution set

2. Elementary row operations: applying those operations will produce an equivalent one
• interchange
• scaling
• row addition

Augmented Matrix

1. Augmented matrix:

2. Reduced Row Echelon Form (): a system of linear equations is easily solvable if its augmented matrix is in RREF
• Row Echelon Form:
• Each nonzero row lied above every zero row
• The leading entries are in echelon form
• Reduced: the columns containing the leading entries are standard vectors

3. RREF is unique: a matrix can be transformed into multiple REF by row operation, but only on R

4. Pivot positions of are (1,1, (2,3) and (3,4) and the pivot columns of are 1st, 3rd and 4th columns

5. Free variables and basic variables / Parametric Representation

6. Gaussian elimination: an algorithm for finding the reduced row echelon form of a matrix.

What can we know from RREF?

RREF & Linear Combination

1. Column Correspondence Theorem:
• For
• if is a linear combination of other columns of is a linear combination of the corresponding columns of with the same coefficients
• Or: The RREF of matrix is , and have the same solution set
• There are no row correspondence theorem
 

2. Span of columns will change but span of rows are equivalent\

3. Conclusion
• Columns
• the relations between the columns are the same
• the span of the columns are different
• Rows:
• the relations between the rows are different
• the span of the rows are the same

RREF & Independent

1. Pivot column independent

2. All columns are independent every column is a pivot column Every column in RREF(a) is standard vector, therefore:
• columns are dependent

RREF & Rank

1. Rank? Maximum number of independent columns = number of pivot column = number non-zero rows

• Matrix is full rank if
• If , the columns of is dependent
• In , you cannot find more than m vectors that are independent

3. Basic, Free Variables & Ranks
• The number of basic variables =
• The number of basic variables = =

RREF & Span

1. is inconsistent (no solution)

2. is consistent for every
RREF of cannot have a row whose only non-zero entry is at the last column
RREF of cannot have zero row
the number of rows
• independent vectors can span More than vectors in must be dependent

3. Full Rank
• the columns of are linearly independent
• has at most one solution
• All columns are pivot columns
RREF of
• always have solution (at least one solution) for every row in
• Independent (square matrix): one solution
• Dependent: infinite solutions
• The columns of generate
 

Matrix Multiplication

Inner product

1. Matrix multiplication: given two matrices and , the-entry of is the inner product of row i of and column j of

2. Linear combination of columns and rows:
• Column
• Composition: given two functions f and g, the function is the composition
• Matrix multiplication is the composition of two linear functions
• Row

Summation of Matrices

1. Block multiplication

Properties

• and are square and symmetric: ;

Practical Issue

Matrix Inverse

Inverse of Matrix

1. is called invertible if there is a matrix
• Non-singular: invertible
• Singular: not invertible
• Non-square matrix cannot be invertible
• Not all the square matrix is invertible
• Unique
• A and B are invertible nxn matrices,
• If A is invertible, is is invertible

2. Solving Linear Equations:
However, this method is computationally inefficient

Invertible

1. Let ben an matrix, is invertible iff.:

• Onto ⇒ One-to-one ⇒ invertible
• The columns of span
• For every in , the system is consistent
• The rank of is n (the number of rows) = n

• One-to-one ⇒ Onto ⇒ invertible
• The columns of are linear independent
• The rank of is n (the number of columns) = n
• The nullity of is zero
• The only solution to is the zero vector
• Simplest: The reduced row echelon form of is

• Others:
• There exists an matrix s.t.
• There exists an matrix s.t.
• is a product of elementary matrices

Inverse of a Matrix

1. Every Elementary Row Operation can be performed by matrix multiplication

2. Elementary matrix

3. How to find an elementary matrix: apply the desired elementary row operation on identity matrix

4. Inverse of elementary matrix

5. RREF v.s. Elementary Matrix
• Let be an matrix with reduced row echelon form
• There exists an invertible matrix s.t.

Inverse of a general matrix

1. matrix:
(if , is not invertible)

2. Algorithm for Matrix Inversion

Subspace v.s. Span

1. The span of a vector set is a subspace
• Let
• Property 1.
• Property 2.
• Property 3.

2. Null Space
• Definition: the null space of a matrix is the solution set of . It is denoted as Null A
• Null A is a subspace

3. Column Space and Row Space
• Column space of a matrix is the span of its columns . It is denoted as Col A.
• Row space of a matrix is the span of its rows. It is denoted as Row A.
• Columns Space = Range: The range of a linear transformation is the same as the column space of its matrix.

4. RREF
• Original Matrix A v.s. its RREF R
• Columns:
• The relations between the columns are the same.
• The span of the columns are different.
• Rows:
• The relations between the rows are changed
• The span of the rows are the same.

5. Consistent
• has solution (consistent)
• is the linear combinations of columns of
• is in the span of the columns of
• is in Col A

6. Conclusion: Subspace is Closed under addition and multiplication

Basis

1. Definition: let be a non zero subspace of . A basis for is a linearly independent generation set of
• any two independent vectors form a basis for

2. The pivot columns of a matrix form a basis for its columns space

3. Property

4. Theorem
• A basis is the smallest generation set
• Reduction Theorem: There is a basis containing in any generation set / can be reduced to a basis for by moving some vectors
• A vector set generates must contain at least vectors
• A basis is the largest independent vector set in the subspace
• Given an independent vector set in the subspace, can be extended to a basis by adding more vectors
• Any independent vector set in contain at most vectors
• Any two bases for a subspace contain the same numbers of vectors.
• The number of vectors in a basis for a nonzero subspace is called dimension of ()
• The dimension of zero subspace is 0
• Every has dimensions

5. Find a basis for : given a subspace , assume that we already know that . Suppose is a subset of with vectors.
• If is independent is basis
• If is a generation set is basis

Column Space, Null Space, Row Space

1. Rank A
• Basis: the pivot columns of form a basis for
• Dimension:

2. Null A
• Solving
• s
• s
• Dimension

3. Row A
• Basis: nonzero rows of RREF(A)
• Dimension

5. Dimension Theorem:

6. Summary

Coordinate System

Outline

1. Coordinate System
• Each coordinate system is a"viewpoint "for vector representation
• The same vector is represented differently in different coordinate systems
• Different vectors can have the same representation in different coordinate systems
• Changing Coordinates
• A vector set can be considered as a coordinate system for if :
• The vector set spans the the - every vector should have representation
• The vector set is independent - unique representation
• is Cartesian coordinate system

• Other System → Cartesian:
• Cartesian → Other System:

2. Changing Coordinates
• Equation of ellipse
• Equation of hyperbola