Note | Linear Algebra
date
Oct 21, 2021
slug
linear-algebra
status
Published
summary
Linear algebra is also a system has input and output (function, transformation, operator). But it is amazing.
tags
Academic
Math
Study
type
Post
Linear System
Matrix-Vector Product
Two perspective:
- Inner product with row
- Weighted sum of columns
- The matrix A represents the system:
- the size of matrix vector should be matched
Properties of Matrix-vector Product
and are matrices and are vectors in , and is a scalar
- is the zero vector
- is also the zero vector
A=B?
and are matrices. If for all in . Is it true that ?
Has Solution or Not?
Only unique and infinite?
Linear Combination
- Definition: Given a vector set
- are scalars (coefficients of linear combination)
- or "weighted sum" 😊
the linear combination of the vectors in the set
- Parallel no solution
- Nonparallel and are nonzero vectors, and
- If and are in :
- Nonparallel has solution
- Nonparallel has solution
- If and are in :
- Nonparallel has solution
Span
- Definition:
- Given a vector set
- Span S is the vector set of all linear combinations of
- Span
- Vector set
- " is a generating set for " or " generates "
- One way to describe a vector set with infinite elements
- has solution or not?
Is the linear combination of the columns of ?
is in the span of the columns of ?
- Redundant vector (in my way, not the teacher's 😊)
- Given a vector set and
How many solutions?
Dependent and independent
- Dependent: A set of n vectors , is linear dependent
If there exist scalars , not all zero, such that
- Independent: A set of n vectors , is linear independent
Only if
- Zero vector is the linear combination of any other vectors
- Any set contains zero vector would be linear dependent
- If dependent, once we have solution, we have infinite
- Homogeneous linear equations: always has as solution
Rank and Nullity
- The rank of a matrix is defined as the maximum number of linearly independent columns in the matrix
- Nullity = Numbers of columns - rank
- If is a matrix, columns of are independent
❓ Conclusion
Solving System of Linear Equations
Equivalent
- Equivalent: two systems of linear equations are equivalent if they have exactly the same solution set
- Elementary row operations: applying those operations will produce an equivalent one
- interchange
- scaling
- row addition
Augmented Matrix
- Augmented matrix:
- Reduced Row Echelon Form (): a system of linear equations is easily solvable if its augmented matrix is in RREF
- Row Echelon Form:
- Each nonzero row lied above every zero row
- The leading entries are in echelon form
- Reduced: the columns containing the leading entries are standard vectors
- RREF is unique: a matrix can be transformed into multiple REF by row operation, but only on R
- Pivot positions of are (1,1, (2,3) and (3,4) and the pivot columns of are 1st, 3rd and 4th columns
- Free variables and basic variables / Parametric Representation
- Gaussian elimination: an algorithm for finding the reduced row echelon form of a matrix.
What can we know from RREF?
RREF & Linear Combination
- Column Correspondence Theorem:
- For
- if is a linear combination of other columns of is a linear combination of the corresponding columns of with the same coefficients
- Or: The RREF of matrix is , and have the same solution set
- There are no row correspondence theorem
- Span of columns will change but span of rows are equivalent\
- Conclusion
- Columns
- the relations between the columns are the same
- the span of the columns are different
- Rows:
- the relations between the rows are different
- the span of the rows are the same
RREF & Independent
- Pivot column independent
- All columns are independent every column is a pivot column Every column in RREF(a) is standard vector, therefore:
- columns are dependent
RREF & Rank
- Rank? Maximum number of independent columns = number of pivot column = number non-zero rows
- Matrix is full rank if
- If , the columns of is dependent
- In , you cannot find more than m vectors that are independent
- Basic, Free Variables & Ranks
- The number of basic variables =
- The number of basic variables = =
RREF & Span
- is inconsistent (no solution)
- is consistent for every
- independent vectors can span More than vectors in must be dependent
RREF of cannot have a row whose only non-zero entry is at the last column
RREF of cannot have zero row
the number of rows
- Full Rank
- the columns of are linearly independent
- has at most one solution
- All columns are pivot columns
- always have solution (at least one solution) for every row in
- Independent (square matrix): one solution
- Dependent: infinite solutions
- The columns of generate
RREF of
Matrix Multiplication
Inner product
- Matrix multiplication: given two matrices and , the-entry of is the inner product of row i of and column j of
- Linear combination of columns and rows:
- Column
- Composition: given two functions f and g, the function is the composition
- Matrix multiplication is the composition of two linear functions
- Row
Summation of Matrices
- Block multiplication
Properties
- and are square and symmetric: ;
Practical Issue
Order affects computation complexity!
Matrix Inverse
Inverse of Matrix
- is called invertible if there is a matrix
- Non-singular: invertible
- Singular: not invertible
- Non-square matrix cannot be invertible
- Not all the square matrix is invertible
- Unique
- A and B are invertible nxn matrices,
- If A is invertible, is is invertible
- Solving Linear Equations:
However, this method is computationally inefficient
Invertible
- Let ben an matrix, is invertible iff.:
- Onto ⇒ One-to-one ⇒ invertible
- The columns of span
- For every in , the system is consistent
- The rank of is n (the number of rows) = n
- One-to-one ⇒ Onto ⇒ invertible
- The columns of are linear independent
- The rank of is n (the number of columns) = n
- The nullity of is zero
- The only solution to is the zero vector
Simplest
: The reduced row echelon form of is
- Others:
- There exists an matrix s.t.
- There exists an matrix s.t.
- is a product of elementary matrices
Inverse of a Matrix
- Every Elementary Row Operation can be performed by matrix multiplication
- Elementary matrix
- How to find an elementary matrix: apply the desired elementary row operation on identity matrix
- Inverse of elementary matrix
- RREF v.s. Elementary Matrix
- Let be an matrix with reduced row echelon form
- There exists an invertible matrix s.t.
Inverse of a general matrix
- matrix:
(if , is not invertible)
- Algorithm for Matrix Inversion
Subspace v.s. Span
- The span of a vector set is a subspace
- Let
- Property 1.
- Property 2.
- Property 3.
- Null Space
- Definition: the null space of a matrix is the solution set of . It is denoted as Null A
- Null A is a subspace
- Column Space and Row Space
- Column space of a matrix is the span of its columns . It is denoted as Col A.
- Row space of a matrix is the span of its rows. It is denoted as Row A.
- Columns Space = Range: The range of a linear transformation is the same as the column space of its matrix.
- RREF
- Original Matrix A v.s. its RREF R
- Columns:
- The relations between the columns are the same.
- The span of the columns are different.
- Rows:
- The relations between the rows are changed
- The span of the rows are the same.
- Consistent
- has solution (consistent)
- is the linear combinations of columns of
- is in the span of the columns of
- is in Col A
- Conclusion: Subspace is Closed under addition and multiplication
Basis
- Definition: let be a non zero subspace of . A basis for is a linearly independent generation set of
- any two independent vectors form a basis for
- The pivot columns of a matrix form a basis for its columns space
- Property
- Theorem
- A basis is the smallest generation set
- Reduction Theorem: There is a basis containing in any generation set / can be reduced to a basis for by moving some vectors
- A vector set generates must contain at least vectors
- A basis is the largest independent vector set in the subspace
- Given an independent vector set in the subspace, can be extended to a basis by adding more vectors
- Any independent vector set in contain at most vectors
- Any two bases for a subspace contain the same numbers of vectors.
- The number of vectors in a basis for a nonzero subspace is called dimension of ()
- The dimension of zero subspace is 0
- Every has dimensions
- Find a basis for : given a subspace , assume that we already know that . Suppose is a subset of with vectors.
- If is independent is basis
- If is a generation set is basis
Column Space, Null Space, Row Space
- Rank A
- Basis: the pivot columns of form a basis for
- Dimension:
- Null A
- Solving
- s
- s
- Dimension
- Row A
- Basis: nonzero rows of RREF(A)
- Dimension
- Dimension Theorem:
- Summary
Coordinate System
Outline
- Coordinate System
- Each coordinate system is a"viewpoint "for vector representation
- The same vector is represented differently in different coordinate systems
- Different vectors can have the same representation in different coordinate systems
- Changing Coordinates
- A vector set can be considered as a coordinate system for if :
- The vector set spans the the - every vector should have representation
- The vector set is independent - unique representation
- is Cartesian coordinate system
- Other System → Cartesian:
- Cartesian → Other System:
- Changing Coordinates
- Equation of ellipse
- Equation of hyperbola