Misplaced Pages

Revision as of 16:00, 8 April 2013

In the mathematical discipline of linear algebra, a matrix decomposition is a factorization of a matrix into a product of matrices. There are many different matrix decompositions; each finds use among a particular class of problems.

Example

In numerical analysis, different decompositions are used to implement efficient matrix algorithms.

For instance, when solving a system of linear equations ${\displaystyle Ax=b}$, the matrix A can be decomposed via the LU decomposition. The LU decomposition factorizes a matrix into a lower triangular matrix L and an upper triangular matrix U. The systems ${\displaystyle L(Ux)=b}$ and ${\displaystyle Ux=L^{-1}b}$ require fewer additions and multiplications to solve, compared with the original system ${\displaystyle Ax=b}$, though one might require significantly more digits in inexact arithmetic such as floating point.

Similarly, the QR decomposition expresses A as QR with Q a unitary matrix and R an upper triangular matrix. The system Q(Rx) = b is solved by Rx = Qb = c, and the system Rx = c is solved by 'back substitution'. The number of additions and multiplications required is about twice that of using the LU solver, but no more digits are required in inexact arithmetic because the QR decomposition is numerically stable.

Decompositions related to solving systems of linear equations

LU decomposition

• Applicable to: square matrix A
• Decomposition: ${\displaystyle A=LU}$, where L is lower triangular and U is upper triangular
• Related: the LDU decomposition is ${\displaystyle A=LDU}$, where L is lower triangular with ones on the diagonal, U is upper triangular with ones on the diagonal, and D is a diagonal matrix.
• Related: the LUP decomposition is ${\displaystyle A=LUP}$, where L is lower triangular, U is upper triangular, and P is a permutation matrix.
• Existence: An LUP decomposition exists for any square matrix A. When P is an identity matrix, the LUP decomposition reduces to the LU decomposition. If the LU decomposition exists, the LDU decomposition does too.
• Comments: The LUP and LU decompositions are useful in solving an n-by-n system of linear equations ${\displaystyle Ax=b}$. These decompositions summarize the process of Gaussian elimination in matrix form. Matrix P represents any row interchanges carried out in the process of Gaussian elimination. If Gaussian elimination produces the row echelon form without requiring any row interchanges, then P=I, so an LU decomposition exists.

LU reduction

Block LU decomposition

Rank factorization

Cholesky decomposition

• Applicable to: square, symmetric, positive definite matrix A
• Decomposition: ${\displaystyle A=U^{T}U}$, where U is upper triangular with positive diagonal entries
• Comment: the Cholesky decomposition is a special case of the symmetric LU decomposition, with ${\displaystyle L=U^{T}}$.
• Comment: the Cholesky decomposition is unique
• Comment: the Cholesky decomposition is also applicable for complex hermitian positive definite matrices
• Comment: An alternative is the LDL decomposition which can avoid extracting square roots.

QR decomposition

• Applicable to: m-by-n matrix A
• Decomposition: ${\displaystyle A=QR}$ where Q is an orthogonal matrix of size m-by-m, and R is an upper triangular matrix of size m-by-n
• Comment: The QR decomposition provides an alternative way of solving the system of equations ${\displaystyle Ax=b}$ without inverting the matrix A. The fact that Q is orthogonal means that ${\displaystyle Q^{T}Q=I}$, so that ${\displaystyle Ax=b}$ is equivalent to ${\displaystyle Rx=Q^{T}b}$, which is easier to solve since R is triangular.

RRQR factorization

Singular value decomposition

• Applicable to: m-by-n matrix A.
• Decomposition: ${\displaystyle A=UDV^{H}}$, where D is a nonnegative diagonal matrix, and U and V are unitary matrices, and ${\displaystyle V^{H}}$ denotes the conjugate transpose of V (or simply the transpose, if V contains real numbers only).
• Comment: The diagonal elements of D are called the singular values of A.
• Comment: like the eigendecomposition below, the singular value decomposition involves finding basis directions along which matrix multiplication is equivalent to scalar multiplication, but it has greater generality since the matrix under consideration need not be square.

Decomposition into clans

Very simple concept of decomposition was introduced in for rings with sign. At first, relation of equations nearness was introduced: two equations are near if they contain a variable with the same sign. Clan relation was introduced as a transitive enclosure of the nearness relation. It decomposes a matrix in block diagonal form with the first block column of a special form that describes relations of clans in very simple form. It was proven that to solve a system it is required to solve it for each clan and than to solve a single composition system. The described method finds applications at solving sparse Diophantine systems especially in monoid of nonnegative integers when finding invariants of a Petri net.

Decompositions based on eigenvalues and related concepts

Eigendecomposition

• Also called spectral decomposition
• Applicable to: square matrix A with distinct eigenvalues.
• Decomposition: ${\displaystyle A=VDV^{-1}}$, where D is a diagonal matrix formed from the eigenvalues of A, and the columns of V are the corresponding eigenvectors of A.
• Existence: An n-by-n matrix A always has n eigenvalues, which can be ordered (in more than one way) to form an n-by-n diagonal matrix D and a corresponding matrix of nonzero columns V that satisfies the eigenvalue equation ${\displaystyle AV=VD}$. If the n eigenvalues are distinct (that is, none is equal to any of the others), then V is invertible, implying the decomposition ${\displaystyle A=VDV^{-1}}$.
• Comment: The condition of having n distinct eigenvalues is sufficient but not necessary. The necessary and sufficient condition is for each eigenvalue to have geometric multiplicity equal to its algebraic multiplicity.
• Comment: The eigendecomposition is useful for understanding the solution of a system of linear ordinary differential equations or linear difference equations. For example, the difference equation ${\displaystyle x_{t+1}=Ax_{t}}$ starting from the initial condition ${\displaystyle x_{0}=c}$ is solved by ${\displaystyle x_{t}=A^{t}c}$, which is equivalent to ${\displaystyle x_{t}=VD^{t}V^{-1}c}$, where V and D are the matrices formed from the eigenvectors and eigenvalues of A. Since D is diagonal, raising it to power ${\displaystyle D^{t}}$, just involves raising each element on the diagonal to the power t. This is much easier to do and to understand than raising A to power t, since A is usually not diagonal.

Jordan decomposition

The Jordan normal form and the Jordan–Chevalley decomposition

• Applicable to: square matrix A
• Comment: the Jordan normal form generalizes the eigendecomposition to cases where there are repeated eigenvalues and cannot be diagonalized, the Jordan–Chevalley decomposition does this without choosing a basis.

Schur decomposition

• Applicable to: square matrix A
• Comment: there are two versions of this decomposition: the complex Schur decomposition and the real Schur decomposition. A complex matrix always has a complex Schur decomposition. A real matrix admits a real Schur decomposition if and only if all of its eigenvalues are real.
• Decomposition (complex version): ${\displaystyle A=UTU^{H}}$, where U is a unitary matrix, ${\displaystyle U^{H}}$ is the conjugate transpose of U, and T is an upper triangular matrix called the complex Schur form which has the eigenvalues of A along its diagonal.
• Decomposition (real version): ${\displaystyle A=VSV^{T}}$, where A, V, S and ${\displaystyle V^{T}}$ are matrices that contain real numbers only. In this case, V is an orthogonal matrix, ${\displaystyle V^{T}}$ is the transpose of V, and S is a block upper triangular matrix called the real Schur form. The blocks on the diagonal of S are of size 1×1 (in which case they represent real eigenvalues) or 2×2 (in which case they are derived from complex conjugate eigenvalue pairs).

QZ decomposition

• Also called: generalized Schur decomposition
• Applicable to: square matrices A and B
• Comment: there are two versions of this decomposition: complex and real.
• Decomposition (complex version): ${\displaystyle A=QSZ^{H}}$ and ${\displaystyle B=QTZ^{H}}$ where Q and Z are unitary matrices, the H superscript represents conjugate transpose, and S and T are upper triangular matrices.
• Comment: in the complex QZ decomposition, the ratios of the diagonal elements of S to the corresponding diagonal elements of T, ${\displaystyle \lambda _{i}=S_{ii}/T_{ii}}$, are the generalized eigenvalues that solve the generalized eigenvalue problem ${\displaystyle Av=\lambda Bv}$ (where ${\displaystyle \lambda }$ is an unknown scalar and v is an unknown nonzero vector).
• Decomposition (real version): ${\displaystyle A=QSZ^{T}}$ and ${\displaystyle B=QTZ^{T}}$ where A, B, Q, Z, S, and T are matrices containing real numbers only. In this case Q and Z are orthogonal matrices, the T superscript represents transposition, and S and T are block upper triangular matrices. The blocks on the diagonal of S and T are of size 1×1 or 2×2.

Takagi's factorization

• Applicable to: square, complex, symmetric matrix A.
• Decomposition: ${\displaystyle A=VDV^{T}}$, where D is a real nonnegative diagonal matrix, and V is unitary. ${\displaystyle V^{T}}$ denotes the matrix transpose of V.
• Comment: the diagonal elements of D are the nonnegative square roots of the eigenvalues of ${\displaystyle AA^{H}}$.
• Comment: V may be complex even if A is real.

Other decompositions

• Polar decomposition
• Proper orthogonal decomposition

See also

• Matrix splitting

References

1. C. Simon and L. Blume (1994). "Chapter 7". Mathematics for Economists. Norton. ISBN 0-393-95733-0.
2. Zaitsev D.A. Solving Linear Systems via Composition of their Clans, Intelligent Information Management, 2009, 1, 73-80.
3. Zaitsev D.A. Functional Petri Nets , Universite Paris-Dauphine, Cahier du Lamsade 224 Avril 2005, 62p.
4. Meyer, Carl D. (2000), Matrix Analysis and Applied Linear Algebra, SIAM, p. 514, ISBN 978-0-89871-454-8.

External links

• Online Matrix Calculator
• Springer Encyclopaedia of Mathematics » Matrix factorization
• GraphLab GraphLab collaborative filtering library, large scale parallel implementation of matrix decomposition methods (in C++) for multicore.

v t e Linear algebra
Outline Glossary
Basic concepts	Scalar Vector Vector space Scalar multiplication Vector projection Linear span Linear map Linear projection Linear independence Linear combination Multilinear map Basis Change of basis Row and column vectors Row and column spaces Kernel Eigenvalues and eigenvectors Transpose Linear equations
Matrices	Block Decomposition Invertible Minor Multiplication Rank Transformation Cramer's rule Gaussian elimination Productive matrix
Bilinear	Orthogonality Dot product Hadamard product Inner product space Outer product Kronecker product Gram–Schmidt process
Multilinear algebra	Determinant Cross product Triple product Seven-dimensional cross product Geometric algebra Exterior algebra Bivector Multivector Tensor Outermorphism
Vector space constructions	Dual Direct sum Function space Quotient Subspace Tensor product
Numerical	Floating-point Numerical stability Basic Linear Algebra Subprograms Sparse matrix Comparison of linear algebra libraries
Category

• Matrix theory
• Matrix decompositions