Singular value decomposition

In linear algebra, the singular value decomposition (SVD) is a factorization of a real or complex matrix into a rotation, followed by a scaling, followed by another rotation. It generalizes the eigendecomposition of a square normal matrix with an orthonormal eigenbasis to any ⁠${\displaystyle m\times n}$⁠ matrix. It is related to the polar decomposition.

Specifically, the singular value decomposition of an ${\displaystyle m\times n}$ complex matrix ⁠${\displaystyle \mathbf {M} }$⁠ is a factorization of the form ⁠${\displaystyle \mathbf {M} =\mathbf {U\Sigma V} ^{*}}$⁠, where ⁠${\displaystyle \mathbf {U} }$⁠ is an ⁠${\displaystyle m\times m}$⁠ complex unitary matrix, ${\displaystyle \mathbf {\Sigma } }$ is an ${\displaystyle m\times n}$ rectangular diagonal matrix with non-negative real numbers on the diagonal, ⁠${\displaystyle \mathbf {V} }$⁠ is an ${\displaystyle n\times n}$ complex unitary matrix, and ${\displaystyle \mathbf {V} ^{*}}$ is the conjugate transpose of ⁠${\displaystyle \mathbf {V} }$⁠. Such decompositions always exist for any complex matrix. If ⁠${\displaystyle \mathbf {M} }$⁠ is real, then ⁠${\displaystyle \mathbf {U} }$⁠ and ⁠${\displaystyle \mathbf {V} }$⁠ can be guaranteed to be real orthogonal matrices; in such contexts, the SVD is often denoted ⁠${\displaystyle \mathbf {U\Sigma V} ^{\mathsf {T}}}$⁠.

The diagonal entries ${\displaystyle \sigma _{i}=\mathbf {\Sigma } _{ii}}$ of ${\displaystyle \mathbf {\Sigma } }$ are uniquely determined by ⁠${\displaystyle \mathbf {M} }$⁠ up to reordering and are known as the singular values of ⁠${\displaystyle \mathbf {M} }$⁠. The number of non-zero singular values is equal to the rank of ⁠${\displaystyle \mathbf {M} }$⁠. The columns of ⁠${\displaystyle \mathbf {U} }$⁠ and the columns of ⁠${\displaystyle \mathbf {V} }$⁠ are called left-singular vectors and right-singular vectors of ⁠${\displaystyle \mathbf {M} }$⁠, respectively. They form two sets of orthonormal bases ⁠${\displaystyle \mathbf {u} _{1},\ldots ,\mathbf {u} _{m}}$⁠ and ⁠${\displaystyle \mathbf {v} _{1},\ldots ,\mathbf {v} _{n}}$⁠, and if they are sorted so that the singular values ${\displaystyle \sigma _{i}}$ equal to zero are all in the highest-numbered columns (or rows), the singular value decomposition can be written as $${\displaystyle \forall (i,j)\in \{1,\dots ,m\}\times \{1,\dots ,n\},}$$ ${\displaystyle ~\mathbf {M} _{ij}=\mathbf {u'} \!_{i}\mathbf {\Sigma } \,\mathbf {v'} \!\!_{j}\!^{*}=\sum _{k=1}^{k=r}\mathbf {u'} \!_{i,k}\,\sigma _{k}{\overline {\mathbf {v'} \!\!_{j,k}}},}$ where ${\displaystyle \mathbf {u'} \!_{i}}$ is the row vector corresponding to the ⁠${\displaystyle i}$⁠-th row of ⁠${\displaystyle U}$⁠, ${\displaystyle ~}$${\displaystyle \mathbf {u'} \!_{i,k}}$ is the ⁠${\displaystyle k}$⁠-th coordinate of ⁠${\displaystyle \mathbf {u'} \!_{i}}$⁠, ${\displaystyle ~}$${\displaystyle \sigma _{k}}$ is the ⁠${\displaystyle k}$⁠-th singular value on ⁠${\displaystyle \mathbf {\Sigma } }$⁠'s diagonal, and ${\displaystyle r\leq \min\{m,n\}}$ is the rank of ⁠${\displaystyle \mathbf {M} }$⁠.

With the matrices of section Example (below), $${\displaystyle {\begin{aligned}\mathbf {M} _{4,5}&=\mathbf {u'} \!_{4}\mathbf {\Sigma } \,\mathbf {v'} \!\!_{5}\!^{*}=\sum _{k=1}^{k=3}\mathbf {u'} \!_{4,k}\,\sigma _{k}{\overline {\mathbf {v'} \!\!_{5,k}}}\\&=(0\cdot 3\cdot 0)+{\Big (}0\cdot {\sqrt {5}}\cdot {\big (}-{\sqrt {0,8}}{\big )}{\Big )}+(-1\cdot 2\cdot 0){\color {Gray}\,+\,{\big (}({\mathit {0}}\cdot {\mathit {0}}\cdot {\mathit {0}}){\big )}}=0.\end{aligned}}}$$

The SVD is not unique. However, it is always possible to choose the decomposition such that the singular values ${\displaystyle \mathbf {\Sigma } _{ii}}$ are in descending order. In this case, ${\displaystyle \mathbf {\Sigma } }$ (but not ⁠${\displaystyle \mathbf {U} }$⁠ and ⁠${\displaystyle \mathbf {V} }$⁠) is uniquely determined by ⁠${\displaystyle \mathbf {M} }$⁠.

The terminology "SVD" sometimes refers to the compact SVD, a similar decomposition ⁠${\displaystyle \mathbf {M} =\mathbf {U} _{r}\mathbf {\Sigma } _{r}\mathbf {V} ^{*}\!\!\!\!_{r}\,}$⁠ in which ⁠${\displaystyle \mathbf {\Sigma } _{r}}$⁠ is square diagonal of size ⁠${\displaystyle r\times r}$⁠, where ⁠${\displaystyle r\leq \min\{m,n\}}$⁠ is the rank of ⁠${\displaystyle \mathbf {M} }$⁠, and has only the non-zero singular values. In this variant, ⁠${\displaystyle \mathbf {U} _{r}}$⁠ is an ⁠${\displaystyle m\times r}$⁠ semi-unitary matrix and ${\displaystyle \mathbf {V} \!_{r}}$ is an ⁠${\displaystyle n\times r}$⁠ semi-unitary matrix, such that ⁠${\displaystyle \mathbf {U} _{r}^{*}\mathbf {U} _{r}=\mathbf {V} ^{*}\!\!\!\!_{r}~\mathbf {V} \!_{r}=\mathbf {I} _{r}}$⁠.

Mathematical applications of the SVD include computing the pseudoinverse, matrix approximation, and determining the rank, range, and null space of a matrix. The SVD is also extremely useful in many areas of science, engineering, and statistics, such as signal processing, least squares fitting of data, and process control.