Proofs involving the Moore–Penrose inverse

In linear algebra, the Moore–Penrose inverse is a matrix that satisfies some but not necessarily all of the properties of an inverse matrix. This article collects together a variety of proofs involving the Moore–Penrose inverse.

Let ${\displaystyle A}$ be an m-by-n matrix over a field ${\displaystyle \mathbb{𝕂}}$ and ${\displaystyle A^{+}}$ be an n-by-m matrix over ${\displaystyle \mathbb{𝕂}}$, where ${\displaystyle \mathbb{𝕂}}$ is either ${\displaystyle \mathbb{ℝ}}$, the real numbers, or ${\displaystyle \mathbb{ℂ}}$, the complex numbers. The following four criteria are called the Moore–Penrose conditions:

1. ${\displaystyle A{A}^{+}A=A}$,
2. ${\displaystyle A^{+}A{A}^{+}=A^{+}}$,
3. ${\displaystyle {\left(A{A}^{+}\right)}^{*}=A{A}^{+}}$,
4. ${\displaystyle {\left(A^{+}A\right)}^{*}=A^{+}A}$.

Given a matrix ${\displaystyle A}$, there exists a unique matrix ${\displaystyle A^{+}}$ that satisfies all four of the Moore–Penrose conditions, which is called the Moore–Penrose inverse of ${\displaystyle A}$.^[1][2][3][4] Notice that ${\displaystyle A}$ is also the Moore–Penrose inverse of ${\displaystyle A^{+}}$. That is, ${\displaystyle {\left(A^{+}\right)}^{+}=A}$.

These results are used in the proofs below. In the following lemmas, A is a matrix with complex elements and n columns, B is a matrix with complex elements and n rows.

The assumption says that all elements of A*A are zero. Therefore,

${\displaystyle 0=\mathrm{Tr}\left(A^{*}A\right)={\displaystyle \sum _{j=1}^n}{\left(A^{*}A\right)}_{jj}={\displaystyle \sum _{j=1}^n}{\displaystyle \sum _{i=1}^m}{\left(A^{*}\right)}_{ji}{A}_{ij}={\displaystyle \sum _{i=1}^m}{\displaystyle \sum _{j=1}^n}{\left|A_{ij}\right|}^2.}$

Therefore, all ${\displaystyle A_{ij}}$ equal 0 i.e. ${\displaystyle A=0}$.

${\displaystyle \begin{array}{rlr}0& =A^{*}AB& \\ \Rightarrow 0& =B^{*}{A}^{*}AB& \\ \Rightarrow 0& =(AB{)}^{*}(AB)& \\ \Rightarrow 0& =AB& (\text{by Lemma 1})\end{array}}$

This is proved in a manner similar to the argument of Lemma 2 (or by simply taking the Hermitian conjugate).

Let ${\displaystyle A}$ be a matrix over ${\displaystyle \mathbb{ℝ}}$ or ${\displaystyle \mathbb{ℂ}}$. Suppose that ${\displaystyle A_1^{+}}$ and ${\displaystyle A_2^{+}}$ are Moore–Penrose inverses of ${\displaystyle A}$. Observe then that

${\displaystyle A{A}_1^{+}(A{A}_2^{+}A)A_1^{+}=(A{A}_2^{+})(A{A}_1^{+})(A{A}_2^{+}{)}^{*}(A{A}_1^{+}{)}^{*}={A_2^{+}}^{*}(A{A}_1^{+}A{)}^{*}{A_2^{+}}^{*}{A}^{*}=(A{A}_2^{+}{)}^{*}A{A}_2^{+}.}$

Analogously we conclude that ${\displaystyle A_1^{+}}A=A_2^{+}A$. The proof is completed by observing that then

${\displaystyle A_1^{+}}{A}_1^{+}A{A}_1^{+}=A_1^{+}A{A}_2^{+}=A_2^{+}A{A}_2^{+}{A}_2^{+}.$

The proof proceeds in stages.

For any ${\displaystyle x\in \mathbb{𝕂}}$, we define:

${\displaystyle x^{+}:=\{\begin{array}{ll}{x}^{-1},& \text{if }x\ne 0\\ 0,& \text{if }x=0\end{array}}$

It is easy to see that ${\displaystyle x^{+}}$ is a pseudoinverse of ${\displaystyle x}$ (interpreted as a 1-by-1 matrix).

Let ${\displaystyle D}$ be an n-by-n matrix over ${\displaystyle \mathbb{𝕂}}$ with zeros off the diagonal. We define ${\displaystyle D^{+}}$ as an n-by-n matrix over ${\displaystyle \mathbb{𝕂}}$ with ${\displaystyle {\left(D^{+}\right)}_{ij}:={\left(D_{ij}\right)}^{+}}$ as defined above. We write simply ${\displaystyle D_{ij}^{+}}$ for ${\displaystyle {\left(D^{+}\right)}_{ij}={\left(D_{ij}\right)}^{+}}$.

Notice that ${\displaystyle D^{+}}$ is also a matrix with zeros off the diagonal.

We now show that ${\displaystyle D^{+}}$ is a pseudoinverse of ${\displaystyle D}$:

1. ${\displaystyle {\left(D{D}^{+}D\right)}_{ij}=D_{ij}{D}_{ij}^{+}{D}_{ij}=D_{ij}\Rightarrow D{D}^{+}D=D}$
2. ${\displaystyle {\left(D^{+}D{D}^{+}\right)}_{ij}=D_{ij}^{+}{D}_{ij}{D}_{ij}^{+}=D_{ij}^{+}\Rightarrow D^{+}D{D}^{+}=D^{+}}$
3. ${\displaystyle {\left(D{D}^{+}\right)}_{ij}^{*}=\overline{{\left(D{D}^{+}\right)}_{ji}}=\overline{D_{ji}{D}_{ji}^{+}}={\left(D_{ji}{D}_{ji}^{+}\right)}^{*}=D_{ji}{D}_{ji}^{+}=D_{ij}{D}_{ij}^{+}\Rightarrow {\left(D{D}^{+}\right)}^{*}=D{D}^{+}}$
4. ${\displaystyle {\left(D^{+}D\right)}_{ij}^{*}=\overline{{\left(D^{+}D\right)}_{ji}}=\overline{D_{ji}^{+}{D}_{ji}}={\left(D_{ji}^{+}{D}_{ji}\right)}^{*}=D_{ji}^{+}{D}_{ji}=D_{ij}^{+}{D}_{ij}\Rightarrow {\left(D^{+}D\right)}^{*}=D^{+}D}$

Let ${\displaystyle D}$ be an m-by-n matrix over ${\displaystyle \mathbb{𝕂}}$ with zeros off the main diagonal, where m and n are unequal. That is, ${\displaystyle D_{ij}=d_i}$ for some ${\displaystyle d_i\in \mathbb{𝕂}}$ when ${\displaystyle i=j}$ and ${\displaystyle D_{ij}=0}$ otherwise.

Consider the case where ${\displaystyle n>m}$. Then we can rewrite ${\displaystyle D=\left[D_0{\mathbf{𝟎}}_{m\times (n-m)}\right]}$ by stacking where ${\displaystyle D_0}$ is a square diagonal m-by-m matrix, and ${\displaystyle {\mathbf{𝟎}}_{m\times (n-m)}}$ is the m-by-(n−m) zero matrix. We define ${\displaystyle D^{+}\equiv \left[\begin{array}{c}{D}_0^{+}\\ {\mathbf{𝟎}}_{(n-m)\times m}\end{array}\right]}$ as an n-by-m matrix over ${\displaystyle \mathbb{𝕂}}$, with ${\displaystyle D_0^{+}}$ the pseudoinverse of ${\displaystyle D_0}$ defined above, and ${\displaystyle {\mathbf{𝟎}}_{(n-m)\times m}}$ the (n−m)-by-m zero matrix. We now show that ${\displaystyle D^{+}}$ is a pseudoinverse of ${\displaystyle D}$:

1. By multiplication of block matrices, ${\displaystyle D{D}^{+}=D_0{D}_0^{+}+{\mathbf{𝟎}}_{m\times (n-m)}{\mathbf{𝟎}}_{(n-m)\times m}=D_0{D}_0^{+},}$ so by property 1 for square diagonal matrices ${\displaystyle D_0}$ proven in the previous section,${\displaystyle D{D}^{+}D=D_0{D}_0^{+}\left[D_0{\mathbf{𝟎}}_{m\times (n-m)}\right]=\left[D_0{D}_0^{+}{D}_0{\mathbf{𝟎}}_{m\times (n-m)}\right]=\left[D_0{\mathbf{𝟎}}_{m\times (n-m)}\right]=D}$.
2. Similarly, ${\displaystyle D^{+}D=\left[\begin{array}{cc}{D}_0^{+}{D}_0& {\mathbf{𝟎}}_{m\times (n-m)}\\ {\mathbf{𝟎}}_{(n-m)\times m}& {\mathbf{𝟎}}_{(n-m)\times (n-m)}\end{array}\right]}$, so ${\displaystyle D^{+}D{D}^{+}=\left[\begin{array}{cc}{D}_0^{+}{D}_0& {\mathbf{𝟎}}_{m\times (n-m)}\\ {\mathbf{𝟎}}_{(n-m)\times m}& {\mathbf{𝟎}}_{(n-m)\times (n-m)}\end{array}\right]\left[\begin{array}{c}{D}_0^{+}\\ {\mathbf{𝟎}}_{(n-m)\times m}\end{array}\right]=\left[\begin{array}{c}{D}_0^{+}{D}_0{D}_0^{+}\\ {\mathbf{𝟎}}_{(n-m)\times m}\end{array}\right]=D^{+}.}$
3. By 1 and property 3 for square diagonal matrices, ${\displaystyle {\left(D{D}^{+}\right)}^{*}={\left(D_0{D}_0^{+}\right)}^{*}=D_0{D}_0^{+}=D{D}^{+}}$.
4. By 2 and property 4 for square diagonal matrices, ${\displaystyle {\left(D^{+}D\right)}^{*}=\left[\begin{array}{cc}{\left(D_0^{+}{D}_0\right)}^{*}& {\mathbf{𝟎}}_{m\times (n-m)}\\ {\mathbf{𝟎}}_{(n-m)\times m}& {\mathbf{𝟎}}_{(n-m)\times (n-m)}\end{array}\right]=\left[\begin{array}{cc}{D}_0^{+}{D}_0& {\mathbf{𝟎}}_{m\times (n-m)}\\ {\mathbf{𝟎}}_{(n-m)\times m}& {\mathbf{𝟎}}_{(n-m)\times (n-m)}\end{array}\right]=D^{+}D.}$

Existence for ${\displaystyle D}$ such that ${\displaystyle m>n}$ follows by swapping the roles of ${\displaystyle D}$ and ${\displaystyle D^{+}}$ in the ${\displaystyle n>m}$ case and using the fact that ${\displaystyle {\left(D^{+}\right)}^{+}=D}$.

The singular value decomposition theorem states that there exists a factorization of the form

${\displaystyle A=U\mathrm{\Sigma }{V}^{*}}$

where:

${\displaystyle U}$ is an m-by-m unitary matrix over ${\displaystyle \mathbb{𝕂}}$.

${\displaystyle \mathrm{\Sigma }}$ is an m-by-n matrix over ${\displaystyle \mathbb{𝕂}}$ with nonnegative real numbers on the diagonal and zeros off the diagonal.

${\displaystyle V}$ is an n-by-n unitary matrix over ${\displaystyle \mathbb{𝕂}}$.^[5]

Define ${\displaystyle A^{+}}$ as ${\displaystyle V{\mathrm{\Sigma }}^{+}{U}^{*}}$.

We now show that ${\displaystyle A^{+}}$ is a pseudoinverse of ${\displaystyle A}$:

1. ${\displaystyle A{A}^{+}A=U\mathrm{\Sigma }{V}^{*}V{\mathrm{\Sigma }}^{+}{U}^{*}U\mathrm{\Sigma }{V}^{*}=U\mathrm{\Sigma }{\mathrm{\Sigma }}^{+}\mathrm{\Sigma }{V}^{*}=U\mathrm{\Sigma }{V}^{*}=A}$
2. ${\displaystyle A^{+}A{A}^{+}=V{\mathrm{\Sigma }}^{+}{U}^{*}U\mathrm{\Sigma }{V}^{*}V{\mathrm{\Sigma }}^{+}{U}^{*}=V{\mathrm{\Sigma }}^{+}\mathrm{\Sigma }{\mathrm{\Sigma }}^{+}{U}^{*}=V{\mathrm{\Sigma }}^{+}{U}^{*}=A^{+}}$
3. ${\displaystyle {\left(A{A}^{+}\right)}^{*}={\left(U\mathrm{\Sigma }{V}^{*}V{\mathrm{\Sigma }}^{+}{U}^{*}\right)}^{*}={\left(U\mathrm{\Sigma }{\mathrm{\Sigma }}^{+}{U}^{*}\right)}^{*}=U{\left(\mathrm{\Sigma }{\mathrm{\Sigma }}^{+}\right)}^{*}{U}^{*}=U\left(\mathrm{\Sigma }{\mathrm{\Sigma }}^{+}\right)U^{*}=U\mathrm{\Sigma }{V}^{*}V{\mathrm{\Sigma }}^{+}{U}^{*}=A{A}^{+}}$
4. ${\displaystyle {\left(A^{+}A\right)}^{*}={\left(V{\mathrm{\Sigma }}^{+}{U}^{*}U\mathrm{\Sigma }{V}^{*}\right)}^{*}={\left(V{\mathrm{\Sigma }}^{+}\mathrm{\Sigma }{V}^{*}\right)}^{*}=V{\left({\mathrm{\Sigma }}^{+}\mathrm{\Sigma }\right)}^{*}{V}^{*}=V\left({\mathrm{\Sigma }}^{+}\mathrm{\Sigma }\right)V^{*}=V{\mathrm{\Sigma }}^{+}{U}^{*}U\mathrm{\Sigma }{V}^{*}=A^{+}A}$

${\displaystyle {A^{*}}^{+}={A^{+}}^{*}}$

The proof works by showing that ${\displaystyle {A^{+}}^{*}}$ satisfies the four criteria for the pseudoinverse of ${\displaystyle A^{*}}$. Since this amounts to just substitution, it is not shown here.

The proof of this relation is given as Exercise 1.18c in.^[6]

${\displaystyle A^{+}=A^{+}A{A}^{+}}$ and ${\displaystyle A{A}^{+}={\left(A{A}^{+}\right)}^{*}}$ imply that ${\displaystyle A^{+}=A^{+}{\left(A{A}^{+}\right)}^{*}=A^{+}{A}^{{+}^{*}}{A}^{*}}$.

${\displaystyle A^{+}=A^{+}A{A}^{+}}$ and ${\displaystyle A^{+}A={\left(A^{+}A\right)}^{*}}$ imply that ${\displaystyle A^{+}={\left(A^{+}A\right)}^{*}{A}^{+}=A^{*}{A}^{+*}{A}^{+}}$.

${\displaystyle A=A{A}^{+}A}$ and ${\displaystyle A{A}^{+}={\left(A{A}^{+}\right)}^{*}}$ imply that ${\displaystyle A={\left(A{A}^{+}\right)}^{*}A=A^{{+}^{*}}{A}^{*}A}$.

${\displaystyle A=A{A}^{+}A}$ and ${\displaystyle A^{+}A={\left(A^{+}A\right)}^{*}}$ imply that ${\displaystyle A=A{\left(A^{+}A\right)}^{*}=A{A}^{*}{A}^{{+}^{*}}}$.

This is the conjugate transpose of ${\displaystyle A=A^{{+}^{*}}{A}^{*}A}$ above.

This is the conjugate transpose of ${\displaystyle A=A{A}^{*}{A}^{{+}^{*}}}$ above.

The results of this section show that the computation of the pseudoinverse is reducible to its construction in the Hermitian case. It suffices to show that the putative constructions satisfy the defining criteria.

This relation is given as exercise 18(d) in,^[6] for the reader to prove, "for every matrix A". Write ${\displaystyle D=A^{*}{\left(A{A}^{*}\right)}^{+}}$. Observe that

${\displaystyle \begin{array}{rlrlr}& & A{A}^{*}& =A{A}^{*}{\left(A{A}^{*}\right)}^{+}A{A}^{*}& \\ & \iff & A{A}^{*}& =ADA{A}^{*}& \\ & \iff & 0& =(AD-I)A{A}^{*}& \\ & \iff & 0& =ADA-A& (\text{by Lemma 3})\\ & \iff & A& =ADA& \end{array}}$

Similarly, ${\displaystyle {\left(A{A}^{*}\right)}^{+}A{A}^{*}{\left(A{A}^{*}\right)}^{+}={\left(A{A}^{*}\right)}^{+}}$ implies that ${\displaystyle A^{*}{\left(A{A}^{*}\right)}^{+}A{A}^{*}{\left(A{A}^{*}\right)}^{+}=A^{*}{\left(A{A}^{*}\right)}^{+}}$ i.e. ${\displaystyle DAD=D}$.

Additionally, ${\displaystyle AD=A{A}^{*}{\left(A{A}^{*}\right)}^{+}}$ so ${\displaystyle AD=(AD{)}^{*}}$.

Finally, ${\displaystyle DA=A^{*}{\left(A{A}^{*}\right)}^{+}A}$ implies that ${\displaystyle (DA{)}^{*}=A^{*}{\left({\left(A{A}^{*}\right)}^{+}\right)}^{*}A=A^{*}\left({\left(A{A}^{*}\right)}^{+}\right)A=DA}$.

Therefore, ${\displaystyle D=A^{+}}$.

This is proved in an analogous manner to the case above, using Lemma 2 instead of Lemma 3.

For the first three proofs, we consider products C = AB.

If ${\displaystyle A}$ has orthonormal columns i.e. ${\displaystyle A^{*}A=I}$ then ${\displaystyle A^{+}=A^{*}}$. Write ${\displaystyle D=B^{+}{A}^{+}=B^{+}{A}^{*}}$. We show that ${\displaystyle D}$ satisfies the Moore–Penrose criteria.

${\displaystyle \begin{array}{rl}CDC& =AB{B}^{+}{A}^{*}AB=AB{B}^{+}B=AB=C,\\ DCD& =B^{+}{A}^{*}AB{B}^{+}{A}^{*}=B^{+}B{B}^{+}{A}^{*}=B^{+}{A}^{*}=D,\\ (CD{)}^{*}& =D^{*}{B}^{*}{A}^{*}=A{\left(B^{+}\right)}^{*}{B}^{*}{A}^{*}=A{\left(B{B}^{+}\right)}^{*}{A}^{*}=AB{B}^{+}{A}^{*}=CD,\\ (DC{)}^{*}& =B^{*}{A}^{*}{D}^{*}=B^{*}{A}^{*}A{\left(B^{+}\right)}^{*}={\left(B^{+}B\right)}^{*}=B^{+}B=B^{+}{A}^{*}AB=DC.\end{array}}$

Therefore, ${\displaystyle D=C^{+}}$.

If B has orthonormal rows i.e. ${\displaystyle B{B}^{*}=I}$ then ${\displaystyle B^{+}=B^{*}}$. Write ${\displaystyle D=B^{+}{A}^{+}=B^{*}{A}^{+}}$. We show that ${\displaystyle D}$ satisfies the Moore–Penrose criteria.

${\displaystyle \begin{array}{rl}CDC& =AB{B}^{*}{A}^{+}AB=A{A}^{+}AB=AB=C,\\ DCD& =B^{*}{A}^{+}AB{B}^{*}{A}^{+}=B^{*}{A}^{+}A{A}^{+}=B^{*}{A}^{+}=D,\\ (CD{)}^{*}& =D^{*}{B}^{*}{A}^{*}={\left(A^{+}\right)}^{*}B{B}^{*}{A}^{*}={\left(A^{+}\right)}^{*}{A}^{*}={\left(A{A}^{+}\right)}^{*}=A{A}^{+}=AB{B}^{*}{A}^{+}=CD,\\ (DC{)}^{*}& =B^{*}{A}^{*}{D}^{*}=B^{*}{A}^{*}{\left(A^{+}\right)}^{*}B=B^{*}{\left(A^{+}A\right)}^{*}B=B^{*}{A}^{+}AB=DC.\end{array}}$

Therefore, ${\displaystyle D=C^{+}.}$

Since ${\displaystyle A}$ has full column rank, ${\displaystyle A^{*}A}$ is invertible so ${\displaystyle {\left(A^{*}A\right)}^{+}={\left(A^{*}A\right)}^{-1}}$. Similarly, since ${\displaystyle B}$ has full row rank, ${\displaystyle B{B}^{*}}$ is invertible so ${\displaystyle {\left(B{B}^{*}\right)}^{+}={\left(B{B}^{*}\right)}^{-1}}$.

Write ${\displaystyle D=B^{+}{A}^{+}=B^{*}{\left(B{B}^{*}\right)}^{-1}{\left(A^{*}A\right)}^{-1}{A}^{*}}$(using reduction to the Hermitian case). We show that ${\displaystyle D}$ satisfies the Moore–Penrose criteria.

${\displaystyle \begin{array}{rl}CDC& =AB{B}^{*}{\left(B{B}^{*}\right)}^{-1}{\left(A^{*}A\right)}^{-1}{A}^{*}AB=AB=C,\\ DCD& =B^{*}{\left(B{B}^{*}\right)}^{-1}{\left(A^{*}A\right)}^{-1}{A}^{*}AB{B}^{*}{\left(B{B}^{*}\right)}^{-1}{\left(A^{*}A\right)}^{-1}{A}^{*}=B^{*}{\left(B{B}^{*}\right)}^{-1}{\left(A^{*}A\right)}^{-1}{A}^{*}=D,\\ CD& =AB{B}^{*}{\left(B{B}^{*}\right)}^{-1}{\left(A^{*}A\right)}^{-1}{A}^{*}=A{\left(A^{*}A\right)}^{-1}{A}^{*}={\left(A{\left(A^{*}A\right)}^{-1}{A}^{*}\right)}^{*},\\ \Rightarrow (CD{)}^{*}& =CD,\\ DC& =B^{*}{\left(B{B}^{*}\right)}^{-1}{\left(A^{*}A\right)}^{-1}{A}^{*}AB=B^{*}{\left(B{B}^{*}\right)}^{-1}B={\left(B^{*}{\left(B{B}^{*}\right)}^{-1}B\right)}^{*},\\ \Rightarrow (DC{)}^{*}& =DC.\end{array}}$

Therefore, ${\displaystyle D=C^{+}}$.

Here, ${\displaystyle B=A^{*}}$, and thus ${\displaystyle C=A{A}^{*}}$ and ${\displaystyle D=A^{+*}{A}^{+}}$. We show that indeed ${\displaystyle D}$ satisfies the four Moore–Penrose criteria.

${\displaystyle \begin{array}{rl}CDC& =A{A}^{*}{A}^{+*}{A}^{+}A{A}^{*}=A{\left(A^{+}A\right)}^{*}{A}^{+}A{A}^{*}=A{A}^{+}A{A}^{+}A{A}^{*}=A{A}^{+}A{A}^{*}=A{A}^{*}=C\\ DCD& =A^{+*}{A}^{+}A{A}^{*}{A}^{+*}{A}^{+}=A^{+*}{A}^{+}A{\left(A^{+}A\right)}^{*}{A}^{+}=A^{+*}{A}^{+}A{A}^{+}A{A}^{+}=A^{+*}{A}^{+}A{A}^{+}=A^{+*}{A}^{+}=D\\ (CD{)}^{*}& ={\left(A{A}^{*}{A}^{+*}{A}^{+}\right)}^{*}=A^{+*}{A}^{+}A{A}^{*}=A^{+*}{\left(A^{+}A\right)}^{*}{A}^{*}=A^{+*}{A}^{*}{A}^{+*}{A}^{*}\\ & ={\left(A{A}^{+}\right)}^{*}{\left(A{A}^{+}\right)}^{*}=A{A}^{+}A{A}^{+}=A{\left(A^{+}A\right)}^{*}{A}^{+}=A{A}^{*}{A}^{+*}{A}^{+}=CD\\ (DC{)}^{*}& ={\left(A^{+*}{A}^{+}A{A}^{*}\right)}^{*}=A{A}^{*}{A}^{+*}{A}^{+}=A{\left(A^{+}A\right)}^{*}{A}^{+}=A{A}^{+}A{A}^{+}\\ & ={\left(A{A}^{+}\right)}^{*}{\left(A{A}^{+}\right)}^{*}=A^{+*}{A}^{*}{A}^{+*}{A}^{*}=A^{+*}{\left(A^{+}A\right)}^{*}{A}^{*}=A^{+*}{A}^{+}A{A}^{*}=DC\end{array}}$

Therefore, ${\displaystyle D=C^{+}}$. In other words:

${\displaystyle {\left(A{A}^{*}\right)}^{+}=A^{+*}{A}^{+}}$

and, since ${\displaystyle {\left(A^{*}\right)}^{*}=A}$

${\displaystyle {\left(A^{*}A\right)}^{+}=A^{+}{A}^{+*}}$

Define ${\displaystyle P=A{A}^{+}}$ and ${\displaystyle Q=A^{+}A}$. Observe that ${\displaystyle P^2=A{A}^{+}A{A}^{+}=A{A}^{+}=P}$. Similarly ${\displaystyle Q^2=Q}$, and finally, ${\displaystyle P=P^{*}}$ and ${\displaystyle Q=Q^{*}}$. Thus ${\displaystyle P}$ and ${\displaystyle Q}$ are orthogonal projection operators. Orthogonality follows from the relations ${\displaystyle P=P^{*}}$ and ${\displaystyle Q=Q^{*}}$. Indeed, consider the operator ${\displaystyle P}$: any vector decomposes as

${\displaystyle x=Px+(I-P)x}$

and for all vectors ${\displaystyle x}$ and ${\displaystyle y}$ satisfying ${\displaystyle Px=x}$ and ${\displaystyle (I-P)y=y}$, we have

${\displaystyle x^{*}y=(Px{)}^{*}(I-P)y=x^{*}{P}^{*}(I-P)y=x^{*}P(I-P)y=0}$.

It follows that ${\displaystyle PA=A{A}^{+}A=A}$ and ${\displaystyle A^{+}P=A^{+}A{A}^{+}=A^{+}}$. Similarly, ${\displaystyle Q{A}^{+}=A^{+}}$ and ${\displaystyle AQ=A}$. The orthogonal components are now readily identified.

If ${\displaystyle y}$ belongs to the range of ${\displaystyle A}$ then for some ${\displaystyle x}$, ${\displaystyle y=Ax}$ and ${\displaystyle Py=PAx=Ax=y}$. Conversely, if ${\displaystyle Py=y}$ then ${\displaystyle y=A{A}^{+}y}$ so that ${\displaystyle y}$ belongs to the range of ${\displaystyle A}$. It follows that ${\displaystyle P}$ is the orthogonal projector onto the range of ${\displaystyle A}$. ${\displaystyle I-P}$ is then the orthogonal projector onto the orthogonal complement of the range of ${\displaystyle A}$, which equals the kernel of ${\displaystyle A^{*}}$.

A similar argument using the relation ${\displaystyle Q{A}^{*}=A^{*}}$ establishes that ${\displaystyle Q}$ is the orthogonal projector onto the range of ${\displaystyle A^{*}}$ and ${\displaystyle (I-Q)}$ is the orthogonal projector onto the kernel of ${\displaystyle A}$.

Using the relations ${\displaystyle P{\left(A^{+}\right)}^{*}=P^{*}{\left(A^{+}\right)}^{*}={\left(A^{+}P\right)}^{*}={\left(A^{+}\right)}^{*}}$ and ${\displaystyle P=P^{*}={\left(A^{+}\right)}^{*}{A}^{*}}$ it follows that the range of P equals the range of ${\displaystyle {\left(A^{+}\right)}^{*}}$, which in turn implies that the range of ${\displaystyle I-P}$ equals the kernel of ${\displaystyle A^{+}}$. Similarly ${\displaystyle Q{A}^{+}=A^{+}}$ implies that the range of ${\displaystyle Q}$ equals the range of ${\displaystyle A^{+}}$. Therefore, we find,

${\displaystyle \begin{array}{rl}\mathrm{Ker}\left(A^{+}\right)& =\mathrm{Ker}\left(A^{*}\right).\\ \mathrm{Im}\left(A^{+}\right)& =\mathrm{Im}\left(A^{*}\right).\\ \end{array}}$

In the general case, it is shown here for any ${\displaystyle m\times n}$ matrix ${\displaystyle A}$ that ${\displaystyle \Vert Ax-b{\Vert }_2\ge \Vert Az-b{\Vert }_2}$ where ${\displaystyle z=A^{+}b}$. This lower bound need not be zero as the system ${\displaystyle Ax=b}$ may not have a solution (e.g. when the matrix A does not have full rank or the system is overdetermined).

To prove this, we first note that (stating the complex case), using the fact that ${\displaystyle P=A{A}^{+}}$ satisfies ${\displaystyle PA=A}$ and ${\displaystyle P=P^{*}}$, we have

${\displaystyle \begin{array}{rl}{A}^{*}(Az-b)& =A^{*}(A{A}^{+}b-b)\\ & =A^{*}(Pb-b)\\ & =A^{*}{P}^{*}b-A^{*}b\\ & =(PA{)}^{*}b-A^{*}b\\ & =0\end{array}}$

so that (${\displaystyle \text{c.c.}}$ stands for the complex conjugate of the previous term in the following)

${\displaystyle \begin{array}{rl}\Vert Ax-b{\Vert }_2^2& =\Vert Az-b{\Vert }_2^2+(A(x-z){)}^{*}(Az-b)+\text{c.c.}+\Vert A(x-z){\Vert }_2^2\\ & =\Vert Az-b{\Vert }_2^2+(x-z{)}^{*}{A}^{*}(Az-b)+\text{c.c.}+\Vert A(x-z){\Vert }_2^2\\ & =\Vert Az-b{\Vert }_2^2+\Vert A(x-z){\Vert }_2^2\\ & \ge \Vert Az-b{\Vert }_2^2\end{array}}$

as claimed.

If ${\displaystyle A}$ is injective i.e. one-to-one (which implies ${\displaystyle m\ge n}$), then the bound is attained uniquely at ${\displaystyle z}$.

The proof above also shows that if the system ${\displaystyle Ax=b}$ is satisfiable i.e. has a solution, then necessarily ${\displaystyle z=A^{+}b}$ is a solution (not necessarily unique). We show here that ${\displaystyle z}$ is the smallest such solution (its Euclidean norm is uniquely minimum).

To see this, note first, with ${\displaystyle Q=A^{+}A}$, that ${\displaystyle Qz=A^{+}A{A}^{+}b=A^{+}b=z}$ and that ${\displaystyle Q^{*}=Q}$. Therefore, assuming that ${\displaystyle Ax=b}$, we have

${\displaystyle \begin{array}{rl}{z}^{*}(x-z)& =(Qz{)}^{*}(x-z)\\ & =z^{*}Q(x-z)\\ & =z^{*}(A^{+}Ax-z)\\ & =z^{*}(A^{+}b-z)\\ & =0.\end{array}}$

Thus

${\displaystyle \begin{array}{rl}\Vert x{\Vert }_2^2& =\Vert z{\Vert }_2^2+z^{*}(x-z)+\text{c.c.}+\Vert x-z{\Vert }_2^2\\ & =\Vert z{\Vert }_2^2+\Vert x-z{\Vert }_2^2\\ & \ge \Vert z{\Vert }_2^2\end{array}}$

with equality if and only if ${\displaystyle x=z}$, as was to be shown.

An immediate consequence of this result is that ${\displaystyle z}$ is also the uniquely smallest solution to the least-squares minimization problem for all ${\displaystyle Ax=b}$, including when ${\displaystyle A}$ is neither injective nor surjective. It can be shown that the least-squares approximation ${\displaystyle Az=y\approx b}$ is unique. Thus it is necessary and sufficient for all ${\displaystyle x}$ that solve the least-squares minimization to satisfy ${\displaystyle Ax=y=Az=A{A}^{+}b}$. This system always has a solution (not necessarily unique) as ${\displaystyle Az}$ lies in the column space of ${\displaystyle A}$. From the above result the smallest ${\displaystyle x}$ which solves this system is ${\displaystyle A^{+}(A{A}^{+}b)=A^{+}b=z}$.

• Ben-Israel, Adi; Greville, Thomas N.E. (2003). Generalized inverses: Theory and applications (2nd ed.). New York, NY: Springer. doi:10.1007/b97366. ISBN 978-0-387-00293-4. 
• Campbell, S. L.; Meyer, Jr., C. D. (1991). Generalized Inverses of Linear Transformations. Dover. ISBN 978-0-486-66693-8. https://archive.org/details/generalizedinver0000camp. 
• Nakamura, Yoshihiko (1991). Advanced Robotics: Redundancy and Optimization. Addison-Wesley. ISBN 978-0201151985. 
• Rao, C. Radhakrishna; Mitra, Sujit Kumar (1971). Generalized Inverse of Matrices and its Applications. New York: John Wiley & Sons. pp. 240. ISBN 978-0-471-70821-6. https://archive.org/details/generalizedinver0000raoc. 

0.00 (0 votes)