Khatri–Rao product

In mathematics, the Khatri–Rao product or block Kronecker product of two partitioned matrices ${\displaystyle \mathbf{𝐀}}$ and ${\displaystyle \mathbf{𝐁}}$ is defined as^[1][2][3]

${\displaystyle \mathbf{𝐀}\ast \mathbf{𝐁}={({\mathbf{𝐀}}_{ij}\otimes {\mathbf{𝐁}}_{ij})}_{ij}}$

in which the ij-th block is the m_ip_i × n_jq_j sized Kronecker product of the corresponding blocks of A and B, assuming the number of row and column partitions of both matrices is equal. The size of the product is then (Σ_i m_ip_i) × (Σ_j n_jq_j).

For example, if A and B both are 2 × 2 partitioned matrices e.g.:

${\displaystyle \mathbf{𝐀}=\left[\begin{array}{cc}{\mathbf{𝐀}}_{11}& {\mathbf{𝐀}}_{12}\\ {\mathbf{𝐀}}_{21}& {\mathbf{𝐀}}_{22}\end{array}\right]=\left[\begin{array}{ccc}1& 2& 3\\ 4& 5& 6\\ 7& 8& 9\end{array}\right],\mathbf{𝐁}=\left[\begin{array}{cc}{\mathbf{𝐁}}_{11}& {\mathbf{𝐁}}_{12}\\ {\mathbf{𝐁}}_{21}& {\mathbf{𝐁}}_{22}\end{array}\right]=\left[\begin{array}{ccc}1& 4& 7\\ 2& 5& 8\\ 3& 6& 9\end{array}\right],}$

we obtain:

${\displaystyle \mathbf{𝐀}\ast \mathbf{𝐁}=\left[\begin{array}{cc}{\mathbf{𝐀}}_{11}\otimes {\mathbf{𝐁}}_{11}& {\mathbf{𝐀}}_{12}\otimes {\mathbf{𝐁}}_{12}\\ {\mathbf{𝐀}}_{21}\otimes {\mathbf{𝐁}}_{21}& {\mathbf{𝐀}}_{22}\otimes {\mathbf{𝐁}}_{22}\end{array}\right]=\left[\begin{array}{cccc}1& 2& 12& 21\\ 4& 5& 24& 42\\ 14& 16& 45& 72\\ 21& 24& 54& 81\end{array}\right].}$

This is a submatrix of the Tracy–Singh product^[4] of the two matrices (each partition in this example is a partition in a corner of the Tracy–Singh product).

The column-wise Kronecker product of two matrices is a special case of the Khatri-Rao product as defined above, and may also be called the Khatri–Rao product. This product assumes the partitions of the matrices are their columns. In this case m₁ = m, p₁ = p, n = q and for each j: n_j = q_j = 1. The resulting product is a mp × n matrix of which each column is the Kronecker product of the corresponding columns of A and B. Using the matrices from the previous examples with the columns partitioned:

${\displaystyle \mathbf{𝐂}=\left[\begin{array}{ccc}{\mathbf{𝐂}}_1& {\mathbf{𝐂}}_2& {\mathbf{𝐂}}_3\end{array}\right]=\left[\begin{array}{ccc}1& 2& 3\\ 4& 5& 6\\ 7& 8& 9\end{array}\right],\mathbf{𝐃}=\left[\begin{array}{ccc}{\mathbf{𝐃}}_1& {\mathbf{𝐃}}_2& {\mathbf{𝐃}}_3\end{array}\right]=\left[\begin{array}{ccc}1& 4& 7\\ 2& 5& 8\\ 3& 6& 9\end{array}\right],}$

so that:

${\displaystyle \mathbf{𝐂}\ast \mathbf{𝐃}=\left[\begin{array}{ccc}{\mathbf{𝐂}}_1\otimes {\mathbf{𝐃}}_1& {\mathbf{𝐂}}_2\otimes {\mathbf{𝐃}}_2& {\mathbf{𝐂}}_3\otimes {\mathbf{𝐃}}_3\end{array}\right]=\left[\begin{array}{ccc}1& 8& 21\\ 2& 10& 24\\ 3& 12& 27\\ 4& 20& 42\\ 8& 25& 48\\ 12& 30& 54\\ 7& 32& 63\\ 14& 40& 72\\ 21& 48& 81\end{array}\right].}$

This column-wise version of the Khatri–Rao product is useful in linear algebra approaches to data analytical processing^[5] and in optimizing the solution of inverse problems dealing with a diagonal matrix.^[6][7]

In 1996 the column-wise Khatri–Rao product was proposed to estimate the angles of arrival (AOAs) and delays of multipath signals^[8] and four coordinates of signals sources^[9] at a digital antenna array.

An alternative concept of the matrix product, which uses row-wise splitting of matrices with a given quantity of rows, was proposed by V. Slyusar^[10] in 1996.^{[9][11][12][13][14]}

This matrix operation was named the "face-splitting product" of matrices^[11][13] or the "transposed Khatri–Rao product". This type of operation is based on row-by-row Kronecker products of two matrices. Using the matrices from the previous examples with the rows partitioned:

${\displaystyle \mathbf{𝐂}=\left[\begin{array}{c}{\mathbf{𝐂}}_1\\ {\mathbf{𝐂}}_2\\ {\mathbf{𝐂}}_3\\ \end{array}\right]=\left[\begin{array}{ccc}1& 2& 3\\ 4& 5& 6\\ 7& 8& 9\end{array}\right],\mathbf{𝐃}=\left[\begin{array}{c}{\mathbf{𝐃}}_1\\ {\mathbf{𝐃}}_2\\ {\mathbf{𝐃}}_3\\ \end{array}\right]=\left[\begin{array}{ccc}1& 4& 7\\ 2& 5& 8\\ 3& 6& 9\end{array}\right],}$

the result can be obtained:^[9][11][13]

${\displaystyle \mathbf{𝐂}\bullet \mathbf{𝐃}=\left[\begin{array}{c}{\mathbf{𝐂}}_1\otimes {\mathbf{𝐃}}_1\\ {\mathbf{𝐂}}_2\otimes {\mathbf{𝐃}}_2\\ {\mathbf{𝐂}}_3\otimes {\mathbf{𝐃}}_3\\ \end{array}\right]=\left[\begin{array}{ccccccccc}1& 4& 7& 2& 8& 14& 3& 12& 21\\ 8& 20& 32& 10& 25& 40& 12& 30& 48\\ 21& 42& 63& 24& 48& 72& 27& 54& 81\end{array}\right].}$

In the following properties, the operator

denotes the row-wise Kronecker product (face-splitting product) and the operator

denotes the column-wise Kronecker product

Source:^[18]

${\displaystyle \begin{array}{rl}& (\left[\begin{array}{cc}1& 0\\ 0& 1\\ 1& 0\end{array}\right]\bullet \left[\begin{array}{cc}1& 0\\ 1& 0\\ 0& 1\end{array}\right])(\left[\begin{array}{cc}1& 1\\ 1& -1\end{array}\right]\otimes \left[\begin{array}{cc}1& 1\\ 1& -1\end{array}\right])(\left[\begin{array}{cc}{\sigma }_1& 0\\ 0& {\sigma }_2\\ \end{array}\right]\otimes \left[\begin{array}{cc}{\rho }_1& 0\\ 0& {\rho }_2\\ \end{array}\right])(\left[\begin{array}{c}{x}_1\\ x_2\end{array}\right]\ast \left[\begin{array}{c}{y}_1\\ y_2\end{array}\right])\\ =& (\left[\begin{array}{cc}1& 0\\ 0& 1\\ 1& 0\end{array}\right]\bullet \left[\begin{array}{cc}1& 0\\ 1& 0\\ 0& 1\end{array}\right])(\left[\begin{array}{cc}1& 1\\ 1& -1\end{array}\right]\left[\begin{array}{cc}{\sigma }_1& 0\\ 0& {\sigma }_2\\ \end{array}\right]\left[\begin{array}{c}{x}_1\\ x_2\end{array}\right]\otimes \left[\begin{array}{cc}1& 1\\ 1& -1\end{array}\right]\left[\begin{array}{cc}{\rho }_1& 0\\ 0& {\rho }_2\\ \end{array}\right]\left[\begin{array}{c}{y}_1\\ y_2\end{array}\right])\\ =& \left[\begin{array}{cc}1& 0\\ 0& 1\\ 1& 0\end{array}\right]\left[\begin{array}{cc}1& 1\\ 1& -1\end{array}\right]\left[\begin{array}{cc}{\sigma }_1& 0\\ 0& {\sigma }_2\\ \end{array}\right]\left[\begin{array}{c}{x}_1\\ x_2\end{array}\right]\circ \left[\begin{array}{cc}1& 0\\ 1& 0\\ 0& 1\end{array}\right]\left[\begin{array}{cc}1& 1\\ 1& -1\end{array}\right]\left[\begin{array}{cc}{\rho }_1& 0\\ 0& {\rho }_2\\ \end{array}\right]\left[\begin{array}{c}{y}_1\\ y_2\end{array}\right].\end{array}}$

Source:^[18]

If ${\displaystyle M=T^{(1)}\bullet \dots \bullet T^{(c)}}$, where ${\displaystyle T^{(1)},\dots ,T^{(c)}}$ are independent components a random matrix ${\displaystyle T}$ with independent identically distributed rows ${\displaystyle T_1,\dots ,T_m\in {\mathbb{ℝ}}^d}$, such that

${\displaystyle E[(T_{1}x{)}^2]={\Vert x\Vert }_2^2}$ and ${\displaystyle E{[(T_{1}x{)}^p]}^{\frac{1}{p}}\le \sqrt{ap}\Vert x{\Vert }_2}$,

then for any vector ${\displaystyle x}$

${\displaystyle |{\Vert Mx\Vert }_2-{\Vert x\Vert }_2|<\epsilon {\Vert x\Vert }_2}$

with probability ${\displaystyle 1-\delta }$ if the quantity of rows

${\displaystyle m=(4a{)}^{2c}{\epsilon }^{-2}\log 1/\delta +(2ae){\epsilon }^{-1}(\log 1/\delta {)}^c.}$

In particular, if the entries of ${\displaystyle T}$ are ${\displaystyle \pm 1}$ can get

${\displaystyle m=O({\epsilon }^{-2}\log 1/\delta +{\epsilon }^{-1}{(\frac{1}{c}\log 1/\delta )}^c)}$

which matches the Johnson–Lindenstrauss lemma of ${\displaystyle m=O({\epsilon }^{-2}\log 1/\delta )}$ when ${\displaystyle \epsilon }$ is small.

According to the definition of V. Slyusar^[9][13] the block face-splitting product of two partitioned matrices with a given quantity of rows in blocks

${\displaystyle \mathbf{𝐀}=\left[\begin{array}{cc}{\mathbf{𝐀}}_{11}& {\mathbf{𝐀}}_{12}\\ {\mathbf{𝐀}}_{21}& {\mathbf{𝐀}}_{22}\end{array}\right],\mathbf{𝐁}=\left[\begin{array}{cc}{\mathbf{𝐁}}_{11}& {\mathbf{𝐁}}_{12}\\ {\mathbf{𝐁}}_{21}& {\mathbf{𝐁}}_{22}\end{array}\right],}$

can be written as :

${\displaystyle \mathbf{𝐀}[\bullet ]\mathbf{𝐁}=\left[\begin{array}{cc}{\mathbf{𝐀}}_{11}\bullet {\mathbf{𝐁}}_{11}& {\mathbf{𝐀}}_{12}\bullet {\mathbf{𝐁}}_{12}\\ {\mathbf{𝐀}}_{21}\bullet {\mathbf{𝐁}}_{21}& {\mathbf{𝐀}}_{22}\bullet {\mathbf{𝐁}}_{22}\end{array}\right].}$

The transposed block face-splitting product (or Block column-wise version of the Khatri–Rao product) of two partitioned matrices with a given quantity of columns in blocks has a view:^[9][13]

${\displaystyle \mathbf{𝐀}[\ast ]\mathbf{𝐁}=\left[\begin{array}{cc}{\mathbf{𝐀}}_{11}\ast {\mathbf{𝐁}}_{11}& {\mathbf{𝐀}}_{12}\ast {\mathbf{𝐁}}_{12}\\ {\mathbf{𝐀}}_{21}\ast {\mathbf{𝐁}}_{21}& {\mathbf{𝐀}}_{22}\ast {\mathbf{𝐁}}_{22}\end{array}\right].}$

1. Transpose:

   ${\displaystyle {(\mathbf{𝐀}[\ast ]\mathbf{𝐁})}^{\text{T}}={\text{A}}^{\text{T}}[\bullet ]{\mathbf{𝐁}}^{\text{T}}}$^[15]

The Face-splitting product and the Block Face-splitting product used in the tensor-matrix theory of digital antenna arrays. These operations are also used in:

• Artificial Intelligence and Machine learning systems to minimization of convolution and tensor sketch operations,^[18]
• A popular Natural Language Processing models, and hypergraph models of similarity,^[22]
• Generalized linear array model in statistics^[21]
• Two- and multidimensional P-spline approximation of data,^[20]
• Studies of genotype x environment interactions.^[23]

• Kronecker product
• Hadamard product

• Khatri C. G., C. R. Rao (1968). "Solutions to some functional equations and their applications to characterization of probability distributions". Sankhya 30: 167–180. http://sankhya.isical.ac.in/search/30a2/30a2019.html. Retrieved 2008-08-21. 
• Rao C.R.; Rao M. Bhaskara (1998), Matrix Algebra and Its Applications to Statistics and Econometrics, World Scientific, pp. 216 
• Zhang X; Yang Z; Cao C. (2002), "Inequalities involving Khatri–Rao products of positive semi-definite matrices", Applied Mathematics E-notes 2: 117–124 
• Liu Shuangzhe; Trenkler Götz (2008), "Hadamard, Khatri-Rao, Kronecker and other matrix products", International Journal of Information and Systems Sciences 4: 160–177 

0.00 (0 votes) Original source: https://en.wikipedia.org/wiki/Khatri–Rao product. Read more