Matrix normal distribution

Short description: Probability distribution

Matrix normalNotation|  ℳ𝒩n,p(𝐌,𝐔,𝐕)

|

Parameters|

𝐌 location (real n×p matrix)
𝐔 scale (positive-definite real n×n matrix)

𝐕 scale (positive-definite real p×p matrix)
Support|  𝐗∈ℝn×p
PDF|  exp⁡(−12tr[𝐕−1(𝐗−𝐌)T𝐔−1(𝐗−𝐌)])(2π)np/2|𝐕|n/2|𝐔|p/2
Mean|  𝐌
Variance|  𝐔 (among-row) and 𝐕 (among-column)

In statistics, the matrix normal distribution or matrix Gaussian distribution is a probability distribution that is a generalization of the multivariate normal distribution to matrix-valued random variables.

## Contents

* 1 Definition
* 1.1 Proof
* 2 Properties
* 2.1 Expected values
* 2.2 Transformation
* 2.3 Composition
* 3 Example
* 4 Maximum likelihood parameter estimation
* 5 Drawing values from the distribution
* 6 Relation to other distributions
* 7 See also
* 8 References

## Definition

The probability density function for the random matrix X (n × p) that follows the matrix normal distribution ℳ𝒩n,p(𝐌,𝐔,𝐕) has the form:

p(𝐗∣𝐌,𝐔,𝐕)=exp⁡(−12tr[𝐕−1(𝐗−𝐌)T𝐔−1(𝐗−𝐌)])(2π)np/2|𝐕|n/2|𝐔|p/2

where tr denotes trace and M is n × p, U is n × n and V is p × p, and the density is understood as the probability density function with respect to the standard Lebesgue measure in ℝn×p, i.e.: the measure corresponding to integration with respect to dx11dx21…dxn1dx12…dxn2…dxnp.

The matrix normal is related to the multivariate normal distribution in the following way:

𝐗∼ℳ𝒩n×p(𝐌,𝐔,𝐕),

if and only if

vec(𝐗)∼𝒩np(vec(𝐌),𝐕⊗𝐔)

where ⊗ denotes the Kronecker product and vec(𝐌) denotes the vectorization of 𝐌.

### Proof

The equivalence between the above matrix normal and multivariate normal density functions can be shown using several properties of the trace and Kronecker product, as follows. We start with the argument of the exponent of the matrix normal PDF:

−12tr[𝐕−1(𝐗−𝐌)T𝐔−1(𝐗−𝐌)]=−12vec(𝐗−𝐌)Tvec(𝐔−1(𝐗−𝐌)𝐕−1)=−12vec(𝐗−𝐌)T(𝐕−1⊗𝐔−1)vec(𝐗−𝐌)=−12[vec(𝐗)−vec(𝐌)]T(𝐕⊗𝐔)−1[vec(𝐗)−vec(𝐌)]

which is the argument of the exponent of the multivariate normal PDF with respect to Lebesgue measure in ℝnp. The proof is completed by using the determinant property: |𝐕⊗𝐔|=|𝐕|n|𝐔|p.

## Properties

If 𝐗∼ℳ𝒩n×p(𝐌,𝐔,𝐕), then we have the following properties:[1][2]

### Expected values

The mean, or expected value is:

E[𝐗]=𝐌

and we have the following second-order expectations:

E[(𝐗−𝐌)(𝐗−𝐌)T]=𝐔tr⁡(𝐕)

E[(𝐗−𝐌)T(𝐗−𝐌)]=𝐕tr⁡(𝐔)

where tr denotes trace.

More generally, for appropriately dimensioned matrices A,B,C:

E[𝐗𝐀𝐗T]=𝐔tr⁡(𝐀T𝐕)+𝐌𝐀𝐌TE[𝐗T𝐁𝐗]=𝐕tr⁡(𝐔𝐁T)+𝐌T𝐁𝐌E[𝐗𝐂𝐗]=𝐕𝐂T𝐔+𝐌𝐂𝐌

### Transformation

Transpose transform:

𝐗T∼ℳ𝒩p×n(𝐌T,𝐕,𝐔)

Linear transform: let D (r-by-n), be of full rank r ≤ n and C (p-by-s), be of full rank s ≤ p, then:

𝐃𝐗𝐂∼ℳ𝒩r×s(𝐃𝐌𝐂,𝐃𝐔𝐃T,𝐂T𝐕𝐂)

### Composition

The product of two matrix normal densities

ℳ𝒩(M𝟏,U𝟏,V𝟏)⋅ℳ𝒩(M𝟐,U𝟐,V𝟐)∝𝒩(μc,Σc)

is proportional to a normal density with parameters:

Σc=(V1−1⊗U1−1+V2−1⊗U2−1)−1,
μc=Σc((V1−1⊗U1−1)vec⁡(M1)+(V2−1⊗U2−1)vec⁡(M2)).

Note: this is distinct from the product of normal variables, which is not generally normal.

## Example

Let's imagine a sample of n independent p-dimensional random variables identically distributed according to a multivariate normal distribution:

𝐘i∼𝒩p(μ,Σ) with i∈{1,…,n}.

When defining the n × p matrix 𝐗 for which the ith row is 𝐘i, we obtain:

𝐗∼ℳ𝒩n×p(𝐌,𝐔,𝐕)

where each row of 𝐌 is equal to μ, that is 𝐌=𝟏n×μT, 𝐔 is the n × n identity matrix, that is the rows are independent, and 𝐕=Σ.

## Maximum likelihood parameter estimation

Given k matrices, each of size n × p, denoted 𝐗1,𝐗2,…,𝐗k, which we assume have been sampled i.i.d. from a matrix normal distribution, the maximum likelihood estimate of the parameters can be obtained by maximizing:

∏i=1kℳ𝒩n×p(𝐗i∣𝐌,𝐔,𝐕).

The solution for the mean has a closed form, namely

𝐌=1k∑i=1k𝐗i

but the covariance parameters do not. However, these parameters can be iteratively maximized by zero-ing their gradients at:

𝐔=1kp∑i=1k(𝐗i−𝐌)𝐕−1(𝐗i−𝐌)T

and

𝐕=1kn∑i=1k(𝐗i−𝐌)T𝐔−1(𝐗i−𝐌),

See for example [3] and references therein. The covariance parameters are non-identifiable in the sense that for any scale factor, s>0, we have:

ℳ𝒩n×p(𝐗∣𝐌,𝐔,𝐕)=ℳ𝒩n×p(𝐗∣𝐌,s𝐔,1s𝐕).

## Drawing values from the distribution

Sampling from the matrix normal distribution is a special case of the sampling procedure for the multivariate normal distribution. Let 𝐗 be an n by p matrix of np independent samples from the standard normal distribution, so that

𝐗∼ℳ𝒩n×p(𝟎,𝐈,𝐈).

Then let

𝐘=𝐌+𝐀𝐗𝐁,

so that

𝐘∼ℳ𝒩n×p(𝐌,𝐀𝐀T,𝐁T𝐁),

where A and B can be chosen by Cholesky decomposition or a similar matrix square root operation.

## Relation to other distributions

Dawid (1981) provides a discussion of the relation of the matrix-valued normal distribution to other distributions, including the Wishart distribution, inverse-Wishart distribution and matrix t-distribution, but uses different notation from that employed here.

## See also

* Multivariate normal distribution

## References

1. ↑ A K Gupta; D K Nagar (22 October 1999). "Chapter 2: MATRIX VARIATE NORMAL DISTRIBUTION". Matrix Variate Distributions. CRC Press. ISBN 978-1-58488-046-2. https://books.google.com/books?id=PQOYnT7P1loC. Retrieved 23 May 2014.
2. ↑ Ding, Shanshan; R. Dennis Cook (2014). "Dimension folding PCA and PFC for matrix-valued predictors". Statistica Sinica 24 (1): 463–492.
3. ↑ Glanz, Hunter; Carvalho, Luis (2013). "An Expectation-Maximization Algorithm for the Matrix Normal Distribution". arXiv:1309.6609 [stat.ME].

* "Some matrix-variate distribution theory: Notational considerations and a Bayesian application". Biometrika 68 (1): 265-274. 1981. doi:10.1093/biomet/68.1.265.
* Dutilleul, P (1999). "The MLE algorithm for the matrix normal distribution". Journal of Statistical Computation and Simulation 64 (2): 105-123. doi:10.1080/00949659908811970.
* Arnold, S.F. (1981), The theory of linear models and multivariate analysis, New York: John Wiley & Sons, ISBN 0471050652

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/Matrix normal distribution. Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=Matrix_normal_distribution&oldid=3984146"