Quadratic form (statistics)

In multivariate statistics, if ε is a vector of n random variables, and Λ is an n-dimensional symmetric matrix, then the scalar quantity εTΛε is known as a quadratic form in ε.

## Contents

* 1 Expectation
* 1.1 Proof
* 2 Variance in the Gaussian case
* 2.1 Computing the variance in the non-symmetric case
* 3 Examples of quadratic forms
* 4 See also
* 5 References

## Expectation

It can be shown that[1]

E⁡[εTΛε]=tr⁡[ΛΣ]+μTΛμ

where μ and Σ are the expected value and variance-covariance matrix of ε, respectively, and tr denotes the trace of a matrix. This result only depends on the existence of μ and Σ; in particular, normality of ε is not required.

A book treatment of the topic of quadratic forms in random variables is that of Mathai and Provost.[2]

### Proof

Since the quadratic form is a scalar quantity, εTΛε=tr⁡(εTΛε).

Next, by the cyclic property of the trace operator,

E⁡[tr⁡(εTΛε)]=E⁡[tr⁡(ΛεεT)].

Since the trace operator is a linear combination of the components of the matrix, it therefore follows from the linearity of the expectation operator that

E⁡[tr⁡(ΛεεT)]=tr⁡(ΛE⁡(εεT)).

A standard property of variances then tells us that this is

tr⁡(Λ(Σ+μμT)).

Applying the cyclic property of the trace operator again, we get

tr⁡(ΛΣ)+tr⁡(ΛμμT)=tr⁡(ΛΣ)+tr⁡(μTΛμ)=tr⁡(ΛΣ)+μTΛμ.

## Variance in the Gaussian case

In general, the variance of a quadratic form depends greatly on the distribution of ε. However, if ε does follow a multivariate normal distribution, the variance of the quadratic form becomes particularly tractable. Assume for the moment that Λ is a symmetric matrix. Then,

var⁡[εTΛε]=2tr⁡[ΛΣΛΣ]+4μTΛΣΛμ.[3]

In fact, this can be generalized to find the covariance between two quadratic forms on the same ε (once again, Λ1 and Λ2 must both be symmetric):

cov⁡[εTΛ1ε,εTΛ2ε]=2tr⁡[Λ1ΣΛ2Σ]+4μTΛ1ΣΛ2μ.[4]

In addition, a quadratic form such as this follows a generalized chi-squared distribution.

### Computing the variance in the non-symmetric case

The case for general Λ can be derived by noting that

εTΛTε=εTΛε

εTΛ~ε=εT(Λ+ΛT)ε/2

is a quadratic form in the symmetric matrix Λ~=(Λ+ΛT)/2, so the mean and variance expressions are the same, provided Λ is replaced by Λ~ therein.

## Examples of quadratic forms

In the setting where one has a set of observations y and an operator matrix H, then the residual sum of squares can be written as a quadratic form in y:

RSS=yT(I−H)T(I−H)y.

For procedures where the matrix H is symmetric and idempotent, and the errors are Gaussian with covariance matrix σ2I, RSS/σ2 has a chi-squared distribution with k degrees of freedom and noncentrality parameter λ, where

k=tr⁡[(I−H)T(I−H)]
λ=μT(I−H)T(I−H)μ/2

may be found by matching the first two central moments of a noncentral chi-squared random variable to the expressions given in the first two sections. If Hy estimates μ with no bias, then the noncentrality λ is zero and RSS/σ2 follows a central chi-squared distribution.

## See also

* Quadratic form
* Covariance matrix
* Matrix representation of conic sections

## References

1. ↑ Bates, Douglas. "Quadratic Forms of Random Variables". STAT 849 lectures. http://www.stat.wisc.edu/~st849-1/lectures/Ch02.pdf.
2. ↑ Mathai, A. M.; Provost, Serge B. (1992). Quadratic Forms in Random Variables. CRC Press. p. 424. ISBN 978-0824786915.
3. ↑ Rencher, Alvin C.; Schaalje, G. Bruce. (2008). Linear models in statistics (2nd ed.). Hoboken, N.J.: Wiley-Interscience. ISBN 9780471754985. OCLC 212120778.
4. ↑ Graybill, Franklin A.. Matrices with applications in statistics (2. ed.). Wadsworth: Belmont, Calif.. p. 367. ISBN 0534980384.

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/Quadratic form (statistics). Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=Quadratic_form_(statistics)&oldid=191721"