Distance correlation

Short description: Statistical measure

In statistics and in probability theory, distance correlation or distance covariance is a measure of dependence between two paired random vectors of arbitrary, not necessarily equal, dimension. The population distance correlation coefficient is zero if and only if the random vectors are independent. Thus, distance correlation measures both linear and nonlinear association between two random variables or random vectors. This is in contrast to Pearson's correlation, which can only detect linear association between two random variables.

Distance correlation can be used to perform a statistical test of dependence with a permutation test. One first computes the distance correlation (involving the re-centering of Euclidean distance matrices) between two random vectors, and then compares this value to the distance correlations of many shuffles of the data.

Several sets of (x, y) points, with the distance correlation coefficient of x and y for each set. Compare to the graph on correlation

## Contents

* 1 Background
* 2 Definitions
* 2.1 Distance covariance
* 2.2 Distance variance and distance standard deviation
* 2.3 Distance correlation
* 3 Properties
* 3.1 Distance correlation
* 3.2 Distance covariance
* 3.3 Distance variance
* 4 Generalization
* 5 Alternative definition of distance covariance
* 6 Alternative formulation: Brownian covariance
* 7 Related metrics
* 8 See also
* 9 Notes
* 10 References
* 11 External links

## Background

The classical measure of dependence, the Pearson correlation coefficient,[1] is mainly sensitive to a linear relationship between two variables. Distance correlation was introduced in 2005 by Gábor J. Székely in several lectures to address this deficiency of Pearson's correlation, namely that it can easily be zero for dependent variables. Correlation = 0 (uncorrelatedness) does not imply independence while distance correlation = 0 does imply independence. The first results on distance correlation were published in 2007 and 2009.[2][3] It was proved that distance covariance is the same as the Brownian covariance.[3] These measures are examples of energy distances.

The distance correlation is derived from a number of other quantities that are used in its specification, specifically: distance variance, distance standard deviation, and distance covariance. These quantities take the same roles as the ordinary moments with corresponding names in the specification of the Pearson product-moment correlation coefficient.

## Definitions

### Distance covariance

Let us start with the definition of the sample distance covariance. Let (Xk, Yk), k = 1, 2, ..., n be a statistical sample from a pair of real valued or vector valued random variables (X, Y). First, compute the n by n distance matrices (aj, k) and (bj, k) containing all pairwise distances

aj,k=‖Xj−Xk‖,j,k=1,2,…,n,bj,k=‖Yj−Yk‖,j,k=1,2,…,n,

where ||⋅ ||denotes Euclidean norm. Then take all doubly centered distances

Aj,k:=aj,k−a‾j⋅−a‾⋅k+a‾⋅⋅,Bj,k:=bj,k−b‾j⋅−b‾⋅k+b‾⋅⋅,

where a‾j⋅ is the j-th row mean, a‾⋅k is the k-th column mean, and a‾⋅⋅ is the grand mean of the distance matrix of the X sample. The notation is similar for the b values. (In the matrices of centered distances (Aj, k) and (Bj,k) all rows and all columns sum to zero.) The squared sample distance covariance (a scalar) is simply the arithmetic average of the products Aj, k Bj, k:

dCovn2(X,Y):=1n2∑j=1n∑k=1nAj,kBj,k.

The statistic Tn = n dCov2n(X, Y) determines a consistent multivariate test of independence of random vectors in arbitrary dimensions. For an implementation see dcov.test function in the energy package for R.[4]

The population value of distance covariance can be defined along the same lines. Let X be a random variable that takes values in a p-dimensional Euclidean space with probability distribution μ and let Y be a random variable that takes values in a q-dimensional Euclidean space with probability distribution ν, and suppose that X and Y have finite expectations. Write

aμ(x):=E⁡[‖X−x‖],D(μ):=E⁡[aμ(X)],dμ(x,x′):=‖x−x′‖−aμ(x)−aμ(x′)+D(μ).

Finally, define the population value of squared distance covariance of X and Y as

dCov2(X,Y):=E⁡[dμ(X,X′)dν(Y,Y′)].

One can show that this is equivalent to the following definition:

dCov2(X,Y):=E⁡[‖X−X′‖‖Y−Y′‖]+E⁡[‖X−X′‖]E⁡[‖Y−Y′‖]−E⁡[‖X−X′‖‖Y−Y″‖]−E⁡[‖X−X″‖‖Y−Y′‖]=E⁡[‖X−X′‖‖Y−Y′‖]+E⁡[‖X−X′‖]E⁡[‖Y−Y′‖]−2E⁡[‖X−X′‖‖Y−Y″‖],

where E denotes expected value, and (X,Y), (X′,Y′), and (X″,Y″) are independent and identically distributed. The primed random variables (X′,Y′) and (X″,Y″) denote independent and identically distributed (iid) copies of the variables X and Y and are similarly iid.[5] Distance covariance can be expressed in terms of the classical Pearson's covariance, cov, as follows:

dCov2(X,Y)=cov⁡(‖X−X′‖,‖Y−Y′‖)−2cov⁡(‖X−X′‖,‖Y−Y″‖).

This identity shows that the distance covariance is not the same as the covariance of distances, cov(||X − X' ||, ||Y − Y'  ||). This can be zero even if X and Y are not independent.

Alternatively, the distance covariance can be defined as the weighted L2 norm of the distance between the joint characteristic function of the random variables and the product of their marginal characteristic functions:[6]

dCov2(X,Y)=1cpcq∫ℝp+q|φX,Y(s,t)−φX(s)φY(t)|2|s|p1+p|t|q1+qdtds

where φX,Y(s,t), φX(s), and φY(t) are the characteristic functions of (X, Y), X, and Y, respectively, p, q denote the Euclidean dimension of X and Y, and thus of s and t, and cp, cq are constants. The weight function (cpcq|s|p1+p|t|q1+q)−1 is chosen to produce a scale equivariant and rotation invariant measure that doesn't go to zero for dependent variables.[6][7] One interpretation of the characteristic function definition is that the variables eisX and eitY are cyclic representations of X and Y with different periods given by s and t, and the expression ϕX, Y(s, t) − ϕX(s) ϕY(t) in the numerator of the characteristic function definition of distance covariance is simply the classical covariance of eisX and eitY. The characteristic function definition clearly shows that dCov2(X, Y) = 0 if and only if X and Y are independent.

### Distance variance and distance standard deviation

The distance variance is a special case of distance covariance when the two variables are identical. The population value of distance variance is the square root of

dVar2(X):=E⁡[‖X−X′‖2]+E2[‖X−X′‖]−2E⁡[‖X−X′‖‖X−X″‖],

where X, X′, and X″ are independent and identically distributed random variables, E denotes the expected value, and f2(⋅)=(f(⋅))2 for function f(⋅), e.g., E2[⋅]=(E⁡[⋅])2.

The sample distance variance is the square root of

dVarn2(X):=dCovn2(X,X)=1n2∑k,ℓAk,ℓ2,

which is a relative of Corrado Gini's mean difference introduced in 1912 (but Gini did not work with centered distances).[8]

The distance standard deviation is the square root of the distance variance.

### Distance correlation

The distance correlation [2][3] of two random variables is obtained by dividing their distance covariance by the product of their distance standard deviations. The distance correlation is the square root of

dCor2(X,Y)=dCov2(X,Y)dVar2(X)dVar2(Y),

and the sample distance correlation is defined by substituting the sample distance covariance and distance variances for the population coefficients above.

For easy computation of sample distance correlation see the dcor function in the energy package for R.[4]

## Properties

### Distance correlation

1. 0≤dCorn(X,Y)≤1 and 0≤dCor⁡(X,Y)≤1; this is in contrast to Pearson's correlation, which can be negative.
2. dCor⁡(X,Y)=0 if and only if X and Y are independent.
3. dCorn(X,Y)=1 implies that dimensions of the linear subspaces spanned by X and Y samples respectively are almost surely equal and if we assume that these subspaces are equal, then in this subspace Y=A+b𝐂X for some vector A, scalar b, and orthonormal matrix 𝐂.

### Distance covariance

1. dCov⁡(X,Y)≥0 and dCovn(X,Y)≥0;
2. dCov2(a1+b1𝐂1X,a2+b2𝐂2Y)=|b1b2|dCov2(X,Y) for all constant vectors a1,a2, scalars b1,b2, and orthonormal matrices 𝐂1,𝐂2.
3. If the random vectors (X1,Y1) and (X2,Y2) are independent then

dCov⁡(X1+X2,Y1+Y2)≤dCov⁡(X1,Y1)+dCov⁡(X2,Y2).
Equality holds if and only if X1 and Y1 are both constants, or X2 and Y2 are both constants, or X1,X2,Y1,Y2 are mutually independent.
4. dCov⁡(X,Y)=0 if and only if X and Y are independent.

This last property is the most important effect of working with centered distances.

The statistic dCovn2(X,Y) is a biased estimator of dCov2(X,Y). Under independence of X and Y [9]

E⁡[dCovn2(X,Y)]=n−1n2{(n−2)dCov2(X,Y)+E⁡[‖X−X′‖]E⁡[‖Y−Y′‖]}=n−1n2E⁡[‖X−X′‖]E⁡[‖Y−Y′‖].

An unbiased estimator of dCov2(X,Y) is given by Székely and Rizzo.[10]

### Distance variance

1. dVar⁡(X)=0 if and only if X=E⁡[X] almost surely.
2. dVarn(X)=0 if and only if every sample observation is identical.
3. dVar⁡(A+b𝐂X)=|b|dVar⁡(X) for all constant vectors A, scalars b, and orthonormal matrices 𝐂.
4. If X and Y are independent then dVar⁡(X+Y)≤dVar⁡(X)+dVar⁡(Y).

Equality holds in (iv) if and only if one of the random variables X or Y is a constant.

## Generalization

Distance covariance can be generalized to include powers of Euclidean distance. Define

dCov2(X,Y;α):=E⁡[‖X−X′‖α‖Y−Y′‖α]+E⁡[‖X−X′‖α]E⁡[‖Y−Y′‖α]−2E⁡[‖X−X′‖α‖Y−Y″‖α].

Then for every 0<α<2, X and Y are independent if and only if dCov2(X,Y;α)=0. It is important to note that this characterization does not hold for exponent α=2; in this case for bivariate (X,Y), dCor⁡(X,Y;α=2) is a deterministic function of the Pearson correlation.[2] If ak,ℓ and bk,ℓ are α powers of the corresponding distances, 0<α≤2, then α sample distance covariance can be defined as the nonnegative number for which

dCovn2(X,Y;α):=1n2∑k,ℓAk,ℓBk,ℓ.

One can extend dCov to metric-space-valued random variables X and Y: If X has law μ in a metric space with metric d, then define aμ(x):=E⁡[d(X,x)], D(μ):=E⁡[aμ(X)], and (provided aμ is finite, i.e., X has finite first moment), dμ(x,x′):=d(x,x′)−aμ(x)−aμ(x′)+D(μ). Then if Y has law ν (in a possibly different metric space with finite first moment), define

dCov2(X,Y):=E⁡[dμ(X,X′)dν(Y,Y′)].

This is non-negative for all such X,Y iff both metric spaces have negative type.[11] Here, a metric space (M,d) has negative type if (M,d1/2) is isometric to a subset of a Hilbert space.[12] If both metric spaces have strong negative type, then dCov2(X,Y)=0 iff X,Y are independent.[11]

## Alternative definition of distance covariance

The original distance covariance has been defined as the square root of dCov2(X,Y), rather than the squared coefficient itself. dCov⁡(X,Y) has the property that it is the energy distance between the joint distribution of X⁡,Y and the product of its marginals. Under this definition, however, the distance variance, rather than the distance standard deviation, is measured in the same units as the X distances.

Alternately, one could define distance covariance to be the square of the energy distance: dCov2(X,Y). In this case, the distance standard deviation of X is measured in the same units as X distance, and there exists an unbiased estimator for the population distance covariance.[10]

Under these alternate definitions, the distance correlation is also defined as the square dCor2(X,Y), rather than the square root.

## Alternative formulation: Brownian covariance

Brownian covariance is motivated by generalization of the notion of covariance to stochastic processes. The square of the covariance of random variables X and Y can be written in the following form:

cov⁡(X,Y)2=E⁡[(X−E⁡(X))(X'−E⁡(X'))(Y−E⁡(Y))(Y'−E⁡(Y'))]

where E denotes the expected value and the prime denotes independent and identically distributed copies. We need the following generalization of this formula. If U(s), V(t) are arbitrary random processes defined for all real s and t then define the U-centered version of X by

XU:=U(X)−EX[U(X)∣{U(t)}]

whenever the subtracted conditional expected value exists and denote by YV the V-centered version of Y.[3][13][14] The (U,V) covariance of (X,Y) is defined as the nonnegative number whose square is

covU,V2(X,Y):=E⁡[XUXU'YVYV']

whenever the right-hand side is nonnegative and finite. The most important example is when U and V are two-sided independent Brownian motions /Wiener processes with expectation zero and covariance |s| \+ |t| − |s − t| = 2 min(s,t) (for nonnegative s, t only). (This is twice the covariance of the standard Wiener process; here the factor 2 simplifies the computations.) In this case the (U,V) covariance is called Brownian covariance and is denoted by

covW(X,Y).

There is a surprising coincidence: The Brownian covariance is the same as the distance covariance:

covW(X,Y)=dCov⁡(X,Y),

and thus Brownian correlation is the same as distance correlation.

On the other hand, if we replace the Brownian motion with the deterministic identity function id then Covid(X,Y) is simply the absolute value of the classical Pearson covariance,

covid(X,Y)=|cov⁡(X,Y)|.

## Related metrics

Other correlational metrics, including kernel-based correlational metrics (such as the Hilbert-Schmidt Independence Criterion or HSIC) can also detect linear and nonlinear interactions. Both distance correlation and kernel-based metrics can be used in methods such as canonical correlation analysis and independent component analysis to yield stronger statistical power.

## See also

* RV coefficient
* For a related third-order statistic, see Distance skewness.

## Notes

1. ↑ Pearson 1895a, 1895b
2. ↑ 2.0 2.1 2.2 Székely, Rizzo & Bakirov 2007.
3. ↑ 3.0 3.1 3.2 3.3 Székely & Rizzo 2009a.
4. ↑ 4.0 4.1 Rizzo & Székely 2021.
5. ↑ Székely & Rizzo 2014, p. 11.
6. ↑ 6.0 6.1 Székely & Rizzo 2009a, p. 1249, Theorem 7, (3.7).
7. ↑ Székely & Rizzo 2012.
8. ↑ Gini 1912.
9. ↑ Székely & Rizzo 2009b.
10. ↑ 10.0 10.1 Székely & Rizzo 2014.
11. ↑ 11.0 11.1 Lyons 2014.
12. ↑ Klebanov 2005, p. [page needed].
13. ↑ Bickel & Xu 2009.
14. ↑ Kosorok 2009.

## References

* Bickel, Peter J.; Xu, Ying (2009). "Discussion of: Brownian distance covariance". The Annals of Applied Statistics 3 (4): 1266–1269. doi:10.1214/09-AOAS312A. http://projecteuclid.org/download/pdfview_1/euclid.aoas/1267453934.
* Gini, C. (1912). Variabilità e Mutabilità. Bologna: Tipografia di Paolo Cuppini. Bibcode: 1912vamu.book.....G.
* Klebanov, L. B. (2005). N-distances and their applications. Prague: Karolinum Press, Charles University. ISBN 9788024611525.
* Kosorok, Michael R. (2009). "Discussion of: Brownian distance covariance". The Annals of Applied Statistics 3 (4): 1270–1278. doi:10.1214/09-AOAS312B.
* Lyons, Russell (2014). "Distance covariance in metric spaces". The Annals of Probability 41 (5): 3284–3305. doi:10.1214/12-AOP803.
* Pearson, K. (1895a). "Note on regression and inheritance in the case of two parents". Proceedings of the Royal Society 58: 240–242. Bibcode: 1895RSPS...58..240P.
* Pearson, K. (1895b). "Notes on the history of correlation". Biometrika 13: 25–45. doi:10.1093/biomet/13.1.25. https://zenodo.org/record/1431597.
* Rizzo, Maria; Székely, Gábor (2021-02-22). "energy: E-Statistics: Multivariate Inference via the Energy of Data". https://cran.r-project.org/web/packages/energy/index.html.
* Székely, Gábor J.; Rizzo, Maria L.; Bakirov, Nail K. (2007). "Measuring and testing independence by correlation of distances". The Annals of Statistics 35 (6): 2769–2794. doi:10.1214/009053607000000505.
* Székely, Gábor J.; Rizzo, Maria L. (2009a). "Brownian distance covariance". The Annals of Applied Statistics 3 (4): 1236–1265. doi:10.1214/09-AOAS312. PMID 20574547. PMC 2889501. http://projecteuclid.org/download/pdfview_1/euclid.aoas/1267453933.
* Székely, Gábor J.; Rizzo, Maria L. (2009b). "Rejoinder: Brownian distance covariance". The Annals of Applied Statistics 3 (4): 1303–1308. doi:10.1214/09-AOAS312REJ.
* Székely, Gábor J.; Rizzo, Maria L. (2012). "On the uniqueness of distance covariance". Statistics & Probability Letters 82 (12): 2278–2282. doi:10.1016/j.spl.2012.08.007.
* Székely, Gabor J.; Rizzo, Maria L. (2014). "Partial Distance Correlation with Methods for Dissimilarities". The Annals of Statistics 42 (6): 2382–2412. doi:10.1214/14-AOS1255. Bibcode: 2014arXiv1310.2926S.

## External links

* E-statistics (energy statistics)

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/Distance correlation. Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=Distance_correlation&oldid=4227257"