Edgeworth series

Short description: Infinite sum approximating a probability distribution in terms of its cumulants

In probability theory, the Gram–Charlier A series (named in honor of Jørgen Pedersen Gram and Carl Charlier), and the Edgeworth series (named in honor of Francis Ysidro Edgeworth) are series that approximate a probability distribution over the real line (−∞,∞) in terms of its cumulants.[1] The series are the same; but, the arrangement of terms (and thus the accuracy of truncating the series) differ.[2] The key idea of these expansions is to write the characteristic function of the distribution whose probability density function f is to be approximated in terms of the characteristic function of a distribution with known and suitable properties, and to recover f through the inverse Fourier transform.

## Contents

* 1 Gram–Charlier A series
* 2 The Edgeworth series
* 3 Illustration: density of the sample mean of three χ² distributions
* 4 Discussion of results
* 5 See also
* 6 References
* 7 Further reading

## Gram–Charlier A series

We examine a continuous random variable. Let f^ be the characteristic function of its distribution whose density function is f, and κr its cumulants. We expand in terms of a known distribution with probability density function ψ, characteristic function ψ^, and cumulants γr. The density ψ is generally chosen to be that of the normal distribution, but other choices are possible as well. By the definition of the cumulants, we have (see Wallace, 1958)[3]

f^(t)=exp⁡[∑r=1∞κr(it)rr!] and
ψ^(t)=exp⁡[∑r=1∞γr(it)rr!],

which gives the following formal identity:

f^(t)=exp⁡[∑r=1∞(κr−γr)(it)rr!]ψ^(t).

By the properties of the Fourier transform, (it)rψ^(t) is the Fourier transform of (−1)r[Drψ](−x), where D is the differential operator with respect to x. Thus, after changing x with −x on both sides of the equation, we find for f the formal expansion

f(x)=exp⁡[∑r=1∞(κr−γr)(−D)rr!]ψ(x).

If ψ is chosen as the normal density

ϕ(x)=12πσexp⁡[−(x−μ)22σ2]

with mean and variance as given by f, that is, mean μ=κ1 and variance σ2=κ2, then the expansion becomes

f(x)=exp⁡[∑r=3∞κr(−D)rr!]ϕ(x),

since γr=0 for all r > 2, as higher cumulants of the normal distribution are 0. By expanding the exponential and collecting terms according to the order of the derivatives, we arrive at the Gram–Charlier A series. Such an expansion can be written compactly in terms of Bell polynomials as

exp⁡[∑r=3∞κr(−D)rr!]=∑n=0∞Bn(0,0,κ3,…,κn)(−D)nn!.

Since the n-th derivative of the Gaussian function ϕ is given in terms of Hermite polynomial as

ϕ(n)(x)=(−1)nσnHen(x−μσ)ϕ(x),

this gives us the final expression of the Gram–Charlier A series as

f(x)=ϕ(x)∑n=0∞1n!σnBn(0,0,κ3,…,κn)Hen(x−μσ).

Integrating the series gives us the cumulative distribution function

F(x)=∫−∞xf(u)du=Φ(x)−ϕ(x)∑n=3∞1n!σn−1Bn(0,0,κ3,…,κn)Hen−1(x−μσ),

where Φ is the CDF of the normal distribution.

If we include only the first two correction terms to the normal distribution, we obtain

f(x)≈12πσexp⁡[−(x−μ)22σ2][1+κ33!σ3He3(x−μσ)+κ44!σ4He4(x−μσ)],

with He3(x)=x3−3x and He4(x)=x4−6x2+3.

Note that this expression is not guaranteed to be positive, and is therefore not a valid probability distribution. The Gram–Charlier A series diverges in many cases of interest—it converges only if f(x) falls off faster than exp⁡(−(x2)/4) at infinity (Cramér 1957). When it does not converge, the series is also not a true asymptotic expansion, because it is not possible to estimate the error of the expansion. For this reason, the Edgeworth series (see next section) is generally preferred over the Gram–Charlier A series.

## The Edgeworth series

Edgeworth developed a similar expansion as an improvement to the central limit theorem.[4] The advantage of the Edgeworth series is that the error is controlled, so that it is a true asymptotic expansion.

Let {Zi} be a sequence of independent and identically distributed random variables with finite mean μ and variance σ2, and let Xn be their standardized sums:

Xn=1n∑i=1nZi−μσ.

Let Fn denote the cumulative distribution functions of the variables Xn. Then by the central limit theorem,

limn->∞Fn(x)=Φ(x)≡∫−∞x12πe−12q2dq

for every x, as long as the mean and variance are finite.

The standardization of {Zi} ensures that the first two cumulants of Xn are κ1Fn=0 and κ2Fn=1. Now assume that, in addition to having mean μ and variance σ2, the i.i.d. random variables Zi have higher cumulants κr. From the additivity and homogeneity properties of cumulants, the cumulants of Xn in terms of the cumulants of Zi are for r≥2,

κrFn=nκrσrnr/2=λrnr/2−1whereλr=κrσr.

If we expand the formal expression of the characteristic function f^n(t) of Fn in terms of the standard normal distribution, that is, if we set

ϕ(x)=12πexp⁡(−12x2),

then the cumulant differences in the expansion are

κ1Fn−γ1=0,
κ2Fn−γ2=0,
κrFn−γr=λrnr/2−1;r≥3.

The Gram–Charlier A series for the density function of Xn is now

fn(x)=ϕ(x)∑r=0∞1r!Br(0,0,λ3n1/2,…,λrnr/2−1)Her(x).

The Edgeworth series is developed similarly to the Gram–Charlier A series, only that now terms are collected according to powers of n. The coefficients of n−m/2 term can be obtained by collecting the monomials of the Bell polynomials corresponding to the integer partitions of m. Thus, we have the characteristic function as

f^n(t)=[1+∑j=1∞Pj(it)nj/2]exp⁡(−t2/2),

where Pj(x) is a polynomial of degree 3j. Again, after inverse Fourier transform, the density function fn follows as

fn(x)=ϕ(x)+∑j=1∞Pj(−D)nj/2ϕ(x).

Likewise, integrating the series, we obtain the distribution function

Fn(x)=Φ(x)+∑j=1∞1nj/2Pj(−D)Dϕ(x).

We can explicitly write the polynomial Pm(−D) as

Pm(−D)=∑∏i1ki!(λlili!)ki(−D)s,

where the summation is over all the integer partitions of m such that ∑iiki=m and li=i+2 and s=∑ikili.

For example, if m = 3, then there are three ways to partition this number: 1 + 1 + 1 = 2 + 1 = 3. As such we need to examine three cases:

* 1 + 1 + 1 = 1 · k1, so we have k1 = 3, l1 = 3, and s = 9.
* 1 + 2 = 1 · k1 \+ 2 · k2, so we have k1 = 1, k2 = 1, l1 = 3, l2 = 4, and s = 7.
* 3 = 3 · k3, so we have k3 = 1, l3 = 5, and s = 5.

Thus, the required polynomial is

P3(−D)=13!(λ33!)3(−D)9+11!1!(λ33!)(λ44!)(−D)7+11!(λ55!)(−D)5=λ331296(−D)9+λ3λ4144(−D)7+λ5120(−D)5.

The first five terms of the expansion are[5]

fn(x)=ϕ(x)−n−12(16λ3ϕ(3)(x))+n−1(124λ4ϕ(4)(x)+172λ32ϕ(6)(x))−n−32(1120λ5ϕ(5)(x)+1144λ3λ4ϕ(7)(x)+11296λ33ϕ(9)(x))+n−2(1720λ6ϕ(6)(x)+(11152λ42+1720λ3λ5)ϕ(8)(x)+11728λ32λ4ϕ(10)(x)+131104λ34ϕ(12)(x))+O(n−52).

Here, φ(j)(x) is the j-th derivative of φ(·) at point x. Remembering that the derivatives of the density of the normal distribution are related to the normal density by ϕ(n)(x)=(−1)nHen(x)ϕ(x), (where Hen is the Hermite polynomial of order n), this explains the alternative representations in terms of the density function. Blinnikov and Moessner (1998) have given a simple algorithm to calculate higher-order terms of the expansion.

Note that in case of a lattice distributions (which have discrete values), the Edgeworth expansion must be adjusted to account for the discontinuous jumps between lattice points.[6]

## Illustration: density of the sample mean of three χ² distributions

Density of the sample mean of three chi2 variables. The chart compares the true density, the normal approximation, and two Edgeworth expansions.

Take Xi∼χ2(k=2),i=1,2,3(n=3) and the sample mean X¯=13∑i=13Xi.

We can use several distributions for X¯:

* The exact distribution, which follows a gamma distribution: X¯∼Gamma(α=n⋅k/2,θ=2/n)=Gamma(α=3,θ=2/3).
* The asymptotic normal distribution: X¯→n->∞N(k,2⋅k/n)=N(2,4/3).
* Two Edgeworth expansions, of degrees 2 and 3.

## Discussion of results

* For finite samples, an Edgeworth expansion is not guaranteed to be a proper probability distribution as the CDF values at some points may go beyond [0,1].
* They guarantee (asymptotically) absolute errors, but relative errors can be easily assessed by comparing the leading Edgeworth term in the remainder with the overall leading term.[2]

## See also

* Cornish–Fisher expansion
* Edgeworth binomial tree

## References

1. ↑ Stuart, A., & Kendall, M. G. (1968). The advanced theory of statistics. Hafner Publishing Company.
2. ↑ 2.0 2.1 Kolassa, John E. (2006). Series approximation methods in statistics (3rd ed.). Springer. ISBN 0387322272.
3. ↑ Wallace, D. L. (1958). "Asymptotic Approximations to Distributions". Annals of Mathematical Statistics 29 (3): 635–654. doi:10.1214/aoms/1177706528.
4. ↑ Hall, P. (2013). The bootstrap and Edgeworth expansion. Springer Science & Business Media.
5. ↑ Weisstein, Eric W.. "Edgeworth Series". http://mathworld.wolfram.com/EdgeworthSeries.html.
6. ↑ Kolassa, John E.; McCullagh, Peter (1990). "Edgeworth series for lattice distributions". Annals of Statistics 18 (2): 981–985. doi:10.1214/aos/1176347637.

## Further reading

* Hazewinkel, Michiel, ed. (2001), "Edgeworth series", Encyclopedia of Mathematics, Springer Science+Business Media B.V. / Kluwer Academic Publishers, ISBN 978-1-55608-010-4, https://www.encyclopediaofmath.org/index.php?title=p/e035060
* Blinnikov, S.; Moessner, R. (1998). "Expansions for nearly Gaussian distributions". Astronomy and Astrophysics Supplement Series 130: 193–205. doi:10.1051/aas:1998221. Bibcode: 1998A&AS..130..193B. http://aas.aanda.org/articles/aas/pdf/1998/10/h0596.pdf.
* Cox, D. R.; Barndorff-Nielsen, O. E. (1989). Asymptotic Techniques for Use in Statistics. London: Chapman and Hall.
* Cramér, H. (1957). Mathematical Methods of Statistics. Princeton: Princeton University Press.
* Hall, P. (1992). The Bootstrap and Edgeworth Expansion. New York: Springer.
* Kendall, M.; Stuart, A. (1977). The advanced theory of statistics. 1: Distribution theory (4th ed.). New York: Macmillan.
* Kolassa, J. E. (2006). Series Approximation Methods in Statistics. Lecture Notes in Statistics. 88 (3rd ed.). New York: Springer.
* Martin, Douglas; Arora, Rohit (2017). "Inefficiency and bias of modified value-at-risk and expected shortfall". Journal of Risk 19 (6): 59–84. doi:10.21314/JOR.2017.365.
* McCullagh, P. (1987). Tensor Methods in Statistics. London: Chapman and Hall.
* Tan, W. Y. (1980). "On Approximating Multivariate Distributions". in Gupta, R. P.. Multivariate Statistical Analysis. North-Holland. pp. 237–249. ISBN 0-444-86019-3.
* Wallace, D. L. (1958). "Asymptotic approximations to distributions". Annals of Mathematical Statistics 29 (3): 635–654. doi:10.1214/aoms/1177706528.

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/Edgeworth series. Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=Edgeworth_series&oldid=3984023"