Empirical measure

Short description: Random measure in probability theory

This article includes a list of references, but its sources remain unclear because it has insufficient inline citations. Please help to improve this article by introducing more precise citations. (March 2011) (Learn how and when to remove this template message)

|

In probability theory, an empirical measure is a random measure arising from a particular realization of a (usually finite) sequence of random variables. The precise definition is found below. Empirical measures are relevant to mathematical statistics.

The motivation for studying empirical measures is that it is often impossible to know the true underlying probability measure P. We collect observations X1,X2,…,Xn and compute relative frequencies. We can estimate P, or a related distribution function F by means of the empirical measure or empirical distribution function, respectively. These are uniformly good estimates under certain conditions. Theorems in the area of empirical processes provide rates of this convergence.

## Contents

* 1 Definition
* 2 Empirical distribution function
* 3 See also
* 4 References
* 5 Further reading

## Definition

Let X1,X2,… be a sequence of independent identically distributed random variables with values in the state space S with probability distribution P.

Definition

The empirical measure Pn is defined for measurable subsets of S and given by

Pn(A)=1n∑i=1nIA(Xi)=1n∑i=1nδXi(A)
where IA is the indicator function and δX is the Dirac measure.

Properties

* For a fixed measurable set A, nPn(A) is a binomial random variable with mean nP(A) and variance nP(A)(1 − P(A)).
* In particular, Pn(A) is an unbiased estimator of P(A).
* For a fixed partition Ai of S, random variables Yi=nPn(Ai) form a multinomial distribution with event probabilities P(Ai)
* The covariance matrix of this multinomial distribution is Cov(Yi,Yj)=nP(Ai)(δij−P(Aj)).

Definition

(Pn(c))c∈𝒞 is the empirical measure indexed by 𝒞, a collection of measurable subsets of S.

To generalize this notion further, observe that the empirical measure Pn maps measurable functions f:S->ℝ to their empirical mean,

f↦Pnf=∫SfdPn=1n∑i=1nf(Xi)

In particular, the empirical measure of A is simply the empirical mean of the indicator function, Pn(A) = Pn IA.

For a fixed measurable function f, Pnf is a random variable with mean 𝔼f and variance 1n𝔼(f−𝔼f)2.

By the strong law of large numbers, Pn(A) converges to P(A) almost surely for fixed A. Similarly Pnf converges to 𝔼f almost surely for a fixed measurable function f. The problem of uniform convergence of Pn to P was open until Vapnik and Chervonenkis solved it in 1968.[1]

If the class 𝒞 (or ℱ) is Glivenko–Cantelli with respect to P then Pn converges to P uniformly over c∈𝒞 (or f∈ℱ). In other words, with probability 1 we have

‖Pn−P‖𝒞=supc∈𝒞|Pn(c)−P(c)|->0,
‖Pn−P‖ℱ=supf∈ℱ|Pnf−𝔼f|->0.

## Empirical distribution function

Main article: Empirical distribution function

The empirical distribution function provides an example of empirical measures. For real-valued iid random variables X1,…,Xn it is given by

Fn(x)=Pn((−∞,x])=PnI(−∞,x].

In this case, empirical measures are indexed by a class 𝒞={(−∞,x]:x∈ℝ}. It has been shown that 𝒞 is a uniform Glivenko–Cantelli class, in particular,

supF‖Fn(x)−F(x)‖∞->0

with probability 1.

## See also

* Empirical risk minimization
* Poisson random measure

## References

1. ↑ Vapnik, V.; Chervonenkis, A (1968). "Uniform convergence of frequencies of occurrence of events to their probabilities". Dokl. Akad. Nauk SSSR 181.

## Further reading

* Billingsley, P. (1995). Probability and Measure (Third ed.). New York: John Wiley and Sons. ISBN 0-471-80478-9.
* Donsker, M. D. (1952). "Justification and extension of Doob's heuristic approach to the Kolmogorov–Smirnov theorems". Annals of Mathematical Statistics 23 (2): 277–281. doi:10.1214/aoms/1177729445.
* Dudley, R. M. (1978). "Central limit theorems for empirical measures". Annals of Probability 6 (6): 899–929. doi:10.1214/aop/1176995384.
* Dudley, R. M. (1999). Uniform Central Limit Theorems. Cambridge Studies in Advanced Mathematics. 63. Cambridge, UK: Cambridge University Press. ISBN 0-521-46102-2.
* Wolfowitz, J. (1954). "Generalization of the theorem of Glivenko–Cantelli". Annals of Mathematical Statistics 25 (1): 131–138. doi:10.1214/aoms/1177728852.

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/Empirical measure. Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=Empirical_measure&oldid=4229398"