Empirical likelihood

Short description: Method of estimating statistical parameters

In probability theory and statistics, empirical likelihood (EL) is a nonparametric method for estimating the parameters of statistical models. It requires fewer assumptions about the error distribution while retaining some of the merits in likelihood-based inference. The estimation method requires that the data are independent and identically distributed (iid). It performs well even when the distribution is asymmetric or censored.[1] EL methods can also handle constraints and prior information on parameters. Art Owen pioneered work in this area with his 1988 paper.[2]

## Contents

* 1 Definition
* 1.1 Discrete random variables
* 2 Estimation Procedure
* 3 Empirical Likelihood Ratio (ELR)
* 4 Other Applications
* 5 See also
* 6 Literature
* 7 References

## Definition

Given a set of n i.i.d. realizations yi of random variables Yi, then the empirical distribution function is F^(y):=∑i=1nπiI(Yi<y), with the indicator function I and the (normalized) weights πi. Then, the empirical likelihood is:[3]

L:=∏i=1nF^(yi)−F^(yi−δy)δy,

where δy is a small number (potentially the difference to the next smaller sample).

Empirical likelihood estimation can be augmented with side information by using further constraints (similar to the generalized estimating equations approach) for the empirical distribution function. E.g. a constraint like the following can be incorporated using a Lagrange multiplier E[h(Y;θ)]=∫−∞∞h(y;θ)dF=0 which implies E^[h(y;θ)]=∑i=1nh(yi;θ)πi=0.

With similar constraints, we could also model correlation.

### Discrete random variables

The empirical-likelihood method can also be also employed for discrete distributions.[4] Given  pi:=F^(yi)−F^(yi−δy), i=1,...,n such that pi≥0 and ∑i=1n pi=1.

Then the empirical likelihood is again L(p1,...,pn)=∏i=1n pi.

Using the Lagrangian multiplier method to maximize the logarithm of the empirical likelihood subject to the trivial normalization constraint, we find pi=1/n as a maximum. Therefore, F^ is the empirical distribution function.

## Estimation Procedure

EL estimates are calculated by maximizing the empirical likelihood function (see above) subject to constraints based on the estimating function and the trivial assumption that the probability weights of the likelihood function sum to 1.[5] This procedure is represented as:

maxπi,θln⁡(L)=maxπi,θ∑i=1nln⁡πi

subject to the constraints

s.t.∑i=1nπi=1,∑i=1nπih(yi;θ)=0,∀i∈[1..n]0≤πi.[6]: Equation (73)

The value of the theta parameter can be found by solving the Lagrangian function

ℒ=∑i=1nln⁡πi+μ(1−∑i=1nπi)−nτ′∑i=1nπih(yi;θ).[6]: Equation (74)

There is a clear analogy between this maximization problem and the one solved for maximum entropy.

The parameters πi are nuisance parameters.

## Empirical Likelihood Ratio (ELR)

An empirical likelihood ratio function is defined and used to obtain confidence intervals parameter of interest θ similar to parametric likelihood ratio confidence intervals.[7][8] Let L(F) be the empirical likelihood of function F, then the ELR would be:

R(F)=L(F)/L(Fn).

Consider sets of the form

C={T(F)|R(F)≥r}.

Under such conditions a test of T(F)=t rejects when t does not belong to C, that is, when no distribution F with T(F)=t has likelihood L(F)≥rL(Fn).

The central result is for the mean of X. Clearly, some restrictions on F are needed, or else C=ℝp whenever r<1. To see this, let:

F=ϵδx+(1−ϵ)Fn

If ϵ is small enough and ϵ>0, then R(F)≥r.

But then, as x ranges through ℝp, so does the mean of F, tracing out C=ℝp. The problem can be solved by restricting to distributions F that are supported in a bounded set. It turns out to be possible to restrict attention t distributions with support in the sample, in other words, to distribution F≪Fn. Such method is convenient since the statistician might not be willing to specify a bounded support for F, and since t converts the construction of C into a finite dimensional problem.

## Other Applications

The use of empirical likelihood is not limited to confidence intervals. In efficient quantile regression, an EL-based categorization[9] procedure helps determine the shape of the true discrete distribution at level p, and also provides a way of formulating a consistent estimator. In addition, EL can be used in place of parametric likelihood to form model selection criteria.[10] Empirical likelihood can naturally be applied in survival analysis[11] or regression problems[12]

## See also

* Bootstrapping (statistics)
* Jackknife (statistics)
* Nelson–Aalen estimator
* Survival analysis

## Literature

* Nordman, Daniel J., and Soumendra N. Lahiri. "A review of empirical likelihood methods for time series." Journal of Statistical Planning and Inference 155 (2014): 1-18. https://doi.org/10.1016/j.jspi.2013.10.001

## References

1. ↑ Owen, Art B. (2001). Empirical likelihood. Boca Raton, Fla.. ISBN 978-1-4200-3615-2. OCLC 71012491.
2. ↑ Owen, Art B. (1988). "Empirical likelihood ratio confidence intervals for a single functional". Biometrika 75 (2): 237–249. doi:10.1093/biomet/75.2.237. ISSN 0006-3444. http://dx.doi.org/10.1093/biomet/75.2.237.
3. ↑ F^(yi)−F^(yi−δy)δy is an estimate of the probability density, compare histogram
4. ↑ Wang, Dong; Chen, Song Xi (2009-02-01). "Empirical likelihood for estimating equations with missing values". The Annals of Statistics 37 (1). doi:10.1214/07-aos585. ISSN 0090-5364. http://dx.doi.org/10.1214/07-aos585.
5. ↑ Mittelhammer, Judge, and Miller (2000), 292.
6. ↑ 6.0 6.1 Bera, Anil K.; Bilias, Yannis (2002). "The MM, ME, ML, EL, EF and GMM approaches to estimation: a synthesis". Journal of Econometrics 107 (1–2): 51–86. doi:10.1016/S0304-4076(01)00113-0.
7. ↑ Owen, Art (1990-03-01). "Empirical Likelihood Ratio Confidence Regions". The Annals of Statistics 18 (1). doi:10.1214/aos/1176347494. ISSN 0090-5364.
8. ↑ Dong, Lauren Bin; Giles, David E. A. (2007-01-30). "An Empirical Likelihood Ratio Test for Normality". Communications in Statistics - Simulation and Computation 36 (1): 197–215. doi:10.1080/03610910601096544. ISSN 0361-0918. http://dx.doi.org/10.1080/03610910601096544.
9. ↑ Chen, Jien; Lazar, Nicole A. (2010-01-27). "Quantile estimation for discrete data via empirical likelihood". Journal of Nonparametric Statistics 22 (2): 237–255. doi:10.1080/10485250903301525. ISSN 1048-5252. http://dx.doi.org/10.1080/10485250903301525.
10. ↑ Chen, Chixiang; Wang, Ming; Wu, Rongling; Li, Runze (2022). "A Robust Consistent Information Criterion for Model Selection Based on Empirical Likelihood". Statistica Sinica. doi:10.5705/ss.202020.0254. ISSN 1017-0405. http://dx.doi.org/10.5705/ss.202020.0254.
11. ↑ Zhou, M. (2015). Empirical Likelihood Method in Survival Analysis (1st ed.). Chapman and Hall/CRC. https://doi.org/10.1201/b18598
12. ↑ Chen, Song Xi, and Ingrid Van Keilegom. "A review on empirical likelihood methods for regression." TEST volume 18, pages 415–447, (2009) https://doi.org/10.1007/s11749-009-0159-5

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/Empirical likelihood. Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=Empirical_likelihood&oldid=4154518"