Sufficient statistic

Short description: Statistical principle

In statistics, sufficiency is a property of a statistic computed on a sample dataset in relation to a parametric model of the dataset. A sufficient statistic contains all of the information that the dataset provides about the model parameters. It is closely related to the concepts of an ancillary statistic which contains no information about the model parameters, and of a complete statistic which only contains information about the parameters and no ancillary information.

A related concept is that of linear sufficiency, which is weaker than sufficiency but can be applied in some cases where there is no sufficient statistic, although it is restricted to linear estimators.[1] The Kolmogorov structure function deals with individual finite data; the related notion there is the algorithmic sufficient statistic.

The concept is due to Sir Ronald Fisher in 1920.[2] Stephen Stigler noted in 1973 that the concept of sufficiency had fallen out of favor in descriptive statistics because of the strong dependence on an assumption of the distributional form (see Pitman–Koopman–Darmois theorem below), but remained very important in theoretical work.[3]

## Contents

* 1 Background
* 2 Mathematical definition
* 2.1 Example
* 3 Fisher–Neyman factorization theorem
* 3.1 Likelihood principle interpretation
* 3.2 Proof
* 3.3 Another proof
* 4 Minimal sufficiency
* 5 Examples
* 5.1 Bernoulli distribution
* 5.2 Uniform distribution
* 5.3 Uniform distribution (with two parameters)
* 5.4 Poisson distribution
* 5.5 Normal distribution
* 5.6 Exponential distribution
* 5.7 Gamma distribution
* 6 Rao–Blackwell theorem
* 7 Exponential family
* 8 Other types of sufficiency
* 8.1 Bayesian sufficiency
* 8.2 Linear sufficiency
* 9 See also
* 10 Notes
* 11 References

## Background

Roughly, given a set 𝐗 of independent identically distributed data conditioned on an unknown parameter θ, a sufficient statistic is a function T(𝐗) whose value contains all the information needed to compute any estimate of the parameter (e.g. a maximum likelihood estimate). Due to the factorization theorem (see below), for a sufficient statistic T(𝐗), the probability density can be written as f𝐗(x;θ)=h(x)g(θ,T(x)). From this factorization, it can easily be seen that the maximum likelihood estimate of θ will interact with 𝐗 only through T(𝐗). Typically, the sufficient statistic is a simple function of the data, e.g. the sum of all the data points.

More generally, the "unknown parameter" may represent a vector of unknown quantities or may represent everything about the model that is unknown or not fully specified. In such a case, the sufficient statistic may be a set of functions, called a jointly sufficient statistic. Typically, there are as many functions as there are parameters. For example, for a Gaussian distribution with unknown mean and variance, the jointly sufficient statistic, from which maximum likelihood estimates of both parameters can be estimated, consists of two functions, the sum of all data points and the sum of all squared data points (or equivalently, the sample mean and sample variance).

In other words, the joint probability distribution of the data is conditionally independent of the parameter given the value of the sufficient statistic for the parameter. Both the statistic and the underlying parameter can be vectors.

## Mathematical definition

A statistic t = T(X) is sufficient for underlying parameter θ precisely if the conditional probability distribution of the data X, given the statistic t = T(X), does not depend on the parameter θ.[4]

Alternatively, one can say the statistic T(X) is sufficient for θ if, for all prior distributions on θ, the mutual information between θ and T(X) equals the mutual information between θ and X.[5] In other words, the data processing inequality becomes an equality:

I(θ;T(X))=I(θ;X)

### Example

As an example, the sample mean is sufficient for the (unknown) mean μ of a normal distribution with known variance. Once the sample mean is known, no further information about μ can be obtained from the sample itself. On the other hand, for an arbitrary distribution the median is not sufficient for the mean: even if the median of the sample is known, knowing the sample itself would provide further information about the population mean. For example, if the observations that are less than the median are only slightly less, but observations exceeding the median exceed it by a large amount, then this would have a bearing on one's inference about the population mean.

## Fisher–Neyman factorization theorem

Fisher's factorization theorem or factorization criterion provides a convenient characterization of a sufficient statistic. If the probability density function is ƒθ(x), then T is sufficient for θ if and only if nonnegative functions g and h can be found such that

f(x;θ)=h(x)g(θ,T(x)),

i.e., the density ƒ can be factored into a product such that one factor, h, does not depend on θ and the other factor, which does depend on θ, depends on x only through T(x). A general proof of this was given by Halmos and Savage[6] and the theorem is sometimes referred to as the Halmos–Savage factorization theorem.[7] The proofs below handle special cases, but an alternative general proof along the same lines can be given.[8] In many simple cases the probability density function is fully specified by θ and T(x), and h(x)=1 (see Examples).

It is easy to see that if F(t) is a one-to-one function and T is a sufficient statistic, then F(T) is a sufficient statistic. In particular we can multiply a sufficient statistic by a nonzero constant and get another sufficient statistic.

### Likelihood principle interpretation

An implication of the theorem is that when using likelihood-based inference, two sets of data yielding the same value for the sufficient statistic T(X) will always yield the same inferences about θ. By the factorization criterion, the likelihood's dependence on θ is only in conjunction with T(X). As this is the same in both cases, the dependence on θ will be the same as well, leading to identical inferences.

### Proof

Due to Hogg and Craig.[9] Let X1,X2,…,Xn, denote a random sample from a distribution having the pdf f(x, θ) for ι < θ < δ. Let Y1 = u1(X1, X2, ..., Xn) be a statistic whose pdf is g1(y1; θ). What we want to prove is that Y1 = u1(X1, X2, ..., Xn) is a sufficient statistic for θ if and only if, for some function H,

∏i=1nf(xi;θ)=g1[u1(x1,x2,…,xn);θ]H(x1,x2,…,xn).

First, suppose that

∏i=1nf(xi;θ)=g1[u1(x1,x2,…,xn);θ]H(x1,x2,…,xn).

We shall make the transformation yi = ui(x1, x2, ..., xn), for i = 1, ..., n, having inverse functions xi = wi(y1, y2, ..., yn), for i = 1, ..., n, and Jacobian J=[wi/yj]. Thus,

∏i=1nf[wi(y1,y2,…,yn);θ]=|J|g1(y1;θ)H[w1(y1,y2,…,yn),…,wn(y1,y2,…,yn)].

The left-hand member is the joint pdf g(y1, y2, ..., yn; θ) of Y1 = u1(X1, ..., Xn), ..., Yn = un(X1, ..., Xn). In the right-hand member, g1(y1;θ) is the pdf of Y1, so that H[w1,…,wn]|J| is the quotient of g(y1,…,yn;θ) and g1(y1;θ); that is, it is the conditional pdf h(y2,…,yn∣y1;θ) of Y2,…,Yn given Y1=y1.

But H(x1,x2,…,xn), and thus H[w1(y1,…,yn),…,wn(y1,…,yn))], was given not to depend upon θ. Since θ was not introduced in the transformation and accordingly not in the Jacobian J, it follows that h(y2,…,yn∣y1;θ) does not depend upon θ and that Y1 is a sufficient statistics for θ.

The converse is proven by taking:

g(y1,…,yn;θ)=g1(y1;θ)h(y2,…,yn∣y1),

where h(y2,…,yn∣y1) does not depend upon θ because Y2...Yn depend only upon X1...Xn, which are independent on Θ when conditioned by Y1, a sufficient statistics by hypothesis. Now divide both members by the absolute value of the non-vanishing Jacobian J, and replace y1,…,yn by the functions u1(x1,…,xn),…,un(x1,…,xn) in x1,…,xn. This yields

g[u1(x1,…,xn),…,un(x1,…,xn);θ]|J*|=g1[u1(x1,…,xn);θ]h(u2,…,un∣u1)|J*|

where J* is the Jacobian with y1,…,yn replaced by their value in terms x1,…,xn. The left-hand member is necessarily the joint pdf f(x1;θ)⋯f(xn;θ) of X1,…,Xn. Since h(y2,…,yn∣y1), and thus h(u2,…,un∣u1), does not depend upon θ, then

H(x1,…,xn)=h(u2,…,un∣u1)|J*|

is a function that does not depend upon θ.

### Another proof

A simpler more illustrative proof is as follows, although it applies only in the discrete case.

We use the shorthand notation to denote the joint probability density of (X,T(X)) by fθ(x,t). Since T is a deterministic function of X, we have fθ(x,t)=fθ(x), as long as t=T(x) and zero otherwise. Therefore:

fθ(x)=fθ(x,t)=fθ(x∣t)fθ(t)=f(x∣t)fθ(t)

with the last equality being true by the definition of sufficient statistics. Thus fθ(x)=a(x)bθ(t) with a(x)=fX∣t(x) and bθ(t)=fθ(t).

Conversely, if fθ(x)=a(x)bθ(t), we have

fθ(t)=∑x:T(x)=tfθ(x,t)=∑x:T(x)=tfθ(x)=∑x:T(x)=ta(x)bθ(t)=(∑x:T(x)=ta(x))bθ(t).

With the first equality by the definition of pdf for multiple variables, the second by the remark above, the third by hypothesis, and the fourth because the summation is not over t.

Let fX∣t(x) denote the conditional probability density of X given T(X). Then we can derive an explicit expression for this:

fX∣t(x)=fθ(x,t)fθ(t)=fθ(x)fθ(t)=a(x)bθ(t)(∑x:T(x)=ta(x))bθ(t)=a(x)∑x:T(x)=ta(x).

With the first equality by definition of conditional probability density, the second by the remark above, the third by the equality proven above, and the fourth by simplification. This expression does not depend on θ and thus T is a sufficient statistic.[10]

## Minimal sufficiency

A sufficient statistic is minimal sufficient if it can be represented as a function of any other sufficient statistic. In other words, S(X) is minimal sufficient if and only if[11]

1. S(X) is sufficient, and
2. if T(X) is sufficient, then there exists a function f such that S(X) = f(T(X)).

Intuitively, a minimal sufficient statistic most efficiently captures all possible information about the parameter θ.

A useful characterization of minimal sufficiency is that when the density fθ exists, S(X) is minimal sufficient if

fθ(x)fθ(y) is independent of θ :⟺ S(x) = S(y)

This follows as a consequence from Fisher's factorization theorem stated above.

A case in which there is no minimal sufficient statistic was shown by Bahadur, 1954.[12] However, under mild conditions, a minimal sufficient statistic does always exist. In particular, in Euclidean space, these conditions always hold if the random variables (associated with Pθ ) are all discrete or are all continuous.

If there exists a minimal sufficient statistic, and this is usually the case, then every complete sufficient statistic is necessarily minimal sufficient[13] (note that this statement does not exclude a pathological case in which a complete sufficient exists while there is no minimal sufficient statistic). While it is hard to find cases in which a minimal sufficient statistic does not exist, it is not so hard to find cases in which there is no complete sufficient statistic.

The collection of likelihood ratios {L(X∣θi)L(X∣θ0)} for i=1,...,k, is a minimal sufficient statistic if the parameter space is discrete {θ0,...,θk}.

## Examples

### Bernoulli distribution

If X1, ...., Xn are independent Bernoulli-distributed random variables with expected value p, then the sum T(X) = X1 + ... + Xn is a sufficient statistic for p (here 'success' corresponds to Xi = 1 and 'failure' to Xi = 0; so T is the total number of successes)

This is seen by considering the joint probability distribution:

Pr⁡{X=x}=Pr⁡{X1=x1,X2=x2,…,Xn=xn}.

Because the observations are independent, this can be written as

px1(1−p)1−x1px2(1−p)1−x2⋯pxn(1−p)1−xn

and, collecting powers of p and 1 − p, gives

p∑xi(1−p)n−∑xi=pT(x)(1−p)n−T(x)

which satisfies the factorization criterion, with h(x) = 1 being just a constant.

Note the crucial feature: the unknown parameter p interacts with the data x only via the statistic T(x) = Σ xi.

As a concrete application, this gives a procedure for distinguishing a fair coin from a biased coin.

### Uniform distribution

See also: German tank problem

If X1, ...., Xn are independent and uniformly distributed on the interval [0,θ], then T(X) = max(X1, ..., Xn) is sufficient for θ — the sample maximum is a sufficient statistic for the population maximum.

To see this, consider the joint probability density function of X (X1,...,Xn). Because the observations are independent, the pdf can be written as a product of individual densities

fθ(x1,…,xn)=1θ𝟏{0≤x1≤θ}⋯1θ𝟏{0≤xn≤θ}=1θn𝟏{0≤min⁡{xi}}𝟏{max⁡{xi}≤θ}

where 1{...} is the indicator function. Thus the density takes form required by the Fisher–Neyman factorization theorem, where h(x) = 1{min{xi}≥0}, and the rest of the expression is a function of only θ and T(x) = max{xi}.

In fact, the minimum-variance unbiased estimator (MVUE) for θ is

n+1nT(X).

This is the sample maximum, scaled to correct for the bias, and is MVUE by the Lehmann–Scheffé theorem. Unscaled sample maximum T(X) is the maximum likelihood estimator for θ.

### Uniform distribution (with two parameters)

If X1,...,Xn are independent and uniformly distributed on the interval [α,β] (where α and β are unknown parameters), then T(X1n)=(min1≤i≤nXi,max1≤i≤nXi) is a two-dimensional sufficient statistic for (α,β).

To see this, consider the joint probability density function of X1n=(X1,…,Xn). Because the observations are independent, the pdf can be written as a product of individual densities, i.e.

fX1n(x1n)=∏i=1n(1β−α)𝟏{α≤xi≤β}=(1β−α)n𝟏{α≤xi≤β,∀i=1,…,n}=(1β−α)n𝟏{α≤min1≤i≤nXi}𝟏{max1≤i≤nXi≤β}.

The joint density of the sample takes the form required by the Fisher–Neyman factorization theorem, by letting

h(x1n)=1,g(α,β)(x1n)=(1β−α)n𝟏{α≤min1≤i≤nXi}𝟏{max1≤i≤nXi≤β}.

Since h(x1n) does not depend on the parameter (α,β) and g(α,β)(x1n) depends only on x1n through the function T(X1n)=(min1≤i≤nXi,max1≤i≤nXi),

the Fisher–Neyman factorization theorem implies T(X1n)=(min1≤i≤nXi,max1≤i≤nXi) is a sufficient statistic for (α,β).

### Poisson distribution

If X1, ...., Xn are independent and have a Poisson distribution with parameter λ, then the sum T(X) = X1 + ... + Xn is a sufficient statistic for λ.

To see this, consider the joint probability distribution:

Pr⁡(X=x)=P(X1=x1,X2=x2,…,Xn=xn).

Because the observations are independent, this can be written as

e−λλx1x1!⋅e−λλx2x2!⋯e−λλxnxn!

which may be written as

e−nλλ(x1+x2+⋯+xn)⋅1x1!x2!⋯xn!

which shows that the factorization criterion is satisfied, where h(x) is the reciprocal of the product of the factorials. Note the parameter λ interacts with the data only through its sum T(X).

### Normal distribution

If X1,…,Xn are independent and normally distributed with expected value θ (a parameter) and known finite variance σ2, then

T(X1n)=x‾=1n∑i=1nXi

is a sufficient statistic for θ.

To see this, consider the joint probability density function of X1n=(X1,…,Xn). Because the observations are independent, the pdf can be written as a product of individual densities, i.e.

fX1n(x1n)=∏i=1n12πσ2exp⁡(−(xi−θ)22σ2)[6pt]=(2πσ2)−n2exp⁡(−∑i=1n(xi−θ)22σ2)[6pt]=(2πσ2)−n2exp⁡(−∑i=1n((xi−x‾)−(θ−x‾))22σ2)[6pt]=(2πσ2)−n2exp⁡(−12σ2(∑i=1n(xi−x‾)2+∑i=1n(θ−x‾)2−2∑i=1n(xi−x‾)(θ−x‾)))[6pt]=(2πσ2)−n2exp⁡(−12σ2(∑i=1n(xi−x‾)2+n(θ−x‾)2))∑i=1n(xi−x‾)(θ−x‾)=0[6pt]=(2πσ2)−n2exp⁡(−12σ2∑i=1n(xi−x‾)2)exp⁡(−n2σ2(θ−x‾)2)

The joint density of the sample takes the form required by the Fisher–Neyman factorization theorem, by letting

h(x1n)=(2πσ2)−n2exp⁡(−12σ2∑i=1n(xi−x‾)2)gθ(x1n)=exp⁡(−n2σ2(θ−x‾)2)

Since h(x1n) does not depend on the parameter θ and gθ(x1n) depends only on x1n through the function

T(X1n)=x‾=1n∑i=1nXi,

the Fisher–Neyman factorization theorem implies T(X1n) is a sufficient statistic for θ.

If σ2 is unknown and since s2=1n−1∑i=1n(xi−x‾)2, the above likelihood can be rewritten as

fX1n(x1n)=(2πσ2)−n/2exp⁡(−n−12σ2s2)exp⁡(−n2σ2(θ−x‾)2).

The Fisher–Neyman factorization theorem still holds and implies that (x‾,s2) is a joint sufficient statistic for (θ,σ2).

### Exponential distribution

If X1,…,Xn are independent and exponentially distributed with expected value θ (an unknown real-valued positive parameter), then T(X1n)=∑i=1nXi is a sufficient statistic for θ.

To see this, consider the joint probability density function of X1n=(X1,…,Xn). Because the observations are independent, the pdf can be written as a product of individual densities, i.e.

fX1n(x1n)=∏i=1n1θe−1θxi=1θne−1θ∑i=1nxi.

The joint density of the sample takes the form required by the Fisher–Neyman factorization theorem, by letting

h(x1n)=1,gθ(x1n)=1θne−1θ∑i=1nxi.

Since h(x1n) does not depend on the parameter θ and gθ(x1n) depends only on x1n through the function T(X1n)=∑i=1nXi

the Fisher–Neyman factorization theorem implies T(X1n)=∑i=1nXi is a sufficient statistic for θ.

### Gamma distribution

If X1,…,Xn are independent and distributed as a Γ(α,β), where α and β are unknown parameters of a Gamma distribution, then T(X1n)=(∏i=1nXi,∑i=1nXi) is a two-dimensional sufficient statistic for (α,β).

To see this, consider the joint probability density function of X1n=(X1,…,Xn). Because the observations are independent, the pdf can be written as a product of individual densities, i.e.

fX1n(x1n)=∏i=1n(1Γ(α)βα)xiα−1e(−1/β)xi=(1Γ(α)βα)n(∏i=1nxi)α−1e−1β∑i=1nxi.

The joint density of the sample takes the form required by the Fisher–Neyman factorization theorem, by letting

h(x1n)=1,g(α,β)(x1n)=(1Γ(α)βα)n(∏i=1nxi)α−1e−1β∑i=1nxi.

Since h(x1n) does not depend on the parameter (α,β) and g(α,β)(x1n) depends only on x1n through the function T(x1n)=(∏i=1nxi,∑i=1nxi),

the Fisher–Neyman factorization theorem implies T(X1n)=(∏i=1nXi,∑i=1nXi) is a sufficient statistic for (α,β).

## Rao–Blackwell theorem

Sufficiency finds a useful application in the Rao–Blackwell theorem, which states that if g(X) is any kind of estimator of θ, then typically the conditional expectation of g(X) given sufficient statistic T(X) is a better (in the sense of having lower variance) estimator of θ, and is never worse. Sometimes one can very easily construct a very crude estimator g(X), and then evaluate that conditional expected value to get an estimator that is in various senses optimal.

## Exponential family

Main article: Exponential family

According to the Pitman–Koopman–Darmois theorem, among families of probability distributions whose domain does not vary with the parameter being estimated, only in exponential families is there a sufficient statistic whose dimension remains bounded as sample size increases. Intuitively, this states that nonexponential families of distributions on the real line require nonparametric statistics to fully capture the information in the data.

Less tersely, suppose Xn,n=1,2,3,… are independent identically distributed real random variables whose distribution is known to be in some family of probability distributions, parametrized by θ, satisfying certain technical regularity conditions, then that family is an exponential family if and only if there is a ℝm-valued sufficient statistic T(X1,…,Xn) whose number of scalar components m does not increase as the sample size n increases.[14]

This theorem shows that the existence of a finite-dimensional, real-vector-valued sufficient statistics sharply restricts the possible forms of a family of distributions on the real line.

When the parameters or the random variables are no longer real-valued, the situation is more complex.[15]

## Other types of sufficiency

### Bayesian sufficiency

An alternative formulation of the condition that a statistic be sufficient, set in a Bayesian context, involves the posterior distributions obtained by using the full data-set and by using only a statistic. Thus the requirement is that, for almost every x,

Pr⁡(θ∣X=x)=Pr⁡(θ∣T(X)=t(x)).

More generally, without assuming a parametric model, we can say that the statistics T is predictive sufficient if

Pr⁡(X′=x′∣X=x)=Pr⁡(X′=x′∣T(X)=t(x)).

It turns out that this "Bayesian sufficiency" is a consequence of the formulation above,[16] however they are not directly equivalent in the infinite-dimensional case.[17] A range of theoretical results for sufficiency in a Bayesian context is available.[18]

### Linear sufficiency

A concept called "linear sufficiency" can be formulated in a Bayesian context,[19] and more generally.[20] First define the best linear predictor of a vector Y based on X as E^[Y∣X]. Then a linear statistic T(x) is linear sufficient[21] if

E^[θ∣X]=E^[θ∣T(X)].

## See also

* Completeness of a statistic
* Basu's theorem on independence of complete sufficient and ancillary statistics
* Lehmann–Scheffé theorem: a complete sufficient estimator is the best estimator of its expectation
* Rao–Blackwell theorem
* Chentsov's theorem
* Sufficient dimension reduction
* Ancillary statistic

## Notes

1. ↑ Dodge, Y. (2003) — entry for linear sufficiency
2. ↑ Fisher, R.A. (1922). "On the mathematical foundations of theoretical statistics". Philosophical Transactions of the Royal Society A 222 (594–604): 309–368. doi:10.1098/rsta.1922.0009. Bibcode: 1922RSPTA.222..309F. http://digital.library.adelaide.edu.au/dspace/handle/2440/15172.
3. ↑ Stigler, Stephen (December 1973). "Studies in the History of Probability and Statistics. XXXII: Laplace, Fisher and the Discovery of the Concept of Sufficiency". Biometrika 60 (3): 439–445. doi:10.1093/biomet/60.3.439.
4. ↑ Casella, George; Berger, Roger L. (2002). Statistical Inference, 2nd ed. Duxbury Press.
5. ↑ Cover, Thomas M. (2006). Elements of Information Theory. Joy A. Thomas (2nd ed.). Hoboken, N.J.: Wiley-Interscience. pp. 36. ISBN 0-471-24195-4. OCLC 59879802.
6. ↑ Halmos, P. R.; Savage, L. J. (1949). "Application of the Radon-Nikodym Theorem to the Theory of Sufficient Statistics" (in en). The Annals of Mathematical Statistics 20 (2): 225–241. doi:10.1214/aoms/1177730032. ISSN 0003-4851. http://projecteuclid.org/euclid.aoms/1177730032.
7. ↑ "Factorization theorem - Encyclopedia of Mathematics". https://encyclopediaofmath.org/wiki/Factorization_theorem.
8. ↑ Taraldsen, G. (2022). "The Factorization Theorem for Sufficiency" (in en). Preprint. doi:10.13140/RG.2.2.15068.87687.
9. ↑ Hogg, Robert V.; Craig, Allen T. (1995). Introduction to Mathematical Statistics. Prentice Hall. ISBN 978-0-02-355722-4.
10. ↑ "The Fisher–Neyman Factorization Theorem". http://cnx.org/content/m11480/1.6/. . Webpage at Connexions (cnx.org)
11. ↑ Dodge (2003) — entry for minimal sufficient statistics
12. ↑ Lehmann and Casella (1998), Theory of Point Estimation, 2nd Edition, Springer, p 37
13. ↑ Lehmann and Casella (1998), Theory of Point Estimation, 2nd Edition, Springer, page 42
14. ↑ Tikochinsky, Y.; Tishby, N. Z.; Levine, R. D. (1984-11-01). "Alternative approach to maximum-entropy inference". Physical Review A 30 (5): 2638–2644. doi:10.1103/physreva.30.2638. ISSN 0556-2791. Bibcode: 1984PhRvA..30.2638T. http://dx.doi.org/10.1103/physreva.30.2638.
15. ↑ Andersen, Erling Bernhard (September 1970). "Sufficiency and Exponential Families for Discrete Sample Spaces". Journal of the American Statistical Association 65 (331): 1248–1255. doi:10.1080/01621459.1970.10481160. ISSN 0162-1459. http://dx.doi.org/10.1080/01621459.1970.10481160.
16. ↑ "Section 5.1.4". Bayesian Theory. Wiley. 1994. ISBN 0-471-92416-4.
17. ↑ "A Bayes but not classically sufficient statistic.". Annals of Statistics 10 (3): 1025–1026. 1982. doi:10.1214/aos/1176345895.
18. ↑ Nogales, A.G.; Oyola, J.A.; Perez, P. (2000). "On conditional independence and the relationship between sufficiency and invariance under the Bayesian point of view". Statistics & Probability Letters 46 (1): 75–84. doi:10.1016/S0167-7152(99)00089-9. https://dialnet.unirioja.es/servlet/oaiart?codigo=118597.
19. ↑ Goldstein, M.; O'Hagan, A. (1996). "Bayes Linear Sufficiency and Systems of Expert Posterior Assessments". Journal of the Royal Statistical Society. Series B 58 (2): 301–316. doi:10.1111/j.2517-6161.1996.tb02083.x.
20. ↑ Godambe, V. P. (1966). "A New Approach to Sampling from Finite Populations. II Distribution-Free Sufficiency". Journal of the Royal Statistical Society. Series B 28 (2): 320–328. doi:10.1111/j.2517-6161.1966.tb00645.x.
21. ↑ Witting, T. (1987). "The linear Markov property in credibility theory". ASTIN Bulletin 17 (1): 71–84. doi:10.2143/ast.17.1.2014984.

## References

* Hazewinkel, Michiel, ed. (2001), "Sufficient statistic", Encyclopedia of Mathematics, Springer Science+Business Media B.V. / Kluwer Academic Publishers, ISBN 978-1-55608-010-4, https://www.encyclopediaofmath.org/index.php?title=S/s091070
* Lehmann, E. L.; Casella, G. (1998). Theory of Point Estimation (2nd ed.). Springer. pp. Chapter 4. ISBN 0-387-98502-6.
* Dodge, Y. (2003) The Oxford Dictionary of Statistical Terms, OUP. ISBN 0-19-920613-9

* v
* t
* e

Statistics

* Outline
* Index

Descriptive statistics

| Continuous data|

| Center|

* Mean
* arithmetic
* geometric
* harmonic
* Median
* Mode

|

Dispersion|

* Variance
* Standard deviation
* Coefficient of variation
* Percentile
* Range
* Interquartile range

Shape|

* Central limit theorem
* Moments
* Skewness
* Kurtosis
* L-moments

Count data|

* Index of dispersion

Summary tables|

* Grouped data
* Frequency distribution
* Contingency table

Dependence|

* Pearson product-moment correlation
* Rank correlation
* Spearman's rho
* Kendall's tau
* Partial correlation
* Scatter plot

Graphics|

* Bar chart
* Biplot
* Box plot
* Control chart
* Correlogram
* Fan chart
* Forest plot
* Histogram
* Pie chart
* Q–Q plot
* Run chart
* Scatter plot
* Stem-and-leaf display
* Radar chart
* Violin plot

Data collection

| Study design|

* Population
* Statistic
* Effect size
* Statistical power
* Optimal design
* Sample size determination
* Replication
* Missing data

|

Survey methodology|

* Sampling
* stratified
* cluster
* Standard error
* Opinion poll
* Questionnaire

Controlled experiments|

* Scientific control
* Randomized experiment
* Randomized controlled trial
* Random assignment
* Blocking
* Interaction
* Factorial experiment

Adaptive Designs|

* Adaptive clinical trial
* Up-and-Down Designs
* Stochastic approximation

Observational Studies|

* Cross-sectional study
* Cohort study
* Natural experiment
* Quasi-experiment

Statistical inference

| Statistical theory|

* Population
* Statistic
* Probability distribution
* Sampling distribution
* Order statistic
* Empirical distribution
* Density estimation
* Statistical model
* Model specification
* Lp space
* Parameter
* location
* scale
* shape
* Parametric family
* Likelihood (monotone)
* Location–scale family
* Exponential family
* Completeness
* Sufficiency
* Statistical functional
* Bootstrap
* U
* V
* Optimal decision
* loss function
* Efficiency
* Statistical distance
* divergence
* Asymptotics
* Robustness

|

Frequentist inference|

| Point estimation|

* Estimating equations
* Maximum likelihood
* Method of moments
* M-estimator
* Minimum distance
* Unbiased estimators
* Mean-unbiased minimum-variance
* Rao–Blackwellization
* Lehmann–Scheffé theorem
* Median unbiased
* Plug-in

|

Interval estimation|

* Confidence interval
* Pivot
* Likelihood interval
* Prediction interval
* Tolerance interval
* Resampling
* Bootstrap
* Jackknife

Testing hypotheses|

* 1- & 2-tails
* Power
* Uniformly most powerful test
* Permutation test
* Randomization test
* Multiple comparisons

Parametric tests|

* Likelihood-ratio
* Score/Lagrange multiplier
* Wald

Specific tests|

* Z-test (normal)
* Student's t-test
* F-test

Goodness of fit|

* Chi-squared
* G-test
* Kolmogorov–Smirnov
* Anderson–Darling
* Lilliefors
* Jarque–Bera
* Normality (Shapiro–Wilk)
* Likelihood-ratio test
* Model selection
* Cross validation
* AIC
* BIC

Rank statistics|

* Sign
* Sample median
* Signed rank (Wilcoxon)
* Hodges–Lehmann estimator
* Rank sum (Mann–Whitney)
* Nonparametric anova
* 1-way (Kruskal–Wallis)
* 2-way (Friedman)
* Ordered alternative (Jonckheere–Terpstra)

Bayesian inference|

* Bayesian probability
* prior
* posterior
* Credible interval
* Bayes factor
* Bayesian estimator
* Maximum posterior estimator

* Correlation
* Regression analysis

| Correlation|

* Pearson product-moment
* Partial correlation
* Confounding variable
* Coefficient of determination

|

Regression analysis|

* Errors and residuals
* Regression validation
* Mixed effects models
* Simultaneous equations models
* Multivariate adaptive regression splines (MARS)

Linear regression|

* Simple linear regression
* Ordinary least squares
* General linear model
* Bayesian regression

Non-standard predictors|

* Nonlinear regression
* Nonparametric
* Semiparametric
* Isotonic
* Robust
* Heteroscedasticity
* Homoscedasticity

Generalized linear model|

* Exponential families
* Logistic (Bernoulli) / Binomial / Poisson regressions

Partition of variance|

* Analysis of variance (ANOVA, anova)
* Analysis of covariance
* Multivariate ANOVA
* Degrees of freedom

Categorical / Multivariate / Time-series / Survival analysis

| Categorical|

* Cohen's kappa
* Contingency table
* Graphical model
* Log-linear model
* McNemar's test

|

Multivariate|

* Regression
* Manova
* Principal components
* Canonical correlation
* Discriminant analysis
* Cluster analysis
* Classification
* Structural equation model
* Factor analysis
* Multivariate distributions
* Elliptical distributions
* Normal

Time-series|

| General|

* Decomposition
* Trend
* Stationarity
* Seasonal adjustment
* Exponential smoothing
* Cointegration
* Structural break
* Granger causality

|

Specific tests|

* Dickey–Fuller
* Johansen
* Q-statistic (Ljung–Box)
* Durbin–Watson
* Breusch–Godfrey

Time domain|

* Autocorrelation (ACF)
* partial (PACF)
* Cross-correlation (XCF)
* ARMA model
* ARIMA model (Box–Jenkins)
* Autoregressive conditional heteroskedasticity (ARCH)
* Vector autoregression (VAR)

Frequency domain|

* Spectral density estimation
* Fourier analysis
* Wavelet
* Whittle likelihood

Survival|

| Survival function|

* Kaplan–Meier estimator (product limit)
* Proportional hazards models
* Accelerated failure time (AFT) model
* First hitting time

|

Hazard function|

* Nelson–Aalen estimator

Test|

* Log-rank test

Applications

| Biostatistics|

* Bioinformatics
* Clinical trials / studies
* Epidemiology
* Medical statistics

|

Engineering statistics|

* Chemometrics
* Methods engineering
* Probabilistic design
* Process / quality control
* Reliability
* System identification

Social statistics|

* Actuarial science
* Census
* Crime statistics
* Demography
* Econometrics
* Jurimetrics
* National accounts
* Official statistics
* Population statistics
* Psychometrics

Spatial statistics|

* Cartography
* Environmental statistics
* Geographic information system
* Geostatistics
* Kriging

* Category
* '
* Commons
* WikiProject

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/Sufficient statistic. Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=Sufficient_statistic&oldid=4156471"
*[v]: View this template
*[t]: Discuss this template
*[e]: Edit this template