Normal-inverse-gamma distribution

Short description: Family of multivariate continuous probability distributions

normal-inverse-gamma

Probability density function

Parameters|  μ location (real)
λ>0 (real)
α>0 (real)
β>0 (real)
Support|  x∈(−∞,∞),σ2∈(0,∞)
PDF|  λ2πσ2βαΓ(α)(1σ2)α+1exp⁡(−2β+λ(x−μ)22σ2)
Mean|

E⁡[x]=μ

E⁡[σ2]=βα−1, for α>1
Mode|

x=μ(univariate),x=μ(multivariate)

σ2=βα+1+1/2(univariate),σ2=βα+1+k/2(multivariate)
Variance|

Var⁡[x]=β(α−1)λ, for α>1
Var⁡[σ2]=β2(α−1)2(α−2), for α>2

Cov⁡[x,σ2]=0, for α>1

In probability theory and statistics, the normal-inverse-gamma distribution (or Gaussian-inverse-gamma distribution) is a four-parameter family of multivariate continuous probability distributions. It is the conjugate prior of a normal distribution with unknown mean and variance.

## Contents

* 1 Definition
* 2 Characterization
* 2.1 Probability density function
* 2.1.1 Alternative parameterization
* 2.2 Cumulative distribution function
* 3 Properties
* 3.1 Marginal distributions
* 3.2 Summation
* 3.3 Scaling
* 3.4 Exponential family
* 3.5 Information entropy
* 3.6 Kullback–Leibler divergence
* 4 Maximum likelihood estimation
* 5 Posterior distribution of the parameters
* 6 Interpretation of the parameters
* 7 Generating normal-inverse-gamma random variates
* 8 Related distributions
* 9 See also
* 10 References

## Definition

Suppose

x∣σ2,μ,λ∼N(μ,σ2/λ)

has a normal distribution with mean μ and variance σ2/λ, where

σ2∣α,β∼Γ−1(α,β)

has an inverse-gamma distribution. Then (x,σ2) has a normal-inverse-gamma distribution, denoted as

(x,σ2)∼N-Γ−1(μ,λ,α,β).

(NIG is also used instead of N-Γ−1.)

The normal-inverse-Wishart distribution is a generalization of the normal-inverse-gamma distribution that is defined over multivariate random variables.

## Characterization

### Probability density function

f(x,σ2∣μ,λ,α,β)=λσ2πβαΓ(α)(1σ2)α+1exp⁡(−2β+λ(x−μ)22σ2)

For the multivariate form where 𝐱 is a k×1 random vector,

f(𝐱,σ2∣μ,𝐕−1,α,β)=|𝐕|−1/2(2π)−k/2βαΓ(α)(1σ2)α+1+k/2exp⁡(−2β+(𝐱−μ)T𝐕−1(𝐱−μ)2σ2).

where |𝐕| is the determinant of the k×k matrix 𝐕. Note how this last equation reduces to the first form if k=1 so that 𝐱,𝐕,μ are scalars.

#### Alternative parameterization

It is also possible to let γ=1/λ in which case the pdf becomes

f(x,σ2∣μ,γ,α,β)=1σ2πγβαΓ(α)(1σ2)α+1exp⁡(−2γβ+(x−μ)22γσ2)

In the multivariate form, the corresponding change would be to regard the covariance matrix 𝐕 instead of its inverse 𝐕−1 as a parameter.

### Cumulative distribution function

F(x,σ2∣μ,λ,α,β)=e−βσ2(βσ2)α(erf⁡(λ(x−μ)2σ)+1)2σ2Γ(α)

## Properties

### Marginal distributions

Given (x,σ2)∼N-Γ−1(μ,λ,α,β). as above, σ2 by itself follows an inverse gamma distribution:

σ2∼Γ−1(α,β)

while αλβ(x−μ) follows a t distribution with 2α degrees of freedom.[1]

Proof for λ=1

For λ=1 probability density function is

f(x,σ2∣μ,α,β)=1σ2πβαΓ(α)(1σ2)α+1exp⁡(−2β+(x−μ)22σ2)

Marginal distribution over x is

f(x∣μ,α,β)=∫0∞dσ2f(x,σ2∣μ,α,β)=12πβαΓ(α)∫0∞dσ2(1σ2)α+1/2+1exp⁡(−2β+(x−μ)22σ2)

Except for normalization factor, expression under the integral coincides with Inverse-gamma distribution

Γ−1(x;a,b)=baΓ(a)e−b/xxa+1,

with x=σ2, a=α+1/2, b=2β+(x−μ)22.

Since ∫0∞dxΓ−1(x;a,b)=1,∫0∞dxx−(a+1)e−b/x=Γ(a)b−a, and

∫0∞dσ2(1σ2)α+1/2+1exp⁡(−2β+(x−μ)22σ2)=Γ(α+1/2)(2β+(x−μ)22)−(α+1/2)

Substituting this expression and factoring dependence on x,

f(x∣μ,α,β)∝x(1+(x−μ)22β)−(α+1/2).

Shape of generalized Student's t-distribution is

t(x|ν,μ^,σ^2)∝x(1+1ν(x−μ^)2σ^2)−(ν+1)/2.

Marginal distribution f(x∣μ,α,β) follows t-distribution with 2α degrees of freedom

f(x∣μ,α,β)=t(x|ν=2α,μ^=μ,σ^2=β/α).

In the multivariate case, the marginal distribution of 𝐱 is a multivariate t distribution:

𝐱∼t2α(μ,βα𝐕)

### Summation

### Scaling

Suppose

(x,σ2)∼N-Γ−1(μ,λ,α,β).

Then for c>0,

(cx,cσ2)∼N-Γ−1(cμ,λ/c,α,cβ).

Proof: To prove this let (x,σ2)∼N-Γ−1(μ,λ,α,β) and fix c>0. Defining Y=(Y1,Y2)=(cx,cσ2), observe that the PDF of the random variable Y evaluated at (y1,y2) is given by 1/c2 times the PDF of a N-Γ−1(μ,λ,α,β) random variable evaluated at (y1/c,y2/c). Hence the PDF of Y evaluated at (y1,y2) is given by :fY(y1,y2)=1c2λ2πy2/cβαΓ(α)(1y2/c)α+1exp⁡(−2β+λ(y1/c−μ)22y2/c)=λ/c2πy2(cβ)αΓ(α)(1y2)α+1exp⁡(−2cβ+(λ/c)(y1−cμ)22y2).

The right hand expression is the PDF for a N-Γ−1(cμ,λ/c,α,cβ) random variable evaluated at (y1,y2), which completes the proof.

### Exponential family

Normal-inverse-gamma distributions form an exponential family with natural parameters θ1=−λ2, θ2=λμ, θ3=α, and θ4=−β+−λμ22 and sufficient statistics T1=x2σ2, T2=xσ2, T3=log⁡(1σ2), and T4=1σ2.

### Information entropy

### Kullback–Leibler divergence

Measures difference between two distributions.

## Maximum likelihood estimation

## Posterior distribution of the parameters

See the articles on normal-gamma distribution and conjugate prior.

## Interpretation of the parameters

See the articles on normal-gamma distribution and conjugate prior.

## Generating normal-inverse-gamma random variates

Generation of random variates is straightforward:

1. Sample σ2 from an inverse gamma distribution with parameters α and β
2. Sample x from a normal distribution with mean μ and variance σ2/λ

## Related distributions

* The normal-gamma distribution is the same distribution parameterized by precision rather than variance
* A generalization of this distribution which allows for a multivariate mean and a completely unknown positive-definite covariance matrix σ2𝐕 (whereas in the multivariate inverse-gamma distribution the covariance matrix is regarded as known up to the scale factor σ2) is the normal-inverse-Wishart distribution

## See also

* Compound probability distribution

## References

1. ↑ Ramírez-Hassan, Andrés. 4.2 Conjugate prior to exponential family | Introduction to Bayesian Econometrics. https://bookdown.org/aramir21/IntroductionBayesianEconometricsGuidedTour/sec42.html#sec42.

* Denison, David G. T. et al. (2002). Bayesian Methods for Nonlinear Classification and Regression. Wiley. ISBN 0471490369.
* Koch, Karl-Rudolf (2007). Introduction to Bayesian Statistics (2nd ed.). Springer. ISBN 354072723X.

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/Normal-inverse-gamma distribution. Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=Normal-inverse-gamma_distribution&oldid=4152989"