Posterior probability

Short description: Conditional probability used in Bayesian statistics

Part of a series on

Bayesian statistics
Theory

* Admissible decision rule
* Bayesian efficiency
* Bayesian probability
* Probability interpretations
* Bayes' theorem
* Bayes factor
* Bayesian inference
* Bayesian network
* Prior
* Posterior
* Likelihood
* Conjugate prior
* Posterior predictive
* Hyperparameter
* Hyperprior
* Principle of indifference
* Principle of maximum entropy
* Empirical Bayes method
* Cromwell's rule
* Bernstein–von Mises theorem
* Schwarz criterion
* Credible interval
* Maximum a posteriori estimation
* Radical probabilism

Techniques

* Bayesian linear regression
* Bayesian estimator
* Approximate Bayesian computation
* Markov chain Monte Carlo

* v
* t
* e

The posterior probability is a type of conditional probability that results from updating the prior probability with information summarized by the likelihood via an application of Bayes' rule.[1] From an epistemological perspective, the posterior probability contains everything there is to know about an uncertain proposition (such as a scientific hypothesis, or parameter values), given prior knowledge and a mathematical model describing the observations available at a particular time.[2] After the arrival of new information, the current posterior probability may serve as the prior in another round of Bayesian updating.[3]

In the context of Bayesian statistics, the posterior probability distribution usually describes the epistemic uncertainty about statistical parameters conditional on a collection of observed data. From a given posterior distribution, various point and interval estimates can be derived, such as the maximum a posteriori (MAP) or the highest posterior density interval (HPDI).[4] But while conceptually simple, the posterior distribution is generally not tractable and therefore needs to be either analytically or numerically approximated.[5]

## Contents

* 1 Definition in the distributional case
* 2 Example
* 3 Calculation
* 4 Credible interval
* 5 Classification
* 6 See also
* 7 References
* 8 Further reading

## Definition in the distributional case

In Bayesian statistics, the posterior probability is the probability distribution of the parameters θ given the evidence X, and is denoted p(θ|X).

It contrasts with the likelihood function, which is the probability of the evidence given the parameters: p(X|θ).

The two are related as follows:

Given a prior belief that a probability distribution function is p(θ) and that the observations x have a likelihood p(x|θ), then the posterior probability is defined as

p(θ|x)=p(x|θ)p(x)p(θ),[6]

where p(x) is the normalizing constant and is calculated as

p(x)=∫p(x|θ)p(θ)dθ

for continuous θ, or by summing p(x|θ)p(θ) over all possible values of θ for discrete θ.[7]

The posterior probability is therefore proportional to the product Likelihood · Prior probability.[8]

## Example

Suppose there is a school with 60% boys and 40% girls as students. The girls wear trousers or skirts in equal numbers; all boys wear trousers. An observer sees a (random) student from a distance; all the observer can see is that this student is wearing trousers. What is the probability this student is a girl? The correct answer can be computed using Bayes' theorem.

The event G is that the student observed is a girl, and the event T is that the student observed is wearing trousers. To compute the posterior probability P(G|T), we first need to know:

* P(G), or the probability that the student is a girl regardless of any other information. Since the observer sees a random student, meaning that all students have the same probability of being observed, and the percentage of girls among the students is 40%, this probability equals 0.4.
* P(B), or the probability that the student is not a girl (i.e. a boy) regardless of any other information (B is the complementary event to G). This is 60%, or 0.6.
* P(T|G), or the probability of the student wearing trousers given that the student is a girl. As they are as likely to wear skirts as trousers, this is 0.5.
* P(T|B), or the probability of the student wearing trousers given that the student is a boy. This is given as 1.
* P(T), or the probability of a (randomly selected) student wearing trousers regardless of any other information. Since P(T)=P(T|G)P(G)+P(T|B)P(B) (via the law of total probability), this is P(T)=0.5×0.4+1×0.6=0.8.

Given all this information, the posterior probability of the observer having spotted a girl given that the observed student is wearing trousers can be computed by substituting these values in the formula:

P(G|T)=P(T|G)P(G)P(T)=0.5×0.40.8=0.25.

An intuitive way to solve this is to assume the school has N students. Number of boys = 0.6N and number of girls = 0.4N. If N is large enough that rounding errors can be ignored, total number of trouser wearers = 0.6N \+ 50% of 0.4N. And number of girl trouser wearers = 50% of 0.4N. Therefore, in the population of trousers, girls are (50% of 0.4N)/(0.6N \+ 50% of 0.4N) = 25%. In other words, if you separated out the group of trouser wearers, a quarter of that group will be girls. Therefore, if you see trousers, the most you can deduce is that you are looking at a single sample from a subset of students where 25% are girls. And by definition, chance of this random student being a girl is 25%. Every Bayes-theorem problem can be solved in this way.[9]

## Calculation

The posterior probability distribution of one random variable given the value of another can be calculated with Bayes' theorem by multiplying the prior probability distribution by the likelihood function, and then dividing by the normalizing constant, as follows:

fX∣Y=y(x)=fX(x)ℒX∣Y=y(x)∫−∞∞fX(u)ℒX∣Y=y(u)du

gives the posterior probability density function for a random variable X given the data Y=y, where

* fX(x) is the prior density of X,
* ℒX∣Y=y(x)=fY∣X=x(y) is the likelihood function as a function of x,
* ∫−∞∞fX(u)ℒX∣Y=y(u)du is the normalizing constant, and
* fX∣Y=y(x) is the posterior density of X given the data Y=y.[10]

## Credible interval

Posterior probability is a conditional probability conditioned on randomly observed data. Hence it is a random variable. For a random variable, it is important to summarize its amount of uncertainty. One way to achieve this goal is to provide a credible interval of the posterior probability.[11]

## Classification

In classification, posterior probabilities reflect the uncertainty of assessing an observation to particular class, see also class-membership probabilities. While statistical classification methods by definition generate posterior probabilities, Machine Learners usually supply membership values which do not induce any probabilistic confidence. It is desirable to transform or rescale membership values to class-membership probabilities, since they are comparable and additionally more easily applicable for post-processing.[12]

## See also

* Prediction interval
* Bernstein–von Mises theorem
* Probability of success
* Bayesian epistemology
* Metropolis–Hastings algorithm

## References

1. ↑ Lambert, Ben (2018). "The posterior – the goal of Bayesian inference". A Student's Guide to Bayesian Statistics. Sage. pp. 121–140. ISBN 978-1-4739-1636-4.
2. ↑ Grossman, Jason (2005). Inferences from observations to simple statistical hypotheses (PhD thesis). University of Sydney. hdl:2123/9107.
3. ↑ Etz, Alex (2015-07-25). "Understanding Bayes: Updating priors via the likelihood" (in en). https://alexanderetz.com/2015/07/25/understanding-bayes-updating-priors-via-the-likelihood/.
4. ↑ Gill, Jeff (2014). "Summarizing Posterior Distributions with Intervals". Bayesian Methods: A Social and Behavioral Sciences Approach (Third ed.). Chapman & Hall. pp. 42–48. ISBN 978-1-4398-6248-3.
5. ↑ Press, S. James (1989). "Approximations, Numerical Methods, and Computer Programs". Bayesian Statistics : Principles, Models, and Applications. New York: John Wiley & Sons. pp. 69–102. ISBN 0-471-63729-7.
6. ↑ Christopher M. Bishop (2006). Pattern Recognition and Machine Learning. Springer. pp. 21–24. ISBN 978-0-387-31073-2.
7. ↑ Andrew Gelman, John B. Carlin, Hal S. Stern, David B. Dunson, Aki Vehtari and Donald B. Rubin (2014). Bayesian Data Analysis. CRC Press. pp. 7. ISBN 978-1-4398-4095-5.
8. ↑ Ross, Kevin. Chapter 8 Introduction to Continuous Prior and Posterior Distributions | An Introduction to Bayesian Reasoning and Methods. https://bookdown.org/kevin_davisross/bayesian-reasoning-and-methods/continuous.html.
9. ↑ "Bayes' theorem - C o r T e x T". https://sites.google.com/site/artificialcortext/others/mathematics/bayes-theorem.
10. ↑ "Posterior probability - formulasearchengine". https://formulasearchengine.com/wiki/Posterior_probability.
11. ↑ Clyde, Merlise; Çetinkaya-Rundel, Mine; Rundel, Colin; Banks, David; Chai, Christine; Huang, Lizzy. Chapter 1 The Basics of Bayesian Statistics | An Introduction to Bayesian Thinking. https://statswithr.github.io/book/the-basics-of-bayesian-statistics.html.
12. ↑ Boedeker, Peter; Kearns, Nathan T. (2019-07-09). "Linear Discriminant Analysis for Prediction of Group Membership: A User-Friendly Primer" (in en). Advances in Methods and Practices in Psychological Science 2 (3): 250–263. doi:10.1177/2515245919849378. ISSN 2515-2459. http://journals.sagepub.com/doi/10.1177/2515245919849378.

## Further reading

* Lancaster, Tony (2004). An Introduction to Modern Bayesian Econometrics. Oxford: Blackwell. ISBN 1-4051-1720-6.
* Lee, Peter M. (2004). Bayesian Statistics : An Introduction (3rd ed.). Wiley. ISBN 0-340-81405-5.

* v
* t
* e

Statistics

* Outline
* Index

Descriptive statistics

| Continuous data|

| Center|

* Mean
* arithmetic
* geometric
* harmonic
* Median
* Mode

|

Dispersion|

* Variance
* Standard deviation
* Coefficient of variation
* Percentile
* Range
* Interquartile range

Shape|

* Central limit theorem
* Moments
* Skewness
* Kurtosis
* L-moments

Count data|

* Index of dispersion

Summary tables|

* Grouped data
* Frequency distribution
* Contingency table

Dependence|

* Pearson product-moment correlation
* Rank correlation
* Spearman's rho
* Kendall's tau
* Partial correlation
* Scatter plot

Graphics|

* Bar chart
* Biplot
* Box plot
* Control chart
* Correlogram
* Fan chart
* Forest plot
* Histogram
* Pie chart
* Q–Q plot
* Run chart
* Scatter plot
* Stem-and-leaf display
* Radar chart
* Violin plot

Data collection

| Study design|

* Population
* Statistic
* Effect size
* Statistical power
* Optimal design
* Sample size determination
* Replication
* Missing data

|

Survey methodology|

* Sampling
* stratified
* cluster
* Standard error
* Opinion poll
* Questionnaire

Controlled experiments|

* Scientific control
* Randomized experiment
* Randomized controlled trial
* Random assignment
* Blocking
* Interaction
* Factorial experiment

Adaptive Designs|

* Adaptive clinical trial
* Up-and-Down Designs
* Stochastic approximation

Observational Studies|

* Cross-sectional study
* Cohort study
* Natural experiment
* Quasi-experiment

Statistical inference

| Statistical theory|

* Population
* Statistic
* Probability distribution
* Sampling distribution
* Order statistic
* Empirical distribution
* Density estimation
* Statistical model
* Model specification
* Lp space
* Parameter
* location
* scale
* shape
* Parametric family
* Likelihood (monotone)
* Location–scale family
* Exponential family
* Completeness
* Sufficiency
* Statistical functional
* Bootstrap
* U
* V
* Optimal decision
* loss function
* Efficiency
* Statistical distance
* divergence
* Asymptotics
* Robustness

|

Frequentist inference|

| Point estimation|

* Estimating equations
* Maximum likelihood
* Method of moments
* M-estimator
* Minimum distance
* Unbiased estimators
* Mean-unbiased minimum-variance
* Rao–Blackwellization
* Lehmann–Scheffé theorem
* Median unbiased
* Plug-in

|

Interval estimation|

* Confidence interval
* Pivot
* Likelihood interval
* Prediction interval
* Tolerance interval
* Resampling
* Bootstrap
* Jackknife

Testing hypotheses|

* 1- & 2-tails
* Power
* Uniformly most powerful test
* Permutation test
* Randomization test
* Multiple comparisons

Parametric tests|

* Likelihood-ratio
* Score/Lagrange multiplier
* Wald

Specific tests|

* Z-test (normal)
* Student's t-test
* F-test

Goodness of fit|

* Chi-squared
* G-test
* Kolmogorov–Smirnov
* Anderson–Darling
* Lilliefors
* Jarque–Bera
* Normality (Shapiro–Wilk)
* Likelihood-ratio test
* Model selection
* Cross validation
* AIC
* BIC

Rank statistics|

* Sign
* Sample median
* Signed rank (Wilcoxon)
* Hodges–Lehmann estimator
* Rank sum (Mann–Whitney)
* Nonparametric anova
* 1-way (Kruskal–Wallis)
* 2-way (Friedman)
* Ordered alternative (Jonckheere–Terpstra)

Bayesian inference|

* Bayesian probability
* prior
* posterior
* Credible interval
* Bayes factor
* Bayesian estimator
* Maximum posterior estimator

* Correlation
* Regression analysis

| Correlation|

* Pearson product-moment
* Partial correlation
* Confounding variable
* Coefficient of determination

|

Regression analysis|

* Errors and residuals
* Regression validation
* Mixed effects models
* Simultaneous equations models
* Multivariate adaptive regression splines (MARS)

Linear regression|

* Simple linear regression
* Ordinary least squares
* General linear model
* Bayesian regression

Non-standard predictors|

* Nonlinear regression
* Nonparametric
* Semiparametric
* Isotonic
* Robust
* Heteroscedasticity
* Homoscedasticity

Generalized linear model|

* Exponential families
* Logistic (Bernoulli) / Binomial / Poisson regressions

Partition of variance|

* Analysis of variance (ANOVA, anova)
* Analysis of covariance
* Multivariate ANOVA
* Degrees of freedom

Categorical / Multivariate / Time-series / Survival analysis

| Categorical|

* Cohen's kappa
* Contingency table
* Graphical model
* Log-linear model
* McNemar's test

|

Multivariate|

* Regression
* Manova
* Principal components
* Canonical correlation
* Discriminant analysis
* Cluster analysis
* Classification
* Structural equation model
* Factor analysis
* Multivariate distributions
* Elliptical distributions
* Normal

Time-series|

| General|

* Decomposition
* Trend
* Stationarity
* Seasonal adjustment
* Exponential smoothing
* Cointegration
* Structural break
* Granger causality

|

Specific tests|

* Dickey–Fuller
* Johansen
* Q-statistic (Ljung–Box)
* Durbin–Watson
* Breusch–Godfrey

Time domain|

* Autocorrelation (ACF)
* partial (PACF)
* Cross-correlation (XCF)
* ARMA model
* ARIMA model (Box–Jenkins)
* Autoregressive conditional heteroskedasticity (ARCH)
* Vector autoregression (VAR)

Frequency domain|

* Spectral density estimation
* Fourier analysis
* Wavelet
* Whittle likelihood

Survival|

| Survival function|

* Kaplan–Meier estimator (product limit)
* Proportional hazards models
* Accelerated failure time (AFT) model
* First hitting time

|

Hazard function|

* Nelson–Aalen estimator

Test|

* Log-rank test

Applications

| Biostatistics|

* Bioinformatics
* Clinical trials / studies
* Epidemiology
* Medical statistics

|

Engineering statistics|

* Chemometrics
* Methods engineering
* Probabilistic design
* Process / quality control
* Reliability
* System identification

Social statistics|

* Actuarial science
* Census
* Crime statistics
* Demography
* Econometrics
* Jurimetrics
* National accounts
* Official statistics
* Population statistics
* Psychometrics

Spatial statistics|

* Cartography
* Environmental statistics
* Geographic information system
* Geostatistics
* Kriging

* Category
* '
* Commons
* WikiProject

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/Posterior probability. Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=Posterior_probability&oldid=4529883"
*[v]: View this template
*[t]: Discuss this template
*[e]: Edit this template