Multilevel regression with poststratification

Statistical regression technique

Part of a series on

Regression analysis
Models

* Linear regression
* Simple regression
* Polynomial regression
* General linear model

* Generalized linear model
* Vector generalized linear model
* Discrete choice
* Binomial regression
* Binary regression
* Logistic regression
* Multinomial logistic regression
* Mixed logit
* Probit
* Multinomial probit
* Ordered logit
* Ordered probit
* Poisson

* Multilevel model
* Fixed effects
* Random effects
* Linear mixed-effects model
* Nonlinear mixed-effects model

* Nonlinear regression
* Nonparametric
* Semiparametric
* Robust
* Quantile
* Isotonic
* Principal components
* Least angle
* Local
* Segmented

* Errors-in-variables

Estimation

* Least squares
* Linear
* Non-linear

* Ordinary
* Weighted
* Generalized
* Generalized estimating equation

* Partial
* Total
* Non-negative
* Ridge regression
* Regularized

* Least absolute deviations
* Iteratively reweighted
* Bayesian
* Bayesian multivariate
* Least-squares spectral analysis

Background

* Regression validation
* Mean and predicted response
* Errors and residuals
* Goodness of fit
* Studentized residual
* Gauss–Markov theorem

* Mathematics portal

* v
* t
* e

Multilevel regression with poststratification (MRP) is a statistical technique used for correcting model estimates for known differences between a sample population (the population of the data one has), and a target population (a population one wishes to estimate for).

The poststratification refers to the process of adjusting the estimates, essentially a weighted average of estimates from all possible combinations of attributes (for example age and sex). Each combination is sometimes called a "cell". The multilevel regression is the use of a multilevel model to smooth noisy estimates in the cells with too little data by using overall or nearby averages.

One application is estimating preferences in sub-regions (e.g., states, individual constituencies) based on individual-level survey data gathered at other levels of aggregation (e.g., national surveys).[1]

## Mathematical formulation

[edit]

Following the MRP model description,[2] assume  Y {\displaystyle Y} represents single outcome measurement and the population mean value of  Y {\displaystyle Y} ,  μ Y {\displaystyle \mu _{Y}} , is the target parameter of interest. In the underlying population, each individual,  i {\displaystyle i} , belongs to one of  j = 1 , 2 , ⋯ , J {\displaystyle j=1,2,\cdots ,J} poststratification cells characterized by a unique set of covariates. The multilevel regression with poststratification model involves the following pair of steps:

MRP step 1 (multilevel regression): The multilevel regression model specifies a linear predictor for the mean  μ Y {\displaystyle \mu _{Y}} , or the logit transform of the mean in the case of a binary outcome, in poststratification cell  j {\displaystyle j} ,

g ( μ j ) = g ( E [ Y j [ i ] ] ) = β 0 + X j T β + ∑ k = 1 K a l [ j ] k , {\displaystyle g{\left({\mathrm {\mu } }_{j}\right)}=g{\left(E{\left[Y_{j{\lbrack i\rbrack }}\right]}\right)}={\mathrm {\beta } }_{0}+{\boldsymbol {X}}_{j}^{T}\mathbf {\beta } +\sum _{k=1}^{K}a_{l{\lbrack j\rbrack }}^{k},}

where  Y j [ i ] {\displaystyle Y_{j\lbrack i\rbrack }} is the outcome measurement for respondent  i {\displaystyle i} in cell  j {\displaystyle j} ,  β 0 {\displaystyle \beta _{0}} is the fixed intercept,  X j {\displaystyle {\boldsymbol {X}}_{j}} is the unique covariate vector for cell  j {\displaystyle j} ,  β {\displaystyle {\mathrm {\beta } }} is a vector of regression coefficients (fixed effects),  a l [ j ] k {\displaystyle a_{l{\lbrack j\rbrack }}^{k}} is the varying coefficient (random effect),  l [ j ] {\displaystyle l{\lbrack j\rbrack }} maps the  j {\displaystyle j} cell index to the corresponding category index  l {\displaystyle l} of variable  k ∈ { 1 , 2 , ⋯ , K } {\displaystyle k\in \\{1,2,\cdots ,K\\}} . All varying coefficients are exchangeable batches with independent normal prior distributions a l k ∼ N ( 0 , σ k 2 ) , l ∈ { 1 , … , L k } {\displaystyle a_{l}^{k}\sim \mathrm {N} \left(0,\mathrm {\sigma } _{k}^{2}\right),\ l\in \\{1,\dots ,L_{k}\\}} .

MRP step 2: poststratification: The poststratification (PS) estimate for the population parameter of interest is  μ ^ P S = ∑ j = 1 J N j μ ^ j ∑ i = 1 J N j {\displaystyle {\hat {\mu }}^{PS}={\frac {\sum _{j=1}^{J}N_{j}{\hat {\mu }}_{j}}{\sum _{i=1}^{J}N_{j}}}} where  μ ^ j {\displaystyle {\hat {\mu }}_{j}} is the estimated outcome of interest for poststratification cell  j {\displaystyle j} and is the size of the  j {\displaystyle j} -th poststratification cell in the population. Estimates at any subpopulation level  s {\displaystyle s} are similarly derived  μ ^ s P S = ∑ j = 1 J s N j μ ^ j ∑ i = 1 J s N j {\displaystyle {\hat {\mu }}_{s}^{PS}={\frac {\sum _{j=1}^{J_{s}}N_{j}{\hat {\mu }}_{j}}{\sum _{i=1}^{J_{s}}N_{j}}}} where  J s {\displaystyle J_{s}} is the subset of all poststratification cells that comprise  s {\displaystyle s} .

## The technique and its advantages

[edit]

The technique essentially involves using data from, for example, censuses relating to various types of people corresponding to different characteristics (e.g. age, race), in a first step to estimate the relationship between those types and individual preferences (i.e., multi-level regression of the dataset). This relationship is then used in a second step to estimate the sub-regional preference based on the number of people having each type or characteristic in that sub-region (a process known as "poststratification").[3] In this way the need to perform surveys at sub-regional level, which can be expensive and impractical in an area (e.g. a country) with many sub-regions (e.g. counties, ridings, or states), is avoided. It also avoids issues with consistency of survey when comparing different surveys performed in different areas.[4][1] Additionally, it allows the estimating of preference within a specific locality based on a survey taken across a wider area that includes relatively few people from the locality in question, or where the sample may be highly unrepresentative.[5]

## History

[edit]

The technique was originally developed by Gelman and T. Little in 1997,[6] building upon ideas of Fay and Herriot[7] and R. Little.[8] It was subsequently expanded on by Park, Gelman, and Bafumi in 2004 and 2006. It was proposed for use in estimating US-state-level voter preference by Lax and Philips in 2009. Warshaw and Rodden subsequently proposed it for use in estimating district-level public opinion in 2012.[1] Later, Wang et al.[9] used survey data of Xbox users to predict the outcome of the 2012 US presidential election. The Xbox gamers were 65% 18- to 29-year-olds and 93% male, while the electorate as a whole was 19% 18- to 29-year-olds and 47% male. Even though the original data was highly biased, after multilevel regression with poststratification the authors were able to get estimates that agreed with those coming from polls using large amounts of random and representative data. Since then it has also been proposed for use in the field of epidemiology.[5]

YouGov used the technique to successfully predict the overall outcome of the 2017 UK general election,[10] correctly predicting the result in 93% of constituencies.[11] In the 2019 and 2024 elections other pollsters used MRP including Survation [12] and Ipsos.[13]

## Limitations and extensions

[edit]

MRP can be extended to estimating the change of opinion over time[4] and when used to predict elections works best when used relatively close to the polling date, after nominations have closed.[14]

Both the "multilevel regression" and "poststratification" ideas of MRP can be generalized. Multilevel regression can be replaced by nonparametric regression[15] or regularized prediction, and poststratification can be generalized to allow for non-census variables, i.e. poststratification totals that are estimated rather than being known.[16]

## References

[edit]

1. ^ a b c Buttice, Matthew K.; Highton, Benjamin (Autumn 2013). "How Does Multilevel Regression and Poststratification Perform with Conventional National Surveys?" (PDF). Political Analysis. 21 (4): 449–451\. doi:10.1093/pan/mpt017. JSTOR 24572674. Archived (PDF) from the original on 12 February 2025. Retrieved 16 February 2025.449-451&rft.date=2013&rft_id=info:doi/10.1093/pan/mpt017&rft_id=https://www.jstor.org/stable/24572674#id-name=JSTOR&rft.aulast=Buttice&rft.aufirst=Matthew+K.&rft.au=Highton,+Benjamin&rft_id=https://escholarship.org/content/qt5wc2g12h/qt5wc2g12h.pdf?t=o18zd3&rfr_id=info:sid/en.wikipedia.org:Multilevel+regression+with+poststratification" class="Z3988">
2. ^ Downes, Marnie Downes; at al. (August 2018). "Multilevel Regression and Poststratification: A Modeling Approach to Estimating Population Quantities From Highly Selected Survey Samples". American Journal of Epidemiology. 187 (8): 1780–1790\. doi:10.1093/aje/kwy070. PMID 29635276.1780-1790&rft.date=2018-08&rft_id=info:doi/10.1093/aje/kwy070&rft_id=info:pmid/29635276&rft.aulast=Downes&rft.aufirst=Marnie+Downes&rft.au=at+al.&rft_id=https://doi.org/10.1093/aje/kwy070&rfr_id=info:sid/en.wikipedia.org:Multilevel+regression+with+poststratification" class="Z3988">
3. ^ "What is MRP?". Survation. 5 November 2018. Archived from the original on 16 February 2025. Retrieved 31 October 2019.
4. ^ a b Gelman, Andrew; Lax, Jeffrey; Phillips, Justin; Gabry, Jonah; Trangucci, Robert (28 August 2018). "Using Multilevel Regression and Poststratification to Estimate Dynamic Public Opinion" (PDF). sites.stat.columbia.edu: 1–3\. Archived (PDF) from the original on 16 February 2025. Retrieved 31 October 2019.1-3&rft.date=2018-08-28&rft.aulast=Gelman&rft.aufirst=Andrew&rft.au=Lax,+Jeffrey&rft.au=Phillips,+Justin&rft.au=Gabry,+Jonah&rft.au=Trangucci,+Robert&rft_id=http://www.stat.columbia.edu/~gelman/research/unpublished/MRT(1).pdf&rfr_id=info:sid/en.wikipedia.org:Multilevel+regression+with+poststratification" class="Z3988">
5. ^ a b Downes, Marnie; Gurrin, Lyle C.; English, Dallas R.; Pirkis, Jane; Currier, Diane; Spital, Matthew J.; Carlin, John B. (August 2018) [9 April 2018]. "Multilevel Regression and Poststratification: A Modeling Approach to Estimating Population Quantities From Highly Selected Survey Samples". American Journal of Epidemiology. 187 (8). Oxford University Press: 1780–1790\. doi:10.1093/aje/kwy070. Retrieved 31 October 2019.1780-1790&rft.date=2018-08&rft_id=info:doi/10.1093/aje/kwy070&rft.aulast=Downes&rft.aufirst=Marnie&rft.au=Gurrin,+Lyle+C.&rft.au=English,+Dallas+R.&rft.au=Pirkis,+Jane&rft.au=Currier,+Diane&rft.au=Spital,+Matthew+J.&rft.au=Carlin,+John+B.&rft_id=https://academic.oup.com/aje/article/187/8/1780/4964985&rfr_id=info:sid/en.wikipedia.org:Multilevel+regression+with+poststratification" class="Z3988">
6. ^ Gelman, Andrew; Little, Thomas (1997). "Poststratification into many categories using hierarchical logistic regression". Survey Methodology. 23: 127–135.127-135&rft.date=1997&rft.aulast=Gelman&rft.aufirst=Andrew&rft.au=Little,+Thomas&rft_id=https://www150.statcan.gc.ca/n1/en/catalogue/12-001-X19970023616&rfr_id=info:sid/en.wikipedia.org:Multilevel+regression+with+poststratification" class="Z3988">
7. ^ Fay, Robert; Herriot, Roger (1979). "Estimates of income for small places: An application of James-Stein procedures to census data". Journal of the American Statistical Association. 74 (423): 1001–1012\. doi:10.1080/01621459.1979.10482505. JSTOR 2286322.1001-1012&rft.date=1979&rft_id=info:doi/10.1080/01621459.1979.10482505&rft_id=https://www.jstor.org/stable/2286322#id-name=JSTOR&rft.aulast=Fay&rft.aufirst=Robert&rft.au=Herriot,+Roger&rfr_id=info:sid/en.wikipedia.org:Multilevel+regression+with+poststratification" class="Z3988">
8. ^ Little, Roderick (1993). "Post-stratification: A modeler's perspective". Journal of the American Statistical Association. 88 (423): 1001–1012\. doi:10.1080/01621459.1993.10476368. JSTOR 2290792.1001-1012&rft.date=1993&rft_id=info:doi/10.1080/01621459.1993.10476368&rft_id=https://www.jstor.org/stable/2290792#id-name=JSTOR&rft.aulast=Little&rft.aufirst=Roderick&rfr_id=info:sid/en.wikipedia.org:Multilevel+regression+with+poststratification" class="Z3988">
9. ^ Wang, Wei; Rothschild, David; Goel, Sharad; Gelman, Andrew (2015). "Forecasting elections with non-representative polls" (PDF). International Journal of Forecasting. 31 (3): 980–991\. doi:10.1016/j.ijforecast.2014.06.001. Archived (PDF) from the original on 1 November 2020. Retrieved 1 December 2019.980-991&rft.date=2015&rft_id=info:doi/10.1016/j.ijforecast.2014.06.001&rft.aulast=Wang&rft.aufirst=Wei&rft.au=Rothschild,+David&rft.au=Goel,+Sharad&rft.au=Gelman,+Andrew&rft_id=https://www.microsoft.com/en-us/research/wp-content/uploads/2016/04/forecasting-with-nonrepresentative-polls.pdf&rfr_id=info:sid/en.wikipedia.org:Multilevel+regression+with+poststratification" class="Z3988">
10. ^ Revell, Timothy (9 June 2017). "How YouGov's experimental poll correctly called the UK election". New Scientist. Archived from the original on 9 June 2017. Retrieved 31 October 2019.
11. ^ Cohen, Daniel (27 September 2019). "'I've never known voters be so promiscuous': the pollsters working to predict the next UK election". The Guardian. Archived from the original on 11 September 2024. Retrieved 31 October 2019.
12. ^ Survation 2019 https://www.survation.com/2019-general-election-mrp-predictions-survation-and-dr-chris-hanretty/ Archived 16 February 2025 at the Wayback Machine
13. ^ Ipsos 2024 https://www.ipsos.com/en-uk/uk-opinion-polls/ipsos-election-mrp Archived 18 June 2024 at the Wayback Machine
14. ^ James, William; MacLellan, Kylie (15 October 2019). "A question of trust: British pollsters battle to call looming election". Reuters. Archived from the original on 31 October 2019. Retrieved 31 October 2019.
15. ^ Bisbee, James (2019). "BARP: Improving Mister P Using Bayesian Additive Regression Trees". American Political Science Review. 113 (4): 1060–1065\. doi:10.1017/S0003055419000480. S2CID 201385400.1060-1065&rft.date=2019&rft_id=info:doi/10.1017/S0003055419000480&rft_id=https://api.semanticscholar.org/CorpusID:201385400#id-name=S2CID&rft.aulast=Bisbee&rft.aufirst=James&rfr_id=info:sid/en.wikipedia.org:Multilevel+regression+with+poststratification" class="Z3988">
16. ^ Gelman, Andrew (28 October 2018). "MRP (or RPP) with non-census variables". Statistical Modeling, Causal Inference, and Social Science. Archived from the original on 22 June 2019. Retrieved 1 December 2019.

*[v]: View this template
*[t]: Discuss this template
*[e]: Edit this template