Score test

Short description: Statistical test based on the gradient of the likelihood function

In statistics, the score test assesses constraints on statistical parameters based on the gradient of the likelihood function—known as the score—evaluated at the hypothesized parameter value under the null hypothesis. Intuitively, if the restricted estimator is near the maximum of the likelihood function, the score should not differ from zero by more than sampling error. While the finite sample distributions of score tests are generally unknown, they have an asymptotic χ2-distribution under the null hypothesis as first proved by C. R. Rao in 1948,[1] a fact that can be used to determine statistical significance.

Since function maximization subject to equality constraints is most conveniently done using a Lagrangean expression of the problem, the score test can be equivalently understood as a test of the magnitude of the Lagrange multipliers associated with the constraints where, again, if the constraints are non-binding at the maximum likelihood, the vector of Lagrange multipliers should not differ from zero by more than sampling error. The equivalence of these two approaches was first shown by S. D. Silvey in 1959,[2] which led to the name Lagrange multiplier test that has become more commonly used, particularly in econometrics, since Breusch and Pagan's much-cited 1980 paper.[3]

The main advantage of the score test over the Wald test and likelihood-ratio test is that the score test only requires the computation of the restricted estimator.[4] This makes testing feasible when the unconstrained maximum likelihood estimate is a boundary point in the parameter space.[citation needed] Further, because the score test only requires the estimation of the likelihood function under the null hypothesis, it is less specific than the likelihood ratio test about the alternative hypothesis.[5]

## Contents

* 1 Single-parameter test
* 1.1 The statistic
* 1.1.1 Note on notation
* 1.2 As most powerful test for small deviations
* 1.3 Relationship with other hypothesis tests
* 2 Multiple parameters
* 3 Special cases
* 4 See also
* 5 References
* 6 Further reading

## Single-parameter test

### The statistic

Let [math]\displaystyle{ L }[/math] be the likelihood function which depends on a univariate parameter [math]\displaystyle{ \theta }[/math] and let [math]\displaystyle{ x }[/math] be the data. The score [math]\displaystyle{ U(\theta) }[/math] is defined as

[math]\displaystyle{ U(\theta)=\frac{\partial \log L(\theta \mid x)}{\partial \theta}. }[/math]

The Fisher information is[6]

[math]\displaystyle{ I(\theta) = - \operatorname{E} \left[\left. \frac{\partial^2}{\partial\theta^2} \log f(X;\theta)\,\right|\,\theta \right]\,, }[/math]

where ƒ is the probability density.

The statistic to test [math]\displaystyle{ \mathcal{H}_0:\theta=\theta_0 }[/math] is [math]\displaystyle{ S(\theta_0) = \frac{U(\theta_0)^2}{I(\theta_0)} }[/math]

which has an asymptotic distribution of [math]\displaystyle{ \chi^2_1 }[/math], when [math]\displaystyle{ \mathcal{H}_0 }[/math] is true. While asymptotically identical, calculating the LM statistic using the outer-gradient-product estimator of the Fisher information matrix can lead to bias in small samples.[7]

#### Note on notation

Note that some texts use an alternative notation, in which the statistic [math]\displaystyle{ S^*(\theta)=\sqrt{ S(\theta) } }[/math] is tested against a normal distribution. This approach is equivalent and gives identical results.

### As most powerful test for small deviations

[math]\displaystyle{ \left(\frac{\partial \log L(\theta \mid x)}{\partial \theta}\right)_{\theta=\theta_0} \geq C }[/math]

where [math]\displaystyle{ L }[/math] is the likelihood function, [math]\displaystyle{ \theta_0 }[/math] is the value of the parameter of interest under the null hypothesis, and [math]\displaystyle{ C }[/math] is a constant set depending on the size of the test desired (i.e. the probability of rejecting [math]\displaystyle{ H_0 }[/math] if [math]\displaystyle{ H_0 }[/math] is true; see Type I error).

The score test is the most powerful test for small deviations from [math]\displaystyle{ H_0 }[/math]. To see this, consider testing [math]\displaystyle{ \theta=\theta_0 }[/math] versus [math]\displaystyle{ \theta=\theta_0+h }[/math]. By the Neyman–Pearson lemma, the most powerful test has the form

[math]\displaystyle{ \frac{L(\theta_0+h\mid x)}{L(\theta_0\mid x)} \geq K; }[/math]

Taking the log of both sides yields

[math]\displaystyle{ \log L(\theta_0 + h \mid x ) - \log L(\theta_0\mid x) \geq \log K. }[/math]

The score test follows making the substitution (by Taylor series expansion)

[math]\displaystyle{ \log L(\theta_0+h\mid x) \approx \log L(\theta_0\mid x) + h\times \left(\frac{\partial \log L(\theta \mid x)}{\partial \theta}\right)_{\theta=\theta_0} }[/math]

and identifying the [math]\displaystyle{ C }[/math] above with [math]\displaystyle{ \log(K) }[/math].

### Relationship with other hypothesis tests

If the null hypothesis is true, the likelihood ratio test, the Wald test, and the Score test are asymptotically equivalent tests of hypotheses.[8][9] When testing nested models, the statistics for each test then converge to a Chi-squared distribution with degrees of freedom equal to the difference in degrees of freedom in the two models. If the null hypothesis is not true, however, the statistics converge to a noncentral chi-squared distribution with possibly different noncentrality parameters.

## Multiple parameters

A more general score test can be derived when there is more than one parameter. Suppose that [math]\displaystyle{ \widehat{\theta}_0 }[/math] is the maximum likelihood estimate of [math]\displaystyle{ \theta }[/math] under the null hypothesis [math]\displaystyle{ H_0 }[/math] while [math]\displaystyle{ U }[/math] and [math]\displaystyle{ I }[/math] are respectively, the score vector and the Fisher information matrix. Then

[math]\displaystyle{ U^T(\widehat{\theta}_0) I^{-1}(\widehat{\theta}_0) U(\widehat{\theta}_0) \sim \chi^2_k }[/math]

asymptotically under [math]\displaystyle{ H_0 }[/math], where [math]\displaystyle{ k }[/math] is the number of constraints imposed by the null hypothesis and

[math]\displaystyle{ U(\widehat{\theta}_0) = \frac{\partial \log L(\widehat{\theta}_0 \mid x)}{\partial \theta} }[/math]

and

[math]\displaystyle{ I(\widehat{\theta}_0) = -\operatorname E\left(\frac{\partial^2 \log L(\widehat{\theta}_0 \mid x)}{\partial \theta \, \partial \theta'} \right). }[/math]

This can be used to test [math]\displaystyle{ H_0 }[/math].

The actual formula for the test statistic depends on which estimator of the Fisher information matrix is being used.[10]

## Special cases

In many situations, the score statistic reduces to another commonly used statistic.[11]

In linear regression, the Lagrange multiplier test can be expressed as a function of the F-test.[12]

When the data follows a normal distribution, the score statistic is the same as the t statistic.[clarification needed]

When the data consists of binary observations, the score statistic is the same as the chi-squared statistic in the Pearson's chi-squared test.

## See also

* Fisher information
* Uniformly most powerful test
* Score (statistics)
* Sup-LM test

## References

1. ↑ Rao, C. Radhakrishna (1948). "Large sample tests of statistical hypotheses concerning several parameters with applications to problems of estimation". Mathematical Proceedings of the Cambridge Philosophical Society 44 (1): 50–57. doi:10.1017/S0305004100023987.
2. ↑ Silvey, S. D. (1959). "The Lagrangian Multiplier Test". Annals of Mathematical Statistics 30 (2): 389–407. doi:10.1214/aoms/1177706259.
3. ↑ Breusch, T. S.; Pagan, A. R. (1980). "The Lagrange Multiplier Test and its Applications to Model Specification in Econometrics". Review of Economic Studies 47 (1): 239–253.
4. ↑ Fahrmeir, Ludwig; Kneib, Thomas; Lang, Stefan; Marx, Brian (2013). Regression : Models, Methods and Applications. Berlin: Springer. pp. 663–664. ISBN 978-3-642-34332-2. https://archive.org/details/regressionmodels00fahr.
5. ↑ Kennedy, Peter (1998). A Guide to Econometrics (Fourth ed.). Cambridge: MIT Press. p. 68. ISBN 0-262-11235-3.
6. ↑ Lehmann and Casella, eq. (2.5.16).
7. ↑ Davidson, Russel; MacKinnon, James G. (1983). "Small sample properties of alternative forms of the Lagrange Multiplier test". Economics Letters 12 (3–4): 269–275. doi:10.1016/0165-1765(83)90048-4.
8. ↑ Engle, Robert F. (1983). "Wald, Likelihood Ratio, and Lagrange Multiplier Tests in Econometrics". in Intriligator, M. D.. Handbook of Econometrics. II. Elsevier. pp. 796–801. ISBN 978-0-444-86185-6.
9. ↑ Burzykowski, Andrzej Gałecki, Tomasz (2013). Linear mixed-effects models using R : a step-by-step approach. New York, NY: Springer. ISBN 1461438993.
10. ↑ Taboga, Marco. "Lectures on Probability Theory and Mathematical Statistics". https://www.statlect.com/fundamentals-of-statistics/score-test.
11. ↑ Cook, T. D.; DeMets, D. L., eds (2007). Introduction to Statistical Methods for Clinical Trials. Chapman and Hall. pp. 296–297. ISBN 1-58488-027-9.
12. ↑ Vandaele, Walter (1981). "Wald, likelihood ratio, and Lagrange multiplier tests as an F test". Economics Letters 8 (4): 361–365. doi:10.1016/0165-1765(81)90026-4.

## Further reading

* Buse, A. (1982). "The Likelihood Ratio, Wald, and Lagrange Multiplier Tests: An Expository Note". The American Statistician 36 (3a): 153–157. doi:10.1080/00031305.1982.10482817.
* Godfrey, L. G. (1988). "The Lagrange Multiplier Test and Testing for Misspecification : An Extended Analysis". Misspecification Tests in Econometrics. New York: Cambridge University Press. pp. 69–99. ISBN 0-521-26616-5.
* Ma, Jun; Nelson, Charles R. (2016). "The superiority of the LM test in a class of econometric models where the Wald test performs poorly". Unobserved Components and Time Series Econometrics. Oxford University Press. pp. 310–330. doi:10.1093/acprof:oso/9780199683666.003.0014. ISBN 978-0-19-968366-6.
* Rao, C. R. (2005). "Score Test: Historical Review and Recent Developments". Advances in Ranking and Selection, Multiple Comparisons, and Reliability. Boston: Birkhäuser. pp. 3–20. ISBN 978-0-8176-3232-8.

* v
* t
* e

Statistics

* Outline
* Index

Descriptive statistics

| Continuous data|

| Center|

* Mean
* arithmetic
* geometric
* harmonic
* Median
* Mode

|

Dispersion|

* Variance
* Standard deviation
* Coefficient of variation
* Percentile
* Range
* Interquartile range

Shape|

* Central limit theorem
* Moments
* Skewness
* Kurtosis
* L-moments

Count data|

* Index of dispersion

Summary tables|

* Grouped data
* Frequency distribution
* Contingency table

Dependence|

* Pearson product-moment correlation
* Rank correlation
* Spearman's rho
* Kendall's tau
* Partial correlation
* Scatter plot

Graphics|

* Bar chart
* Biplot
* Box plot
* Control chart
* Correlogram
* Fan chart
* Forest plot
* Histogram
* Pie chart
* Q–Q plot
* Run chart
* Scatter plot
* Stem-and-leaf display
* Radar chart
* Violin plot

Data collection

| Study design|

* Population
* Statistic
* Effect size
* Statistical power
* Optimal design
* Sample size determination
* Replication
* Missing data

|

Survey methodology|

* Sampling
* stratified
* cluster
* Standard error
* Opinion poll
* Questionnaire

Controlled experiments|

* Scientific control
* Randomized experiment
* Randomized controlled trial
* Random assignment
* Blocking
* Interaction
* Factorial experiment

Adaptive Designs|

* Adaptive clinical trial
* Up-and-Down Designs
* Stochastic approximation

Observational Studies|

* Cross-sectional study
* Cohort study
* Natural experiment
* Quasi-experiment

Statistical inference

| Statistical theory|

* Population
* Statistic
* Probability distribution
* Sampling distribution
* Order statistic
* Empirical distribution
* Density estimation
* Statistical model
* Model specification
* Lp space
* Parameter
* location
* scale
* shape
* Parametric family
* Likelihood (monotone)
* Location–scale family
* Exponential family
* Completeness
* Sufficiency
* Statistical functional
* Bootstrap
* U
* V
* Optimal decision
* loss function
* Efficiency
* Statistical distance
* divergence
* Asymptotics
* Robustness

|

Frequentist inference|

| Point estimation|

* Estimating equations
* Maximum likelihood
* Method of moments
* M-estimator
* Minimum distance
* Unbiased estimators
* Mean-unbiased minimum-variance
* Rao–Blackwellization
* Lehmann–Scheffé theorem
* Median unbiased
* Plug-in

|

Interval estimation|

* Confidence interval
* Pivot
* Likelihood interval
* Prediction interval
* Tolerance interval
* Resampling
* Bootstrap
* Jackknife

Testing hypotheses|

* 1- & 2-tails
* Power
* Uniformly most powerful test
* Permutation test
* Randomization test
* Multiple comparisons

Parametric tests|

* Likelihood-ratio
* Score/Lagrange multiplier
* Wald

Specific tests|

* Z-test (normal)
* Student's t-test
* F-test

Goodness of fit|

* Chi-squared
* G-test
* Kolmogorov–Smirnov
* Anderson–Darling
* Lilliefors
* Jarque–Bera
* Normality (Shapiro–Wilk)
* Likelihood-ratio test
* Model selection
* Cross validation
* AIC
* BIC

Rank statistics|

* Sign
* Sample median
* Signed rank (Wilcoxon)
* Hodges–Lehmann estimator
* Rank sum (Mann–Whitney)
* Nonparametric anova
* 1-way (Kruskal–Wallis)
* 2-way (Friedman)
* Ordered alternative (Jonckheere–Terpstra)

Bayesian inference|

* Bayesian probability
* prior
* posterior
* Credible interval
* Bayes factor
* Bayesian estimator
* Maximum posterior estimator

* Correlation
* Regression analysis

| Correlation|

* Pearson product-moment
* Partial correlation
* Confounding variable
* Coefficient of determination

|

Regression analysis|

* Errors and residuals
* Regression validation
* Mixed effects models
* Simultaneous equations models
* Multivariate adaptive regression splines (MARS)

Linear regression|

* Simple linear regression
* Ordinary least squares
* General linear model
* Bayesian regression

Non-standard predictors|

* Nonlinear regression
* Nonparametric
* Semiparametric
* Isotonic
* Robust
* Heteroscedasticity
* Homoscedasticity

Generalized linear model|

* Exponential families
* Logistic (Bernoulli) / Binomial / Poisson regressions

Partition of variance|

* Analysis of variance (ANOVA, anova)
* Analysis of covariance
* Multivariate ANOVA
* Degrees of freedom

Categorical / Multivariate / Time-series / Survival analysis

| Categorical|

* Cohen's kappa
* Contingency table
* Graphical model
* Log-linear model
* McNemar's test

|

Multivariate|

* Regression
* Manova
* Principal components
* Canonical correlation
* Discriminant analysis
* Cluster analysis
* Classification
* Structural equation model
* Factor analysis
* Multivariate distributions
* Elliptical distributions
* Normal

Time-series|

| General|

* Decomposition
* Trend
* Stationarity
* Seasonal adjustment
* Exponential smoothing
* Cointegration
* Structural break
* Granger causality

|

Specific tests|

* Dickey–Fuller
* Johansen
* Q-statistic (Ljung–Box)
* Durbin–Watson
* Breusch–Godfrey

Time domain|

* Autocorrelation (ACF)
* partial (PACF)
* Cross-correlation (XCF)
* ARMA model
* ARIMA model (Box–Jenkins)
* Autoregressive conditional heteroskedasticity (ARCH)
* Vector autoregression (VAR)

Frequency domain|

* Spectral density estimation
* Fourier analysis
* Wavelet
* Whittle likelihood

Survival|

| Survival function|

* Kaplan–Meier estimator (product limit)
* Proportional hazards models
* Accelerated failure time (AFT) model
* First hitting time

|

Hazard function|

* Nelson–Aalen estimator

Test|

* Log-rank test

Applications

| Biostatistics|

* Bioinformatics
* Clinical trials / studies
* Epidemiology
* Medical statistics

|

Engineering statistics|

* Chemometrics
* Methods engineering
* Probabilistic design
* Process / quality control
* Reliability
* System identification

Social statistics|

* Actuarial science
* Census
* Crime statistics
* Demography
* Econometrics
* Jurimetrics
* National accounts
* Official statistics
* Population statistics
* Psychometrics

Spatial statistics|

* Cartography
* Environmental statistics
* Geographic information system
* Geostatistics
* Kriging

* Category
* '
* Commons
* WikiProject

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/Score test. Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=Score_test&oldid=3357130"
*[v]: View this template
*[t]: Discuss this template
*[e]: Edit this template