General linear model

Short description: Statistical linear model

Not to be confused with Multiple linear regression, Generalized linear model or General linear methods.

Part of a series on

Regression analysis
Models

* Linear regression
* Simple regression
* Polynomial regression
* General linear model

* Generalized linear model
* Discrete choice
* Binomial regression
* Binary regression
* Logistic regression
* Multinomial logit
* Mixed logit
* Probit
* Multinomial probit
* Ordered logit
* Ordered probit
* Poisson

* Multilevel model
* Fixed effects
* Random effects
* Linear mixed-effects model
* Nonlinear mixed-effects model

* Nonlinear regression
* Nonparametric
* Semiparametric
* Robust
* Quantile
* Isotonic
* Principal components
* Least angle
* Local
* Segmented

* Errors-in-variables

Estimation

* Least squares
* Linear
* Non-linear

* Ordinary
* Weighted
* Generalized

* Partial
* Total
* Non-negative
* Ridge regression
* Regularized

* Least absolute deviations
* Iteratively reweighted
* Bayesian
* Bayesian multivariate

Background

* Regression validation
* Mean and predicted response
* Errors and residuals
* Goodness of fit
* Studentized residual
* Gauss–Markov theorem

* v
* t
* e

The general linear model or general multivariate regression model is a compact way of simultaneously writing several multiple linear regression models. In that sense it is not a separate statistical linear model. The various multiple linear regression models may be compactly written as[1]

𝐘=𝐗𝐁+𝐔,

where Y is a matrix with series of multivariate measurements (each column being a set of measurements on one of the dependent variables), X is a matrix of observations on independent variables that might be a design matrix (each column being a set of observations on one of the independent variables), B is a matrix containing parameters that are usually to be estimated and U is a matrix containing errors (noise). The errors are usually assumed to be uncorrelated across measurements, and follow a multivariate normal distribution. If the errors do not follow a multivariate normal distribution, generalized linear models may be used to relax assumptions about Y and U.

The general linear model (GLM) encompasses several statistical models, including ANOVA, ANCOVA, MANOVA, MANCOVA, and ordinary linear regression. Within this framework, both the t-test and the F-test can be applied. The general linear model is a generalization of multiple linear regression to the case of more than one dependent variable. If Y, B, and U were column vectors, the matrix equation above would represent multiple linear regression.

Hypothesis tests with the general linear model can be made in two ways: multivariate or as several independent univariate tests. In multivariate tests the columns of Y are tested together, whereas in univariate tests the columns of Y are tested independently, i.e., as multiple univariate tests with the same design matrix.

## Contents

* 1 Comparison to multiple linear regression
* 2 Comparison to generalized linear model
* 3 Applications
* 4 See also
* 5 Notes
* 6 References
* 7 External links

## Comparison to multiple linear regression

Multiple linear regression is a generalization of simple linear regression to the case of more than one independent variable, and a special case of general linear models, restricted to one dependent variable. The basic model for multiple linear regression is

Yi=β0+β1Xi1+β2Xi2+…+βpXip+ϵi or more compactly Yi=β0+∑k=1pβkXik+ϵi

for each observation i = 1, ... , n.

In the formula above we consider n observations of one dependent variable and p independent variables. Thus, Yi is the ith observation of the dependent variable, Xik is ith observation of the kth independent variable, k = 1, 2, ..., p. The values βk represent parameters to be estimated, and εi is the ith independent identically distributed normal error.

In the more general multivariate linear regression, there is one equation of the above form for each of m > 1 dependent variables that share the same set of explanatory variables and hence are estimated simultaneously with each other:

Yij=β0j+β1jXi1+β2jXi2+…+βpjXip+ϵij or more compactly Yij=β0j+∑k=1pβkjXik+ϵij

for all observations indexed as i = 1, ... , n and for all dependent variables indexed as j = 1, ... , m.

Note that, since each dependent variable has its own set of regression parameters to be fitted, from a computational point of view the general multivariate regression is simply a sequence of standard multiple linear regressions using the same explanatory variables.

## Comparison to generalized linear model

The general linear model and the generalized linear model (GLM)[2][3] are two commonly used families of statistical methods to relate some number of continuous and/or categorical predictors to a single outcome variable.

The main difference between the two approaches is that the general linear model strictly assumes that the residuals will follow a conditionally normal distribution,[4] while the GLM loosens this assumption and allows for a variety of other distributions from the exponential family for the residuals.[2] The general linear model is a special case of the GLM in which the distribution of the residuals follow a conditionally normal distribution.

The distribution of the residuals largely depends on the type and distribution of the outcome variable; different types of outcome variables lead to the variety of models within the GLM family. Commonly used models in the GLM family include binary logistic regression[5] for binary or dichotomous outcomes, Poisson regression[6] for count outcomes, and linear regression for continuous, normally distributed outcomes. This means that GLM may be spoken of as a general family of statistical models or as specific models for specific outcome types.

| General linear model  | Generalized linear model

|
|

Typical estimation method  | Least squares, best linear unbiased prediction | Maximum likelihood or Bayesian
Examples  | ANOVA, ANCOVA, linear regression
Extensions and related methods  | MANOVA, MANCOVA, linear mixed model | generalized linear mixed model (GLMM), generalized estimating equations (GEE)
R package and function  | lm() in stats package (base R)  | glm() in stats package (base R) manova,
MATLAB function  | mvregress()  | glmfit()
SAS procedures  | PROC GLM, PROC REG | PROC GENMOD, PROC LOGISTIC (for binary & ordered or unordered categorical outcomes)
Stata command  | regress  | glm
SPSS command  | regression, glm | genlin, logistic
Wolfram Language & Mathematica function  | LinearModelFit[][7] | GeneralizedLinearModelFit[][8]
EViews command  | ls[9] | glm[10]
statsmodels Python Package  | regression-and-linear-models | GLM

## Applications

An application of the general linear model appears in the analysis of multiple brain scans in scientific experiments where Y contains data from brain scanners, X contains experimental design variables and confounds. It is usually tested in a univariate way (usually referred to a mass-univariate in this setting) and is often referred to as statistical parametric mapping.[11]

## See also

* Bayesian multivariate linear regression
* F-test
* t-test

## Notes

1. ↑ Mardia, K. V.; Kent, J. T.; Bibby, J. M. (1979). Multivariate Analysis. Academic Press. ISBN 0-12-471252-5.
2. ↑ 2.0 2.1 McCullagh, P.; Nelder, J. A. (January 1, 1983). "An outline of generalized linear models". Generalized Linear Models. Springer US. pp. 21–47. doi:10.1007/978-1-4899-3242-6_2. ISBN 9780412317606.
3. ↑ Fox, J. (2015). Applied regression analysis and generalized linear models. Sage Publications.
4. ↑ Cohen, J.; Cohen, P.; West, S. G.; Aiken, L. S. (2003). ﻿Applied multiple regression/correlation analysis for the behavioral sciences﻿ (Report).
5. ↑ Hosmer Jr, D. W., Lemeshow, S., & Sturdivant, R. X. (2013). Applied logistic regression (Vol. 398). John Wiley & Sons.
6. ↑ Gardner, W.; Mulvey, E. P.; Shaw, E. C. (1995). "Regression analyses of counts and rates: Poisson, overdispersed Poisson, and negative binomial models". Psychological Bulletin 118 (3): 392–404. doi:10.1037/0033-2909.118.3.392. PMID 7501743.
7. ↑ LinearModelFit, Wolfram Language Documentation Center.
8. ↑ GeneralizedLinearModelFit, Wolfram Language Documentation Center.
9. ↑ ls, EViews Help.
10. ↑ glm, EViews Help.
11. ↑ Friston, K.J.; Holmes, A.P.; Worsley, K.J.; Poline, J.-B.; Frith, C.D.; Frackowiak, R.S.J. (1995). "Statistical Parametric Maps in functional imaging: A general linear approach". Human Brain Mapping 2 (4): 189–210. doi:10.1002/hbm.460020402.

## References

* Christensen, Ronald (2020). Plane Answers to Complex Questions: The Theory of Linear Models (5th ed.). New York: Springer. ISBN 978-3-030-32096-6.
* Wichura, Michael J. (2006). The coordinate-free approach to linear models. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge: Cambridge University Press. pp. xiv+199. ISBN 978-0-521-86842-6.
* Rawlings, John O.; Pantula, Sastry G.; Dickey, David A., eds (1998). Applied Regression Analysis. Springer Texts in Statistics. doi:10.1007/b98890. ISBN 0-387-98454-2.

* v
* t
* e

Statistics

* Outline
* Index

Descriptive statistics

| Continuous data|

| Center|

* Mean
* arithmetic
* geometric
* harmonic
* Median
* Mode

|

Dispersion|

* Variance
* Standard deviation
* Coefficient of variation
* Percentile
* Range
* Interquartile range

Shape|

* Central limit theorem
* Moments
* Skewness
* Kurtosis
* L-moments

Count data|

* Index of dispersion

Summary tables|

* Grouped data
* Frequency distribution
* Contingency table

Dependence|

* Pearson product-moment correlation
* Rank correlation
* Spearman's rho
* Kendall's tau
* Partial correlation
* Scatter plot

Graphics|

* Bar chart
* Biplot
* Box plot
* Control chart
* Correlogram
* Fan chart
* Forest plot
* Histogram
* Pie chart
* Q–Q plot
* Run chart
* Scatter plot
* Stem-and-leaf display
* Radar chart
* Violin plot

Data collection

| Study design|

* Population
* Statistic
* Effect size
* Statistical power
* Optimal design
* Sample size determination
* Replication
* Missing data

|

Survey methodology|

* Sampling
* stratified
* cluster
* Standard error
* Opinion poll
* Questionnaire

Controlled experiments|

* Scientific control
* Randomized experiment
* Randomized controlled trial
* Random assignment
* Blocking
* Interaction
* Factorial experiment

Adaptive Designs|

* Adaptive clinical trial
* Up-and-Down Designs
* Stochastic approximation

Observational Studies|

* Cross-sectional study
* Cohort study
* Natural experiment
* Quasi-experiment

Statistical inference

| Statistical theory|

* Population
* Statistic
* Probability distribution
* Sampling distribution
* Order statistic
* Empirical distribution
* Density estimation
* Statistical model
* Model specification
* Lp space
* Parameter
* location
* scale
* shape
* Parametric family
* Likelihood (monotone)
* Location–scale family
* Exponential family
* Completeness
* Sufficiency
* Statistical functional
* Bootstrap
* U
* V
* Optimal decision
* loss function
* Efficiency
* Statistical distance
* divergence
* Asymptotics
* Robustness

|

Frequentist inference|

| Point estimation|

* Estimating equations
* Maximum likelihood
* Method of moments
* M-estimator
* Minimum distance
* Unbiased estimators
* Mean-unbiased minimum-variance
* Rao–Blackwellization
* Lehmann–Scheffé theorem
* Median unbiased
* Plug-in

|

Interval estimation|

* Confidence interval
* Pivot
* Likelihood interval
* Prediction interval
* Tolerance interval
* Resampling
* Bootstrap
* Jackknife

Testing hypotheses|

* 1- & 2-tails
* Power
* Uniformly most powerful test
* Permutation test
* Randomization test
* Multiple comparisons

Parametric tests|

* Likelihood-ratio
* Score/Lagrange multiplier
* Wald

Specific tests|

* Z-test (normal)
* Student's t-test
* F-test

Goodness of fit|

* Chi-squared
* G-test
* Kolmogorov–Smirnov
* Anderson–Darling
* Lilliefors
* Jarque–Bera
* Normality (Shapiro–Wilk)
* Likelihood-ratio test
* Model selection
* Cross validation
* AIC
* BIC

Rank statistics|

* Sign
* Sample median
* Signed rank (Wilcoxon)
* Hodges–Lehmann estimator
* Rank sum (Mann–Whitney)
* Nonparametric anova
* 1-way (Kruskal–Wallis)
* 2-way (Friedman)
* Ordered alternative (Jonckheere–Terpstra)

Bayesian inference|

* Bayesian probability
* prior
* posterior
* Credible interval
* Bayes factor
* Bayesian estimator
* Maximum posterior estimator

* Correlation
* Regression analysis

| Correlation|

* Pearson product-moment
* Partial correlation
* Confounding variable
* Coefficient of determination

|

Regression analysis|

* Errors and residuals
* Regression validation
* Mixed effects models
* Simultaneous equations models
* Multivariate adaptive regression splines (MARS)

Linear regression|

* Simple linear regression
* Ordinary least squares
* General linear model
* Bayesian regression

Non-standard predictors|

* Nonlinear regression
* Nonparametric
* Semiparametric
* Isotonic
* Robust
* Heteroscedasticity
* Homoscedasticity

Generalized linear model|

* Exponential families
* Logistic (Bernoulli) / Binomial / Poisson regressions

Partition of variance|

* Analysis of variance (ANOVA, anova)
* Analysis of covariance
* Multivariate ANOVA
* Degrees of freedom

Categorical / Multivariate / Time-series / Survival analysis

| Categorical|

* Cohen's kappa
* Contingency table
* Graphical model
* Log-linear model
* McNemar's test

|

Multivariate|

* Regression
* Manova
* Principal components
* Canonical correlation
* Discriminant analysis
* Cluster analysis
* Classification
* Structural equation model
* Factor analysis
* Multivariate distributions
* Elliptical distributions
* Normal

Time-series|

| General|

* Decomposition
* Trend
* Stationarity
* Seasonal adjustment
* Exponential smoothing
* Cointegration
* Structural break
* Granger causality

|

Specific tests|

* Dickey–Fuller
* Johansen
* Q-statistic (Ljung–Box)
* Durbin–Watson
* Breusch–Godfrey

Time domain|

* Autocorrelation (ACF)
* partial (PACF)
* Cross-correlation (XCF)
* ARMA model
* ARIMA model (Box–Jenkins)
* Autoregressive conditional heteroskedasticity (ARCH)
* Vector autoregression (VAR)

Frequency domain|

* Spectral density estimation
* Fourier analysis
* Wavelet
* Whittle likelihood

Survival|

| Survival function|

* Kaplan–Meier estimator (product limit)
* Proportional hazards models
* Accelerated failure time (AFT) model
* First hitting time

|

Hazard function|

* Nelson–Aalen estimator

Test|

* Log-rank test

Applications

| Biostatistics|

* Bioinformatics
* Clinical trials / studies
* Epidemiology
* Medical statistics

|

Engineering statistics|

* Chemometrics
* Methods engineering
* Probabilistic design
* Process / quality control
* Reliability
* System identification

Social statistics|

* Actuarial science
* Census
* Crime statistics
* Demography
* Econometrics
* Jurimetrics
* National accounts
* Official statistics
* Population statistics
* Psychometrics

Spatial statistics|

* Cartography
* Environmental statistics
* Geographic information system
* Geostatistics
* Kriging

* Category
* '
* Commons
* WikiProject

## External links

* General Linear Models: A Practical Guide for Data-Driven Decisions

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/General linear model. Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=General_linear_model&oldid=4299334"
*[v]: View this template
*[t]: Discuss this template
*[e]: Edit this template