Working–Hotelling procedure

Short description: Method of simultaneous inference

Part of a series on

Regression analysis
Models

* Linear regression
* Simple regression
* Polynomial regression
* General linear model

* Generalized linear model
* Discrete choice
* Binomial regression
* Binary regression
* Logistic regression
* Multinomial logit
* Mixed logit
* Probit
* Multinomial probit
* Ordered logit
* Ordered probit
* Poisson

* Multilevel model
* Fixed effects
* Random effects
* Linear mixed-effects model
* Nonlinear mixed-effects model

* Nonlinear regression
* Nonparametric
* Semiparametric
* Robust
* Quantile
* Isotonic
* Principal components
* Least angle
* Local
* Segmented

* Errors-in-variables

Estimation

* Least squares
* Linear
* Non-linear

* Ordinary
* Weighted
* Generalized

* Partial
* Total
* Non-negative
* Ridge regression
* Regularized

* Least absolute deviations
* Iteratively reweighted
* Bayesian
* Bayesian multivariate

Background

* Regression validation
* Mean and predicted response
* Errors and residuals
* Goodness of fit
* Studentized residual
* Gauss–Markov theorem

* v
* t
* e

In statistics, particularly regression analysis, the Working–Hotelling procedure, named after Holbrook Working and Harold Hotelling, is a method of simultaneous estimation in linear regression models. One of the first developments in simultaneous inference, it was devised by Working and Hotelling for the simple linear regression model in 1929.[1] It provides a confidence region for multiple mean responses, that is, it gives the upper and lower bounds of more than one value of a dependent variable at several levels of the independent variables at a certain confidence level. The resulting confidence bands are known as the Working–Hotelling–Scheffé confidence bands.

Like the closely related Scheffé's method in the analysis of variance, which considers all possible contrasts, the Working–Hotelling procedure considers all possible values of the independent variables; that is, in a particular regression model, the probability that all the Working–Hotelling confidence intervals cover the true value of the mean response is the confidence coefficient. As such, when only a small subset of the possible values of the independent variable is considered, it is more conservative and yields wider intervals than competitors like the Bonferroni correction at the same level of confidence. It outperforms the Bonferroni correction as more values are considered.

## Contents

* 1 Statement
* 1.1 Simple linear regression
* 1.2 Multiple linear regression
* 2 Graphical representation
* 3 Numerical example
* 4 Comparison with other methods
* 5 See also
* 6 Footnotes
* 7 Bibliography

## Statement

### Simple linear regression

Consider a simple linear regression model Y=β0+β1X+ε, where Y is the response variable and X the explanatory variable, and let b0 and b1 be the least-squares estimates of β0 and β1 respectively. Then the least-squares estimate of the mean response E(Yi) at the level X=xi is Yi^=b0+b1xi. It can then be shown, assuming that the errors independently and identically follow the normal distribution, that an 1−α confidence interval of the mean response at a certain level of X is as follows:

y^i∈[b0+b1xi±tα/2,df=n−2(1n−2∑j=1nej2)⋅(1n+(xi−x¯)2∑j=1n(xj−x¯)2)],

where (1n−2∑j=1nej2) is the mean squared error and tα/2,df=n−2 denotes the upper α2th percentile of Student's t-distribution with n−2 degrees of freedom.

However, as multiple mean responses are estimated, the confidence level declines rapidly. To fix the confidence coefficient at 1−α, the Working–Hotelling approach employs an F-statistic:[2][3]

y^i∈[b0+b1xi±W(1n−2∑j=1nej2)⋅(1n+(xi−x¯)2∑j=1n(xj−x¯)2)],

where W2=2Fα,df=(2,n−2) and F denotes the upper αth percentile of the F-distribution with (2,n−2) degrees of freedom. The confidence level of is 1−α over all values of X, i.e. xi∈ℝ.

### Multiple linear regression

The Working–Hotelling confidence bands can be easily generalised to multiple linear regression. Consider a general linear model as defined in the linear regressions article, that is,

𝐘=𝐗β+ε,

where

𝐘=(Y1Y2⋮Yn),𝐗=(𝐱1T𝐱2T⋮𝐱nT)=(x11⋯x1px21⋯x2p⋮⋱⋮xn1⋯xnp),β=(β1β2⋮βp),ε=(ε1ε2⋮εn).

Again, it can be shown that the least-squares estimate of the mean response E(Yi)=𝐱iTβ is Y^i=𝐱iT𝐛, where 𝐛 consists of least-square estimates of the entries in β, i.e. 𝐛=(𝐗T𝐗)−1𝐗T𝐘. Likewise, it can be shown that a 1−α confidence interval for a single mean response estimate is as follows:[4]

y^i∈[𝐱iT𝐛±tα/2,df=n−pMSE⁡(𝐱iT(𝐗T𝐗)−1𝐱i)],

where MSE is the observed value of the mean squared error (YTY−𝐛TXTY).

The Working–Hotelling approach to multiple estimations is similar to that of simple linear regression, with only a change in the degrees of freedom:[3]

y^i∈[𝐱iT𝐛±WMSE⁡(𝐱iT(𝐗T𝐗)−1𝐱i)],

where W2=2Fα,df=(p,n−p).

## Graphical representation

In the simple linear regression case, Working–Hotelling–Scheffé confidence bands, drawn by connecting the upper and lower limits of the mean response at every level, take the shape of hyperbolas. In drawing, they are sometimes approximated by the Graybill–Bowden confidence bands, which are linear and hence easier to graph:[2]

β0+β1(xi−x¯)∈[b0+b1(xi−x¯)±mα,2,df=n−2⋅(1n+|xi−x¯|∑j=1n(xj−x¯))]

where mα,2,df=n−2denotes the upper αth percentile of the Studentized maximum modulus distribution with two means and n−2 degrees of freedom.

The simple linear regression model with a Working–Hotelling confidence band.

## Numerical example

The same data in ordinary least squares are utilised in this example:

Height (m)  | 1.47 | 1.50 | 1.52 | 1.55 | 1.57 | 1.60 | 1.63 | 1.65 | 1.68 | 1.70 | 1.73 | 1.75 | 1.78 | 1.80 | 1.83

|
|
|
|
|
|
|
|
|
|
|
|
|
|
|

Weight (kg)  | 52.21 | 53.12 | 54.48 | 55.84 | 57.20 | 58.57 | 59.93 | 61.29 | 63.11 | 64.47 | 66.28 | 68.10 | 69.92 | 72.19 | 74.46

A simple linear regression model is fit to this data. The values of b0 and b1 have been found to be −39.06 and 61.27 respectively. The goal is to estimate the mean mass of women given their heights at the 95% confidence level. The value of W2 was found to be F0.95,df=(2,15−2)=2.758828. It was also found that x¯=1.651, ∑j=1nej2=7.490558, MSE⁡=0.5761968 and ∑j=1n(xj−x¯)2=693.3726. Then, to predict the mean mass of all women of a particular height, the following Working–Hotelling–Scheffé band has been derived:

y^i∈[−39.06+61.27xi±2.758828⋅0.5761968⋅(115+(xi−1.651)2693.3726)],

which results in the graph on the left.

## Comparison with other methods

Bonferroni bands for the same linear regression model, based on estimating the response variable given the observed values of X. The confidence bands are noticeably tighter.

The Working–Hotelling approach may give tighter or looser confidence limits compared to the Bonferroni correction. In general, for small families of statements, the Bonferroni bounds may be tighter, but when the number of estimated values increases, the Working–Hotelling procedure will yield narrower limits. This is because the confidence level of Working–Hotelling–Scheffé bounds is exactly 1−α when all values of the independent variables, i.e. xi∈ℝ, are considered. Alternatively, from an algebraic perspective, the critical value ±W remains constant as the number estimates of increases, whereas the corresponding values in Bonferonni estimates, ±t1−α/g,df=n−p, will be increasingly divergent as the number g of estimates increases. Therefore, the Working–Hotelling method is more suited for large-scale comparisons, whereas Bonferroni is preferred if only a few mean responses are to be estimated. In practice, both methods are usually used first and the narrower interval chosen.[4]

Another alternative to the Working–Hotelling–Scheffé band is the Gavarian band, which is used when a confidence band is needed that maintains equal widths at all levels.[5]

The Working–Hotelling procedure is based on the same principles as Scheffé's method, which gives family confidence intervals for all possible contrasts.[6] Their proofs are almost identical.[5] This is because both methods estimate linear combinations of mean response at all factor levels. However, the Working–Hotelling procedure does not deal with contrasts but with different levels of the independent variable, so there is no requirement that the coefficients of the parameters sum up to zero. Therefore, it has one more degree of freedom.[6]

## See also

* Multiple comparisons

## Footnotes

1. ↑ Miller (1966), p. 1
2. ↑ 2.0 2.1 Miller (2014)
3. ↑ 3.0 3.1 Neter, Wasserman and Kutner, pp. 163–165
4. ↑ 4.0 4.1 Neter, Wasserman and Kutner, pp. 244–245
5. ↑ 5.0 5.1 Miller (1966), pp. 123–127
6. ↑ 6.0 6.1 Westfall, Tobias and Wolfinger, pp. 277–280

## Bibliography

* Graybill, Franklin A.; Bowden, David C. (1967-06-01). "Linear Segment Confidence Bands for Simple Linear Models". Journal of the American Statistical Association 62 (318): 403–408. doi:10.1080/01621459.1967.10482917. ISSN 0162-1459.
* Miller, Rupert G. (1966). Simultaneous Statistical Inference. New York: Springer-Verlag. ISBN 978-1-4613-8124-2.
* Miller, R. (2014). "Multiple Comparisons I". Encyclopedia of Statistical Sciences. doi:10.1002/0471667196. ISBN 9780471667193. http://onlinelibrary.wiley.com/book/10.1002/0471667196;jsessionid=52A4434648CAAFAC4B5CF6F525F38E57.f04t04.
* Neter, John; Wasserman, William; Kutner, Michael (1990). Applied Linear Statistical Models. Tokyo: Richard D Irwin, Inc.. ISBN 978-0-256-08338-5.
* Westfall, Peter H; Tobias, R D; Wolfinger, Russell Dean (2011). Multiple comparisons and multiple tests using SAS. Cary, N.C.: SAS Pub.. ISBN 9781607648857.
* Working, Holbrook; Hotelling, Harold (1929-03-01). "Applications of the Theory of Error to the Interpretation of Trends". Journal of the American Statistical Association 24 (165A): 73–85. doi:10.1080/01621459.1929.10506274. ISSN 0162-1459.

* v
* t
* e

Least squares and regression analysis

Computational statistics|

* Least squares
* Linear least squares
* Non-linear least squares
* Iteratively reweighted least squares

Correlation and dependence|

* Pearson product-moment correlation
* Rank correlation (Spearman's rho
* Kendall's tau)
* Partial correlation
* Confounding variable

Regression analysis|

* Ordinary least squares
* Partial least squares
* Total least squares
* Ridge regression

Regression as a
statistical model|

| Linear regression|

* Simple linear regression
* Ordinary least squares
* Generalized least squares
* Weighted least squares
* General linear model

|

Predictor structure|

* Polynomial regression
* Growth curve (statistics)
* Segmented regression
* Local regression

Non-standard|

* Nonlinear regression
* Nonparametric
* Semiparametric
* Robust
* Quantile
* Isotonic

Non-normal errors|

* Generalized linear model
* Binomial
* Poisson
* Logistic

Decomposition of variance|

* Analysis of variance
* Analysis of covariance
* Multivariate AOV

Model exploration|

* Stepwise regression
* Model selection
* Mallows's Cp
* AIC
* BIC
* Model specification
* Regression validation

Background|

* Mean and predicted response
* Gauss–Markov theorem
* Errors and residuals
* Goodness of fit
* Studentized residual
* Minimum mean-square error

Design of experiments|

* Response surface methodology
* Optimal design
* Bayesian design

Numerical approximation|

* Numerical analysis
* Approximation theory
* Numerical integration
* Gaussian quadrature
* Orthogonal polynomials
* Chebyshev polynomials
* Chebyshev nodes

Applications|

* Curve fitting
* Calibration curve
* Numerical smoothing and differentiation
* System identification
* Moving least squares

* Regression analysis category
* Statistics category
*   * Statistics outline
* Statistics topics

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/Working–Hotelling procedure. Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=Working–Hotelling_procedure&oldid=4153503"
*[v]: View this template
*[t]: Discuss this template
*[e]: Edit this template