Partially linear model

Short description: Type of statistical model

This article may be too technical for most readers to understand. Please help improve it to make it understandable to non-experts, without removing the technical details. (June 2019) (Learn how and when to remove this template message)

|

A partially linear model is a form of semiparametric model, since it contains parametric and nonparametric elements. Application of the least squares estimators is available to partially linear model, if the hypothesis of the known of nonparametric element is valid. Partially linear equations were first used in the analysis of the relationship between temperature and usage of electricity by Engle, Granger, Rice and Weiss (1986).[1] Typical application of partially linear model in the field of Microeconomics is presented by Tripathi in the case of profitability of firm's production in 1997. Also, partially linear model applied successfully in some other academic field. In 1994, Zeger and Diggle introduced partially linear model into biometrics. In environmental science, Parda-Sanchez et al. used partially linear model to analysis collected data in 2000. So far, partially linear model was optimized in many other statistic methods. In 1988, Robinson applied Nadaraya-Waston kernel estimator to test the nonparametric element to build a least-squares estimator  After that, in 1997, local linear method was found by Truong.

## Contents

* 1 Partially linear model
* 1.1 Synopsis
* 1.1.1 Algebra equation
* 1.1.2 Equation components outline
* 1.2 Assumption
* 1.3 The least square (LS) estimators
* 1.4 History and applications of partially linear model
* 1.5 Development of partially linear model
* 1.6 Other econometrics tools in partially linear model
* 1.6.1 Bandwidth selection in partially linear model
* 1.6.2 Kernel-based profile and backfitting method
* 1.6.3 The penalized spline method
* 2 References

## Partially linear model

thumb

### Synopsis

#### Algebra equation

The algebra expression of partially linear model is written as:

yi=δTiβ+f(Ti)+μi[2]

#### Equation components outline

δTi and Ti: Vectors of explanatory variables. Independently random or fixed distributed variables.

β: To be measured Parameter.

μi: The random error in statistics with 0 mean.

f(Ti): To be measured part in partially linear model.

### Assumption

Source:[2]

Wolfgang, Hua Liang and Jiti Gao consider the assumptions and remarks of partially linear model under fixed and random design conditions.

When randomly distributed, introduce

Lj(Ti)=E(δi,j|Ti) and μi,j=δi,j−E(δi,j|Ti) (1)

E(||δ1||3)|T=t)is smaller than positive infinity when t value between 0 and 1, and the sum of covariance of δ1−E(δ1|T1) is positive. The random errors μ are independent of (δi,Ti),

When δi and Ti are fixed distributed, Ljvalued between 0 and 1, and δisatisfies δij=Lj(Ti)+μij, where factor i values between 1 and n, and factor j value between 1 and p, Error factor μijsatisfies, limn->∞1/n∑i=1nμiμiT=∑.

### The least square (LS) estimators

Source:[2]

The precondition of application of the least squares estimators is the existence of nonparametric component, and running at random distributed and fixed distributed cases.

Engle, Granger, Rice and Weiss's (1986) smoothing model should be first introduced, before applying the least squares estimators. The algebra function of their model is expressed as Y=δTβ+f(t) (2).

Wolfgang, Liang and Gao (1988) make an assumption that the pair (ß,g) satisfies 1/n∑i=1nE{Yi−δiTβ−f(Ti)}2=min1/n∑i=1nE{Yi−δiT−f(Ti)}2 (3).

This means that for all 1≤i≤n, δiTβ1+f1(Ti)=δiTβ2+f2(Ti).

So, f1=f2 and β1=β2.

Under random distributed case, Wolfgang, Hua Liang and Jiti Gao assume that for all 1 ≤ i ≤ n, E[Yi|(δi,Ti)]=δiTβ1+f1(Ti)=δiTβ2+f2(Ti) (4)

so, E{Yi−δiTβ1−f1(Ti)}2=E{Yi−δiTβ2−f2(Ti)}2+(β1−β2)TE{(δi−E[δi|Ti])(δi−E[δi|Ti]T)}(β1−β2)β1=β2, due to the fact that E{(δi−E[δi|Ti])(δi−E[δi|Ti]T)} is a positive number, as proved by function (1). So, fj(Ti)=E[Yi|Ti]−E[δiTβj|Ti]established for all 1≤i≤n and j equals to 1 and 2 when f1=f2.

Under fixed distributed case, By parameterizing factor  from smoothing model (2) as {f(T1),.........,f(Tn)}T=ωrandωr=Q(Y−xβ)where Q=ω(ωTω)−1ωT.

By making same assumption as (4), which follows from assumption (1), β1=β2and f1=f2under the fact of 1/nE{(Y−Xβ1−ωr1)T(Y−Xβ1−ωr1)}=1/nE(Y−Xβ2−ωr2)T(Y−Xβ2−ωr2)+1/n(β1−β2)TXT(1−Q)X(β1−β2).

Assuming factors δi,Ti,Yi(i here are positive integers) satisfies yi=δiTβ+f(Ti)+μiand establish positive weight functions ψni(t). Any estimators of f(t), for every β, we have fn(t;β)=∑i=1nψni(t)(Yi−δiTβ). By applying LS criterion, the LS estimator of βLS={(δ~Tδ~)}−1δ~TY~. The nonparametric estimator of f(n) is expressed as fn^(t)=∑i=1nψni(t)(Yi−δiTβLS). So, When the random errors are identically distributed, the estimators of variance σ2 is expressed as, σ^n2=1/n∑i=1n(Yi~−δiT~βLS).

### History and applications of partially linear model

The real-world application of partially linear model was first considered for analyzing data by Engle, Granger, Rice and Weiss in 1986.[2]

In their point of view, the relevance between temperature and the consumption of electricity cannot be expressed in a linear model, because there are massive of confounding factors, such as average income, goods price, consumer purchase ability and some other economic activities. Some of the factors are relevance with each other and might influence the observed result. Therefore, they introduced partially linear model, which contained both with parametric and nonparametric factors. The partially linear model enables and simplifies the linear transformation of data (Engle, Granger, Rice and Weiss, 1986). They also applied the smoothing spline technique for their research.

There was a case of application of partially linear model in biometrics by Zeger and Diggle in 1994. The research objective of their paper is the evolution period cycle of CD4 cell amounts in HIV (Human immune-deficiency virus) seroconverters (Zeger and Diggle, 1994).[3] CD4 cell plays a significant role in immune function in human body. Zeger and Diggle aimed to assess the proceed of disease by measuring the changing amount of CD4 cells. The number of CD4 cell is associated with body age and smoking behavior and so on. To clear the group of observation data in their experiment, Zeger and Diggle applied partially linear model for their work. Partially linear model primarily contributes to the estimation of average loss time of CD4 cells and adjusts the time dependence of some other covariables in order to simplify the proceed of data comparison, and also, the partially linear model characterizes the deviation of typical curve for their observed group to estimate the progression curve of the changing amount of CD4 cell. The deviation, granted by partially linear model, potentially helps to recognize the observed targets who had a slow progression on the amounting change of CD4 cells.

In 1999, Schmalensee and Stoker (1999) have used partially linear model in the field of economics. The independent variable of their research is the demand for gasoline in The United States. The primary research target in their paper is the relationship between gasoline consumption and long-run income elasticity in the U.S. Similarly, there are also massive of confounding variables, which might mutually affect. Hence, Schmalemsee and Stoker chose to deal with the issues of linear transformation of data between parametric and nonparametric by applying partially linear model.[4]

In the field of environment science, Prada-Sanchez used partially linear model to predict the sulfur dioxide pollution in 2000 (Prada-Sanchez, 2000),[5] and in the next year, Lin and Carroll applied partially linear model for clustered data (Lin and Carroall, 2001).[6]

### Development of partially linear model

According to Liang's paper in 2010 (Liang, 2010), The smoothing spline technique was introduced in partially linear model by Engle, Heckman and Rice in 1986. After that, Robinson found an available LS estimator for nonparametric factors in partially linear model in 1988. At the same year, profile LS method was recommended by Speckman.[7]

### Other econometrics tools in partially linear model

Kernel regression also was introduced in partially linear model. The local constant method, which is developed by Speckman, and local linear techniques, which was found by Hamilton and Truong in 1997 and was revised by Opsomer and Ruppert in 1997, are all included in kernel regression. Green et al., Opsomer and Ruppert found that one of the significant characteristic of kernel-based methods is that under-smoothing has been taken in order to find root-n estimator of beta. However, Speckman's research in 1988 and Severini's and Staniswalis's research in 1994 proved that those restriction might be canceled.

#### Bandwidth selection in partially linear model

Bandwidth selection in partially linear model is a confusing issue. Liang addressed a possible solution for this bandwidth selection in his literature by applying profile-kernel based method and backfitting methods. Also the necessity of undersmoothing for backfitting method and the reason why profile-kernel based method can work out the optimal bandwidth selection were justified by Liang. The general computation strategy is applied in Liang's literature for estimating nonparametric function. Moreover, the penalized spline method for partially linear models and intensive simulation experiments were introduced to discover the numerical feature of the penalized spline method, profile and backfitting methods.[7]

#### Kernel-based profile and backfitting method

Source:[7]

By introducing E(Y|T)=E(X|T)Tβ+g(T)

Following with Y−E(Y|T)=(X−E(X|T))Tβ+ϵ

The intuitive estimator of ß can be defined as the LS estimator after appropriately estimating E(Y|T) and E(X|T).

Then, for all random vector variable ξ, assume E^(ξ|T)is a kernel regression estimator of E(ξ|T). Let ξ~=ξ−E(ξ|T),∑X|T=covX−E(X|T). For example, X~i=Xi−E(Xi|Ti). Denote Y=(Y1,...,Yn)TX,g and T similarly. Let mx(t)=E(X|T=t),my(t)=E(Y|T=t). So ψ(mx,my,β,Y,X,T)=X−mx(T)[Y−my(T)−X−mx(T)Tβ]

The profile-kernel based estimators βp^solves,

0=∑i=1nψ(mx^,my^,β,Yi,Xi,Ti)

where mx^,my^are kernel estimators of mx and my.

#### The penalized spline method

Source:[7]

The penalized spline method was developed by Eilers and Marx in 1996. Ruppert and Carroll in 2000 and Brumback, Ruppert and Wand in 1999 employed this method in LME framework.

Assuming function g(t)can be estimated by g(t,τ)=τ0+τ1t+...+τptp+∑k=1Kbk(t−ξk)p

where p⩾1is an integer, and ξ1<...<ξkare fixed knots, a+=max(a,0).Denote τ=(tau0,...,τp)TConsider Y=XTβ+g(T,τ)+ϵ. The penalized spline estimator (βpsT^,τpsT^)Tof(βT,τT)Tis defined as follow

∑i=1n[Yi−XiTβi−g(Ti,τ)]2+α∑k=1Kbk2

Where αis a smoothing parameter.

As Brumback et al. mentioned in 1999,[8] the estimator (βpsT^,τpsT^)Tis same as the estimator of β based on LME model.

y=Λ(βT,τT)T+Zb+ϵ,

where Λ=(x11...x1d1T1...T1px21...x2d1T2...T2p.................................xn1...xnd1Tn...Tnp), Z=((T1−ξ1)p...(T1−ξK)+p(T2−ξ1)p...(T2−ξK)+p...............(Tn−ξ1)p...(Tn−ξK)+p)

Where b=(b1,...,bk)T∽(0,σb2),ϵ=(ϵ1,...,ϵn)T∼(0,σϵ2), and α=σϵ2/σb2. The matrix shows the penalized spline smoother for up above framework.

## References

1. ↑ Engle, Robert F.; Granger, C. W. J.; Rice, John; Weiss, Andrew (1986). "Semiparametric Estimates of the Relation Between Weather and Electricity Sales". Journal of the American Statistical Association 81 (394): 310–20. doi:10.2307/2289218.
2. ↑ 2.0 2.1 2.2 2.3 Hardle, Liang, JiTi, WolfGang, Hua, Gao (2000). Partially linear model. PHYSICA-VERLAG.
3. ↑ Zeger, Scott L.; Diggle, Peter J. (1994). "Semiparametric Models for Longitudinal Data with Application to CD4 Cell Numbers in HIV Seroconverters". Biometrics 50 (3): 689–699. doi:10.2307/2532783. ISSN 0006-341X. PMID 7981395.
4. ↑ Schmalensee, Richard; Stoker, Thomas M. (1999). "Household Gasoline Demand in the United States" (in en). Econometrica 67 (3): 645–662. doi:10.1111/1468-0262.00041. ISSN 1468-0262. https://dspace.mit.edu/bitstream/1721.1/50215/1/35721241.pdf.
5. ↑ Prada‐Sánchez, J. M.; Febrero‐Bande, M.; Cotos‐Yáñez, T.; González‐Manteiga, W.; Bermúdez‐Cela, J. L.; Lucas‐Domínguez, T. (2000). "Prediction of SO2 pollution incidents near a power station using partially linear models and an historical matrix of predictor-response vectors" (in en). Environmetrics 11 (2): 209–225. doi:10.1002/(SICI)1099-095X(200003/04)11:2<209::AID-ENV403>3.0.CO;2-Z. ISSN 1099-095X.
6. ↑ Carroll, Raymond J.; Lin, Xihong (2001-12-01). "Semiparametric regression for clustered data" (in en). Biometrika 88 (4): 1179–1185. doi:10.1093/biomet/88.4.1179. ISSN 0006-3444.
7. ↑ 7.0 7.1 7.2 7.3 Liang, Hua (2006-02-10). "Estimation in Partially Linear Models and Numerical Comparisons". Computational Statistics & Data Analysis 50 (3): 675–687. doi:10.1016/j.csda.2004.10.007. ISSN 0167-9473. PMID 20174596.
8. ↑ Brumback, Babette A.; Ruppert, David; Wand, M. P. (1999). "Variable Selection and Function Estimation in Additive Nonparametric Regression Using a Data-Based Prior: Comment". Journal of the American Statistical Association 94 (447): 794–797. doi:10.2307/2669991. ISSN 0162-1459.

* v
* t
* e

Statistics

* Outline
* Index

Descriptive statistics

| Continuous data|

| Center|

* Mean
* arithmetic
* geometric
* harmonic
* Median
* Mode

|

Dispersion|

* Variance
* Standard deviation
* Coefficient of variation
* Percentile
* Range
* Interquartile range

Shape|

* Central limit theorem
* Moments
* Skewness
* Kurtosis
* L-moments

Count data|

* Index of dispersion

Summary tables|

* Grouped data
* Frequency distribution
* Contingency table

Dependence|

* Pearson product-moment correlation
* Rank correlation
* Spearman's rho
* Kendall's tau
* Partial correlation
* Scatter plot

Graphics|

* Bar chart
* Biplot
* Box plot
* Control chart
* Correlogram
* Fan chart
* Forest plot
* Histogram
* Pie chart
* Q–Q plot
* Run chart
* Scatter plot
* Stem-and-leaf display
* Radar chart
* Violin plot

Data collection

| Study design|

* Population
* Statistic
* Effect size
* Statistical power
* Optimal design
* Sample size determination
* Replication
* Missing data

|

Survey methodology|

* Sampling
* stratified
* cluster
* Standard error
* Opinion poll
* Questionnaire

Controlled experiments|

* Scientific control
* Randomized experiment
* Randomized controlled trial
* Random assignment
* Blocking
* Interaction
* Factorial experiment

Adaptive Designs|

* Adaptive clinical trial
* Up-and-Down Designs
* Stochastic approximation

Observational Studies|

* Cross-sectional study
* Cohort study
* Natural experiment
* Quasi-experiment

Statistical inference

| Statistical theory|

* Population
* Statistic
* Probability distribution
* Sampling distribution
* Order statistic
* Empirical distribution
* Density estimation
* Statistical model
* Model specification
* Lp space
* Parameter
* location
* scale
* shape
* Parametric family
* Likelihood (monotone)
* Location–scale family
* Exponential family
* Completeness
* Sufficiency
* Statistical functional
* Bootstrap
* U
* V
* Optimal decision
* loss function
* Efficiency
* Statistical distance
* divergence
* Asymptotics
* Robustness

|

Frequentist inference|

| Point estimation|

* Estimating equations
* Maximum likelihood
* Method of moments
* M-estimator
* Minimum distance
* Unbiased estimators
* Mean-unbiased minimum-variance
* Rao–Blackwellization
* Lehmann–Scheffé theorem
* Median unbiased
* Plug-in

|

Interval estimation|

* Confidence interval
* Pivot
* Likelihood interval
* Prediction interval
* Tolerance interval
* Resampling
* Bootstrap
* Jackknife

Testing hypotheses|

* 1- & 2-tails
* Power
* Uniformly most powerful test
* Permutation test
* Randomization test
* Multiple comparisons

Parametric tests|

* Likelihood-ratio
* Score/Lagrange multiplier
* Wald

Specific tests|

* Z-test (normal)
* Student's t-test
* F-test

Goodness of fit|

* Chi-squared
* G-test
* Kolmogorov–Smirnov
* Anderson–Darling
* Lilliefors
* Jarque–Bera
* Normality (Shapiro–Wilk)
* Likelihood-ratio test
* Model selection
* Cross validation
* AIC
* BIC

Rank statistics|

* Sign
* Sample median
* Signed rank (Wilcoxon)
* Hodges–Lehmann estimator
* Rank sum (Mann–Whitney)
* Nonparametric anova
* 1-way (Kruskal–Wallis)
* 2-way (Friedman)
* Ordered alternative (Jonckheere–Terpstra)

Bayesian inference|

* Bayesian probability
* prior
* posterior
* Credible interval
* Bayes factor
* Bayesian estimator
* Maximum posterior estimator

* Correlation
* Regression analysis

| Correlation|

* Pearson product-moment
* Partial correlation
* Confounding variable
* Coefficient of determination

|

Regression analysis|

* Errors and residuals
* Regression validation
* Mixed effects models
* Simultaneous equations models
* Multivariate adaptive regression splines (MARS)

Linear regression|

* Simple linear regression
* Ordinary least squares
* General linear model
* Bayesian regression

Non-standard predictors|

* Nonlinear regression
* Nonparametric
* Semiparametric
* Isotonic
* Robust
* Heteroscedasticity
* Homoscedasticity

Generalized linear model|

* Exponential families
* Logistic (Bernoulli) / Binomial / Poisson regressions

Partition of variance|

* Analysis of variance (ANOVA, anova)
* Analysis of covariance
* Multivariate ANOVA
* Degrees of freedom

Categorical / Multivariate / Time-series / Survival analysis

| Categorical|

* Cohen's kappa
* Contingency table
* Graphical model
* Log-linear model
* McNemar's test

|

Multivariate|

* Regression
* Manova
* Principal components
* Canonical correlation
* Discriminant analysis
* Cluster analysis
* Classification
* Structural equation model
* Factor analysis
* Multivariate distributions
* Elliptical distributions
* Normal

Time-series|

| General|

* Decomposition
* Trend
* Stationarity
* Seasonal adjustment
* Exponential smoothing
* Cointegration
* Structural break
* Granger causality

|

Specific tests|

* Dickey–Fuller
* Johansen
* Q-statistic (Ljung–Box)
* Durbin–Watson
* Breusch–Godfrey

Time domain|

* Autocorrelation (ACF)
* partial (PACF)
* Cross-correlation (XCF)
* ARMA model
* ARIMA model (Box–Jenkins)
* Autoregressive conditional heteroskedasticity (ARCH)
* Vector autoregression (VAR)

Frequency domain|

* Spectral density estimation
* Fourier analysis
* Wavelet
* Whittle likelihood

Survival|

| Survival function|

* Kaplan–Meier estimator (product limit)
* Proportional hazards models
* Accelerated failure time (AFT) model
* First hitting time

|

Hazard function|

* Nelson–Aalen estimator

Test|

* Log-rank test

Applications

| Biostatistics|

* Bioinformatics
* Clinical trials / studies
* Epidemiology
* Medical statistics

|

Engineering statistics|

* Chemometrics
* Methods engineering
* Probabilistic design
* Process / quality control
* Reliability
* System identification

Social statistics|

* Actuarial science
* Census
* Crime statistics
* Demography
* Econometrics
* Jurimetrics
* National accounts
* Official statistics
* Population statistics
* Psychometrics

Spatial statistics|

* Cartography
* Environmental statistics
* Geographic information system
* Geostatistics
* Kriging

* Category
* '
* Commons
* WikiProject

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/Partially linear model. Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=Partially_linear_model&oldid=4226797"
*[v]: View this template
*[t]: Discuss this template
*[e]: Edit this template