Nonlinear regression

Short description: Regression analysis

Part of a series on

Regression analysis
Models

* Linear regression
* Simple regression
* Polynomial regression
* General linear model

* Generalized linear model
* Discrete choice
* Binomial regression
* Binary regression
* Logistic regression
* Multinomial logit
* Mixed logit
* Probit
* Multinomial probit
* Ordered logit
* Ordered probit
* Poisson

* Multilevel model
* Fixed effects
* Random effects
* Linear mixed-effects model
* Nonlinear mixed-effects model

* Nonlinear regression
* Nonparametric
* Semiparametric
* Robust
* Quantile
* Isotonic
* Principal components
* Least angle
* Local
* Segmented

* Errors-in-variables

Estimation

* Least squares
* Linear
* Non-linear

* Ordinary
* Weighted
* Generalized

* Partial
* Total
* Non-negative
* Ridge regression
* Regularized

* Least absolute deviations
* Iteratively reweighted
* Bayesian
* Bayesian multivariate

Background

* Regression validation
* Mean and predicted response
* Errors and residuals
* Goodness of fit
* Studentized residual
* Gauss–Markov theorem

* v
* t
* e

See Michaelis–Menten kinetics for details

In statistics, nonlinear regression is a form of regression analysis in which observational data are modeled by a function which is a nonlinear combination of the model parameters and depends on one or more independent variables. The data are fitted by a method of successive approximations (iterations).

## Contents

* 1 General
* 2 Regression statistics
* 3 Ordinary and weighted least squares
* 4 Linearization
* 4.1 Transformation
* 4.2 Segmentation
* 5 See also
* 6 Notes
* 7 References
* 8 Further reading

## General

In nonlinear regression, a statistical model of the form,

𝐲∼f(𝐱,β)

relates a vector of independent variables, 𝐱, and its associated observed dependent variables, 𝐲. The function f is nonlinear in the components of the vector of parameters β, but otherwise arbitrary. For example, the Michaelis–Menten model for enzyme kinetics has two parameters and one independent variable, related by f by:[lower-alpha 1]

f(x,β)=β1xβ2+x

This function, which is a rectangular hyperbola, is nonlinear because it cannot be expressed as a linear combination of the two βs.

Systematic error may be present in the independent variables but its treatment is outside the scope of regression analysis. If the independent variables are not error-free, this is an errors-in-variables model, also outside this scope.

Other examples of nonlinear functions include exponential functions, logarithmic functions, trigonometric functions, power functions, Gaussian function, and Lorentz distributions. Some functions, such as the exponential or logarithmic functions, can be transformed so that they are linear. When so transformed, standard linear regression can be performed but must be applied with caution. See Lua error: no page name found in parameter |1=., below, for more details. In microbiology and biotechnology, nonlinear regression is used to model complex microbial growth kinetics. While simple growth follows monoauxic functions (such as the Gompertz or Boltzmann models), multiphasic (polyauxic) growth is modeled using linear combinations of these nonlinear functions. Estimating parameters for these complex models often requires robust regression techniques (e.g., using a Lorentzian loss function to mitigate outliers) and global optimization algorithms (such as Differential evolution with L-BFGS-B) to avoid local minima and ensure biologically interpretable results.[1][2]

In general, there is no closed-form expression for the best-fitting parameters, as there is in linear regression. Usually numerical optimization algorithms are applied to determine the best-fitting parameters. Again in contrast to linear regression, there may be many local minima of the function to be optimized and even the global minimum may produce a biased estimate. In practice, estimated values of the parameters are used, in conjunction with the optimization algorithm, to attempt to find the global minimum of a sum of squares.

For details concerning nonlinear data modeling see least squares and non-linear least squares.

## Regression statistics

The assumption underlying this procedure is that the model can be approximated by a linear function, namely a first-order Taylor series:

f(xi,β)≈f(xi,0)+∑jJijβj

where Jij=∂f(xi,β)∂βj are Jacobian matrix elements. It follows from this that the least squares estimators are given by

β^≈(𝐉𝐓𝐉)−𝟏𝐉𝐓𝐲, compare generalized least squares with covariance matrix proportional to the unit matrix. The nonlinear regression statistics are computed and used as in linear regression statistics, but using J in place of X in the formulas.

When the function f(xi,β) itself is not known analytically, but needs to be linearly approximated from n+1, or more, known values (where n is the number of estimators), the best estimator is obtained directly from the Linear Template Fit as [3]β^=((𝐘M~)TΩ−1𝐘M~)−1(𝐘M~)TΩ−1(𝐝−𝐘m¯) (see also linear least squares).

The linear approximation introduces bias into the statistics. Therefore, more caution than usual is required in interpreting statistics derived from a nonlinear model.

## Ordinary and weighted least squares

The best-fit curve is often assumed to be that which minimizes the sum of squared residuals. This is the ordinary least squares (OLS) approach. However, in cases where the dependent variable does not have constant variance, or there are some outliers, a sum of weighted squared residuals may be minimized; see weighted least squares. Each weight should ideally be equal to the reciprocal of the variance of the observation, or the reciprocal of the dependent variable to some power in the outlier case,[4] but weights may be recomputed on each iteration, in an iteratively weighted least squares algorithm.

## Linearization

### Transformation

Some nonlinear regression problems can be moved to a linear domain by a suitable transformation of the model formulation.

For example, consider the nonlinear regression problem

y=aebxU

with parameters a and b and with multiplicative error term U. If we take the logarithm of both sides, this becomes

ln⁡(y)=ln⁡(a)+bx+u,

where u = ln(U), suggesting estimation of the unknown parameters by a linear regression of ln(y) on x, a computation that does not require iterative optimization. However, use of a nonlinear transformation requires caution. The influences of the data values will change, as will the error structure of the model and the interpretation of any inferential results. These may not be desired effects. On the other hand, depending on what the largest source of error is, a nonlinear transformation may distribute the errors in a Gaussian fashion, so the choice to perform a nonlinear transformation must be informed by modeling considerations.

For Michaelis–Menten kinetics, the linear Lineweaver–Burk plot

1v=1Vmax+KmVmax[S]

of 1/v against 1/[S] has been much used. However, since it is very sensitive to data error and is strongly biased toward fitting the data in a particular range of the independent variable, [S], its use is strongly discouraged.

For error distributions that belong to the exponential family, a link function may be used to transform the parameters under the Generalized linear model framework.

### Segmentation

File:MUSTARD.JPGYield of mustard and soil salinity

Main article: Segmented regression

The independent or explanatory variable (say X) can be split up into classes or segments and linear regression can be performed per segment. Segmented regression with confidence analysis may yield the result that the dependent or response variable (say Y) behaves differently in the various segments.[5] For example, the figure shows that the soil salinity (X) initially exerts no influence on the crop yield (Y) of mustard, until a critical or threshold value (breakpoint), after which the yield is affected negatively.[6]

## See also

* Non-linear least squares
* Curve fitting
* Generalized linear model
* Local regression
* Response modeling methodology
* Genetic programming
* Multi expression programming
* Linear or quadratic template fit

## Notes

1. ↑ This model can also be expressed in the conventional biological notation: v=Vmax [S]Km+[S]

## References

1. ↑ Mockaitis, Gustavo (2025). "Mono- and Polyauxic Growth Kinetics: A Semi-Mechanistic Framework for Complex Biological Dynamics". arXiv:2507.05960 [q-bio.QM].
2. ↑ Mockaitis, Gustavo (2026). "Polyauxic Modeling Platform v1.0.0 (Streamlit)". doi:10.5281/zenodo.18025828. https://doi.org/10.5281/zenodo.18025828.
3. ↑ Britzger, Daniel (2022). "The Linear Template Fit". Eur. Phys. J. C 82 (8). doi:10.1140/epjc/s10052-022-10581-w. Bibcode: 2022EPJC...82..731B.
4. ↑ Motulsky, H.J.; Ransnas, L.A. (1987). "Fitting curves to data using nonlinear regression: a practical and nonmathematical review". The FASEB Journal 1 (5): 365–374. doi:10.1096/fasebj.1.5.3315805. PMID 3315805.
5. ↑ Oosterbaan, R. J. (1994). "Frequency and Regression Analysis". Drainage Principles and Applications. Wageningen: International Institute for Land Reclamation and Improvement (ILRI). pp. 175–224. ISBN 90-70754-33-9. http://www.waterlog.info/pdf/regtxt.pdf.
6. ↑ Oosterbaan, R. J. (2002). "Drainage research in farmers' fields: analysis of data.". Part of project "Liquid Gold" of the International Institute for Land Reclamation and Improvement (ILRI), Wageningen, The Netherlands.. http://www.waterlog.info/pdf/analysis.pdf.

## Further reading

* Bethea, R. M.; Duran, B. S.; Boullion, T. L. (1985). Statistical Methods for Engineers and Scientists. New York: Marcel Dekker. ISBN 0-8247-7227-X.
* Meade, N.; Islam, T. (1995). "Prediction Intervals for Growth Curve Forecasts". Journal of Forecasting 14 (5): 413–430. doi:10.1002/for.3980140502.
* Schittkowski, K. (2002). Data Fitting in Dynamical Systems. Boston: Kluwer. ISBN 1402010796.
* Seber, G. A. F.; Wild, C. J. (1989). Nonlinear Regression. New York: John Wiley and Sons. ISBN 0471617601.

* v
* t
* e

Statistics

* Outline
* Index

Descriptive statistics

| Continuous data|

| Center|

* Mean
* arithmetic
* geometric
* harmonic
* Median
* Mode

|

Dispersion|

* Variance
* Standard deviation
* Coefficient of variation
* Percentile
* Range
* Interquartile range

Shape|

* Central limit theorem
* Moments
* Skewness
* Kurtosis
* L-moments

Count data|

* Index of dispersion

Summary tables|

* Grouped data
* Frequency distribution
* Contingency table

Dependence|

* Pearson product-moment correlation
* Rank correlation
* Spearman's rho
* Kendall's tau
* Partial correlation
* Scatter plot

Graphics|

* Bar chart
* Biplot
* Box plot
* Control chart
* Correlogram
* Fan chart
* Forest plot
* Histogram
* Pie chart
* Q–Q plot
* Run chart
* Scatter plot
* Stem-and-leaf display
* Radar chart
* Violin plot

Data collection

| Study design|

* Population
* Statistic
* Effect size
* Statistical power
* Optimal design
* Sample size determination
* Replication
* Missing data

|

Survey methodology|

* Sampling
* stratified
* cluster
* Standard error
* Opinion poll
* Questionnaire

Controlled experiments|

* Scientific control
* Randomized experiment
* Randomized controlled trial
* Random assignment
* Blocking
* Interaction
* Factorial experiment

Adaptive Designs|

* Adaptive clinical trial
* Up-and-Down Designs
* Stochastic approximation

Observational Studies|

* Cross-sectional study
* Cohort study
* Natural experiment
* Quasi-experiment

Statistical inference

| Statistical theory|

* Population
* Statistic
* Probability distribution
* Sampling distribution
* Order statistic
* Empirical distribution
* Density estimation
* Statistical model
* Model specification
* Lp space
* Parameter
* location
* scale
* shape
* Parametric family
* Likelihood (monotone)
* Location–scale family
* Exponential family
* Completeness
* Sufficiency
* Statistical functional
* Bootstrap
* U
* V
* Optimal decision
* loss function
* Efficiency
* Statistical distance
* divergence
* Asymptotics
* Robustness

|

Frequentist inference|

| Point estimation|

* Estimating equations
* Maximum likelihood
* Method of moments
* M-estimator
* Minimum distance
* Unbiased estimators
* Mean-unbiased minimum-variance
* Rao–Blackwellization
* Lehmann–Scheffé theorem
* Median unbiased
* Plug-in

|

Interval estimation|

* Confidence interval
* Pivot
* Likelihood interval
* Prediction interval
* Tolerance interval
* Resampling
* Bootstrap
* Jackknife

Testing hypotheses|

* 1- & 2-tails
* Power
* Uniformly most powerful test
* Permutation test
* Randomization test
* Multiple comparisons

Parametric tests|

* Likelihood-ratio
* Score/Lagrange multiplier
* Wald

Specific tests|

* Z-test (normal)
* Student's t-test
* F-test

Goodness of fit|

* Chi-squared
* G-test
* Kolmogorov–Smirnov
* Anderson–Darling
* Lilliefors
* Jarque–Bera
* Normality (Shapiro–Wilk)
* Likelihood-ratio test
* Model selection
* Cross validation
* AIC
* BIC

Rank statistics|

* Sign
* Sample median
* Signed rank (Wilcoxon)
* Hodges–Lehmann estimator
* Rank sum (Mann–Whitney)
* Nonparametric anova
* 1-way (Kruskal–Wallis)
* 2-way (Friedman)
* Ordered alternative (Jonckheere–Terpstra)

Bayesian inference|

* Bayesian probability
* prior
* posterior
* Credible interval
* Bayes factor
* Bayesian estimator
* Maximum posterior estimator

* Correlation
* Regression analysis

| Correlation|

* Pearson product-moment
* Partial correlation
* Confounding variable
* Coefficient of determination

|

Regression analysis|

* Errors and residuals
* Regression validation
* Mixed effects models
* Simultaneous equations models
* Multivariate adaptive regression splines (MARS)

Linear regression|

* Simple linear regression
* Ordinary least squares
* General linear model
* Bayesian regression

Non-standard predictors|

* Nonlinear regression
* Nonparametric
* Semiparametric
* Isotonic
* Robust
* Heteroscedasticity
* Homoscedasticity

Generalized linear model|

* Exponential families
* Logistic (Bernoulli) / Binomial / Poisson regressions

Partition of variance|

* Analysis of variance (ANOVA, anova)
* Analysis of covariance
* Multivariate ANOVA
* Degrees of freedom

Categorical / Multivariate / Time-series / Survival analysis

| Categorical|

* Cohen's kappa
* Contingency table
* Graphical model
* Log-linear model
* McNemar's test

|

Multivariate|

* Regression
* Manova
* Principal components
* Canonical correlation
* Discriminant analysis
* Cluster analysis
* Classification
* Structural equation model
* Factor analysis
* Multivariate distributions
* Elliptical distributions
* Normal

Time-series|

| General|

* Decomposition
* Trend
* Stationarity
* Seasonal adjustment
* Exponential smoothing
* Cointegration
* Structural break
* Granger causality

|

Specific tests|

* Dickey–Fuller
* Johansen
* Q-statistic (Ljung–Box)
* Durbin–Watson
* Breusch–Godfrey

Time domain|

* Autocorrelation (ACF)
* partial (PACF)
* Cross-correlation (XCF)
* ARMA model
* ARIMA model (Box–Jenkins)
* Autoregressive conditional heteroskedasticity (ARCH)
* Vector autoregression (VAR)

Frequency domain|

* Spectral density estimation
* Fourier analysis
* Wavelet
* Whittle likelihood

Survival|

| Survival function|

* Kaplan–Meier estimator (product limit)
* Proportional hazards models
* Accelerated failure time (AFT) model
* First hitting time

|

Hazard function|

* Nelson–Aalen estimator

Test|

* Log-rank test

Applications

| Biostatistics|

* Bioinformatics
* Clinical trials / studies
* Epidemiology
* Medical statistics

|

Engineering statistics|

* Chemometrics
* Methods engineering
* Probabilistic design
* Process / quality control
* Reliability
* System identification

Social statistics|

* Actuarial science
* Census
* Crime statistics
* Demography
* Econometrics
* Jurimetrics
* National accounts
* Official statistics
* Population statistics
* Psychometrics

Spatial statistics|

* Cartography
* Environmental statistics
* Geographic information system
* Geostatistics
* Kriging

* Category
* '
* Commons
* WikiProject

* v
* t
* e

Least squares and regression analysis

Computational statistics|

* Least squares
* Linear least squares
* Non-linear least squares
* Iteratively reweighted least squares

Correlation and dependence|

* Pearson product-moment correlation
* Rank correlation (Spearman's rho
* Kendall's tau)
* Partial correlation
* Confounding variable

Regression analysis|

* Ordinary least squares
* Partial least squares
* Total least squares
* Ridge regression

Regression as a
statistical model|

| Linear regression|

* Simple linear regression
* Ordinary least squares
* Generalized least squares
* Weighted least squares
* General linear model

|

Predictor structure|

* Polynomial regression
* Growth curve (statistics)
* Segmented regression
* Local regression

Non-standard|

* Nonlinear regression
* Nonparametric
* Semiparametric
* Robust
* Quantile
* Isotonic

Non-normal errors|

* Generalized linear model
* Binomial
* Poisson
* Logistic

Decomposition of variance|

* Analysis of variance
* Analysis of covariance
* Multivariate AOV

Model exploration|

* Stepwise regression
* Model selection
* Mallows's Cp
* AIC
* BIC
* Model specification
* Regression validation

Background|

* Mean and predicted response
* Gauss–Markov theorem
* Errors and residuals
* Goodness of fit
* Studentized residual
* Minimum mean-square error

Design of experiments|

* Response surface methodology
* Optimal design
* Bayesian design

Numerical approximation|

* Numerical analysis
* Approximation theory
* Numerical integration
* Gaussian quadrature
* Orthogonal polynomials
* Chebyshev polynomials
* Chebyshev nodes

Applications|

* Curve fitting
* Calibration curve
* Numerical smoothing and differentiation
* System identification
* Moving least squares

* Regression analysis category
* Statistics category
*   * Statistics outline
* Statistics topics

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/Nonlinear regression. Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=Nonlinear_regression&oldid=4227625"
*[v]: View this template
*[t]: Discuss this template
*[e]: Edit this template