Singular perturbation theory

Singular perturbation theory concerns the study of problems featuring a parameter for which the solutions of the problem at a limiting value of the parameter are different in character from the limit of the solutions of the general problem; namely, the limit is singular. In contrast, for regular perturbation problems, the solutions of the general problem converge to the solutions of the limit-problem as the parameter approaches the limit-value.

Though classically having their origins in the study of differential equations, singular perturbation problems occur in a broad array of contexts. It is not possible to provide an exhaustive list, but we discuss some of the common features below and provide references for further reading.

[edit] History and key ideas

A great deal of the early motivation in this area arose from studies of physical problems (O'Malley 1991, Cronin and O'Malley 1999). Notable examples are:

Poincare's work on time-scales for periodic phenomena in celestial mechanics
Prandtl's work on fluid flow (Van Dyke 1975)
van der Pol's work on electric circuits and oscillatory dynamics
studies of biological systems and chemical reaction kinetics by Segel and others

Each of these areas yield problems whose solutions have features that vary on disparate length- or time-scales. The philosophy behind singular perturbation theory is to take advantage of this separation of scales to obtain reduced problems that are simpler than the original full problem. Constructing an approximation of the full (global) solution of a singular problem in terms of the solutions of the reduced problems is a key element of the work in this field.

[edit] Algebraic equations

Although we do not include many fully worked problems herein, it is illustrative to consider a basic example in singularly perturbed algebraic problems. Algebraic equations serve as a good setting for introducing many of the fundamental issues also present in more general types of singular problems.

For comparison purposes, let us first consider the regular perturbation problem \[\tag{1} x^2 + 3\epsilon x -4=0\quad\mbox{as}\quad \epsilon\to 0. \]

The two solutions can be expressed in the form of regular expansions, \[\tag{2} x= x_0 + \delta_1(\epsilon) x_1 + \delta_2(\epsilon) x_2 + \delta_3(\epsilon)x_3 +\cdots \]

where the \(\delta(\epsilon)\) gauge functions define the asymptotic ordering of terms, \[\tag{3} 1\gg \delta_1 \gg \delta_2 \gg \delta_3 \gg \cdots \qquad \mbox{as}\quad\epsilon \to 0. \]

This defines the nontrivial leading order term \(x_0\) as the asymptotic approximation to the full solution for \(\epsilon\to 0\) with the higher order terms in the expansions being viewed as successively smaller corrections to \(x_0\) in that limit. The leading-order problem is obtained by evaluating (1) at the limit-value, \(\epsilon=0\ ,\) \[\tag{4} x_0^2 -4=0, \]

yielding two solutions, \(x_0=\pm 2\ ,\) as expected since the full problem is a quadratic equation. Substituting (2) into (1) and balancing terms for \(\epsilon\to 0\) determines the gauge functions as \(\delta_n=\epsilon^n\) and values for the coefficients \(x_n\) for the higher order terms in each of the two roots, \(x\sim \pm 2-{3\over 2} \epsilon \pm {9\over 16}\epsilon^2\ ,\) where the symbol \(\sim\) identifies two functions as being asymptotically equivalent, say \(f(\epsilon)\sim g(\epsilon)\ ,\) if \(\lim_{\epsilon\to 0} f(\epsilon)/g(\epsilon)=1\ .\)

In contrast, consider the singularly perturbed equation, \[\tag{5} \epsilon x^2 +3x -4=0\qquad \epsilon\to 0. \]

Note that the small parameter now multiplies the highest order term. Consequently, using (2) yields the leading order problem \[\tag{6} 3x_0-4=0, \]

which has only one solution \(x_0=4/3\ .\) The loss of one root of the equation is a consequence of the other solution of (5) being singular for \(\epsilon\to 0\ ,\) and hence not of the form of the regular expansion ansatz (2).

[edit] Rescaling and distinguished limits

Solutions of many singular problems can be found by use of appropriate rescalings. In this case, consider \(x=\delta(\epsilon) X(\epsilon)\ ,\) where \(X(\epsilon)\) can be written as a regular expansion. Substituting into (5) yields \[\tag{7} \epsilon \delta^2 X_0^2 + 3\delta X_0 -4=0. \]

We must now decide which terms constitute the leading-order dominant balance (i.e. which terms are the dominant ones for a limit of the equation). For this problem there are two choices for \(\delta(\epsilon)\) that yield consistent results (with all neglected terms being asymptotically subdominant):

\(\delta=1\ ,\) obtained by balancing the second and third terms in (7), yielding \(3X_0-4=0\ .\) This is the leading order problem for the regular solution, as obtained earlier (6).
\(\delta=1/\epsilon\ ,\) obtained by balancing the first and second terms in (7), yielding \(X_0^2+3X_0=0\) and producing \(X_0= -3\) the leading order term in the expansion of the singular solution \(x\sim -3/\epsilon.\)

The second case is referred to as the singular distinguished limit. For the regular example (1), in going to the leading order equation (4) the term \(\epsilon x=O(\epsilon)\to 0\) is dropped. For both solutions of (1) this term is indeed smaller than the \(O(1)\) terms in the dominant balance for (4). In going from (5) to (6) the \(\epsilon x^2\) term is similarly assumed to be vanishingly small relative to the two other terms comprising the leading order balance. This condition is true for the regular solution, with \(\epsilon x^2=O(\epsilon)\to 0\ ,\) but is violated for the singular solution, with \(\epsilon x^2 =O(1/\epsilon)\to \infty\ .\) This is a consequence of attempting to use the regular distinguished limit with the singular solution; the appropriate form of (7) for the singular solution is \(X^2+3X-4\epsilon=0\ ,\) corresponding to the singular dominant balance.

This elementary example captures many aspects that are common across singular perturbations for many classes of more complicated systems such as integral equations and generalized eigenvalue problems for matrices, \({ A}\vec{ x} = \lambda { B}(\epsilon)\vec{ x}\ .\) We also note that the above approach can be considered graphically in terms of so-called Kruskal-Newton graphs (White 2005); this graphical approach can also be extended to some of the differential equation problems described below.

[edit] Differential equations

Singular perturbation problems for differential equations can arise in a number of ways and are typically more complicated than their algebraic counterparts. Analogously though, solutions to the full equations when \(\epsilon=0\) can differ substantially (in number or form) from the limiting solutions as \(\epsilon\to 0\ ;\) in particular, solutions will have a non-uniform dependence on \(\epsilon\ .\)

In what follows, we describe three methodologies for investigating solutions to singularly perturbed differential equations. The choice of technique to be applied depends on the form of the problem and also on the desired properties to be studied.

[edit] WKB analysis

Singularly perturbed differential equations arise in many applications, such as wave propagation and quantum mechanics. A powerful approach coming from these areas applicable to linear homogeneous differential equations is the WKB method (after Wentzel-Kramers-Brillouin, also known as WKBJ for WKB+Jeffreys) (see Bender and Orszag 1999). Consider the second order equation, \[\tag{8} \epsilon {d^2y\over dx^2} +p(x) {dy\over dx} + q(x) y=0\quad\mbox{as}\quad\epsilon\rightarrow0. \]

Expressing the solution in terms of a phase function, \(y(x)=\exp(s(x))\ ,\) transforms the problem to a nonlinear equation for the phase, \[ \epsilon(s'' + {s'}^2) + p s' + q =0. \] Setting \(\epsilon=0\) reduces the problem to a first order ODE for \(s(x)\) where, analogous to the singular algebraic problem considered above.

Seeking distinguished limits, we scale the phase as \(s(x)=\delta(\epsilon) S(x)\) and expand \(S\) as a regular perturbation series. For this problem, \(S\sim S_0(x) + \epsilon S_1(x) + \epsilon^2 S_2(x)\ ,\) and there are two distinguished limits for \(\epsilon\to 0\ :\)

\(\delta=1\ ,\) with leading order problem \(p(x) S'_0 + q(x)=0\ ,\) corresponding to the regular solution, \(py_0' + q y_0=0\ .\)
\(\delta=1/\epsilon\ ,\) with leading order problem \({S_0'}^2 + p(x) S_0'=0\ ,\) sometimes called the eikonal equation.

In each case, the problem is reduced to a simpler algebraic equation in terms of \(S_0'\) with only parametric dependence on \(x\ .\) Obtaining uniform solutions of (8) from the the solutions for \(S(x)\) can be challenging when non-uniformities are introduced due to the forms of the coefficient functions in (8) in neighborhoods of so-called turning points (Wasow 1965). Progress can be made by studying Stokes phenomena and exponential asymptotics for the solutions in the complex plane (Chapman et al 1998).

[edit] Boundary layers and matched asymptotic expansions

Figure 1: Schematic of a boundary layer

Singularly perturbed differential equations can yield solutions containing regions of rapid variation (rapid compared to the regular length scale for the problem). These regions, which may be apparent in the solution or in its derivatives, are called `layers' and often appear at the boundary of the domain (as illustrated in Figure 1).

Constructing a solution to a differential equation or system involves several steps: identifying the locations of layers (boundary or internal), deriving asymptotic approximations to the solution in the different regions (corresponding to different distinguished limits in the equations), and ultimately, forming a uniformly valid solution over the entire domain. Solutions obtained for the layers (singular distinguished limits) are usually termed inner solutions while the slowly varying solutions for the regular distinguished limits are referred to as outer solutions.

The uniformly valid solution can be constructed through asymptotic matching of the inner and outer solutions, which relies on the fundamental assumption that the different solution forms overlap at on some identifiable region (see Figure 1). Procedures for matching asymptotic expansions have been examined by Kaplun, Van Dyke and others (Lagerstrom 1988, Van Dyke 1975, Kevorkian and Cole 1996, Eckhaus 1979), but there are still some fundamental theoretical issues to be resolved.

Unlike WKB theory, this approach can also be applied to nonlinear equations, and this versatility allows a wide range of problems to be tackled:

In dynamical systems, boundary layers-in-time commonly occur as initial layers, connecting the initial conditions to the regular or "slow" dynamics of the outer solution.
Relaxation oscillators are dynamical systems with periodic solutions composed of "fast" (interior layers) and "slow" (outer) dynamics. Canards are special solutions of this type.
Geometric singular perturbation theory provides a rigorous approach for describing solutions of singularly perturbed dynamical systems, based on Fenichel's analysis of the manifolds underlying the system (Jones 1995, Kaper in Cronin and O'Malley 1999, pp 85-132).
In some problems (especially in fluid dynamics and combustion), boundary layers can exist in nested forms; these are often known as triple-deck problems (Murdock 1999, Van Dyke 1975)
An important problem in partial differential equations is the construction and analysis of viscous regularizations of hyperbolic conservation laws –- shocks appearing in the solutions of the unregularized equations can be considered as singular limits of viscous interior layers (Kevorkian and Cole 1996, Zauderer 2007).
These and similar methods may be applied to problems involving difference equations and integral equations (Hoppensteadt, 2010).

[edit] Multiple scales analysis for long-time dynamics

A prototype of another class of singular perturbation problems is the van der Pol oscillator: \[\tag{9} {d^2y \over dt^2} + \epsilon (y^2-1) {dy\over dt} + y=0\quad\epsilon\to0. \]

This equation is not of the type described in the previous sections, as the small parameter does not multiply the highest order term; the equation does not reduce in order as \(\epsilon\to 0\ .\)

Equation (9) is a weakly-nonlinear oscillator, reducing to the linear oscillator equation at leading order. For finite, bounded times, \( t=O(1)\) as \(\epsilon\to 0\ ,\) solutions can be asymptotically approximated by application of the regular expansion, \(y(t)\sim y_0(t) +\epsilon y_1(t) +\cdots\ ,\) with the leading order solution, \(y_0(t)=A\cos(t+\phi)\ .\) However, at large times (\(t\to\infty\)), the naive regular expansion breaks-down due to the appearance of secular terms (terms which grow with time). This failure of the regular expansion can be traced to the fact that the limits \(\epsilon\to 0\) and \(t\to\infty\) do not commute, and this indicates that the expansion is not uniformly valid in time. The growing cumulative error in phase and amplitude apparent in the regular expansion is a consequence of the limitations of the ansatz, particularly in deviations from the unperturbed natural frequency of the system.

Various perturbation methods have been developed for dealing with such problems. These include:

Multiple scale analysis (sometimes known as two-timing) (Kevorkian and Cole 1996)
- The Poincare-Lindstedt method
- The method of strained parameters
Normal forms and the method of averaging (Sanders et al 2007)
Renormalization theory (DeVille et al 2008, Mudavanhu and O'Malley 2002)

Similar ideas also arise in homogenization theory, considering the averaging of spatially periodic structures of materials (Bensoussan et al 1978, Holmes 1995).

[edit] Concluding remarks

We have only scratched the surface of this research area, but it is hoped that the above illustrates the power and usefulness of the methods grouped under singular perturbation theory.

As discussed above, singular perturbation theory tackles difficult problems by investigating various reduced problems and then assembling the results together in an appropriate form. These reductions could, for example, simply be to a lower order polynomial in an algebraic problem, or could be more significant, such as in the reduction of a PDE to an ODE, or a functional equation to one of algebraic form. The reduced problems can still be mathematically challenging, with the construction of a uniformly valid solution requiring an involved analysis.

While some singular perturbation methods are based on rigorous analysis, the vast range of applications and available techniques typically restrict against such results. Consequently, the methods are often classed as formal techniques. However, this is not considered to be a significant problem: any a priori assumptions can be checked for consistency once suitable expansions have been derived; furthermore, formal results obtained by these techniques have been known to provide direction for additional rigorous theory (Smith 1985, Eckhaus 1979). In fact, some authors have seen the generality of the methods described above as representative of some more fundamental notion. For example, Kruskal went as far as to introduce the term asymptotology in referring to the art of dealing with applied mathematical systems in limiting cases (Kruskal 1963) and considered singular perturbation theory (and asymptotic methods in general) as a component of asymptotology.

[edit] References

Internal references

Jan A. Sanders (2006) Averaging. Scholarpedia, 1(11):1760.

Martin Wechselberger (2007) Canards. Scholarpedia, 2(4):1356.

Sylvio Ferraz-Mello (2009) Celestial mechanics. Scholarpedia, 4(1):4416.

Richard J. Field (2008) Chemical reaction kinetics. Scholarpedia, 3(10):4051.

James Meiss (2007) Dynamical systems. Scholarpedia, 2(2):1629.

Carson C. Chow (2007) Multiple scale analysis. Scholarpedia, 2(10):1617.

James Murdock (2006) Normal forms. Scholarpedia, 1(10):1902.

Andrei D. Polyanin, William E. Schiesser, Alexei I. Zhurov (2008) Partial differential equation. Scholarpedia, 3(10):4605.

Jeff Moehlis, Kresimir Josic, Eric T. Shea-Brown (2006) Periodic orbit. Scholarpedia, 1(7):1358.

Takashi Kanamaru (2007) Van der Pol oscillator. Scholarpedia, 2(1):2202.

[edit] Introductory Texts

Bender, C. M.; Orszag, S. A. Advanced mathematical methods for scientists and engineers. I. Asymptotic methods and perturbation theory. Springer-Verlag, New York, 1999.

Hinch, E. J. Perturbation methods. Cambridge Texts in Applied Mathematics. Cambridge University Press, Cambridge, 1991.

Holmes, M. H. Introduction to Perturbation Methods. Texts in Applied Mathematics, 20. Springer-Verlag, New York, 1995.

Lin, C. C. and Segel, L. A. Mathematics applied to deterministic problems in the natural sciences. Classics in Applied Mathematics 1, SIAM, Philadelphia, 1988.

Murdock, J. A. Perturbations: Theory and methods. Classics in Applied Mathematics, 27. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 1999.

Logan, J. D. Applied mathematics, 3rd ed. Wiley, New York, 2006.

Nayfeh, A. H. Perturbation methods. Wiley Classics Library. Wiley-Interscience, New York, 2000.

O'Malley, R. E., Jr. Singular perturbation methods for ordinary differential equations. Applied Mathematical Sciences, 89. Springer-Verlag, New York, 1991.

Verhulst, F. Methods and applications of singular perturbations: Boundary layers and multiple timescale dynamics. Texts in Applied Mathematics, 50. Springer, New York, 2005.

White, R. B. Asymptotic Analysis of Differential Equations, Imperial College Press, London, 2005.

Zauderer, E. Partial Differential Equations of Applied Mathematics, 3rd Ed, Wiley-Interscience, New Jersey, 2006.

[edit] Advanced Texts

Bensoussan, A. and Lions, J.-L. and Papanicolaou, G. Asymptotic analysis for periodic structures. North-Holland, Amsterdam, 1978.

Eckhaus, W. Asymptotic Analysis of Singular Perturbations, North-Holland, Amsterdam, 1979.

Hoppensteadt, F.C., Quasi-Steady State Analysis of Differential, Difference, Integral and Gradient Systems, Courant Lecture Notes 21, Amer. Math. Soc., 2010

Kevorkian, J. and Cole, J. D. Multiple scale and singular perturbation methods. Applied Mathematical Sciences, 114. Springer-Verlag, New York, 1996.

Lagerstrom, P. A. Matched asymptotic expansions: Ideas and techniques. Applied Mathematical Sciences, 76. Springer-Verlag, New York, 1988.

Sanders, J. A. and Verhulst, F. and Murdock, J. Averaging Methods in Nonlinear Dynamical Systems, 2nd Ed, Springer-Verlag, New York, 2007.

Smith, D. R. Singular-perturbation theory, Cambridge University Press, Cambridge, 1985.

Van Dyke, M. Perturbation methods in fluid mechanics. The Parabolic Press, Stanford, Calif., 1975.

Wasow, W. Asymptotic expansions for ordinary differential equations, Wiley-Interscience, New York, 1965.

[edit] Articles

Chapman, S. J. and King, J. R. and Adams K. L. Exponential asymptotics and Stokes lines in nonlinear ordinary differential equations. Royal Soc. Lond. Proc. Series A 454, 1978 (1998) p. 2733-2755.

Analyzing multiscale phenomena using singular perturbation methods. Edited by J. Cronin and R. E. O'Malley, Jr. Proceedings of Symposia in Applied Mathematics, 56. American Mathematical Society, Providence, RI, 1999.

DeVille, R.E.L. and Harkin, A. and Holzer, M. and Kresimir J. and Kaper, T. J. Analysis of a renormalization group method and normal form theory for perturbed ordinary differential equations. Physica D 237(2008) pp. 1029-1052.

Jones, C.K.R.T, Geometric Singular Perturbation Theory. In Dynamical Systems, Lecture Notes in Mathematics, Volume 1609, R. Johnson editor, Springer-Verlag, Berlin (1995) pp. 44-118.

Kruskal, M. D. Asymptotology. In Conference on Mathematical Models in Physical Sciences (University of Notre Dame, April 1962), S. Drobot and P. A. Viebrock editors, Prentice-Hall, New Jersey (1963) pp. 17-48.

Mudvanhu, B and O'Malley, R. E., Jr. A new renormalization method for the asymptotic solution of weakly nonlinear vector systems. SIAM J. Appl. Math. 63, 2 (2002) p. 373-397.

Segel, L. A. and Slemrod, M. The quasi-steady-state assumption: a case study in perturbation, SIAM Review 31, 3 (1989) p. 446-477.