Kolmogorov equations

Short description: Equations characterizing continuous-time Markov processes

For the equations in population dynamics, see Kolmogorov population model.

In probability theory, Kolmogorov equations characterize continuous-time Markov processes. In particular, they describe how the probability of a continuous-time Markov process in a certain state changes over time. There are four distinct equations: the Kolmogorov forward equation for continuous processes, now understood to be identical to the Fokker–Planck equation, the Kolmogorov forward equation for jump processes, and two Kolmogorov backward equations for processes with and without discontinuous jumps.

## Contents

* 1 Diffusion processes vs. jump processes
* 2 History
* 3 The modern view
* 4 Continuous-time Markov chains
* 4.1 Relation with the generating function
* 5 An example from biology
* 6 See also
* 7 References

## Diffusion processes vs. jump processes

Writing in 1931, Andrey Kolmogorov started from the theory of discrete time Markov processes, which are described by the Chapman–Kolmogorov equation, and sought to derive a theory of continuous time Markov processes by extending this equation. He found that there are two kinds of continuous time Markov processes, depending on the assumed behavior over small intervals of time:

If you assume that "in a small time interval there is an overwhelming probability that the state will remain unchanged; however, if it changes, the change may be radical",[1] then you are led to what are called jump processes.

The other case leads to processes such as those "represented by diffusion and by Brownian motion; there it is certain that some change will occur in any time interval, however small; only, here it is certain that the changes during small time intervals will be also small".[1]

For each of these two kinds of processes, Kolmogorov derived a forward and a backward system of equations (four in all).

## History

The equations are named after Andrey Kolmogorov since they were highlighted in his 1931 foundational work.[2]

William Feller, in 1949, used the names "forward equation" and "backward equation" for his more general version of the Kolmogorov's pair, in both jump and diffusion processes.[1] Much later, in 1956, he referred to the equations for the jump process as "Kolmogorov forward equations" and "Kolmogorov backward equations".[3]

Other authors, such as Motoo Kimura,[4] referred to the diffusion (Fokker–Planck) equation as Kolmogorov forward equation, a name that has persisted.

## The modern view

* In the context of a continuous-time Markov process with jumps, see Kolmogorov equations (Markov jump process). In particular, in natural sciences the forward equation is also known as master equation.
* In the context of a diffusion process, for the backward Kolmogorov equations see Kolmogorov backward equations (diffusion). The forward Kolmogorov equation is also known as Fokker–Planck equation.

## Continuous-time Markov chains

The original derivation of the equations by Kolmogorov starts with the Chapman–Kolmogorov equation (Kolmogorov called it fundamental equation) for time-continuous and differentiable Markov processes on a finite, discrete state space.[2] In this formulation, it is assumed that the probabilities P(x,s;y,t) are continuous and differentiable functions of t>s, where x,y∈Ω (the state space) and t>s,t,s∈ℝ≥0 are the final and initial times, respectively. Also, adequate limit properties for the derivatives are assumed. Feller derives the equations under slightly different conditions, starting with the concept of purely discontinuous Markov process and then formulating them for more general state spaces.[5] Feller proves the existence of solutions of probabilistic character to the Kolmogorov forward equations and Kolmogorov backward equations under natural conditions.[5]

For the case of a countable state space we put i,j in place of x,y. The Kolmogorov forward equations read

∂Pij∂t(s;t)=∑kPik(s;t)Akj(t),

where A(t) is the transition rate matrix (also known as the generator matrix),

while the Kolmogorov backward equations are

∂Pij∂s(s;t)=−∑kPkj(s;t)Aik(s)

The functions Pij(s;t) are continuous and differentiable in both time arguments. They represent the probability that the system that was in state i at time s jumps to state j at some later time t>s. The continuous quantities Aij(t) satisfy

Aij(t)=[∂Pij∂u(t;u)]u=t,Ajk(t)≥0, j≠k,∑kAjk(t)=0.

### Relation with the generating function

Still in the discrete state case, letting s=0 and assuming that the system initially is found in state i, the Kolmogorov forward equations describe an initial-value problem for finding the probabilities of the process, given the quantities Ajk(t). We write pk(t)=Pik(0;t) where ∑kpk(t)=1, then

dpkdt(t)=∑jAjk(t)pj(t);pk(0)=δik,k=0,1,….

For the case of a pure death process with constant rates the only nonzero coefficients are Aj,j−1=μj, j≥1. Letting

Ψ(x,t)=∑kxkpk(t),

the system of equations can in this case be recast as a partial differential equation for Ψ(x,t) with initial condition Ψ(x,0)=xi. After some manipulations, the system of equations reads,[6]

∂Ψ∂t(x,t)=μ(1−x)∂Ψ∂x(x,t);Ψ(x,0)=xi,Ψ(1,t)=1.

## An example from biology

One example from biology is given below:[7]

pn′(t)=(n−1)βpn−1(t)−nβpn(t)

This equation is applied to model population growth with birth. Where n is the population index, with reference the initial population, β is the birth rate, and finally pn(t)=Pr⁡(N(t)=n), i.e. the probability of achieving a certain population size.

The analytical solution is:[7]

pn(t)=(n−1)βe−nβt∫0tpn−1(s)enβsds

This is a formula for the probability pn(t) in terms of the preceding ones, i.e. pn−1(t).

## See also

* Feynman–Kac formula
* Fokker–Planck equation
* Kolmogorov backward equation

## References

1. ↑ 1.0 1.1 1.2 Feller, W. (1949). "On the Theory of Stochastic Processes, with Particular Reference to Applications". Proceedings of the (First) Berkeley Symposium on Mathematical Statistics and Probability. 1. University of California Press. pp. 403–432. https://projecteuclid.org/euclid.bsmsp/1166219215.
2. ↑ 2.0 2.1 Kolmogorov, Andrei (1931). "Über die analytischen Methoden in der Wahrscheinlichkeitsrechnung" (in de). Mathematische Annalen 104: 415–458. doi:10.1007/BF01457949.
3. ↑ Feller, William (1957). "On Boundaries and Lateral Conditions for the Kolmogorov Differential Equations". Annals of Mathematics 65 (3): 527–570. doi:10.2307/1970064.
4. ↑ Kimura, Motoo (1957). "Some Problems of Stochastic Processes in Genetics". Annals of Mathematical Statistics 28 (4): 882–901. doi:10.1214/aoms/1177706791.
5. ↑ 5.0 5.1 Feller, Willy (1940) "On the Integro-Differential Equations of Purely Discontinuous Markoff Processes", Transactions of the American Mathematical Society, 48 (3), 488-515 JSTOR 1990095
6. ↑ Bailey, Norman T.J. (1990) The Elements of Stochastic Processes with Applications to the Natural Sciences, Wiley. ISBN 0-471-52368-2 (page 90)
7. ↑ 7.0 7.1 Logan, J. David; Wolesensky, William R. (2009). Mathematical Methods in Biology. Pure and Applied Mathematics. John Wiley& Sons. pp. 325–327. ISBN 978-0-470-52587-6.

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/Kolmogorov equations. Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=Kolmogorov_equations&oldid=4247800"