Markov chain

	Main Article	Discussion	Related Articles  [?]	Bibliography  [?]	External Links  [?]	Citable Version  [?]
	This editable Main Article is under development and subject to a disclaimer. [edit intro]

A Markov chain is a Markov process with a discrete time parameter ^[1]. The Markov chain is a useful way to model systems with no long-term memory of previous states. That is, the state of the system at time ${\displaystyle (t+1)}$ is solely a function of the state ${\displaystyle t}$, and not of any previous states ^[2].

A Formal Model[edit]

The influence of the values of ${\displaystyle X^{\left(0\right)},X^{\left(1\right)},\dots ,X^{\left(n\right)}}$ on the distribution of ${\displaystyle X^{(n+1)}}$ can be formally modelled as:

In this model, ${\displaystyle E}$ is any desired subset of the series ${\displaystyle \{0,\dots ,n-1\}}$. These ${\displaystyle t}$ indexes commonly represent the time component, and the range of ${\displaystyle X^{\left(t\right)}}$ is the Markov chain's state space ^[1].

Probability Density[edit]

The Markov chain can also be specified using a series of probabilities. If the initial probability of the state ${\displaystyle x}$ is ${\displaystyle p_{\left(0\right)}\left(x\right)}$, then the transition probability for state ${\displaystyle y}$ occurring at time ${\displaystyle n+1}$ can be expressed as:

In words, this states that the probability of the system entering state ${\displaystyle y}$ at time ${\displaystyle n+1}$ is a function of the summed products of the initial probability density and the probability of state ${\displaystyle y}$ given state ${\displaystyle x}$ ^[2].

Invariant Distributions[edit]

In many cases, the density will approach a limit ${\displaystyle p\left(y\right)}$ that is uniquely determined by ${\displaystyle p(y\mid x)}$ (and not ${\displaystyle p_{\left(n\right)}\left(x\right)}$). This limiting distribution is referred to as the invariant (or stationary) distribution over the states of the Markov chain. When such a distribution is reached, it persists forever^[2].

References[edit]