Matrix of transition probabilities

The matrix $P_t=\Vert p_{ij}(t)\Vert$ of transition probabilities in time $t$ for a homogeneous Markov chain $\xi (t)$ with at most a countable set of states $S$:

$$p_{ij}(t)=\mathsf{P}\{\xi (t)=j\mid \xi (0)=i\},i,j\in S.$$

The matrices $\Vert p_{ij}(t)\Vert$ of a Markov chain with discrete time or a regular Markov chain with continuous time satisfy the following conditions for any $t>0$ and $i,j\in S$:

$$p_{ij}(t)\ge 0,\sum _{j\in S}{p}_{ij}(t)=1,$$

i.e. they are stochastic matrices (cf. Stochastic matrix), while for irregular chains

$$p_{ij}(t)\ge 0,\sum _{j\in S}{p}_{ij}(t)\le 1,$$

such matrices are called sub-stochastic.

By virtue of the basic (Chapman–Kolmogorov) property of a homogeneous Markov chain,

$$p_{ij}(s+t)=\sum _{k\in S}{p}_{ik}(s)p_{kj}(t),$$

the family of matrices $\{P_t:t>0\}$ forms a multiplicative semi-group; if the time is discrete, this semi-group is uniquely determined by $P_1$.

References[edit]