Regular conditional probability

In probability theory, regular conditional probability is a concept that formalizes the notion of conditioning on the outcome of a random variable. The resulting conditional probability distribution is a parametrized family of probability measures called a Markov kernel.

## Definition

[edit]

### Conditional probability distribution

[edit]

Consider two random variables  X , Y : Ω → R {\displaystyle X,Y:\Omega \to \mathbb {R} } . The conditional probability distribution of Y given X is a two variable function  κ Y ∣ X : R × B ( R ) → [ 0 , 1 ] {\displaystyle \kappa _{Y\mid X}:\mathbb {R} \times {\mathcal {B}}(\mathbb {R} )\to [0,1]}

If the random variable X is discrete

κ Y ∣ X ( x , A ) = P ( Y ∈ A ∣ X = x ) = { P ( Y ∈ A , X = x ) P ( X = x )  if  P ( X = x ) > 0 arbitrary value  otherwise . {\displaystyle \kappa _{Y\mid X}(x,A)=P(Y\in A\mid X=x)={\begin{cases}{\frac {P(Y\in A,X=x)}{P(X=x)}}&{\text{ if }}P(X=x)>0\\\\[3pt]{\text{arbitrary value}}&{\text{ otherwise}}.\end{cases}}}

If the random variables X, Y are continuous with density  f X , Y ( x , y ) {\displaystyle f_{X,Y}(x,y)} .

κ Y ∣ X ( x , A ) = { ∫ A f X , Y ( x , y ) d y ∫ R f X , Y ( x , y ) d y  if  ∫ R f X , Y ( x , y ) d y > 0 arbitrary value  otherwise . {\displaystyle \kappa _{Y\mid X}(x,A)={\begin{cases}{\frac {\int _{A}f_{X,Y}(x,y)\,\mathrm {d} y}{\int _{\mathbb {R} }f_{X,Y}(x,y)\mathrm {d} y}}&{\text{ if }}\int _{\mathbb {R} }f_{X,Y}(x,y)\,\mathrm {d} y>0\\\\[3pt]{\text{arbitrary value}}&{\text{ otherwise}}.\end{cases}}}

A more general definition can be given in terms of conditional expectation. Consider a function  e Y ∈ A : R → [ 0 , 1 ] {\displaystyle e_{Y\in A}:\mathbb {R} \to [0,1]} satisfying

e Y ∈ A ( X ( ω ) ) = E ⁡ [ 1 Y ∈ A ∣ X ] ( ω ) {\displaystyle e_{Y\in A}(X(\omega ))=\operatorname {E} [1_{Y\in A}\mid X](\omega )}

for almost all  ω {\displaystyle \omega } . Then the conditional probability distribution is given by

κ Y ∣ X ( x , A ) = e Y ∈ A ( x ) . {\displaystyle \kappa _{Y\mid X}(x,A)=e_{Y\in A}(x).}

As with conditional expectation, this can be further generalized to conditioning on a sigma algebra  F {\displaystyle {\mathcal {F}}} . In that case the conditional distribution is a function  Ω × B ( R ) → [ 0 , 1 ] {\displaystyle \Omega \times {\mathcal {B}}(\mathbb {R} )\to [0,1]} :

κ Y ∣ F ( ω , A ) = E ⁡ [ 1 Y ∈ A ∣ F ] ( ω ) {\displaystyle \kappa _{Y\mid {\mathcal {F}}}(\omega ,A)=\operatorname {E} [1_{Y\in A}\mid {\mathcal {F}}](\omega )}

### Regularity

[edit]

For working with  κ Y ∣ X {\displaystyle \kappa _{Y\mid X}} , it is important that it be regular, that is:

1. For almost all x,  A ↦ κ Y ∣ X ( x , A ) {\displaystyle A\mapsto \kappa _{Y\mid X}(x,A)} is a probability measure
2. For all A,  x ↦ κ Y ∣ X ( x , A ) {\displaystyle x\mapsto \kappa _{Y\mid X}(x,A)} is a measurable function

In other words  κ Y ∣ X {\displaystyle \kappa _{Y\mid X}} is a Markov kernel.

The second condition holds trivially, but the proof of the first is more involved. It can be shown that if Y is a random element  Ω → S {\displaystyle \Omega \to S} in a Radon space S, there exists a  κ Y ∣ X {\displaystyle \kappa _{Y\mid X}} that satisfies the first condition.[1] It is possible to construct more general spaces where a regular conditional probability distribution does not exist.[2]

### Relation to conditional expectation

[edit]

For discrete and continuous random variables, the conditional expectation can be expressed as

E ⁡ [ Y ∣ X = x ] = ∑ y y P ( Y = y ∣ X = x ) E ⁡ [ Y ∣ X = x ] = ∫ y f Y ∣ X ( x , y ) d y {\displaystyle {\begin{aligned}\operatorname {E} [Y\mid X=x]&=\sum _{y}y\,P(Y=y\mid X=x)\\\\\operatorname {E} [Y\mid X=x]&=\int y\,f_{Y\mid X}(x,y)\,\mathrm {d} y\end{aligned}}}

where  f Y ∣ X ( x , y ) {\displaystyle f_{Y\mid X}(x,y)} is the conditional density of Y given X.

This result can be extended to measure theoretical conditional expectation using the regular conditional probability distribution:

E ⁡ [ Y ∣ X ] ( ω ) = ∫ y κ Y ∣ σ ( X ) ( ω , d y ) . {\displaystyle \operatorname {E} [Y\mid X](\omega )=\int y\,\kappa _{Y\mid \sigma (X)}(\omega ,\mathrm {d} y).}

## Formal definition

[edit]

Let  ( Ω , F , P ) {\displaystyle (\Omega ,{\mathcal {F}},P)} be a probability space, and let  T : Ω → E {\displaystyle T:\Omega \rightarrow E} be a random variable, defined as a Borel-measurable function from  Ω {\displaystyle \Omega } to its state space ( E , E ) {\displaystyle (E,{\mathcal {E}})} . One should think of  T {\displaystyle T} as a way to "disintegrate" the sample space  Ω {\displaystyle \Omega } into  { T − 1 ( x ) } x ∈ E {\displaystyle \\{T^{-1}(x)\\}_{x\in E}} . Using the disintegration theorem from the measure theory, it allows us to "disintegrate" the measure  P {\displaystyle P} into a collection of measures, one for each  x ∈ E {\displaystyle x\in E} . Formally, a regular conditional probability is defined as a function  ν : E × F → [ 0 , 1 ] , {\displaystyle \nu :E\times {\mathcal {F}}\rightarrow [0,1],} called a "transition probability", where:

* For every  x ∈ E {\displaystyle x\in E} ,  ν ( x , ⋅ ) {\displaystyle \nu (x,\cdot )} is a probability measure on  F {\displaystyle {\mathcal {F}}} . Thus we provide one measure for each  x ∈ E {\displaystyle x\in E} .
* For all  A ∈ F {\displaystyle A\in {\mathcal {F}}} ,  ν ( ⋅ , A ) {\displaystyle \nu (\cdot ,A)} (a mapping  E → [ 0 , 1 ] {\displaystyle E\to [0,1]} ) is  E {\displaystyle {\mathcal {E}}} -measurable, and
* For all  A ∈ F {\displaystyle A\in {\mathcal {F}}} and all  B ∈ E {\displaystyle B\in {\mathcal {E}}} [3]

P ( A ∩ T − 1 ( B ) ) = ∫ B ν ( x , A ) ( P ∘ T − 1 ) ( d x ) {\displaystyle P{\big (}A\cap T^{-1}(B){\big )}=\int _{B}\nu (x,A)\,(P\circ T^{-1})(\mathrm {d} x)}

where  P ∘ T − 1 {\displaystyle P\circ T^{-1}} is the pushforward measure T ∗ P {\displaystyle T_{*}P} of the distribution of the random element  T {\displaystyle T} ,  x ∈ supp ⁡ T , {\displaystyle x\in \operatorname {supp} T,} i.e. the support of the  T ∗ P {\displaystyle T_{*}P} . Specifically, if we take  B = E {\displaystyle B=E} , then  A ∩ T − 1 ( E ) = A {\displaystyle A\cap T^{-1}(E)=A} , and so

P ( A ) = ∫ E ν ( x , A ) ( P ∘ T − 1 ) ( d x ) , {\displaystyle P(A)=\int _{E}\nu (x,A)\,(P\circ T^{-1})(\mathrm {d} x),}

where  ν ( x , A ) {\displaystyle \nu (x,A)} can be denoted, using more familiar terms  P ( A   |   T = x ) {\displaystyle P(A\ |\ T=x)} .

## Alternate definition

[edit]

The factual accuracy of part of this article is disputed. The dispute is about this way leads to irregular conditional probability. Please help to ensure that disputed statements are reliably sourced. See the relevant discussion on the talk page. (September 2009) (Learn how and when to remove this message)

|

Consider a Radon space Ω {\displaystyle \Omega } (that is a probability measure defined on a Radon space endowed with the Borel sigma-algebra) and a real-valued random variable T. As discussed above, in this case there exists a regular conditional probability with respect to T. Moreover, we can alternatively define the regular conditional probability for an event A given a particular value t of the random variable T in the following manner:

P ( A ∣ T = t ) = lim U ⊃ { T = t } P ( A ∩ U ) P ( U ) , {\displaystyle P(A\mid T=t)=\lim _{U\supset \\{T=t\\}}{\frac {P(A\cap U)}{P(U)}},}

where the limit is taken over the net of open neighborhoods U of t as they become smaller with respect to set inclusion. This limit is defined if and only if the probability space is Radon, and only in the support of T, as described in the article. This is the restriction of the transition probability to the support of T. To describe this limiting process rigorously:

For every  ε > 0 , {\displaystyle \varepsilon >0,} there exists an open neighborhood U of the event {T = t}, such that for every open V with  { T = t } ⊂ V ⊂ U , {\displaystyle \\{T=t\\}\subset V\subset U,}

| P ( A ∩ V ) P ( V ) − L | < ε , {\displaystyle \left|{\frac {P(A\cap V)}{P(V)}}-L\right|<\varepsilon ,}

where  L = P ( A ∣ T = t ) {\displaystyle L=P(A\mid T=t)} is the limit.

## See also

[edit]

* Conditioning (probability)
* Disintegration theorem
* Adherent point
* Limit point

## References

[edit]

1. ^ Klenke, Achim (30 August 2013). Probability theory : a comprehensive course (Second ed.). London. ISBN 978-1-4471-5361-0.`{{cite book}}`: CS1 maint: location missing publisher (link)
2. ^ Faden, A.M., 1985. The existence of regular conditional probabilities: necessary and sufficient conditions. The Annals of Probability, 13(1), pp. 288–298.
3. ^ D. Leao Jr. et al. Regular conditional probability, disintegration of probability and Radon spaces. Proyecciones. Vol. 23, No. 1, pp. 15–29, May 2004, Universidad Católica del Norte, Antofagasta, Chile PDF

## External links

[edit]

* Regular Conditional Probability on PlanetMath