Probability vector

Short description: Vector with non-negative entries that add up to one

"Stochastic vector" redirects here. For the concept of a random vector, see Multivariate random variable.

In mathematics and statistics, a probability vector or stochastic vector is a vector with non-negative entries that add up to one.

Underlying every probability vector is an experiment that can produce an outcome. To connect this experiment to mathematics, one introduces a discrete random variable, which is a function that assigns a numerical value to each possible outcome. For example, if the experiment consists of rolling a single die, the possible values of this random variable are the integers 1,2,…,6. The associated probability vector has six components, each representing the probability of obtaining the corresponding outcome. More generally, a probability vector of length n represents the distribution of probabilities across the n possible numerical outcomes of a random variable.[1]

The vector gives us the probability mass function of that random variable, which is the standard way of characterizing a discrete probability distribution.[2]

## Contents

* 1 Examples
* 2 Properties
* 3 Significance of the bounds on variance
* 4 Geometry of the probability simplex
* 5 Properties of the probability simplex
* 6 See also
* 7 References

## Examples

Here are some examples of probability vectors. The vectors can be either columns or rows.[3]

* x0=[0.50.250.25],
* x1=[010],
* x2=[0.650.35],
* x3=[0.30.50.070.10.03].

## Properties

* The mean of the components of any probability vector is 1/n.[4]

* The Euclidean length of a probability vector is related to the variance of its components by [5]

‖p‖=nσ2+1n.

* This expression for length reaches its minimum value of1n when all components are equal, with pi=1/n.[3]
* The longest probability vector has the value 1 in a single component and 0 in all others, and has a length of 1.[3]
* The shortest vector corresponds to maximum uncertainty, the longest to maximum certainty.
* The variance σ2 of a probability vector p=(p1,p2,…,pn) satisfies:

σ2∈[0,n−1n2].

The lower bound occurs when all components are equal pi=1/n, and the upper bound when one component equals 1 and the rest are 0.[6]

## Significance of the bounds on variance

The bounds on variance show that as the number of possible outcomes n increases, the variance necessarily decreases toward zero. As a result, the uncertainty associated with any single outcome increases because the components of the probability vector become more nearly equal. In empirical work, this often motivates binning the outcomes to reduce n; although this discards some information contained in the original outcomes, it allows the coarser-grained structure of the distribution to be revealed. The decrease in variance with increasing n reflects the same tendency toward uniformity that underlies entropy in information theory and statistical mechanics.[7]

## Geometry of the probability simplex

A simplex is the simplest geometric object that fully occupies the region of a given dimension defined by its vertices. It is constructed as the convex hull of n affinely independent points: for n=2 it is a line segment, for n=3 a triangle, for n=4 a tetrahedron, and so on.

The probability simplex (or standard simplex) is the canonical example of a simplex. It is obtained by taking the n standard basis vectors e1=(1,0,0,…,0),e2=(0,1,0,…,0),e3=(0,0,1,0,…,0), … as vertices and forming their convex hull: Δn−1={p∈ℝn∣pi≥0, ∑i=1npi=1}.

This is an (n−1)-dimensional simplex lying on the affine hyperplane ∑ipi=1. A random variable with n possible outcomes naturally lives in this (n−1)-simplex rather than an n-simplex, because the requirement that all probabilities sum to 1 removes one degree of freedom.

The components pi serve as barycentric coordinates, giving this simplex an immediate interpretation in probability theory: each vertex corresponds to a certain outcome, and each interior point represents a mixture or distribution over the n outcomes. Every possible discrete probability distribution on n outcomes corresponds to exactly one point in this simplex, and conversely each point of the simplex defines a unique distribution. Moving toward a vertex along barycentric coordinates corresponds to increasing certainty about the outcome, while movement toward the center represents increasing uncertainty resulting from a more uniform distribution.

The probability simplex serves as the canonical simplex in ℝn, since any other simplex can be obtained from it by an affine transformation, making it the standard reference for geometric and probabilistic analyses.[8][9]

## Properties of the probability simplex

Every probability vector of dimension n lies within an (n − 1)-dimensional simplex. The convex hull of this simplex does not form a smooth, gradually changing surface in ℝn; instead, it has sharp vertices, straight edges, and flat faces.

* Zero probability:

Assigning a zero probability to an outcome corresponds to moving onto a lower-dimensional face of the simplex, since that outcome is no longer possible.

* Adding a vertex:

Adding one new possible outcome to the random variable increases n by one and introduces a new orthogonal dimension. A new vertex appears in that dimension, and each face of the previous simplex combines with this vertex to form a new facet of one higher dimension. For example, when a triangle (2-simplex) gains a new vertex, connecting it to each of its three edges produces three new triangular facets, forming a tetrahedron. In the next step, adding another vertex would produce a 4-simplex, whose facets are tetrahedra.

* Affine hyperplane:

The probability simplex lies on the affine hyperplane ∑ipi=1 in ℝn. Its normal vector is a=(1,1,…,1) with norm ‖a‖=n. Thus, all points on the hyperplane lie within the positive orthant at the same perpendicular distance 1/n from the origin. This is because the projection of any point on the hyperplane onto its normal vector a is constant—by definition of a plane. The Euclidean distance from the origin to individual points on the plane varies, but the length of their perpendicular projection (the component along a) remains fixed. Affine independence means that the defining points in hyperspace are located relative to one another, not with respect to the origin as in the case of linear independence. This allows affinely independent objects to “float” relative to the origin, since their defining equation includes a constant term that specifies their offset along the normal direction. Changing this constant translates the entire object parallel to itself, preserving its internal relationships while changing its position in space.

* Centroid:

The centroid, corresponding to the uniform distribution, is u=(1/n,1/n,…,1/n). It lies at both a Euclidean and perpendicular distance ‖u‖=1/n from the origin, since the line from the origin to the centroid coincides with the simplex’s normal vector. Each vertex is at an equal Euclidean distance (n−1)/n from the centroid.

* Containing hypercube:

The (n − 1)-dimensional probability simplex lies entirely within the n-dimensional unit hypercube. That hypercube has a total content (or measure) of one unit. However, the meaning of that measure changes with dimension: a unit square (dimension 2) has an area of one (1 × 1 = 1), a unit cube (dimension 3) has a volume of one (1 × 1 × 1 = 1), and a unit hypercube of dimension 4 has a measure of one (1⁴ = 1), and so on. Although the total content remains constant, the cube’s Euclidean diagonal length increases as n, so the hypercube becomes geometrically “sparser”[10] as n increases—its corners move farther apart even though its content is unchanged. The probability simplex occupies only a thin (n − 1)-dimensional slice across this hypercube, forming a cross-section at a perpendicular distance of 1/n from the origin.[8][9]

* Volume (Content):

The (n−1)-dimensional content (volume) of the standard probability simplex can be computed using the Gram–determinant formula for simplex volume. Choosing the vertex en as a base point, the remaining vertices define edge vectors

vi=ei−en,i=1,…,n−1,

which lie in the (n−1)-dimensional affine hyperplane where the coordinates pi sum to one. The parallelepiped spanned by these vectors has content

Volpara=det⁡(G),

where G is the Gram matrix Gij=vi⋅vj. This Gram matrix has determinant n. [11][12] Since this simplex occupies exactly 1/(n−1)! of the volume of its parallelepiped, its content is

Vn−1=n(n−1)!.

This quantity decreases factorially with n, so although the simplex lies inside the unit n-cube (which has volume 1), the fraction of the hypercube’s volume contained in the simplex becomes super-exponentially small as n increases.

## See also

* Stochastic matrix
* Dirichlet distribution

## References

1. ↑ Bertsekas, D. P., & Tsitsiklis, J. N. (2008). Introduction to Probability (2nd ed.). Athena Scientific. Available as MIT lecture notes PDF. Chapter 2, p. 3.
2. ↑ Jacobs, Konrad (1992), Discrete Stochastics, Basler Lehrbücher [Basel Textbooks], 3, Birkhäuser Verlag, Basel, p. 45, doi:10.1007/978-3-0348-8645-1, ISBN 3-7643-2591-7, https://books.google.com/books?id=2Rv_i4-01JEC&pg=PA45 .
3. ↑ 3.0 3.1 3.2 Lee, Geoffrey (2016). "MATH1014 Linear Algebra Lecture 10 Notes". Australian National University. https://tqft.net/web/teaching/2016/1014/MATH1014-LinearAlgebra-Lecture10.slides.pdf.
4. ↑ StatisticsHowTo, Probability Vector: Definition, Examples, Properties
5. ↑ "Length of a Probability Vector". 2021. https://stats.stackexchange.com/questions/520751/length-of-a-probability-vector.
6. ↑ Bertsekas, D. P. & Tsitsiklis, J. N. (2008). Introduction to Probability. 2nd ed. Athena Scientific. pp. 53–54.
7. ↑ Source needed. probably Cover & Thomas
8. ↑ 8.0 8.1 Boyd, Stephen; Vandenberghe, Lieven (2004). Convex Optimization. Cambridge University Press. p. 32. https://stanford.edu/~boyd/cvxbook/bv_cvxbook.pdf.
9. ↑ 9.0 9.1 Murphy, Kevin P. (2022). Probabilistic Machine Learning: An Introduction. MIT Press. p. 108. https://probml.github.io/pml-book/book1.html.
10. ↑ “Sparser” in this context means that two points chosen at random in the n-dimensional unit hypercube have an expected Root Mean Square distance from each other of n/6, so as n increases the average separation between random points grows proportionally to n.
11. ↑ Cover, Thomas M.; Thomas, Joy A. (2006). "12". Elements of Information Theory (2nd ed.). Wiley.
12. ↑ Gallier, Jean; Quaintance, Jocelyn (2020). "7.8". Linear Algebra and Optimization with Applications to Machine Learning. Springer. https://www.cis.upenn.edu/~jean/gbooks/vol1.pdf.

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/Probability vector. Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=Probability_vector&oldid=4227916"