Maximal entropy random walk

Maximal entropy random walk (MERW) is a popular type of biased random walk on a graph, in which transition probabilities are chosen accordingly to the principle of maximum entropy, which says that the probability distribution which best represents the current state of knowledge is the one with largest entropy. While standard random walk chooses for every vertex uniform probability distribution among its outgoing edges, locally maximizing entropy rate, MERW maximizes it globally (average entropy production) by assuming uniform probability distribution among all paths in a given graph.

MERW is used in various fields of science. A direct application is choosing probabilities to maximize transmission rate through a constrained channel, analogously to Fibonacci coding. Its properties also made it useful for example in analysis of complex networks,[1] like link prediction,[2] community detection,[3] robust transport over networks[4] and centrality measures.[5] Also in image analysis, for example for detecting visual saliency regions,[6] object localization,[7] tampering detection[8] or tractography problem.[9]

Additionally, it recreates some properties of quantum mechanics, suggesting a way to repair the discrepancy between diffusion models and quantum predictions, like Anderson localization.[10]

## Contents

* 1 Basic model
* 1.1 Sketch of derivation
* 1.2 Weighted MERW: Boltzmann path ensemble
* 2 Examples
* 3 See also
* 4 References
* 5 External links

## Basic model

Left: basic concept of the generic random walk (GRW) and maximal entropy random walk (MERW)
Right: example of their evolution on the same inhomogeneous 2D lattice with cyclic boundary conditions – probability density after 10, 100 and 1000 steps while starting from the same vertex. The small boxes represent defects: all vertices but the marked ones have additional self-loop (edge to itself). For regular lattices (no defects), GRW and MERW are identical. While defects do not strongly affect the local beha­vior, they lead to a completely different global stationary probability here. While GRW (and based on it standard diffusion) leads to nearly uniform stationary density, MERW has strong localization property, imprisoning the walkers in entropic wells in analogy to electrons in defected lattice of semi-conductor.

Consider a graph with [math]\displaystyle{ n }[/math] vertices, defined by an adjacency matrix [math]\displaystyle{ A \in \left\\{0, 1\right\\}^{n \times n} }[/math]: [math]\displaystyle{ A_{ij}=1 }[/math] if there is an edge from vertex [math]\displaystyle{ i }[/math] to [math]\displaystyle{ j }[/math], 0 otherwise. For simplicity assume it is an undirected graph, which corresponds to a symmetric [math]\displaystyle{ A }[/math]; however, MERW can also be generalized for directed and weighted graphs (for example Boltzmann distribution among paths instead of uniform).

We would like to choose a random walk as a Markov process on this graph: for every vertex [math]\displaystyle{ i }[/math] and its outgoing edge to [math]\displaystyle{ j }[/math], choose probability [math]\displaystyle{ S_{ij} }[/math] of the walker randomly using this edge after visiting [math]\displaystyle{ i }[/math]. Formally, find a stochastic matrix [math]\displaystyle{ S }[/math] (containing the transition probabilities of a Markov chain) such that

* [math]\displaystyle{ 0\leq S_{ij} \leq A_{ij} }[/math] for all [math]\displaystyle{ i, j }[/math] and
* [math]\displaystyle{ \sum_{j=1}^n S_{ij}=1 }[/math] for all [math]\displaystyle{ i }[/math].

Assuming this graph is connected and not periodic, ergodic theory says that evolution of this stochastic process leads to some stationary probability distribution [math]\displaystyle{ \rho }[/math] such that [math]\displaystyle{ \rho S = \rho }[/math].

Using Shannon entropy for every vertex and averaging over probability of visiting this vertex (to be able to use its entropy), we get the following formula for average entropy production (entropy rate) of the stochastic process:

[math]\displaystyle{ H(S)=\sum_{i=1}^n \rho_i \sum_{j=1}^n S_{ij} \log(1/S_{ij}) }[/math]

This definition turns out to be equivalent to the asymptotic average entropy (per length) of the probability distribution in the space of paths for this stochastic process.

In the standard random walk, referred to here as generic random walk (GRW), we naturally choose that each outgoing edge is equally probable:

[math]\displaystyle{ S_{ij} = \frac{A_{ij}}{\sum\limits_{k=1}^n A_{ik}} }[/math].

For a symmetric [math]\displaystyle{ A }[/math] it leads to a stationary probability distribution [math]\displaystyle{ \rho }[/math] with

[math]\displaystyle{ \rho_i = \frac{\sum\limits_{j=1}^n A_{ij}}{\sum\limits_{i=1}^n \sum\limits_{j=1}^n A_{ij}} }[/math].

It locally maximizes entropy production (uncertainty) for every vertex, but usually leads to a suboptimal averaged global entropy rate [math]\displaystyle{ H(S) }[/math].

MERW chooses the stochastic matrix which maximizes [math]\displaystyle{ H(S) }[/math], or equivalently assumes uniform probability distribution among all paths in a given graph. Its formula is obtained by first calculating the dominant eigenvalue [math]\displaystyle{ \lambda }[/math] and corresponding eigenvector [math]\displaystyle{ \psi }[/math] of the adjacency matrix, i.e. the largest [math]\displaystyle{ \lambda \in \mathbb{R} }[/math] with corresponding [math]\displaystyle{ \psi \in \mathbb{R}^n }[/math] such that [math]\displaystyle{ \psi A = \lambda \psi }[/math]. Then stochastic matrix and stationary probability distribution are given by

[math]\displaystyle{ S_{ij} = \frac{A_{ij}}{\lambda} \frac{\psi_j}{\psi_i} }[/math]

for which every possible path of length [math]\displaystyle{ l }[/math] from the [math]\displaystyle{ i }[/math]-th to [math]\displaystyle{ j }[/math]-th vertex has probability

[math]\displaystyle{ \frac{1}{\lambda^l} \frac{\psi_j}{\psi_i} }[/math].

Its entropy rate is [math]\displaystyle{ \log(\lambda) }[/math] and the stationary probability distribution [math]\displaystyle{ \rho }[/math] is

[math]\displaystyle{ \rho_i = \frac{\psi_i^2}{\|\psi\|_2^2} }[/math].

In contrast to GRW, the MERW transition probabilities generally depend on the structure of the entire graph (are nonlocal). Hence, they should not be imagined as directly applied by the walker – if random-looking decisions are made based on the local situation, like a person would make, the GRW approach is more appropriate. MERW is based on the principle of maximum entropy, making it the safest assumption when we don't have any additional knowledge about the system. For example, it would be appropriate for modelling our knowledge about an object performing some complex dynamics – not necessarily random, like a particle.

### Sketch of derivation

Assume for simplicity that the considered graph is indirected, connected and aperiodic, allowing to conclude from the Perron–Frobenius theorem that the dominant eigenvector is unique. Hence [math]\displaystyle{ A^l }[/math] can be asymptotically ([math]\displaystyle{ l\rightarrow\infty }[/math]) approximated by [math]\displaystyle{ \lambda^l \psi \psi^T }[/math] (or [math]\displaystyle{ \lambda^l |\psi\rangle \langle \psi| }[/math] in bra–ket notation).

MERW requires uniform distribution along paths. The number [math]\displaystyle{ m_{il} }[/math] of paths with length [math]\displaystyle{ 2l }[/math] and vertex [math]\displaystyle{ i }[/math] in the center is

[math]\displaystyle{ m_{il} = \sum_{j=1}^n \sum_{k=1}^n \left(A^l\right)_{ji} \left(A^l\right)_{ik} \approx \sum_{j=1}^n \sum_{k=1}^n \left(\lambda^l \psi \psi^\top\right)_{ji} \left(\lambda^l \psi \psi^\top\right)_{ik} = \sum_{j=1}^n \sum_{k=1}^n \lambda^{2l} \psi_j \psi_i \psi_i \psi_k = \lambda^{2l} \psi_i^2 \underbrace{\sum_{j=1}^n \psi_j \sum_{k=1}^n \psi_k}_{=: b} }[/math],

hence for all [math]\displaystyle{ i }[/math],

[math]\displaystyle{ \rho_i = \lim_{l \rightarrow \infty} \frac{m_{il}}{\sum\limits_{k=1}^n m_{kl}} = \lim_{l \rightarrow \infty} \frac{\lambda^{2l} \psi_i^2 b}{\sum\limits_{k=1}^n \lambda^{2l} \psi_k^2 b} = \lim_{l \rightarrow \infty} \frac{\psi_i^2}{\sum\limits_{k=1}^n \psi_k^2} = \frac{\psi_i^2}{\sum\limits_{k=1}^n \psi_k^2} = \frac{\psi_i^2}{\|\psi\|_2^2} }[/math].

Analogously calculating probability distribution for two succeeding vertices, one obtains that the probability of being at the [math]\displaystyle{ i }[/math]-th vertex and next at the [math]\displaystyle{ j }[/math]-th vertex is

[math]\displaystyle{ \frac{\psi_i A_{ij} \psi_j}{\sum\limits_{i'=1}^n \sum\limits_{j'=1}^n \psi_{i'} A_{i'j'} \psi_{j'}} = \frac{\psi_i A_{ij} \psi_j}{\psi A \psi^\top} = \frac{\psi_i A_{ij} \psi_j}{\lambda \|\psi\|_2^2} }[/math].

Dividing by the probability of being at the [math]\displaystyle{ i }[/math]-th vertex, i.e. [math]\displaystyle{ \rho_i }[/math], gives for the conditional probability [math]\displaystyle{ S_{ij} }[/math] of the [math]\displaystyle{ j }[/math]-th vertex being next after the [math]\displaystyle{ i }[/math]-th vertex

[math]\displaystyle{ S_{ij} = \frac{A_{ij}}{\lambda} \frac{\psi_j}{\psi_i} }[/math].

### Weighted MERW: Boltzmann path ensemble

We have assumed that [math]\displaystyle{ A_{ij} \in \\{0,1\\} }[/math] for MERW corresponding to uniform ensemble among paths. However, the above derivation works for real nonnegative [math]\displaystyle{ A }[/math]. Parametrizing [math]\displaystyle{ A_{ij} = \exp(-E_{ij}) }[/math] and asking for probability of length [math]\displaystyle{ l }[/math] path [math]\displaystyle{ (\gamma_0, \ldots,\gamma_l) }[/math], we get:

[math]\displaystyle{ \textrm{Pr}(\gamma_0, \ldots,\gamma_l)=\rho_{\gamma_0} S_{\gamma_0 \gamma_1}\ldots S_{\gamma_{l-1}\gamma_l}= \psi_{\gamma_0} \frac{A_{\gamma_0 \gamma_1}\ldots A_{\gamma_{l-1}\gamma_l}}{\lambda^l} \psi_{\gamma_l}=\psi_{\gamma_0}\frac{\exp(-(E_{\gamma_0 \gamma_1}+\ldots +E_{\gamma_{l-1}\gamma_l}))}{\lambda^l} \psi_{\gamma_l} }[/math]

As in Boltzmann distribution of paths for energy defined as sum of [math]\displaystyle{ E_{ij} }[/math] over given path. For example, it allows to calculate probability distribution of patterns in Ising model.

## Examples

Left: choosing the optimal probability after symbol 0 in Fibonacci coding. Right: one-dimensional defected lattice and its stationary density for length 1000 cycle (it has three defects). While in standard random walk the stationary density is proportional to degree of a vertex, leading to 3/2 difference here, in MERW density is nearly completely localized in the largest defect-free region, analogous to the ground state predicted by quantum mechanics.

Let us first look at a simple nontrivial situation: Fibonacci coding, where we want to transmit a message as a sequence of 0s and 1s, but not using two successive 1s: after a 1 there has to be a 0. To maximize the amount of information transmitted in such sequence, we should assume uniform probability distribution in the space of all possible sequences fulfilling this constraint. To practically use such long sequences, after 1 we have to use 0, but there remains a freedom of choosing the probability of 0 after 0. Let us denote this probability by [math]\displaystyle{ q }[/math], then entropy coding would allow encoding a message using this chosen probability distribution. The stationary probability distribution of symbols for a given [math]\displaystyle{ q }[/math] turns out to be [math]\displaystyle{ \rho=(1/(2-q),1-1/(2-q)) }[/math]. Hence, entropy production is [math]\displaystyle{ H(S)=\rho_0 \left(q\log(1/q)+(1-q)\log(1/(1-q))\right) }[/math], which is maximized for [math]\displaystyle{ q=(\sqrt{5}-1)/2\approx 0.618 }[/math], known as the golden ratio. In contrast, standard random walk would choose suboptimal [math]\displaystyle{ q=0.5 }[/math]. While choosing larger [math]\displaystyle{ q }[/math] reduces the amount of information produced after 0, it also reduces frequency of 1, after which we cannot write any information.

A more complex example is the defected one-dimensional cyclic lattice: let say 1000 nodes connected in a ring, for which all nodes but the defects have a self-loop (edge to itself). In standard random walk (GRW) the stationary probability distribution would have defect probability being 2/3 of probability of the non-defect vertices – there is nearly no localization, also analogously for standard diffusion, which is infinitesimal limit of GRW. For MERW we have to first find the dominant eigenvector of the adjacency matrix – maximizing [math]\displaystyle{ \lambda }[/math] in:

[math]\displaystyle{ (\lambda \psi)_x = (A\psi)_x = \psi_{x-1}+ (1-V_x) \psi_x +\psi_{x+1} }[/math]

for all positions [math]\displaystyle{ x }[/math], where [math]\displaystyle{ V_x=1 }[/math] for defects, 0 otherwise. Substituting [math]\displaystyle{ 3\psi_x }[/math] and multiplying the equation by −1 we get:

[math]\displaystyle{ E \psi_x =-(\psi_{x-1} -2 \psi_x +\psi_{x+1}) + V_x \psi_x }[/math]

where [math]\displaystyle{ E = 3-\lambda }[/math] is minimized now, becoming the analog of energy. The formula inside the bracket is discrete Laplace operator, making this equation a discrete analogue of stationary Schrodinger equation. As in quantum mechanics, MERW predicts that the probability distribution should lead exactly to the one of quantum ground state: [math]\displaystyle{ \rho_x \propto \psi_x^2 }[/math] with its strongly localized density (in contrast to standard diffusion). Taking the infinitesimal limit, we can get standard continuous stationary (time-independent) Schrodinger equation ([math]\displaystyle{ E\psi=-C\psi_{xx}+V\psi }[/math] for [math]\displaystyle{ C=\hbar^2/2m }[/math]) here.[11]

## See also

* Principle of maximum entropy
* Eigenvector centrality
* Markov chain
* Anderson localization

## References

1. ↑ Sinatra, Roberta; Gómez-Gardeñes, Jesús; Lambiotte, Renaud; Nicosia, Vincenzo; Latora, Vito (2011). "Maximal-entropy random walks in complex networks with limited information". Physical Review E 83 (3): 030103. doi:10.1103/PhysRevE.83.030103. ISSN 1539-3755. PMID 21517435. Bibcode: 2011PhRvE..83c0103S. http://www.robertasinatra.com/pdf/sinatra_maximal_entropy.pdf.
2. ↑ Li, Rong-Hua; Yu, Jeffrey Xu; Liu, Jianquan (2011). "Link prediction: the power of maximal entropy random walk". Association for Computing Machinery Conference on Information and Knowledge Management. pp. 1147. doi:10.1145/2063576.2063741. https://pdfs.semanticscholar.org/f185/bc2499be95b4312169a7e722bac570c2d509.pdf.
3. ↑ Ochab, J.K.; Burda, Z. (2013). "Maximal entropy random walk in community detection". The European Physical Journal Special Topics 216 (1): 73–81. doi:10.1140/epjst/e2013-01730-6. ISSN 1951-6355. Bibcode: 2013EPJST.216...73O.
4. ↑ Chen, Y.; Georgiou, T.T.; Pavon, M.; Tannenbaum, A. (2016). "Robust transport over networks". IEEE Transactions on Automatic Control 62 (9): 4675–4682. doi:10.1109/TAC.2016.2626796. PMID 28924302. Bibcode: 2016arXiv160308129C.
5. ↑ Delvenne, Jean-Charles; Libert, Anne-Sophie (2011). "Centrality measures and thermodynamic formalism for complex networks". Physical Review E 83 (4): 046117. doi:10.1103/PhysRevE.83.046117. ISSN 1539-3755. PMID 21599250. Bibcode: 2011PhRvE..83d6117D.
6. ↑ Jin-Gang Yu; Ji Zhao; Jinwen Tian; Yihua Tan (2014). "Maximal Entropy Random Walk for Region-Based Visual Saliency". IEEE Transactions on Cybernetics (Institute of Electrical and Electronics Engineers (IEEE)) 44 (9): 1661–1672. doi:10.1109/tcyb.2013.2292054. ISSN 2168-2267. PMID 25137693.
7. ↑ L. Wang, J. Zhao, X. Hu, J. Lu, Weakly supervised object localization via maximal entropy random walk, ICIP, 2014.
8. ↑ Korus, Pawel; Huang, Jiwu (2016). "Improved Tampering Localization in Digital Image Forensics Based on Maximal Entropy Random Walk". IEEE Signal Processing Letters (Institute of Electrical and Electronics Engineers (IEEE)) 23 (1): 169–173. doi:10.1109/lsp.2015.2507598. ISSN 1070-9908. Bibcode: 2016ISPL...23..169K.
9. ↑ Galinsky, Vitaly L.; Frank, Lawrence R. (2015). "Simultaneous Multi-Scale Diffusion Estimation and Tractography Guided by Entropy Spectrum Pathways". IEEE Transactions on Medical Imaging (Institute of Electrical and Electronics Engineers (IEEE)) 34 (5): 1177–1193. doi:10.1109/tmi.2014.2380812. ISSN 0278-0062. PMID 25532167.
10. ↑ Burda, Z.; Duda, J.; Luck, J. M.; Waclaw, B. (2009-04-23). "Localization of the Maximal Entropy Random Walk". Physical Review Letters 102 (16): 160602. doi:10.1103/physrevlett.102.160602. ISSN 0031-9007. PMID 19518691. Bibcode: 2009PhRvL.102p0602B.
11. ↑ J. Duda, Extended Maximal Entropy Random Walk, PhD Thesis, 2012.

## External links

* Gábor Simonyi, Y. Lin, Z. Zhang, "Mean first-passage time for maximal-entropy random walks in complex networks". Scientific Reports, 2014.
* Electron Conductance Models Using Maximal Entropy Random Walks Wolfram Demonstration Project

* v
* t
* e

Stochastic processes

Discrete time|

* Bernoulli process
* Branching process
* Chinese restaurant process
* Galton–Watson process
* Independent and identically distributed random variables
* Markov chain
* Moran process
* Random walk
* Loop-erased
* Self-avoiding
* Biased
* Maximal entropy

Continuous time|

* Bessel process
* Birth–death process
* Brownian motion
* Bridge
* Excursion
* Fractional
* Geometric
* Meander
* Cauchy process
* Contact process
* Continuous-time random walk
* Cox process
* Diffusion process
* Empirical process
* Feller process
* Fleming–Viot process
* Gamma process
* Geometric process
* Hunt process
* Interacting particle systems
* Itô diffusion
* Itô process
* Jump diffusion
* Jump process
* Lévy process
* Local time
* Markov additive process
* McKean–Vlasov process
* Ornstein–Uhlenbeck process
* Poisson process
* Compound
* Non-homogeneous
* Point process
* Schramm–Loewner evolution
* Semimartingale
* Sigma-martingale
* Stable process
* Superprocess
* Telegraph process
* Variance gamma process
* Wiener process
* Wiener sausage

Both|

* Branching process
* Galves–Löcherbach model
* Gaussian process
* Hidden Markov model (HMM)
* Markov process
* Martingale
* Differences
* Local
* Sub-
* Super-
* Random dynamical system
* Regenerative process
* Renewal process
* Stochastic chains with memory of variable length
* White noise

Fields and other|

* Dirichlet process
* Gaussian random field
* Gibbs measure
* Hopfield model
* Ising model
* Potts model
* Boolean network
* Markov random field
* Percolation
* Pitman–Yor process
* Point process
* Cox
* Poisson
* Random field
* Random graph

Time series models|

* Autoregressive conditional heteroskedasticity (ARCH) model
* Autoregressive integrated moving average (ARIMA) model
* Autoregressive (AR) model
* Autoregressive–moving-average (ARMA) model
* Generalized autoregressive conditional heteroskedasticity (GARCH) model
* Moving-average (MA) model

Financial models|

* Black–Derman–Toy
* Black–Karasinski
* Black–Scholes
* Chen
* Constant elasticity of variance (CEV)
* Cox–Ingersoll–Ross (CIR)
* Garman–Kohlhagen
* Heath–Jarrow–Morton (HJM)
* Heston
* Ho–Lee
* Hull–White
* LIBOR market
* Rendleman–Bartter
* SABR volatility
* Vašíček
* Wilkie

Actuarial models|

* Bühlmann
* Cramér–Lundberg
* Risk process
* Sparre–Anderson

Queueing models|

* Bulk
* Fluid
* Generalized queueing network
* M/G/1
* M/M/1
* M/M/c

Properties|

* Càdlàg paths
* Continuous
* Continuous paths
* Ergodic
* Exchangeable
* Feller-continuous
* Gauss–Markov
* Markov
* Mixing
* Piecewise deterministic
* Predictable
* Progressively measurable
* Self-similar
* Stationary
* Time-reversible

Limit theorems|

* Central limit theorem
* Donsker's theorem
* Doob's martingale convergence theorems
* Ergodic theorem
* Fisher–Tippett–Gnedenko theorem
* Large deviation principle
* Law of large numbers (weak/strong)
* Law of the iterated logarithm
* Maximal ergodic theorem
* Sanov's theorem

Inequalities|

* Burkholder–Davis–Gundy
* Doob's martingale
* Kunita–Watanabe

Tools|

* Cameron–Martin formula
* Convergence of random variables
* Doléans-Dade exponential
* Doob decomposition theorem
* Doob–Meyer decomposition theorem
* Doob's optional stopping theorem
* Dynkin's formula
* Feynman–Kac formula
* Filtration
* Girsanov theorem
* Infinitesimal generator
* Itô integral
* Itô's lemma
* Karhunen–Loève_theorem
* Kolmogorov continuity theorem
* Kolmogorov extension theorem
* Lévy–Prokhorov metric
* Malliavin calculus
* Martingale representation theorem
* Optional stopping theorem
* Prokhorov's theorem
* Quadratic variation
* Reflection principle
* Skorokhod integral
* Skorokhod's representation theorem
* Skorokhod space
* Snell envelope
* Stochastic differential equation
* Tanaka
* Stopping time
* Stratonovich integral
* Uniform integrability
* Usual hypotheses
* Wiener space
* Classical
* Abstract

Disciplines|

* Actuarial mathematics
* Econometrics
* Ergodic theory
* Extreme value theory (EVT)
* Large deviations theory
* Mathematical finance
* Mathematical statistics
* Probability theory
* Queueing theory
* Renewal theory
* Ruin theory
* Statistics
* System on Chip design
* Stochastic analysis
* Time series analysis
* Machine learning

* List of topics
* Category

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/Maximal entropy random walk. Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=Physics:Maximal_entropy_random_walk&oldid=3281684"
*[v]: View this template
*[t]: Discuss this template
*[e]: Edit this template