随机算法 (Fall 2011)/Rapid mixing random walks

We see that the mixing performance of a random walk on an undirected graph is determined by the expansion ratio of the graph. We now consider the random walks in a more general setting, and study the mixing performance of a general class of Markov chains.

Mixing Time

The mixing time of a Markov chain gives the time of the chain to approach the stationary distribution. To formally define the mixing time, we need a notion of the distance between two probability distributions.

Let [math]\displaystyle{ p }[/math] and [math]\displaystyle{ q }[/math] be two probability distributions over the same finite state space [math]\displaystyle{ \mathcal{S} }[/math], the total variation distance between [math]\displaystyle{ p }[/math] and [math]\displaystyle{ q }[/math] is defined as

[math]\displaystyle{ \frac{1}{2}\sum_{i\in\mathcal{S}}|p_i-q_i| }[/math],

which we can express as the [math]\displaystyle{ \ell_1 }[/math]-distance [math]\displaystyle{ \frac{1}{2}\|p-q\|_1. }[/math]

You may have encountered the concept of total variation before, and it might be defined differently, as

[math]\displaystyle{ \max_{A\subset\mathcal{S}}|p(A)-q(A)|. }[/math]

It is not hard to see that the two definitions are equivalent and

[math]\displaystyle{ \max_{A\subset\mathcal{S}}|p(A)-q(A)|=\frac{1}{2}\sum_{i\in \mathcal{S}}|p_i-q_i|=\frac{1}{2}\|p-q\|_1. }[/math]

Here we prefer to use our version, because it is convenient to use the tools for norms to analyze it.

Definition (mixing time)

For a Markov chain with finite state space [math]\displaystyle{ \mathcal{S} }[/math], transition matrix [math]\displaystyle{ P }[/math], and stationary distribution [math]\displaystyle{ \pi }[/math], the total variation distance at time [math]\displaystyle{ t }[/math] with initial state [math]\displaystyle{ i\in\mathcal{S} }[/math] is defined as [math]\displaystyle{ \Delta_i(t)=\frac{1}{2}\sum_{j\in\mathcal{S}}|P^t(i,j)-\pi(j)|=\frac{1}{2}\|\boldsymbol{e}_iP^t-\pi\|_1 }[/math] where [math]\displaystyle{ \boldsymbol{e}_i }[/math] is the vector that [math]\displaystyle{ \boldsymbol{e}_i(i)=1 }[/math] and [math]\displaystyle{ \boldsymbol{e}_i(j)=0 }[/math] for [math]\displaystyle{ j\neq i }[/math]. We define that [math]\displaystyle{ \tau_i(\epsilon)=\min\{t\mid \Delta_i(t)\le \epsilon\} }[/math] and [math]\displaystyle{ \tau(\epsilon)=\max_{i\in\mathcal{S}}\tau_i(\epsilon) }[/math].

[math]\displaystyle{ \tau_i(\epsilon) }[/math] is the first time that a chain starting from state [math]\displaystyle{ i }[/math] approaches its stationary distribution within a total variation distance of [math]\displaystyle{ \epsilon }[/math], and [math]\displaystyle{ \tau(\epsilon) }[/math] is the maximum of these values over all states. While [math]\displaystyle{ \tau(\epsilon) }[/math] is described as a function of [math]\displaystyle{ \epsilon }[/math], it is generally referred as the mixing time of the Markov chain.

For the efficiency of randomized algorithms, we are interested in the random walks that converges "fast". Measured by the mixing time, we need the mixing time to be "small". Recall that the mixing time [math]\displaystyle{ \tau(\epsilon) }[/math] is a function. So what we really mean is that as a function, the mixing time grows slowly as its input become larger.

The mixing time [math]\displaystyle{ \tau(\epsilon) }[/math] has an input [math]\displaystyle{ \epsilon }[/math] which is the distance to the stationary distribution, and there is another hidden parameter for [math]\displaystyle{ \tau(\epsilon) }[/math], namely, the size of the state space [math]\displaystyle{ N=|\mathcal{S}| }[/math]. The parameter [math]\displaystyle{ \epsilon }[/math] gives the error bound, and [math]\displaystyle{ N }[/math] reflects the size of the problem.

A random walk is called rapid mixing if its mixing time [math]\displaystyle{ \tau(\epsilon) }[/math] is poly-logarithmic of both [math]\displaystyle{ N }[/math] and [math]\displaystyle{ \frac{1}{\epsilon} }[/math], i.e. when

[math]\displaystyle{ \tau(\epsilon)=O((\log N+\log(1/\epsilon))^{C})\, }[/math]

for some constant [math]\displaystyle{ C }[/math].

The eigenvalue approach

Let [math]\displaystyle{ P }[/math] be the transition matrix of a Markov chain, and let [math]\displaystyle{ \pi }[/math] be the stationary distribution, such that [math]\displaystyle{ \pi P=\pi }[/math]. For any initial distribution [math]\displaystyle{ q }[/math], the distribution of the chain at time [math]\displaystyle{ t }[/math] is give by [math]\displaystyle{ qP^t }[/math]. The total variation at time [math]\displaystyle{ t }[/math] is governed by the [math]\displaystyle{ \ell_1 }[/math]-distance

[math]\displaystyle{ \|qP^t-\pi\|_1 }[/math].

But how to estimate this? Not surprisingly, it can be answered by looking at the eigenvalues of [math]\displaystyle{ P }[/math].

Let [math]\displaystyle{ \lambda_1\ge\lambda_2\ge\cdots\ge\lambda_n }[/math] be the eigenvalues of [math]\displaystyle{ P }[/math].

Remark:

The eigenvalues are now of the transition matrix [math]\displaystyle{ P }[/math] instead of the adjacency matrix of a graph. With the same argument as the spectrum of graphs, we can show that [math]\displaystyle{ \lambda_1=1 }[/math] and [math]\displaystyle{ |\lambda_i|\le 1 }[/math] for all [math]\displaystyle{ i }[/math], and for irreducible chains, [math]\displaystyle{ \lambda_1\gt \lambda_2 }[/math]. Therefore, for irreducible Markov chains, [math]\displaystyle{ 1=\lambda_1\gt \lambda_2\ge\cdots\ge\lambda_n\ge -1 }[/math].

Why should we care about eigenvalues of [math]\displaystyle{ P }[/math]? Recall that [math]\displaystyle{ \lambda\neq 0 }[/math] is an eigenvalue of [math]\displaystyle{ P }[/math] if for some vector [math]\displaystyle{ v }[/math],

[math]\displaystyle{ Pv=\lambda v }[/math],

where [math]\displaystyle{ v }[/math] is called an eigenvector. The eigenvalues are the solutions to the [math]\displaystyle{ \det(A-\lambda I)=0 }[/math].

For our purpose, we are interested in the left-eigenvalues and left-eigenvectors, such that

[math]\displaystyle{ vP=\lambda v }[/math].

Note that the left-eigenvalues are equal to the right-eigenvalues, because

[math]\displaystyle{ \det(A-\lambda I)=\det(A^T-\lambda I) }[/math],

however, the left-eigenvectors may be different from right-eigenvectors.

Let [math]\displaystyle{ v_1,v_2,\ldots,v_n }[/math] be the (left)eigenvectors corresponds to the eigenvalues [math]\displaystyle{ \lambda_1\ge\lambda_2\ge\cdots\ge\lambda_n }[/math]. A key observation is that if [math]\displaystyle{ P }[/math] is symmetric (that is, [math]\displaystyle{ P^T=P }[/math]), the eigenvectors are orthogonal to each other, thus can be treated as orthogonal basis, which means that any vector [math]\displaystyle{ q }[/math] can be uniquely represented as

[math]\displaystyle{ q=c_1v_1+c_2v_2+\cdots+c_nv_n }[/math],

for some scalars [math]\displaystyle{ c_1,c_2,\ldots,c_n }[/math]. Furthermore, we can choose the first component as [math]\displaystyle{ c_1v_1=\pi }[/math], because we know that [math]\displaystyle{ \pi }[/math] is the left-eigenvector with the largest eigenvalue [math]\displaystyle{ 1 }[/math]. Thus,

[math]\displaystyle{ q=\pi+c_2v_2+\cdots+c_nv_n }[/math].

Then by the linearity, an action of [math]\displaystyle{ P }[/math] can be computed by

[math]\displaystyle{ qP=\left(\pi+\sum_{i=2}^n c_i v_i\right)P=\pi P+\sum_{i=2}^n (c_i v_iP)=\pi+\sum_{i=2}^n \lambda_i c_i v_i. }[/math]

Thus, multiplying [math]\displaystyle{ P }[/math] corresponds to multiplying an eigenvalue to the scalar corresponding to each basis. Repeating this process, we have

[math]\displaystyle{ qP^t=\pi+\sum_{i=2}^n \lambda_i^t c_i v_i. }[/math]

So the difference between the distribution of the chain and the stationary distribution is shrinking by a factor of [math]\displaystyle{ \lambda_\max=\max(|\lambda_2|,|\lambda_n|) }[/math] at each step. This explain why we care about [math]\displaystyle{ \lambda_\max }[/math], because it dominates the rate at which the difference shrinks.

However, right now this beautiful theory holds only when the transition matrix [math]\displaystyle{ P }[/math] is symmetric. In some special case, such as the random walk on a [math]\displaystyle{ d }[/math]-regular graph, the transition matrix is indeed symmetric, but for various applications of Markov chains, the transition matrix is not necessarily symmetric. We will see that there is a more general class of Markov chains for which we can apply the similar technique as when the transition matrix is symmetric.

Reversibility

We restrict ourselves to a special class of Markov chains call time-reversible Markov chains.

Definition (time-reversible)

A Markov chain with finite state space [math]\displaystyle{ \mathcal{S} }[/math], transition matrix [math]\displaystyle{ P }[/math], and stationary distribution [math]\displaystyle{ \pi }[/math] is said to be time-reversible if for all [math]\displaystyle{ i,j\in\mathcal{S} }[/math] [math]\displaystyle{ \pi_{i}P_{i,j}=\pi_{j}P_{j,i}.\, }[/math]

For reversible chains, its stationary distribution shows a stronger equilibrium property: not only the stationary is unchanged under the action of transition, but also when the chain is stationary, it has equal chance to move from [math]\displaystyle{ i }[/math] to [math]\displaystyle{ j }[/math] and from [math]\displaystyle{ j }[/math] to [math]\displaystyle{ i }[/math].

Example

• A symmetric random walk ([math]\displaystyle{ P }[/math] is symmetric) is time-reversible.
• Random walks on undirected graphs (not necessarily [math]\displaystyle{ d }[/math]-regular) are time-reversible.

The name "time-reversible" is due to the following fact:

Let [math]\displaystyle{ X_0,X_1,\ldots,X_n }[/math] be a time-reversible Markov chain whose initial distribution is the stationary distribution [math]\displaystyle{ \pi }[/math], then the distribution of the reversed sequence [math]\displaystyle{ X_n,X_{n-1},\ldots,X_0 }[/math] is exactly the same as [math]\displaystyle{ X_0,X_1,\ldots,X_n }[/math], formally, for any states [math]\displaystyle{ x_0,x_1,\ldots,x_n }[/math],

[math]\displaystyle{ \Pr[X_0=x_0\wedge X_1=x_1\wedge \ldots \wedge X_n=x_n]=\Pr[X_0=x_n\wedge X_1=x_{n-1}\wedge \ldots \wedge X_n=x_0] }[/math].

Although for a time-reversible Markov chain, the transition matrix [math]\displaystyle{ P }[/math] is not necessarily symmetric, we can make a symmetric matrix out of it.

Mixing time of reversible chains

For a time-reversible [math]\displaystyle{ P }[/math] and stationary distribution [math]\displaystyle{ \pi }[/math], it holds that [math]\displaystyle{ \pi_iP_{i,j}=\pi_jP_{j,i} }[/math]. Divide both sides by [math]\displaystyle{ \sqrt{\pi_i\pi_j} }[/math], we have

[math]\displaystyle{ \sqrt{\frac{\pi_i}{\pi_j}}P_{i,j}=\sqrt{\frac{\pi_j}{\pi_i}}P_{j,i}. }[/math]

This shows that the matrix [math]\displaystyle{ S }[/math] with entries

[math]\displaystyle{ S_{i,j}=\sqrt{\frac{\pi_i}{\pi_j}}P_{i,j}, }[/math]

is symmetric. Let [math]\displaystyle{ \Pi }[/math] be the diagonal matrix given by [math]\displaystyle{ \Pi_{i,i}=\sqrt{\pi_i} }[/math]. Then [math]\displaystyle{ S }[/math] can be written as [math]\displaystyle{ S=\Pi P \Pi^{-1}\, }[/math], therefore, for any time-reversible Markov chain, its transition matrix [math]\displaystyle{ P }[/math] can be written as

[math]\displaystyle{ P=\Pi^{-1}S\Pi\, }[/math]

where [math]\displaystyle{ S }[/math] is symmetric, and has the same eigenvalues as [math]\displaystyle{ P }[/math] (although the eigenvectors may be different), and for any initial distribution [math]\displaystyle{ q }[/math],

[math]\displaystyle{ qP^t=q(\Pi^{-1}S\Pi)^t=q\Pi^{-1}S^t\Pi\, }[/math].

Because [math]\displaystyle{ S }[/math] is symmetric, its eigenvectors are orthogonal basis and the same spectral technique works. This again will give us a nice characterization of the mixing time by [math]\displaystyle{ \lambda_\max=\max(|\lambda_2|,|\lambda_n|) }[/math] of [math]\displaystyle{ P }[/math], and prove the following theorem (the details of the proof are omitted).

Theorem

For a time-reversible Markov chain with finite state space [math]\displaystyle{ \mathcal{S} }[/math] and transition matrix [math]\displaystyle{ P }[/math], let [math]\displaystyle{ \lambda_1\ge\lambda_2\ge\cdots\ge\lambda_n }[/math] be the eigenvalues of [math]\displaystyle{ P }[/math]. For any state [math]\displaystyle{ i\in\mathcal{S} }[/math], [math]\displaystyle{ \Delta_i(t)\le\frac{1}{2}\lambda_\max^t\sqrt{\frac{1-\pi_i}{\pi_i}} }[/math], where [math]\displaystyle{ \lambda_\max=\max(|\lambda_2|,|\lambda_n|) }[/math] is the largest absolute value of eigenvalues other than [math]\displaystyle{ \lambda_1=1 }[/math].

The theorem about the mixing time of random walks on expander graphs is a special case of the above theorem.

It is convenient to express the mixing rate as a function in the form of [math]\displaystyle{ \exp(\cdot) }[/math], so its natural logarithm looks nicer. We observe that

[math]\displaystyle{ \lambda_\max=1-(1-\lambda_\max)\le e^{-(1-\lambda_\max)}, }[/math]

and thus

[math]\displaystyle{ \lambda_\max^t\le e^{-(1-\lambda_\max)t}. }[/math]

The theorem is turned to

[math]\displaystyle{ \Delta_i(t)\le\frac{1}{2}e^{-(1-\lambda_\max)t}\sqrt{\frac{1-\pi_i}{\pi_i}}. }[/math]

Solving the [math]\displaystyle{ \Delta_i(t)=\epsilon }[/math] gives us the mixing time:

Corollary (mixing time of reversible chain) For a time-reversible Markov chain, [math]\displaystyle{ \tau_i(\epsilon)=O\left(\frac{\ln(1/\pi_i)+\ln(1/\epsilon)}{1-\lambda_\max}\right) }[/math]

For a lazy random walk, where the transition matrix is [math]\displaystyle{ Q=\frac{1}{2}(I+P) }[/math], it is easy to see that [math]\displaystyle{ Q }[/math] is also time-reversible and has the same stationary distribution and [math]\displaystyle{ P }[/math], and the eigenvalues of [math]\displaystyle{ Q }[/math] are all nonnegative, thus [math]\displaystyle{ \lambda_\max=\lambda_2 }[/math].

From now on, we only consider lazy random walks and always assume that [math]\displaystyle{ \lambda_\max=\lambda_2 }[/math].

Theorem (mixing time of reversible lazy random walk)

For a time-reversible lazy random walk, that is, for a time-reversible Markov chain whose transition matrix [math]\displaystyle{ P\, }[/math] has [math]\displaystyle{ P_{i,i}\ge\frac{1}{2} }[/math] for all [math]\displaystyle{ i\in\mathcal{S} }[/math], [math]\displaystyle{ \tau_i(\epsilon)=O\left(\frac{\ln(1/\pi_i)+\ln(1/\epsilon)}{1-\lambda_2}\right) }[/math]. In particular, when the stationary distribution is uniform, the mixing time [math]\displaystyle{ \tau(\epsilon)=O\left(\frac{\ln N+\ln(1/\epsilon)}{1-\lambda_2}\right) }[/math], where [math]\displaystyle{ N=|\mathcal{S}| }[/math].

The random walk is rapid mixing if the spectral gap [math]\displaystyle{ (1-\lambda_2) }[/math] is larger than [math]\displaystyle{ \frac{1}{(\log N)^c} }[/math] for some constant [math]\displaystyle{ c }[/math].

Conductance

Spectral gap and the mixing time

• We consider a Markov chain with finite space [math]\displaystyle{ \mathcal{S} }[/math], transition matrix [math]\displaystyle{ P }[/math], and stationary distribution [math]\displaystyle{ \pi }[/math]. Let [math]\displaystyle{ N=|\mathcal{S}| }[/math] denote the size of the state space, and let [math]\displaystyle{ \lambda_1\ge\cdots\ge\lambda_N }[/math] be the eigenvalues of [math]\displaystyle{ P }[/math]. For any stochastic matrix [math]\displaystyle{ P }[/math], it holds that [math]\displaystyle{ 1=\lambda_1\ge\cdots\ge\lambda_N\ge -1 }[/math].
• The mixing time [math]\displaystyle{ \tau(\epsilon)\, }[/math] of an irreducible and aperiodic Markov chain is given by the time to be within total variation distance [math]\displaystyle{ \epsilon }[/math] from [math]\displaystyle{ \pi }[/math], starting from a worst-case state. Formally,

[math]\displaystyle{ \tau_i(\epsilon)=\min\left\{t\,\,\bigg|\,\, \frac{1}{2}\sum_{j\in\mathcal{S}}|P^t(i,j)-\pi(i)|\le\epsilon\right\} }[/math] and [math]\displaystyle{ \tau(\epsilon)=\max_{i\in\mathcal{S}}\tau_i(\epsilon) }[/math].

Conditions:

• The Markov chain is time-reversible: [math]\displaystyle{ \pi_{i}P_{i,j}=\pi_{j}P_{j,i} }[/math] for all [math]\displaystyle{ i,j\in\mathcal{S} }[/math].

For a time-reversible Markov chain, the transition matrix [math]\displaystyle{ P }[/math] can be transformed to a symmetric matrix, thus the orthogonal diagonalization for symmetric matrices can be applied, and the convergence rate is determined by the second largest absolute value of eigenvalues [math]\displaystyle{ \lambda_{\max}=\max(|\lambda_2|,|\lambda_N|) }[/math] as

[math]\displaystyle{ \tau_i(\epsilon)=O\left(\frac{\ln(1/\pi_i)+\ln(1/\epsilon)}{1-\lambda_{\max}}\right) }[/math].

• Lazy random walk: [math]\displaystyle{ P_{i,i}\ge\frac{1}{2} }[/math] for any [math]\displaystyle{ i\in\mathcal{S} }[/math].

A lazy walk is aperiodic. The eigenvalues of a lazy walk are nonnegative, and thus [math]\displaystyle{ \lambda_{\max}=\max(|\lambda_2|,|\lambda_N|)=\lambda_2 }[/math]. The mixing time is determined by the second largest eigenvalue [math]\displaystyle{ \lambda_2 }[/math] as

[math]\displaystyle{ \tau_i(\epsilon)=O\left(\frac{\ln(1/\pi_i)+\ln(1/\epsilon)}{1-\lambda_{2}}\right) }[/math].

• The stationary [math]\displaystyle{ \pi }[/math] is the uniform distribution, that is, [math]\displaystyle{ \pi_i=\frac{1}{N} }[/math] for all [math]\displaystyle{ i\in\mathcal{S} }[/math].

Then:

Theorem

The mixing time of a time-reversible lazy random walk with uniform stationary distribution over a state space of size [math]\displaystyle{ N }[/math], is [math]\displaystyle{ \tau(\epsilon)=O\left(\frac{\ln N+\ln(1/\epsilon)}{1-\lambda_2}\right) }[/math].

From the previous section, we see how to construct a time-reversible lazy random walk with uniform stationary distribution. The mixing time is now determined by the spectral gap [math]\displaystyle{ (1-\lambda_2) }[/math]. In order to upper bound the mixing time, we need to lower bound the spectral gap. However, it is difficult to directly bound the eigenvalues of a usually exponentially large transition matrix.

Conductance and the mixing time

For many problems, such as card shuffling, the state space is exponentially large, so the estimation of [math]\displaystyle{ \lambda_2 }[/math] becomes very difficult. The following technique based on conductance overcomes this issue by considering the conductance of a Markov chain.

Definition (conductance)

The conductance of a irreducible Markov chain with finite state space [math]\displaystyle{ \Omega }[/math], transition matrix [math]\displaystyle{ P }[/math], and stationary distribution [math]\displaystyle{ \pi }[/math], is defined as [math]\displaystyle{ \Phi=\min_{\overset{S\subset\Omega}{0\lt \pi(S)\le\frac{1}{2}}} \frac{\sum_{i\in S, j\not\in S}\pi_iP_{ij}}{\pi(S)} }[/math] where [math]\displaystyle{ \pi(S)=\sum_{i\in S}\pi_i }[/math] is the probability density of [math]\displaystyle{ S\subset \Omega }[/math] under the stationary distribution [math]\displaystyle{ \pi }[/math].

The definition of conductance looks quite similar to the expansion ratio of graphs. Very informally, the conductance can be seen as the weighted normalized version of expansion ratio.

The following lemma states a Cheeger's inequality for the conductance.

Lemma (Jerrum-Sinclair 1988)

In a time-reversible Markov chain, [math]\displaystyle{ 1-2\Phi\le\lambda_2\le 1-\frac{\Phi^2}{2} }[/math].

The inequality can be equivalent written for the spectral gap:

[math]\displaystyle{ \frac{\Phi^2}{2}\le1-\lambda_2\le 2\Phi }[/math].

Thus [math]\displaystyle{ \Phi^2/2 }[/math] is a lower bound of the spectral gap, which in turn gives an upper bound of the mixing time via conductance.

Proposition

The mixing time of a time-reversible lazy random walk with uniform stationary distribution over a state space of size [math]\displaystyle{ N }[/math], is [math]\displaystyle{ \tau(\epsilon)=O\left(\frac{\ln N+\ln(1/\epsilon)}{\Phi^2}\right) }[/math].

So analyzing of mixing time is reduced to analyzing of the second largest eigenvalue, which is reduced to analyzing of the conductance. For some simple Markv chain, this can be done directly. For some more complicated Markov chain, this is done by a technique called the canonical paths argument.

Canonical Paths

We will first show how to lower bound the expansion ratio of a graph by canonical paths argument, and then we will generalize this technique to the conductance.

As we know, the expansion ratio of an undirected graph [math]\displaystyle{ G(V,E) }[/math],

[math]\displaystyle{ \phi(G)=\min_{\overset{ S\subset V}{|S|\le\frac{|V|}{2}}}\frac{|\partial|}{|S|} }[/math],

is hard to compute, because its definition involves an optimization over exponentially many subsets. We will see that it can be lower bounded by computing the maximum congestion caused by a set of paths.

For every pair of vertices [math]\displaystyle{ x,y\in V }[/math], we construct a simple path [math]\displaystyle{ \gamma_{xy} }[/math] from [math]\displaystyle{ x }[/math] to [math]\displaystyle{ v }[/math]. Each ordered pair [math]\displaystyle{ (x,y) }[/math] of distinct vertices corresponds to exact one path, called the canonical path from [math]\displaystyle{ x }[/math] to [math]\displaystyle{ y }[/math]. Let [math]\displaystyle{ \Gamma=\{\gamma_{xy}\mid x,y\in V, x\neq y\} }[/math] be the collection of canonical paths.

The set of canonical paths that an edge [math]\displaystyle{ uv }[/math] is on is

[math]\displaystyle{ \mathrm{CP}(uv)=\{\gamma_{xy}\in\Gamma\mid uv\in\gamma_{xy}\} }[/math].

The congestion of an edge is given by [math]\displaystyle{ |\mathrm{CP}(uv)| }[/math], which is the number of canonical paths crossing the edge [math]\displaystyle{ uv }[/math].

For any [math]\displaystyle{ S\subset V }[/math] where [math]\displaystyle{ |S|\le\frac{|V|}{2} }[/math], there are [math]\displaystyle{ |S||\bar{S}| }[/math] ordered pairs [math]\displaystyle{ (x,y) }[/math] of vertices such that [math]\displaystyle{ x\in S }[/math] and [math]\displaystyle{ y\not\in S }[/math], thus there are [math]\displaystyle{ |S||\bar{S}| }[/math] canonical paths going from [math]\displaystyle{ S }[/math] to [math]\displaystyle{ \bar{S} }[/math]. The average congestion of edges crossing from [math]\displaystyle{ S }[/math] and [math]\displaystyle{ \bar{S} }[/math] is [math]\displaystyle{ \frac{|S||\bar{S}|}{|\partial S|} }[/math], which must be no bigger than the maximum congestion. Thus,

[math]\displaystyle{ \sigma\overset{\mathrm{def}}{=}\max_{uv\in E}|\mathrm{CP}(uv)|\ge\frac{|S||\bar{S}|}{|\partial S|} }[/math]

and

[math]\displaystyle{ \frac{|\partial S|}{|S|}\ge\frac{|\bar{S}|}{\sigma}\ge\frac{|V|}{2\sigma} }[/math].

This inequality holds for any [math]\displaystyle{ S\subset V }[/math] where [math]\displaystyle{ |S|\le\frac{|V|}{2} }[/math], by the definition of expansion ratio,

[math]\displaystyle{ \phi(G)\ge\frac{|V|}{2\sigma} }[/math]

Therefore, for any collection of canonical paths [math]\displaystyle{ \Gamma=\{\gamma_{xy}\} }[/math], the maximum congestion [math]\displaystyle{ \sigma=\max_{uv\in E}|\mathrm{CP}(uv)| }[/math] caused by [math]\displaystyle{ \Gamma }[/math] can be used to lower bound [math]\displaystyle{ \phi(G) }[/math]. By choosing cleverer paths, we can make the bound tighter.

The intuition behind this inequality is that large expansion ratio (the graph is well-connected) is a necessary condition for the existence of a pairwise routing scheme with low-congestion.

---

We now show how to apply the canonical paths argument to the Markov chain and lower bound the conductance of the chain.

For a Markov chain with finite state space [math]\displaystyle{ \mathcal{S} }[/math], and transition matrix [math]\displaystyle{ P }[/math]. The chain corresponds to a transition graph [math]\displaystyle{ \mathcal{G}(\mathcal{S},E) }[/math] on the state space [math]\displaystyle{ \mathcal{S} }[/math]. For any states [math]\displaystyle{ u,v\in\mathcal{S} }[/math], [math]\displaystyle{ uv\in E }[/math] if [math]\displaystyle{ P_{uv}\gt 0 }[/math].

The canonical paths for the chain is a collection of simple paths [math]\displaystyle{ \Gamma=\{\gamma_{xy}\mid x,y\in \mathcal{S}, x\neq y\} }[/math] in the transition graph [math]\displaystyle{ \mathcal{G} }[/math], such that for any ordered pair [math]\displaystyle{ (x,y) }[/math] of distinct states, there is exact one path [math]\displaystyle{ \gamma_{xy} }[/math] going from [math]\displaystyle{ x }[/math] to [math]\displaystyle{ y }[/math] in [math]\displaystyle{ \mathcal{G} }[/math].

Let [math]\displaystyle{ \Gamma=\{\gamma_{xy}\} }[/math] be a collection of canonical paths. The congestion caused by [math]\displaystyle{ \Gamma }[/math] is computed as

[math]\displaystyle{ \rho=\max_{uv \in E}\frac{1}{\pi_u P_{uv}}\sum_{\gamma_{xy}\ni uv}\pi_x\pi_y }[/math].

Note that compared to the example of lower bounding the expansion ratio, in which we directly count the number of paths crossing an edge as the congestion, this time the congestion is defined by the probability "flows". This is analogous to the difference between the definitions of expansion ratio of a graph and the conductance of a Markov chain.

Lemma

Let [math]\displaystyle{ \Gamma=\{\gamma_{xy}\} }[/math] be a collection of canonical path. The conductance is lower bounded by [math]\displaystyle{ \Phi\ge\frac{1}{2\rho} }[/math].

Proof. [math]\displaystyle{ \square }[/math]

Due to the upper bound of the mixing time via the conductance, we have the following proposition.

Proposition

Consider a time-reversible lazy random walk with uniform stationary distribution over a state space of size [math]\displaystyle{ N }[/math]. Let [math]\displaystyle{ \Gamma=\{\gamma_{xy}\} }[/math] be an arbitrary collection of canonical paths. Assuming that the congestion of [math]\displaystyle{ \Gamma }[/math] is [math]\displaystyle{ \rho\, }[/math], the mixing time of the chain is bounded by [math]\displaystyle{ \tau(\epsilon)=O\left(\rho^2\left(\ln N+\ln(1/\epsilon)\right)\right) }[/math].

Example: random walk on a cube

We then apply the canonical path argument on a simple Markov chain: the random walk on high dimensional cube.

Let the state space be [math]\displaystyle{ \mathcal{S}=[d]^n }[/math]. Consider the random walk:

• Start from an arbitrary [math]\displaystyle{ x\in \mathcal{S} }[/math]
• At each step, assuming the current state is [math]\displaystyle{ x\in \mathcal{S} }[/math], choose an random [math]\displaystyle{ i\in\{1,2,\ldots, n\} }[/math] and a random [math]\displaystyle{ b\in\{-1,1\} }[/math] and move to the state [math]\displaystyle{ y\in\mathcal{S} }[/math] such that [math]\displaystyle{ y_i=(x_i+b)\bmod d }[/math] and [math]\displaystyle{ y_j=x_j }[/math] for [math]\displaystyle{ j\neq i }[/math].

The Markov chain is a random walk on an [math]\displaystyle{ n }[/math]-dimensional [math]\displaystyle{ d }[/math]-ary cube [math]\displaystyle{ G(V,E) }[/math], where [math]\displaystyle{ V=[d]^n }[/math], and [math]\displaystyle{ uv\in E }[/math] if [math]\displaystyle{ u }[/math] and [math]\displaystyle{ v }[/math] differ in one coordinate by 1 (modular [math]\displaystyle{ d }[/math]). And for any [math]\displaystyle{ uv\in E }[/math], the transition probability [math]\displaystyle{ P_{uv}=\frac{1}{2n} }[/math].

We will show that the walk is rapid mixing by the canonical paths argument. The argument involves the following steps:

• Construct a collection of canonical paths.
• Bound the congestion of a single edge. This step is usually done by "encoding" the canonical paths crossing the edge.
• Compute the [math]\displaystyle{ \rho\, }[/math] and thus bound the mixing time.

Construct the canonical paths

Consider the canonical paths [math]\displaystyle{ \Gamma=\{\gamma_{xy}\} }[/math] defined by the following "fixing" scheme:

• For each [math]\displaystyle{ x,y \in \mathcal{S} }[/math], [math]\displaystyle{ \gamma_{xy} }[/math] is the path obtained by fixing the 1st, 2nd, ... [math]\displaystyle{ n }[/math]th coordinate of [math]\displaystyle{ x }[/math], by increasing each coordinate by one (modula [math]\displaystyle{ d }[/math]) at a time. The path is described by the following pseudocode.

for(i=1; i<=d; i++){
    while(x[i]!=y[i]){
        x[i]=(x[i]+1)mod d;
    }
}

Bound the congestion [math]\displaystyle{ |\mathrm{CP}(uv)| }[/math]

For any edge [math]\displaystyle{ uv\in E }[/math], i.e. such [math]\displaystyle{ u,v\in[d]^n }[/math] that differ at one coordinate by 1, let

[math]\displaystyle{ \mathrm{CP}(uv)=\{\gamma_{xy}\in\Gamma\mid uv\in\gamma_{xy}\} }[/math]

be the set of canonical paths crossing the edge [math]\displaystyle{ uv }[/math]. The value [math]\displaystyle{ |\mathrm{CP}(uv)| }[/math] is the number of canonical paths crossing [math]\displaystyle{ uv }[/math].

We claim that for any [math]\displaystyle{ uv\in E }[/math]

[math]\displaystyle{ |\mathrm{CP}(uv)|\le d^{n+1} }[/math].

To see this, consider an edge from [math]\displaystyle{ u=(u_1,u_2,\ldots,u_n)\, }[/math] to [math]\displaystyle{ v=(u_1,u_2,\ldots,u_{i-1},(u_i+1)\bmod d,u_{i+1},\ldots,u_n)\, }[/math]. The idea of the argument is that the [math]\displaystyle{ \gamma_{xy} }[/math] that cross the edge [math]\displaystyle{ uv }[/math] must satisfy:

• [math]\displaystyle{ y_1=u_1, \ldots,y_{i-1}=u_{i-1}\, }[/math] and
• [math]\displaystyle{ x_{i+1}=u_{i+1},\ldots, x_n=u_n\, }[/math],

because we are modifying the coordinates in order [math]\displaystyle{ 1,2,\ldots, n }[/math]. More precisely, corresponding to any [math]\displaystyle{ \gamma_{xy} }[/math] that cross the edge [math]\displaystyle{ uv }[/math], we may define a complementary point [math]\displaystyle{ w\in[d]^n }[/math] as

[math]\displaystyle{ w=(x_1,x_2,\ldots,x_i,y_{i+1},y_{i+2},\ldots,y_n)\, }[/math].

Then, for any fixed edge [math]\displaystyle{ uv }[/math], given [math]\displaystyle{ w }[/math] and [math]\displaystyle{ y_i }[/math], a pair of [math]\displaystyle{ x }[/math] and [math]\displaystyle{ y }[/math] whose canonical path cross the [math]\displaystyle{ uv }[/math], can be uniquely determined. In other words, a canonical path [math]\displaystyle{ \gamma_{xy}\in\mathrm{CP}(uv) }[/math] can be encoded as a pair of [math]\displaystyle{ w\in[d]^n }[/math] and [math]\displaystyle{ y_i\in [d] }[/math]. Thus,

[math]\displaystyle{ |\mathrm{CP}(uv)|\le|[d]^n\times [d]|=d^{n+1} }[/math].

Compute the [math]\displaystyle{ \rho\, }[/math] from [math]\displaystyle{ |\mathrm{CP}(uv)| }[/math]

Since the cube is a regular graph, the stationary distribution is uniform, so [math]\displaystyle{ \pi_x=d^{-n} }[/math] for any [math]\displaystyle{ x\in[d]^n }[/math]. Recall that [math]\displaystyle{ P_{uv}=\frac{1}{2n} }[/math] for any [math]\displaystyle{ uv\in E }[/math]. The [math]\displaystyle{ \rho\, }[/math] can be computed by

[math]\displaystyle{ \rho=\max_{uv \in E}\frac{1}{\pi_u P_{uv}}\sum_{\gamma_{xy}\ni uv}\pi_x\pi_y=\max_{uv\in E}\frac{|\mathrm{CP}(uv)|}{d^n\cdot P_{uv}}\le 2nd }[/math].

Therefore, the mixing time is bounded by [math]\displaystyle{ O(n^2d^2\cdot \log (d^n))=O(n^3d^2\log d) }[/math], which is logarithmic to the size of the state space [math]\displaystyle{ |\mathcal{S}|=d^n }[/math].

Generalizing the technique:

• For a Markov chain with [math]\displaystyle{ N }[/math] states, the chain can be represented as a transition graph [math]\displaystyle{ \mathcal{G}(\mathcal{S}, E) }[/math] on the state space. Let [math]\displaystyle{ d }[/math] be the maximum degree of [math]\displaystyle{ \mathcal{G} }[/math].
• If the stationary is uniform, the [math]\displaystyle{ \rho\, }[/math] becomes

[math]\displaystyle{ \rho=\max_{uv \in E}\frac{1}{\pi_u P_{uv}}\sum_{\gamma_{xy}\ni uv}\pi_x\pi_y=\max_{uv\in E}\frac{|\mathrm{CP}(uv)|}{N\cdot P_{uv}} }[/math].

• For Metropolis algorithms which generate uniform stationary distribution, [math]\displaystyle{ P_{uv} }[/math] are equal for all [math]\displaystyle{ uv\in E }[/math] and [math]\displaystyle{ 1/P_{uv} }[/math] can be made [math]\displaystyle{ \le 2d }[/math] for a lazy random walk.
• Therefore, the Markov chain is rapid mixing if the maximum degree [math]\displaystyle{ d }[/math] of the transition graph is within polynomial of [math]\displaystyle{ \log N }[/math], and [math]\displaystyle{ \frac{|\mathrm{CP}(uv)|}{N} }[/math] is within polynomial of [math]\displaystyle{ \log N }[/math] for any edge [math]\displaystyle{ uv\in E }[/math]. Usually, the maximum degree of [math]\displaystyle{ \mathcal{G} }[/math] can be made small, thus the only problem is to bound the maximum [math]\displaystyle{ \frac{|\mathrm{CP}(uv)|}{N} }[/math] within polynomial of [math]\displaystyle{ \log N }[/math].
• The [math]\displaystyle{ |\mathrm{CP}(uv)|\, }[/math] for any [math]\displaystyle{ uv }[/math] can be bound by encoding the pair [math]\displaystyle{ (x,y) }[/math] that [math]\displaystyle{ \gamma_{xy}\in\mathrm{CP}(uv) }[/math] by a complementary state [math]\displaystyle{ w\in\mathcal{S} }[/math] plus some parameter. There are [math]\displaystyle{ N }[/math] states, and assuming the parameter ranging over [math]\displaystyle{ M }[/math] different values, the [math]\displaystyle{ \frac{|\mathrm{CP}(uv)|}{N} }[/math] is bounded by [math]\displaystyle{ M }[/math]. If [math]\displaystyle{ M }[/math] is poly-log of [math]\displaystyle{ N }[/math], then the Markov chain is rapid mixing.