imported>WikiSysop |
|
| Line 1: |
Line 1: |
| We see that the mixing performance of a random walk on an undirected graph is determined by the expansion ratio of the graph. We now consider the random walks in a more general setting, and study the mixing performance of a general class of Markov chains.
| | The following is the pseudocode of the famous [http://en.wikipedia.org/wiki/Quicksort Quicksort] algorithm, whose input is a set <math>S</math> of numbers. |
| | * if <math>|S|>1</math> do: |
| | ** pick an element <math>x</math> from <math>S</math> as the ''pivot''; |
| | ** partition <math>S</math> into <math>S_1</math>, <math>\{x\}</math>, and <math>S_2</math>, where all elements in <math>S_1</math> are smaller than <math>x</math> and all elements in <math>S_2</math> are larger than <math>x</math>; |
| | ** recursively sort <math>S_1</math> and <math>S_2</math>; |
|
| |
|
| == Mixing Time ==
| | The time complexity of this sorting algorithm is measured by the '''number of comparisons'''. |
|
| |
|
| The '''mixing time''' of a Markov chain gives the time of the chain to approach the stationary distribution. To formally define the mixing time, we need a notion of the distance between two probability distributions.
| | For the '''deterministic''' quicksort algorithm, the pivot element is usually the element in a fixed position (e.g. the first one) of the <math>S</math>. This will make the worst-case time complexity <math>\Omega(n^2)</math>, which means there exists a bad case <math>S</math>, sorting which will cost us <math>\Omega(n^2)</math> comparisons, ''every time''! |
|
| |
|
| Let <math>p</math> and <math>q</math> be two probability distributions over the same finite state space <math>\mathcal{S}</math>, the '''total variation distance''' between <math>p</math> and <math>q</math> is defined as
| | It is just so unfair to have an unbeatable input for this brilliant algorithm. So we tweak the algorithm a little bit: |
| :<math>\frac{1}{2}\sum_{i\in\mathcal{S}}|p_i-q_i|</math>,
| | === Algorithm: RandQSort === |
| which we can express as the <math>\ell_1</math>-distance
| | * if <math>|S|>1</math> do: |
| <math>
| | ** ''uniformly'' pick a ''random'' element <math>x</math> from <math>S</math> as the pivot; |
| \frac{1}{2}\|p-q\|_1.
| | ** partition <math>S</math> into <math>S_1</math>, <math>\{x\}</math>, and <math>S_2</math>, where all elements in <math>S_1</math> are smaller than <math>x</math> and all elements in <math>S_2</math> are larger than <math>x</math>; |
| </math> | | ** recursively sort <math>S_1</math> and <math>S_2</math>; |
| | |
| You may have encountered the concept of total variation before, and it might be defined differently, as
| |
| :<math>
| |
| \max_{A\subset\mathcal{S}}|p(A)-q(A)|.
| |
| </math> | |
| It is not hard to see that the two definitions are equivalent and
| |
| :<math>
| |
| \max_{A\subset\mathcal{S}}|p(A)-q(A)|=\frac{1}{2}\sum_{i\in \mathcal{S}}|p_i-q_i|=\frac{1}{2}\|p-q\|_1.
| |
| </math> | |
| Here we prefer to use our version, because it is convenient to use the tools for norms to analyze it.
| |
| | |
| {{Theorem
| |
| |Definition (mixing time)|
| |
| : For a Markov chain with finite state space <math>\mathcal{S}</math>, transition matrix <math>P</math>, and stationary distribution <math>\pi</math>, the total variation distance at time <math>t</math> with initial state <math>i\in\mathcal{S}</math> is defined as
| |
| ::<math>
| |
| \Delta_i(t)=\frac{1}{2}\sum_{j\in\mathcal{S}}|P^t(i,j)-\pi(j)|=\frac{1}{2}\|\boldsymbol{e}_iP^t-\pi\|_1
| |
| </math> | |
| :where <math>\boldsymbol{e}_i</math> is the vector that <math>\boldsymbol{e}_i(i)=1</math> and <math>\boldsymbol{e}_i(j)=0</math> for <math>j\neq i</math>.
| |
| :We define that
| |
| ::<math>\tau_i(\epsilon)=\min\{t\mid \Delta_i(t)\le \epsilon\}</math> and <math>\tau(\epsilon)=\max_{i\in\mathcal{S}}\tau_i(\epsilon)</math>.
| |
| }}
| |
| | |
| <math>\tau_i(\epsilon)</math> is the first time that a chain starting from state <math>i</math> approaches its stationary distribution within a total variation distance of <math>\epsilon</math>, and <math>\tau(\epsilon)</math> is the maximum of these values over all states. While <math>\tau(\epsilon)</math> is described as a function of <math>\epsilon</math>, it is generally referred as the '''mixing time''' of the Markov chain.
| |
|
| |
|
| For the efficiency of randomized algorithms, we are interested in the random walks that converges "fast". Measured by the mixing time, we need the mixing time to be "small". Recall that the mixing time <math>\tau(\epsilon)</math> is a function. So what we really mean is that as a function, the mixing time grows slowly as its input become larger.
| | === Analysis === |
| | Our goal is to analyze the expected number of comparisons during an execution of RandQSort with an arbitrary input <math>S</math>. We achieve this by measuring the chance that each pair of elements are compared, and summing all of them up due to [http://en.wikipedia.org/wiki/Expected_value#Linearity Linearity of Expectation]. |
|
| |
|
| The mixing time <math>\tau(\epsilon)</math> has an input <math>\epsilon</math> which is the distance to the stationary distribution, and there is another hidden parameter for <math>\tau(\epsilon)</math>, namely, the size of the state space <math>N=|\mathcal{S}|</math>. The parameter <math>\epsilon</math> gives the error bound, and <math>N</math> reflects the size of the problem.
| | Let <math>a_i</math> denote the <math>i</math>th smallest element in <math>S</math>. |
| | Let <math>X_{ij}\in\{0,1\}</math> be the random variable which indicates whether <math>a_i</math> and <math>a_j</math> are compared during the execution of RandQSort. That is: |
|
| |
|
| A random walk is called '''rapid mixing''' if its mixing time <math>\tau(\epsilon)</math> is poly-logarithmic of both <math>N</math> and <math>\frac{1}{\epsilon}</math>, i.e. when
| |
| :<math> | | :<math> |
| \tau(\epsilon)=O((\log N+\log(1/\epsilon))^{C})\, | | \begin{align} |
| | X_{ij} &= |
| | \begin{cases} |
| | 1 & a_i\mbox{ and }a_j\mbox{ are compared}\\ |
| | 0 & \mbox{otherwise} |
| | \end{cases}. |
| | \end{align} |
| </math> | | </math> |
| for some constant <math>C</math>.
| |
|
| |
|
| === Coupling ===
| | Elements <math>a_i</math> and <math>a_j</math> are compared only if one of them is chosen as pivot. After comparison they are separated (thus are never compared again). So we have the following observation: |
|
| |
|
| == The eigenvalue approach ==
| | '''Claim 1: Every pair of <math>a_i</math> and <math>a_j</math> are compared at most once.''' |
|
| |
|
| Let <math>P</math> be the transition matrix of a Markov chain, and let <math>\pi</math> be the stationary distribution, such that <math>\pi P=\pi</math>. For any initial distribution <math>q</math>, the distribution of the chain at time <math>t</math> is give by <math>qP^t</math>. The total variation at time <math>t</math> is governed by the <math>\ell_1</math>-distance
| | Therefore the sum of <math>X_{ij}</math> for all pair <math>\{i, j\}</math> gives the total number of comparisons. The expected number of comparisons is <math>\mathbf{E}\left[\sum_{i=1}^n\sum_{j>i}X_{ij}\right]</math>. Due to [http://en.wikipedia.org/wiki/Expected_value#Linearity Linearity of Expectation], <math>\mathbf{E}\left[\sum_{i=1}^n\sum_{j>i}X_{ij}\right] = \sum_{i=1}^n\sum_{j>i}\mathbf{E}\left[X_{ij}\right]</math>. |
| :<math>\|qP^t-\pi\|_1</math>.
| | Our next step is to analyze <math>\mathbf{E}\left[X_{ij}\right]</math> for each <math>\{i, j\}</math>. |
| But how to estimate this? Not surprisingly, it can be answered by looking at the eigenvalues of <math>P</math>.
| |
|
| |
|
| | By the definition of expectation and <math>X_{ij}</math>, |
|
| |
|
| Let <math>\lambda_1\ge\lambda_2\ge\cdots\ge\lambda_n</math> be the eigenvalues of <math>P</math>.
| | :<math>\begin{align} |
| {{Theorem|Remark<nowiki>:</nowiki>| | | \mathbf{E}\left[X_{ij}\right] |
| The eigenvalues are now of the transition matrix <math>P</math> instead of the adjacency matrix of a graph. With the same argument as the spectrum of graphs, we can show that <math>\lambda_1=1</math> and <math>|\lambda_i|\le 1</math> for all <math>i</math>, and for irreducible chains, <math>\lambda_1>\lambda_2</math>. Therefore, for irreducible Markov chains,
| | &= 1\cdot \Pr[a_i\mbox{ and }a_j\mbox{ are compared}] + 0\cdot \Pr[a_i\mbox{ and }a_j\mbox{ are not compared}]\\ |
| :<math>1=\lambda_1>\lambda_2\ge\cdots\ge\lambda_n\ge -1</math>.
| | &= \Pr[a_i\mbox{ and }a_j\mbox{ are compared}]. |
| }}
| | \end{align}</math> |
|
| |
|
| '''Why should we care about eigenvalues of <math>P</math>?''' Recall that <math>\lambda\neq 0</math> is an '''eigenvalue''' of <math>P</math> if for some vector <math>v</math>,
| | We are going to bound this probability. |
| :<math>Pv=\lambda v</math>,
| |
| where <math>v</math> is called an '''eigenvector'''. The eigenvalues are the solutions to the <math>\det(A-\lambda I)=0</math>.
| |
|
| |
|
| For our purpose, we are interested in the '''left-eigenvalues''' and '''left-eigenvectors''', such that
| | '''Claim 2: <math>a_i</math> and <math>a_j</math> are compared if and only if one of them is chosen as pivot when they are still in the same subset.''' |
| :<math>vP=\lambda v</math>. | |
| Note that the left-eigenvalues are equal to the right-eigenvalues, because
| |
| :<math>\det(A-\lambda I)=\det(A^T-\lambda I)</math>,
| |
| however, the left-eigenvectors may be different from right-eigenvectors.
| |
|
| |
|
| Let <math>v_1,v_2,\ldots,v_n</math> be the (left)eigenvectors corresponds to the eigenvalues <math>\lambda_1\ge\lambda_2\ge\cdots\ge\lambda_n</math>. A key observation is that <font color="red">if <math>P</math> is '''symmetric'''</font> (that is, <math>P^T=P</math>), the eigenvectors are '''orthogonal''' to each other, thus can be treated as '''orthogonal basis''', which means that any vector <math>q</math> can be uniquely represented as
| | This is easy to verify: just check the algorithm. The next one is a bit complicated. |
| :<math>q=c_1v_1+c_2v_2+\cdots+c_nv_n</math>, | |
| for some scalars <math>c_1,c_2,\ldots,c_n</math>. Furthermore, we can choose the first component as <math>c_1v_1=\pi</math>, because we know that <math>\pi</math> is the left-eigenvector with the largest eigenvalue <math>1</math>. Thus,
| |
| :<math>q=\pi+c_2v_2+\cdots+c_nv_n</math>.
| |
|
| |
|
| Then by the linearity, an action of <math>P</math> can be computed by
| | '''Claim 3: If <math>a_i</math> and <math>a_j</math> are still in the same subset then all <math>\{a_i, a_{i+1}, \ldots, a_{j-1}, a_{j}\}</math> are in the same subset.''' |
| :<math> | |
| qP=\left(\pi+\sum_{i=2}^n c_i v_i\right)P=\pi P+\sum_{i=2}^n (c_i v_iP)=\pi+\sum_{i=2}^n \lambda_i c_i v_i.
| |
| </math> | |
| Thus, multiplying <math>P</math> corresponds to multiplying an eigenvalue to the scalar corresponding to each basis. Repeating this process, we have
| |
| :<math>
| |
| qP^t=\pi+\sum_{i=2}^n \lambda_i^t c_i v_i.
| |
| </math> | |
|
| |
|
| So the difference between the distribution of the chain and the stationary distribution is shrinking by a factor of <math>\lambda_\max=\max(|\lambda_2|,|\lambda_n|)</math> at each step. This explain why we care about <math>\lambda_\max</math>, because it dominates the rate at which the difference shrinks.
| | We can verify this by induction. Initially, <math>S</math> itself has the property described above; and partitioning any <math>S</math> with the property into <math>S_1</math> and <math>S_2</math> will preserve the property for both <math>S_1</math> and <math>S_2</math>. Therefore Claim 3 holds. |
|
| |
|
| However, right now this beautiful theory holds only when the transition matrix <math>P</math> is symmetric. In some special case, such as the random walk on a <math>d</math>-regular graph, the transition matrix is indeed symmetric, but for various applications of Markov chains, the transition matrix is not necessarily symmetric. We will see that there is a more general class of Markov chains for which we can apply the similar technique as when the transition matrix is symmetric.
| | Combining Claim 2 and 3, we have: |
|
| |
|
| ===Reversibility ===
| | '''Claim 4: <math>a_i</math> and <math>a_j</math> are compared only if one of <math>\{a_i, a_j\}</math> is chosen from <math>\{a_i, a_{i+1}, \ldots, a_{j-1}, a_{j}\}</math>.''' |
| We restrict ourselves to a special class of Markov chains call time-reversible Markov chains.
| |
| {{Theorem
| |
| |Definition (time-reversible)|
| |
| :A Markov chain with finite state space <math>\mathcal{S}</math>, transition matrix <math>P</math>, and stationary distribution <math>\pi</math> is said to be '''time-reversible''' if for all <math>i,j\in\mathcal{S}</math> | |
| ::<math>
| |
| \pi_{i}P_{i,j}=\pi_{j}P_{j,i}.\, | |
| </math> | |
| }}
| |
|
| |
|
| For reversible chains, its stationary distribution shows a stronger equilibrium property: not only the stationary is unchanged under the action of transition, but also when the chain is stationary, it has equal chance to move from <math>i</math> to <math>j</math> and from <math>j</math> to <math>i</math>.
| | And apparently, |
|
| |
|
| ;Example
| | '''Claim 5: Every one of <math>\{a_i, a_{i+1}, \ldots, a_{j-1}, a_{j}\}</math> is chosen equal-probably.''' |
| * A symmetric random walk (<math>P</math> is symmetric) is time-reversible.
| |
| * Random walks on undirected graphs (not necessarily <math>d</math>-regular) are time-reversible.
| |
|
| |
|
| The name "time-reversible" is due to the following fact:
| | This is because our RandQSort chooses the pivot ''uniformly at random''. |
| :Let <math>X_0,X_1,\ldots,X_n</math> be a time-reversible Markov chain whose initial distribution is the stationary distribution <math>\pi</math>, then the distribution of the reversed sequence <math>X_n,X_{n-1},\ldots,X_0</math> is exactly the same as <math>X_0,X_1,\ldots,X_n</math>, formally, for any states <math>x_0,x_1,\ldots,x_n</math>,
| |
| ::<math>\Pr[X_0=x_0\wedge X_1=x_1\wedge \ldots \wedge X_n=x_n]=\Pr[X_0=x_n\wedge X_1=x_{n-1}\wedge \ldots \wedge X_n=x_0]</math>.
| |
|
| |
|
| Although for a time-reversible Markov chain, the transition matrix <math>P</math> is not necessarily symmetric, we can make a symmetric matrix out of it.
| | Claim 4 and Claim 5 together imply: |
|
| |
|
| ===Mixing time of reversible chains ===
| | :<math>\begin{align} |
| | \Pr[a_i\mbox{ and }a_j\mbox{ are compared}] |
| | &\le \frac{2}{j-i+1}. |
| | \end{align}</math> |
|
| |
|
| For a time-reversible <math>P</math> and stationary distribution <math>\pi</math>, it holds that <math>\pi_iP_{i,j}=\pi_jP_{j,i}</math>. Divide both sides by <math>\sqrt{\pi_i\pi_j}</math>, we have
| | {|border="1" |
| :<math> | | |'''Remark:''' Perhaps you feel confused about the above argument. You may ask: "''The algorithm chooses pivots for many times during the execution. Why in the above argument, it looks like the pivot is chosen only once?''" Good question! Let's see what really happens by looking closely. |
| \sqrt{\frac{\pi_i}{\pi_j}}P_{i,j}=\sqrt{\frac{\pi_j}{\pi_i}}P_{j,i}.
| |
| </math>
| |
| This shows that the matrix <math>S</math> with entries
| |
| :<math>
| |
| S_{i,j}=\sqrt{\frac{\pi_i}{\pi_j}}P_{i,j},
| |
| </math>
| |
| is symmetric. Let <math>\Pi</math> be the diagonal matrix given by <math>\Pi_{i,i}=\sqrt{\pi_i}</math>. Then <math>S</math> can be written as <math>S=\Pi P \Pi^{-1}\,</math>, therefore, for any time-reversible Markov chain, its transition matrix <math>P</math> can be written as
| |
| :<math> | |
| P=\Pi^{-1}S\Pi\,
| |
| </math>
| |
| where <math>S</math> is symmetric, and has the same eigenvalues as <math>P</math> (although the eigenvectors may be different),
| |
| and for any initial distribution <math>q</math>,
| |
| :<math>
| |
| qP^t=q(\Pi^{-1}S\Pi)^t=q\Pi^{-1}S^t\Pi\,
| |
| </math>.
| |
| Because <math>S</math> is symmetric, its eigenvectors are orthogonal basis and the same spectral technique works. This again will give us a nice characterization of the mixing time by <math>\lambda_\max=\max(|\lambda_2|,|\lambda_n|)</math> of <math>P</math>, and prove the following theorem (the details of the proof are omitted).
| |
| | |
| {{Theorem
| |
| |Theorem|
| |
| :For a time-reversible Markov chain with finite state space <math>\mathcal{S}</math> and transition matrix <math>P</math>, let <math>\lambda_1\ge\lambda_2\ge\cdots\ge\lambda_n</math> be the eigenvalues of <math>P</math>.
| |
| :For any state <math>i\in\mathcal{S}</math>,
| |
| ::<math>\Delta_i(t)\le\frac{1}{2}\lambda_\max^t\sqrt{\frac{1-\pi_i}{\pi_i}}</math>,
| |
| :where <math>\lambda_\max=\max(|\lambda_2|,|\lambda_n|)</math> is the largest absolute value of eigenvalues other than <math>\lambda_1=1</math>.
| |
| }}
| |
|
| |
|
| The theorem about the mixing time of random walks on expander graphs is a special case of the above theorem.
| | For any pair <math>a_i</math> and <math>a_j</math>, initially <math>\{a_i, a_{i+1}, \ldots, a_{j-1}, a_{j}\}</math> are all in the same set <math>S</math> (obviously!). During the execution of the algorithm, the set which containing <math>\{a_i, a_{i+1}, \ldots, a_{j-1}, a_{j}\}</math> are shrinking (due to the pivoting), until one of <math>\{a_i, a_{i+1}, \ldots, a_{j-1}, a_{j}\}</math> is chosen, and the set is partitioned into different subsets. We ask for the probability that the chosen one is among <math>\{a_i, a_j\}</math>. So we really care about "the last" pivoting before <math>\{a_i, a_{i+1}, \ldots, a_{j-1}, a_{j}\}</math> is split. |
|
| |
|
| It is convenient to express the mixing rate as a function in the form of <math>\exp(\cdot)</math>, so its natural logarithm looks nicer. We observe that
| | Formally, let <math>Y</math> be the random variable denoting the pivot element. We know that for each <math>a_k\in\{a_i, a_{i+1}, \ldots, a_{j-1}, a_{j}\}</math>, <math>Y=a_k</math> with the same probability, and <math>Y\not\in\{a_i, a_{i+1}, \ldots, a_{j-1}, a_{j}\}</math> with an unknown probability (remember that there might be other elements in the same subset with <math>\{a_i, a_{i+1}, \ldots, a_{j-1}, a_{j}\}</math>). The probability we are looking for is actually |
| :<math>
| | <math>\Pr[Y\in \{a_i, a_j\}\mid Y\in\{a_i, a_{i+1}, \ldots, a_{j-1}, a_{j}\}]</math>, which is always <math>\frac{2}{j-i+1}</math>, provided that <math>Y</math> is uniform over <math>\{a_i, a_{i+1}, \ldots, a_{j-1}, a_{j}\}</math>. |
| \lambda_\max=1-(1-\lambda_\max)\le e^{-(1-\lambda_\max)}, | |
| </math> | |
| and thus
| |
| :<math>
| |
| \lambda_\max^t\le e^{-(1-\lambda_\max)t}. | |
| </math> | |
|
| |
|
| The theorem is turned to | | The '''conditional probability''' rules out the ''irrelevant'' events in a probabilistic argument. |
| :<math>
| |
| \Delta_i(t)\le\frac{1}{2}e^{-(1-\lambda_\max)t}\sqrt{\frac{1-\pi_i}{\pi_i}}.
| |
| </math>
| |
| Solving the <math>\Delta_i(t)=\epsilon</math> gives us the mixing time:
| |
| {|border="1"
| |
| |'''Corollary (mixing time of reversible chain)'''
| |
| :For a time-reversible Markov chain,
| |
| :<math>\tau_i(\epsilon)=O\left(\frac{\ln(1/\pi_i)+\ln(1/\epsilon)}{1-\lambda_\max}\right) </math>
| |
| |} | | |} |
|
| |
|
| For a lazy random walk, where the transition matrix is <math>Q=\frac{1}{2}(I+P)</math>, it is easy to see that <math>Q</math> is also time-reversible and has the same stationary distribution and <math>P</math>, and the eigenvalues of <math>Q</math> are all nonnegative, thus <math>\lambda_\max=\lambda_2</math>.
| | Summing all up: |
| | |
| From now on, we only consider lazy random walks and always assume that <math>\lambda_\max=\lambda_2</math>.
| |
| {{Theorem
| |
| |Theorem (mixing time of reversible lazy random walk)|
| |
| :For a time-reversible lazy random walk, that is, for a time-reversible Markov chain whose transition matrix <math>P\,</math> has <math>P_{i,i}\ge\frac{1}{2}</math> for all <math>i\in\mathcal{S}</math>, | |
| ::<math>\tau_i(\epsilon)=O\left(\frac{\ln(1/\pi_i)+\ln(1/\epsilon)}{1-\lambda_2}\right) </math>.
| |
| :In particular, when the stationary distribution is uniform, the mixing time
| |
| ::<math>\tau(\epsilon)=O\left(\frac{\ln N+\ln(1/\epsilon)}{1-\lambda_2}\right) </math>,
| |
| :where <math>N=|\mathcal{S}|</math>.
| |
| }}
| |
| The random walk is rapid mixing if the spectral gap <math>(1-\lambda_2)</math> is larger than <math>\frac{1}{(\log N)^c}</math> for some constant <math>c</math>.
| |
| | |
| == Conductance ==
| |
| === Spectral gap and the mixing time ===
| |
| * We consider a Markov chain with finite space <math>\mathcal{S}</math>, transition matrix <math>P</math>, and stationary distribution <math>\pi</math>. Let <math>N=|\mathcal{S}|</math> denote the size of the state space, and let <math>\lambda_1\ge\cdots\ge\lambda_N</math> be the eigenvalues of <math>P</math>. For any stochastic matrix <math>P</math>, it holds that <math>1=\lambda_1\ge\cdots\ge\lambda_N\ge -1</math>.
| |
| * The '''mixing time''' <math>\tau(\epsilon)\,</math> of an irreducible and aperiodic Markov chain is given by the time to be within total variation distance <math>\epsilon</math> from <math>\pi</math>, starting from a worst-case state. Formally,
| |
| ::<math>
| |
| \tau_i(\epsilon)=\min\left\{t\,\,\bigg|\,\, \frac{1}{2}\sum_{j\in\mathcal{S}}|P^t(i,j)-\pi(i)|\le\epsilon\right\}
| |
| </math> and <math>\tau(\epsilon)=\max_{i\in\mathcal{S}}\tau_i(\epsilon)</math>.
| |
| Conditions:
| |
| * The Markov chain is '''time-reversible''': <math>\pi_{i}P_{i,j}=\pi_{j}P_{j,i}</math> for all <math>i,j\in\mathcal{S}</math>.
| |
| :For a time-reversible Markov chain, the transition matrix <math>P</math> can be transformed to a symmetric matrix, thus the orthogonal diagonalization for symmetric matrices can be applied, and the convergence rate is determined by the second largest absolute value of eigenvalues <math>\lambda_{\max}=\max(|\lambda_2|,|\lambda_N|)</math> as
| |
| ::<math>\tau_i(\epsilon)=O\left(\frac{\ln(1/\pi_i)+\ln(1/\epsilon)}{1-\lambda_{\max}}\right)</math>.
| |
| * '''Lazy random walk''': <math>P_{i,i}\ge\frac{1}{2}</math> for any <math>i\in\mathcal{S}</math>.
| |
| : A lazy walk is aperiodic. The eigenvalues of a lazy walk are nonnegative, and thus <math>\lambda_{\max}=\max(|\lambda_2|,|\lambda_N|)=\lambda_2</math>. The mixing time is determined by the second largest eigenvalue <math>\lambda_2</math> as
| |
| ::<math>\tau_i(\epsilon)=O\left(\frac{\ln(1/\pi_i)+\ln(1/\epsilon)}{1-\lambda_{2}}\right)</math>.
| |
| * The stationary <math>\pi</math> is the '''uniform distribution''', that is, <math>\pi_i=\frac{1}{N}</math> for all <math>i\in\mathcal{S}</math>.
| |
| Then:
| |
| {{Theorem
| |
| |Theorem|
| |
| :The mixing time of a time-reversible lazy random walk with uniform stationary distribution over a state space of size <math>N</math>, is
| |
| ::<math>\tau(\epsilon)=O\left(\frac{\ln N+\ln(1/\epsilon)}{1-\lambda_2}\right) </math>.
| |
| }}
| |
| | |
| From the previous section, we see how to construct a time-reversible lazy random walk with uniform stationary distribution. The mixing time is now determined by the spectral gap <math>(1-\lambda_2)</math>. In order to upper bound the mixing time, we need to lower bound the spectral gap. However, it is difficult to directly bound the eigenvalues of a usually exponentially large transition matrix.
| |
| | |
| === Conductance and the mixing time ===
| |
| | |
| For many problems, such as card shuffling, the state space is exponentially large, so the estimation of <math>\lambda_2</math> becomes very difficult. The following technique based on '''conductance''' overcomes this issue by considering the conductance of a Markov chain.
| |
| | |
| {{Theorem
| |
| |Definition (conductance)|
| |
| :The '''conductance''' of a irreducible Markov chain with finite state space <math>\Omega</math>, transition matrix <math>P</math>, and stationary distribution <math>\pi</math>, is defined as
| |
| ::<math>
| |
| \Phi=\min_{\overset{S\subset\Omega}{0<\pi(S)\le\frac{1}{2}}} \frac{\sum_{i\in S, j\not\in S}\pi_iP_{ij}}{\pi(S)}
| |
| </math>
| |
| :where <math>\pi(S)=\sum_{i\in S}\pi_i</math> is the probability density of <math>S\subset \Omega</math> under the stationary distribution <math>\pi</math>.
| |
| }}
| |
| | |
| The definition of conductance looks quite similar to the expansion ratio of graphs.
| |
| Very informally, the conductance can be seen as the weighted normalized version of expansion ratio.
| |
| | |
| The following lemma states a Cheeger's inequality for the conductance.
| |
| | |
| {{Theorem
| |
| |Lemma (Jerrum-Sinclair 1988)|
| |
| :In a time-reversible Markov chain, <math>1-2\Phi\le\lambda_2\le 1-\frac{\Phi^2}{2}</math>.
| |
| }}
| |
| | |
| The inequality can be equivalent written for the spectral gap:
| |
| :<math>\frac{\Phi^2}{2}\le1-\lambda_2\le 2\Phi</math>.
| |
| Thus <math>\Phi^2/2</math> is a lower bound of the spectral gap, which in turn gives an upper bound of the mixing time via conductance.
| |
| | |
| {{Theorem
| |
| |Proposition|
| |
| :The mixing time of a time-reversible lazy random walk with uniform stationary distribution over a state space of size <math>N</math>, is
| |
| ::<math>\tau(\epsilon)=O\left(\frac{\ln N+\ln(1/\epsilon)}{\Phi^2}\right) </math>.
| |
| }}
| |
| | |
| So analyzing of mixing time is reduced to analyzing of the second largest eigenvalue, which is reduced to analyzing of the conductance. For some simple Markv chain, this can be done directly. For some more complicated Markov chain, this is done by a technique called the canonical paths argument.
| |
| | |
| == Canonical Paths ==
| |
| We will first show how to lower bound the expansion ratio of a graph by canonical paths argument, and then we will generalize this technique to the conductance.
| |
| | |
| As we know, the expansion ratio of an undirected graph <math>G(V,E)</math>,
| |
| :<math>\phi(G)=\min_{\overset{ S\subset V}{|S|\le\frac{|V|}{2}}}\frac{|\partial|}{|S|}</math>,
| |
| is hard to compute, because its definition involves an optimization over exponentially many subsets. We will see that it can be lower bounded by computing the maximum congestion caused by a set of paths.
| |
| | |
| For every pair of vertices <math>x,y\in V</math>, we construct a '''simple''' path <math>\gamma_{xy}</math> from <math>x</math> to <math>v</math>. Each ordered pair <math>(x,y)</math> of distinct vertices corresponds to exact one path, called the '''canonical path''' from <math>x</math> to <math>y</math>. Let <math>\Gamma=\{\gamma_{xy}\mid x,y\in V, x\neq y\}</math> be the collection of canonical paths.
| |
| | |
| The set of canonical paths that an edge <math>uv</math> is on is
| |
| :<math>\mathrm{CP}(uv)=\{\gamma_{xy}\in\Gamma\mid uv\in\gamma_{xy}\}</math>.
| |
| The '''congestion''' of an edge is given by <math>|\mathrm{CP}(uv)|</math>, which is the number of canonical paths crossing the edge <math>uv</math>.
| |
| | |
| For any <math>S\subset V</math> where <math>|S|\le\frac{|V|}{2}</math>, there are <math>|S||\bar{S}|</math> ordered pairs <math>(x,y)</math> of vertices such that <math>x\in S</math> and <math>y\not\in S</math>, thus there are <math>|S||\bar{S}|</math> canonical paths going from <math>S</math> to <math>\bar{S}</math>. The average congestion of edges crossing from <math>S</math> and <math>\bar{S}</math> is <math>\frac{|S||\bar{S}|}{|\partial S|}</math>, which must be no bigger than the maximum congestion. Thus,
| |
| :<math>\sigma\overset{\mathrm{def}}{=}\max_{uv\in E}|\mathrm{CP}(uv)|\ge\frac{|S||\bar{S}|}{|\partial S|}</math>
| |
| and
| |
| :<math>\frac{|\partial S|}{|S|}\ge\frac{|\bar{S}|}{\sigma}\ge\frac{|V|}{2\sigma}</math>.
| |
| This inequality holds for any <math>S\subset V</math> where <math>|S|\le\frac{|V|}{2}</math>, by the definition of expansion ratio,
| |
| :<math>\phi(G)\ge\frac{|V|}{2\sigma}</math>
| |
| Therefore, for any collection of canonical paths <math>\Gamma=\{\gamma_{xy}\}</math>, the maximum congestion <math>\sigma=\max_{uv\in E}|\mathrm{CP}(uv)|</math> caused by <math>\Gamma</math> can be used to lower bound <math>\phi(G)</math>. By choosing cleverer paths, we can make the bound tighter.
| |
| | |
| The intuition behind this inequality is that large expansion ratio (the graph is well-connected) is a necessary condition for the existence of a pairwise routing scheme with low-congestion.
| |
| | |
| ------
| |
| | |
| We now show how to apply the canonical paths argument to the Markov chain and lower bound the conductance of the chain.
| |
| | |
| For a Markov chain with finite state space <math>\mathcal{S}</math>, and transition matrix <math>P</math>. The chain corresponds to a transition graph <math>\mathcal{G}(\mathcal{S},E)</math> on the state space <math>\mathcal{S}</math>. For any states <math>u,v\in\mathcal{S}</math>, <math>uv\in E</math> if <math>P_{uv}>0</math>.
| |
| | |
| The '''canonical paths''' for the chain is a collection of '''simple''' paths <math>\Gamma=\{\gamma_{xy}\mid x,y\in \mathcal{S}, x\neq y\}</math> in the transition graph <math>\mathcal{G}</math>, such that for any ordered pair <math>(x,y)</math> of distinct states, there is exact one path <math>\gamma_{xy}</math> going from <math>x</math> to <math>y</math> in <math>\mathcal{G}</math>.
| |
| | |
| Let <math>\Gamma=\{\gamma_{xy}\}</math> be a collection of canonical paths. The '''congestion''' caused by <math>\Gamma</math> is computed as
| |
| :<math>\rho=\max_{uv \in E}\frac{1}{\pi_u P_{uv}}\sum_{\gamma_{xy}\ni uv}\pi_x\pi_y</math>.
| |
| Note that compared to the example of lower bounding the expansion ratio, in which we directly count the number of paths crossing an edge as the congestion, this time the congestion is defined by the probability "flows". This is analogous to the difference between the definitions of expansion ratio of a graph and the conductance of a Markov chain.
| |
| | |
| {{Theorem
| |
| |Lemma|
| |
| :Let <math>\Gamma=\{\gamma_{xy}\}</math> be a collection of '''canonical path'''. The conductance is lower bounded by
| |
| ::<math>\Phi\ge\frac{1}{2\rho}</math>.
| |
| }}
| |
| {{Proof|}}
| |
| | |
| Due to the upper bound of the mixing time via the conductance, we have the following proposition.
| |
| | |
| {{Theorem
| |
| |Proposition|
| |
| :Consider a time-reversible lazy random walk with uniform stationary distribution over a state space of size <math>N</math>. Let <math>\Gamma=\{\gamma_{xy}\}</math> be an arbitrary collection of canonical paths. Assuming that the congestion of <math>\Gamma</math> is <math>\rho\,</math>, the mixing time of the chain is bounded by
| |
| ::<math>\tau(\epsilon)=O\left(\rho^2\left(\ln N+\ln(1/\epsilon)\right)\right) </math>.
| |
| }}
| |
| | |
| === Example<nowiki>:</nowiki> random walk on a cube===
| |
| We then apply the canonical path argument on a simple Markov chain: the random walk on high dimensional cube.
| |
| | |
| Let the state space be <math>\mathcal{S}=[d]^n</math>. Consider the random walk:
| |
| * Start from an arbitrary <math>x\in \mathcal{S}</math>
| |
| * At each step, assuming the current state is <math>x\in \mathcal{S}</math>, choose an random <math>i\in\{1,2,\ldots, n\}</math> and a random <math>b\in\{-1,1\}</math> and move to the state <math>y\in\mathcal{S}</math> such that <math>y_i=(x_i+b)\bmod d</math> and <math>y_j=x_j</math> for <math>j\neq i</math>.
| |
| | |
| The Markov chain is a random walk on an <math>n</math>-dimensional <math>d</math>-ary cube <math>G(V,E)</math>, where <math>V=[d]^n</math>, and <math>uv\in E</math> if <math>u</math> and <math>v</math> differ in one coordinate by 1 (modular <math>d</math>). And for any <math>uv\in E</math>, the transition probability <math>P_{uv}=\frac{1}{2n}</math>.
| |
| | |
| We will show that the walk is rapid mixing by the canonical paths argument. The argument involves the following steps:
| |
| *Construct a collection of canonical paths.
| |
| *Bound the congestion of a single edge. This step is usually done by "encoding" the canonical paths crossing the edge.
| |
| *Compute the <math>\rho\,</math> and thus bound the mixing time.
| |
| | |
| ;Construct the canonical paths
| |
| Consider the canonical paths <math>\Gamma=\{\gamma_{xy}\}</math> defined by the following "fixing" scheme:
| |
| *For each <math>x,y \in \mathcal{S}</math>, <math>\gamma_{xy}</math> is the path obtained by fixing the 1st, 2nd, ... <math>n</math>th coordinate of <math>x</math>, by increasing each coordinate by one (modula <math>d</math>) at a time. The path is described by the following pseudocode.
| |
| for(i=1; i<=d; i++){
| |
| while(x[i]!=y[i]){
| |
| x[i]=(x[i]+1)mod d;
| |
| }
| |
| }
| |
| | |
| ;Bound the congestion <math>|\mathrm{CP}(uv)|</math>
| |
| For any edge <math>uv\in E</math>, i.e. such <math>u,v\in[d]^n</math> that differ at one coordinate by 1, let
| |
| :<math>\mathrm{CP}(uv)=\{\gamma_{xy}\in\Gamma\mid uv\in\gamma_{xy}\}</math>
| |
| be the set of canonical paths crossing the edge <math>uv</math>. The value <math>|\mathrm{CP}(uv)|</math> is the number of canonical paths crossing <math>uv</math>.
| |
|
| |
|
| We claim that for any <math>uv\in E</math>
| | :<math>\begin{align} |
| :<math>|\mathrm{CP}(uv)|\le d^{n+1}</math>.
| | \mathbf{E}\left[\sum_{i=1}^n\sum_{j>i}X_{ij}\right] |
| To see this, consider an edge from <math>u=(u_1,u_2,\ldots,u_n)\,</math> to <math>v=(u_1,u_2,\ldots,u_{i-1},(u_i+1)\bmod d,u_{i+1},\ldots,u_n)\,</math>. The idea of the argument is that the <math>\gamma_{xy}</math> that cross the edge <math>uv</math> must satisfy:
| | &= |
| *<math>y_1=u_1, \ldots,y_{i-1}=u_{i-1}\,</math> and
| | \sum_{i=1}^n\sum_{j>i}\mathbf{E}\left[X_{ij}\right]\\ |
| *<math>x_{i+1}=u_{i+1},\ldots, x_n=u_n\,</math>,
| | &\le \sum_{i=1}^n\sum_{j>i}\frac{2}{j-i+1}\\ |
| because we are modifying the coordinates in order <math>1,2,\ldots, n</math>. More precisely, corresponding to any <math>\gamma_{xy}</math> that cross the edge <math>uv</math>, we may define a '''complementary''' point <math>w\in[d]^n</math> as
| | &= \sum_{i=1}^n\sum_{k=2}^{n-i+1}\frac{2}{k} & & (\mbox{Let }k=j-i+1)\\ |
| :<math>
| | &\le \sum_{i=1}^n\sum_{k=1}^{n}\frac{2}{k}\\ |
| w=(x_1,x_2,\ldots,x_i,y_{i+1},y_{i+2},\ldots,y_n)\,
| | &= 2n\sum_{k=1}^{n}\frac{1}{k}\\ |
| </math>.
| | &= 2n H(n). |
| Then, for any fixed edge <math>uv</math>, given <math>w</math> and <math>y_i</math>, a pair of <math>x</math> and <math>y</math> whose canonical path cross the <math>uv</math>, can be uniquely determined. In other words, a canonical path <math>\gamma_{xy}\in\mathrm{CP}(uv)</math> can be encoded as a pair of <math>w\in[d]^n</math> and <math>y_i\in [d]</math>. Thus,
| | \end{align}</math> |
| :<math>|\mathrm{CP}(uv)|\le|[d]^n\times [d]|=d^{n+1}</math>.
| |
|
| |
|
| | <math>H(n)</math> is the <math>n</math>th [http://en.wikipedia.org/wiki/Harmonic_number Harmonic number]. It holds that |
|
| |
|
| ;Compute the <math>\rho\,</math> from <math>|\mathrm{CP}(uv)|</math>
| | :<math>\begin{align}H(n) = \ln n+O(1)\end{align}</math>. |
| Since the cube is a regular graph, the stationary distribution is uniform, so <math>\pi_x=d^{-n}</math> for any <math>x\in[d]^n</math>. Recall that <math>P_{uv}=\frac{1}{2n}</math> for any <math>uv\in E</math>. The <math>\rho\,</math> can be computed by
| |
| :<math>\rho=\max_{uv \in E}\frac{1}{\pi_u P_{uv}}\sum_{\gamma_{xy}\ni uv}\pi_x\pi_y=\max_{uv\in E}\frac{|\mathrm{CP}(uv)|}{d^n\cdot P_{uv}}\le 2nd</math>.
| |
| Therefore, the mixing time is bounded by <math>O(n^2d^2\cdot \log (d^n))=O(n^3d^2\log d)</math>, which is logarithmic to the size of the state space <math>|\mathcal{S}|=d^n</math>.
| |
|
| |
|
| Generalizing the technique:
| | Therefore, for an arbitrary input <math>S</math> of <math>n</math> numbers, the expected number of comparisons taken by RandQSort to sort <math>S</math> is <math>\mathrm{O}(n\log n)</math>. |
| * For a Markov chain with <math>N</math> states, the chain can be represented as a transition graph <math>\mathcal{G}(\mathcal{S}, E)</math> on the state space. Let <math>d</math> be the maximum degree of <math>\mathcal{G}</math>.
| |
| * If the stationary is uniform, the <math>\rho\,</math> becomes
| |
| ::<math>\rho=\max_{uv \in E}\frac{1}{\pi_u P_{uv}}\sum_{\gamma_{xy}\ni uv}\pi_x\pi_y=\max_{uv\in E}\frac{|\mathrm{CP}(uv)|}{N\cdot P_{uv}}</math>.
| |
| * For Metropolis algorithms which generate uniform stationary distribution, <math>P_{uv}</math> are equal for all <math>uv\in E</math> and <math>1/P_{uv}</math> can be made <math>\le 2d</math> for a lazy random walk.
| |
| * Therefore, the Markov chain is rapid mixing if the maximum degree <math>d</math> of the transition graph is within polynomial of <math>\log N</math>, and <math>\frac{|\mathrm{CP}(uv)|}{N}</math> is within polynomial of <math>\log N</math> for any edge <math>uv\in E</math>. Usually, the maximum degree of <math>\mathcal{G}</math> can be made small, thus the only problem is to bound the maximum <math>\frac{|\mathrm{CP}(uv)|}{N}</math> within polynomial of <math>\log N</math>.
| |
| * The <math>|\mathrm{CP}(uv)|\,</math> for any <math>uv</math> can be bound by encoding the pair <math>(x,y)</math> that <math>\gamma_{xy}\in\mathrm{CP}(uv)</math> by a complementary state <math>w\in\mathcal{S}</math> plus some parameter. There are <math>N</math> states, and assuming the parameter ranging over <math>M</math> different values, the <math>\frac{|\mathrm{CP}(uv)|}{N}</math> is bounded by <math>M</math>. If <math>M</math> is poly-log of <math>N</math>, then the Markov chain is rapid mixing.
| |