# Expander Graphs

According to wikipedia:

"Expander graphs have found extensive applications in computer science, in designing algorithms, error correcting codes, extractors, pseudorandom generators, sorting networks and robust computer networks. They have also been used in proofs of many important results in computational complexity theory, such as SL=L and the PCP theorem. In cryptography too, expander graphs are used to construct hash functions."

We will not explore everything about expander graphs, but will focus on the performances of random walks on expander graphs.

Consider an undirected (multi-)graph ${\displaystyle G(V,E)}$, where the parallel edges between two vertices are allowed.

Some notations:

• For ${\displaystyle S,T\subset V}$, let ${\displaystyle E(S,T)=\{uv\in E\mid u\in S,v\in T\}}$.
• The Edge Boundary of a set ${\displaystyle S\subset V}$, denoted ${\displaystyle \partial S\,}$, is ${\displaystyle \partial S=E(S,{\bar {S}})}$.
 Definition (Graph expansion) The expansion ratio of an undirected graph ${\displaystyle G}$ on ${\displaystyle n}$ vertices, is defined as ${\displaystyle \phi (G)=\min _{\overset {S\subset V}{|S|\leq {\frac {n}{2}}}}{\frac {|\partial S|}{|S|}}.}$

Expander graphs are ${\displaystyle d}$-regular (multi)graphs with ${\displaystyle d=O(1)}$ and ${\displaystyle \phi (G)=\Omega (1)}$.

This definition states the following properties of expander graphs:

• Expander graphs are sparse graphs. This is because the number of edges is ${\displaystyle dn/2=O(n)}$.
• Despite the sparsity, expander graphs have good connectivity. This is supported by the expansion ratio.
• This one is implicit: expander graph is a family of graphs ${\displaystyle \{G_{n}\}}$, where ${\displaystyle n}$ is the number of vertices. The asymptotic order ${\displaystyle O(1)}$ and ${\displaystyle \Omega (1)}$ in the definition is relative to the number of vertices ${\displaystyle n}$, which grows to infinity.

The following fact is directly implied by the definition.

An expander graph has diameter ${\displaystyle O(\log n)}$.

The proof is left for an exercise.

For a vertex set ${\displaystyle S}$, the size of the edge boundary ${\displaystyle |\partial S|}$ can be seen as the "perimeter" of ${\displaystyle S}$, and ${\displaystyle |S|}$ can be seen as the "volume" of ${\displaystyle S}$. The expansion property can be interpreted as a combinatorial version of isoperimetric inequality.

Vertex expansion
We can alternatively define the vertex expansion. For a vertex set ${\displaystyle S\subset V}$, its vertex boundary, denoted ${\displaystyle \delta S\,}$ is defined as that
${\displaystyle \delta S=\{u\not \in S\mid uv\in E{\mbox{ and }}v\in S\}}$,
and the vertex expansion of a graph ${\displaystyle G}$ is ${\displaystyle \psi (G)=\min _{\overset {S\subset V}{|S|\leq {\frac {n}{2}}}}{\frac {|\delta S|}{|S|}}}$.

## Existence of expander graph

We will show the existence of expander graphs by the probabilistic method. In order to do so, we need to generate random ${\displaystyle d}$-regular graphs.

Suppose that ${\displaystyle d}$ is even. We can generate a random ${\displaystyle d}$-regular graph ${\displaystyle G(V,E)}$ as follows:

• Let ${\displaystyle V}$ be the vertex set. Uniformly and independently choose ${\displaystyle {\frac {d}{2}}}$ cycles of ${\displaystyle V}$.
• For each vertex ${\displaystyle v}$, for every cycle, assuming that the two neighbors of ${\displaystyle v}$ in that cycle is ${\displaystyle w}$ and ${\displaystyle u}$, add two edges ${\displaystyle wv}$ and ${\displaystyle uv}$ to ${\displaystyle E}$.

The resulting ${\displaystyle G(V,E)}$ is a multigraph. That is, it may have multiple edges between two vertices. We will show that ${\displaystyle G(V,E)}$ is an expander graph with high probability. Formally, for some constant ${\displaystyle d}$ and constant ${\displaystyle \alpha }$,

${\displaystyle \Pr[\phi (G)\geq \alpha ]=1-o(1)}$.

By the probabilistic method, this shows that there exist expander graphs. In fact, the above probability bound shows something much stronger: it shows that almost every regular graph is an expander.

Recall that ${\displaystyle \phi (G)=\min _{S:|S|\leq {\frac {n}{2}}}{\frac {|\partial S|}{|S|}}}$. We call such ${\displaystyle S\subset V}$ that ${\displaystyle {\frac {|\partial S|}{|S|}}<\alpha }$ a "bad ${\displaystyle S}$". Then ${\displaystyle \phi (G)<\alpha }$ if and only if there exists a bad ${\displaystyle S}$ of size at most ${\displaystyle {\frac {n}{2}}}$. Therefore,

{\displaystyle {\begin{aligned}\Pr[\phi (G)<\alpha ]&=\Pr \left[\min _{S:|S|\leq {\frac {n}{2}}}{\frac {|\partial S|}{|S|}}<\alpha \right]\\&=\sum _{k=1}^{\frac {n}{2}}\Pr[\,\exists {\mbox{bad }}S{\mbox{ of size }}k\,]\\&\leq \sum _{k=1}^{\frac {n}{2}}\sum _{S\in {V \choose k}}\Pr[\,S{\mbox{ is bad}}\,]\end{aligned}}}

Let ${\displaystyle R\subset S}$ be the set of vertices in ${\displaystyle S}$ which has neighbors in ${\displaystyle {\bar {S}}}$, and let ${\displaystyle r=|R|}$. It is obvious that ${\displaystyle |\partial S|\geq r}$, thus, for a bad ${\displaystyle S}$, ${\displaystyle r<\alpha k}$. Therefore, there are at most ${\displaystyle \sum _{r=1}^{\alpha k}{k \choose r}}$ possible choices such ${\displaystyle R}$. For any fixed choice of ${\displaystyle R}$, the probability that an edge picked by a vertex in ${\displaystyle S-R}$ connects to a vertex in ${\displaystyle S}$ is at most ${\displaystyle k/n}$, and there are ${\displaystyle d(k-r)}$ such edges. For any fixed ${\displaystyle S}$ of size ${\displaystyle k}$ and ${\displaystyle R}$ of size ${\displaystyle r}$, the probability that all neighbors of all vertices in ${\displaystyle S-R}$ are in ${\displaystyle S}$ is at most ${\displaystyle \left({\frac {k}{n}}\right)^{d(k-r)}}$. Due to the union bound, for any fixed ${\displaystyle S}$ of size ${\displaystyle k}$,

{\displaystyle {\begin{aligned}\Pr[\,S{\mbox{ is bad}}\,]&\leq \sum _{r=1}^{\alpha k}{k \choose r}\left({\frac {k}{n}}\right)^{d(k-r)}\leq \alpha k{k \choose \alpha k}\left({\frac {k}{n}}\right)^{dk(1-\alpha )}\end{aligned}}}

Therefore,

{\displaystyle {\begin{aligned}\Pr[\phi (G)<\alpha ]&\leq \sum _{k=1}^{\frac {n}{2}}\sum _{S\in {V \choose k}}\Pr[\,S{\mbox{ is bad}}\,]\\&\leq \sum _{k=1}^{\frac {n}{2}}{n \choose k}\alpha k{k \choose \alpha k}\left({\frac {k}{n}}\right)^{dk(1-\alpha )}\\&\leq \sum _{k=1}^{\frac {n}{2}}\left({\frac {en}{k}}\right)^{k}\alpha k\left({\frac {ek}{\alpha k}}\right)^{\alpha k}\left({\frac {k}{n}}\right)^{dk(1-\alpha )}&\quad ({\mbox{Stirling formula }}{n \choose k}\leq \left({\frac {en}{k}}\right)^{k})\\&\leq \sum _{k=1}^{\frac {n}{2}}\exp(O(k))\left({\frac {k}{n}}\right)^{k(d(1-\alpha )-1)}.\end{aligned}}}

The last line is ${\displaystyle o(1)}$ when ${\displaystyle d\geq {\frac {2}{1-\alpha }}}$. Therefore, ${\displaystyle G}$ is an expander graph with expansion ratio ${\displaystyle \alpha }$ with high probability for suitable choices of constant ${\displaystyle d}$ and constant ${\displaystyle \alpha }$.

## Computing graph expansion

Computation of graph expansion seems hard, because the definition involves the minimum over exponentially many subsets of vertices. In fact, the problem of deciding whether a graph is an expander is co-NP-complete. For a non-expander ${\displaystyle G}$, the vertex set ${\displaystyle S\subset V}$ which has low expansion ratio is a proof of the fact that ${\displaystyle G}$ is not an expander, which can be verified in poly-time. However, there is no efficient algorithm for computing the ${\displaystyle \phi (G)}$ unless NP=P.

The expansion ratio of a graph is closely related to the sparsest cut of the graph, which is the dual problem of the multicommodity flow problem, both NP-complete. Studies of these two problems revolutionized the area of approximation algorithms.

We will see right now that although it is hard to compute the expansion ratio exactly, the expansion ratio can be approximated by some efficiently computable algebraic identity of the graph.

# Spectral Graph Theory

## Graph spectrum

The adjacency matrix of an ${\displaystyle n}$-vertex graph ${\displaystyle G}$, denoted ${\displaystyle A=A(G)}$, is an ${\displaystyle n\times n}$ matrix where ${\displaystyle A(u,v)}$ is the number of edges in ${\displaystyle G}$ between vertex ${\displaystyle u}$ and vertex ${\displaystyle v}$. Because ${\displaystyle A}$ is a symmetric matrix with real entries, due to the Perron-Frobenius theorem, it has real eigenvalues ${\displaystyle \lambda _{1}\geq \lambda _{2}\geq \cdots \geq \lambda _{n}}$, which associate with an orthonormal system of eigenvectors ${\displaystyle v_{1},v_{2},\ldots ,v_{n}\,}$ with ${\displaystyle Av_{i}=\lambda _{i}v_{i}\,}$. We call the eigenvalues of ${\displaystyle A}$ the spectrum of the graph ${\displaystyle G}$.

The spectrum of a graph contains a lot of information about the graph. For example, supposed that ${\displaystyle G}$ is ${\displaystyle d}$-regular, the following lemma holds.

 Lemma ${\displaystyle |\lambda _{i}|\leq d}$ for all ${\displaystyle 1\leq i\leq n}$. ${\displaystyle \lambda _{1}=d}$ and the corresponding eigenvector is ${\displaystyle ({\frac {1}{\sqrt {n}}},{\frac {1}{\sqrt {n}}},\ldots ,{\frac {1}{\sqrt {n}}})}$. ${\displaystyle G}$ is connected if and only if ${\displaystyle \lambda _{1}>\lambda _{2}}$. If ${\displaystyle G}$ is bipartite then ${\displaystyle \lambda _{1}=-\lambda _{n}}$.
Proof.
 Let ${\displaystyle A}$ be the adjacency matrix of ${\displaystyle G}$, with entries ${\displaystyle a_{ij}}$. It is obvious that ${\displaystyle \sum _{j}a_{ij}=d\,}$ for any ${\displaystyle j}$. (1) Suppose that ${\displaystyle Ax=\lambda x,x\neq \mathbf {0} }$, and let ${\displaystyle x_{i}}$ be an entry of ${\displaystyle x}$ with the largest absolute value. Since ${\displaystyle (Ax)_{i}=\lambda x_{i}}$, we have ${\displaystyle \sum _{j}a_{ij}x_{j}=\lambda x_{i},\,}$ and so ${\displaystyle |\lambda ||x_{i}|=\left|\sum _{j}a_{ij}x_{j}\right|\leq \sum _{j}a_{ij}|x_{j}|\leq \sum _{j}a_{ij}|x_{i}|\leq d|x_{i}|.}$ Thus ${\displaystyle |\lambda |\leq d}$. (2) is easy to check. (3) Let ${\displaystyle x}$ be the nonzero vector for which ${\displaystyle Ax=dx}$, and let ${\displaystyle x_{i}}$ be an entry of ${\displaystyle x}$ with the largest absolute value. Since ${\displaystyle (Ax)_{i}=dx_{i}}$, we have ${\displaystyle \sum _{j}a_{ij}x_{j}=dx_{i}.\,}$ Since ${\displaystyle \sum _{j}a_{ij}=d\,}$ and by the maximality of ${\displaystyle x_{i}}$, it follows that ${\displaystyle x_{j}=x_{i}}$ for all ${\displaystyle j}$ that ${\displaystyle a_{ij}>0}$. Thus, ${\displaystyle x_{i}=x_{j}}$ if ${\displaystyle i}$ and ${\displaystyle j}$ are adjacent, which implies that ${\displaystyle x_{i}=x_{j}}$ if ${\displaystyle i}$ and ${\displaystyle j}$ are connected. For connected ${\displaystyle G}$, all vertices are connected, thus all ${\displaystyle x_{i}}$ are equal. This shows that if ${\displaystyle G}$ is connected, the eigenvalue ${\displaystyle d=\lambda _{1}}$ has multiplicity 1, thus ${\displaystyle \lambda _{1}>\lambda _{2}}$. If otherwise, ${\displaystyle G}$ is disconnected, then for two different components, we have ${\displaystyle Ax=dx}$ and ${\displaystyle Ay=dy}$, where the entries of ${\displaystyle x}$ and ${\displaystyle y}$ are nonzero only for the vertices in their components components. Then ${\displaystyle A(\alpha x+\beta y)=d(\alpha x+\beta y)}$. Thus, the multiplicity of ${\displaystyle d}$ is greater than 1, so ${\displaystyle \lambda _{1}=\lambda _{2}}$. (4) If ${\displaystyle G}$ if bipartite, then the vertex set can be partitioned into two disjoint nonempty sets ${\displaystyle V_{1}}$ and ${\displaystyle V_{2}}$ such that all edges have one endpoint in each of ${\displaystyle V_{1}}$ and ${\displaystyle V_{2}}$. Algebraically, this means that the adjacency matrix can be organized into the form ${\displaystyle P^{T}AP={\begin{bmatrix}0&B\\B^{T}&0\end{bmatrix}}}$ where ${\displaystyle P}$ is a permutation matrix, which has no change on the eigenvalues. If ${\displaystyle x}$ is an eigenvector corresponding to the eigenvalue ${\displaystyle \lambda }$, then ${\displaystyle x'}$ which is obtained from ${\displaystyle x}$ by changing the sign of the entries corresponding to vertices in ${\displaystyle V_{2}}$, is an eigenvector corresponding to the eigenvalue ${\displaystyle -\lambda }$. It follows that the spectrum of a bipartite graph is symmetric with respect to 0.
${\displaystyle \square }$

## Cheeger's Inequality

One of the most exciting results in spectral graph theory is the following theorem which relate the graph expansion to the spectral gap.

 Theorem (Cheeger's inequality) Let ${\displaystyle G}$ be a ${\displaystyle d}$-regular graph with spectrum ${\displaystyle \lambda _{1}\geq \lambda _{2}\geq \cdots \geq \lambda _{n}}$. Then ${\displaystyle {\frac {d-\lambda _{2}}{2}}\leq \phi (G)\leq {\sqrt {2d(d-\lambda _{2})}}.}$

The theorem was first stated for Riemannian manifolds, and was proved by Cheeger and Buser (for different directions of the inequalities). The discrete case is proved independently by Dodziuk and Alon-Milman.

For a ${\displaystyle d}$-regular graph, the quantity ${\displaystyle (d-\lambda _{2})}$ is called the spectral gap. The name is due to the fact that it is the gap between the first and the second largest eigenvalues of a graph.

If we write ${\displaystyle \alpha =1-{\frac {\lambda _{2}}{d}}}$ (sometimes it is called the normalized spectral gap), the Cheeger's inequality is turned into a nicer form:

${\displaystyle {\frac {\alpha }{2}}\leq {\frac {\phi }{d}}\leq {\sqrt {2\alpha }}}$ or equivalently ${\displaystyle {\frac {1}{2}}\left({\frac {\phi }{d}}\right)^{2}\leq \alpha \leq 2\left({\frac {\phi }{d}}\right)}$.

### Optimization Characterization of Eigenvalues

 Theorem (Rayleigh-Ritz theorem) Let ${\displaystyle A}$ be a symmetric ${\displaystyle n\times n}$ matrix. Let ${\displaystyle \lambda _{1}\geq \lambda _{2}\geq \cdots \geq \lambda _{n}}$ be the eigen values of ${\displaystyle A}$ and ${\displaystyle v_{1},v_{2},\ldots ,v_{n}}$ be the corresponding eigenvectors. Then {\displaystyle {\begin{aligned}\lambda _{1}&=\max _{x\in \mathbb {R} ^{n}}{\frac {x^{T}Ax}{x^{T}x}}\end{aligned}}} and {\displaystyle {\begin{aligned}\lambda _{2}&=\max _{x\bot v_{1}}{\frac {x^{T}Ax}{x^{T}x}}.\end{aligned}}}
Proof.
 Without loss of generality, we may assume that ${\displaystyle v_{1},v_{2},\ldots ,v_{n}}$ are orthonormal eigen-basis. Then it holds that ${\displaystyle {\frac {v_{1}^{T}Av_{1}}{v_{1}^{T}v_{1}}}=\lambda _{1}v_{1}^{T}v_{1}=\lambda _{1}}$, thus we have ${\displaystyle \max _{x\in \mathbb {R} ^{n}}{\frac {x^{T}Ax}{x^{T}x}}\geq \lambda _{1}}$. Let ${\displaystyle x\in \mathbb {R} ^{n}}$ be an arbitrary vector and let ${\displaystyle y={\frac {x}{\sqrt {x^{T}x}}}={\frac {x}{\|x\|}}}$ be its normalization. Since ${\displaystyle v_{1},v_{2},\ldots ,v_{n}}$ are orthonormal basis, ${\displaystyle y}$ can be expressed as ${\displaystyle y=\sum _{i=1}^{n}c_{i}v_{i}}$. Then {\displaystyle {\begin{aligned}{\frac {x^{T}Ax}{x^{T}x}}&=y^{T}Ay=\left(\sum _{i=1}^{n}c_{i}v_{i}\right)^{T}A\left(\sum _{i=1}^{n}c_{i}v_{i}\right)=\left(\sum _{i=1}^{n}c_{i}v_{i}\right)^{T}\left(\sum _{i=1}^{n}\lambda _{i}c_{i}v_{i}\right)\\&=\sum _{i=1}^{n}\lambda _{i}c_{i}^{2}\leq \lambda _{1}\sum _{i=1}^{n}c_{i}^{2}=\lambda _{1}\|y\|=\lambda _{1}.\end{aligned}}} Therefore, ${\displaystyle \max _{x\in \mathbb {R} ^{n}}{\frac {x^{T}Ax}{x^{T}x}}\leq \lambda _{1}}$. Altogether we have ${\displaystyle \max _{x\in \mathbb {R} ^{n}}{\frac {x^{T}Ax}{x^{T}x}}=\lambda _{1}}$ It is similar to prove ${\displaystyle \max _{x\bot v_{1}}{\frac {x^{T}Ax}{x^{T}x}}=\lambda _{2}}$. In the first part take ${\displaystyle x=v_{2}}$ to show that ${\displaystyle \max _{x\bot v_{1}}{\frac {x^{T}Ax}{x^{T}x}}\geq \lambda _{2}}$; and in the second part take an arbitrary ${\displaystyle x\bot v_{1}}$ and ${\displaystyle y={\frac {x}{\|x\|}}}$. Notice that ${\displaystyle y\bot v_{1}}$, thus ${\displaystyle y=\sum _{i=1}^{n}c_{i}v_{i}}$ with ${\displaystyle c_{1}=0}$.
${\displaystyle \square }$

The Rayleigh-Ritz Theorem is a special case of a fundamental theorem in linear algebra, called the Courant-Fischer theorem, which characterizes the eigenvalues of a symmetric matrix by a series of optimizations:

 Theorem (Courant-Fischer theorem) Let ${\displaystyle A}$ be a symmetric matrix with eigenvalues ${\displaystyle \lambda _{1}\geq \lambda _{2}\geq \cdots \geq \lambda _{n}}$. Then {\displaystyle {\begin{aligned}\lambda _{k}&=\max _{v_{1},v_{2},\ldots ,v_{n-k}\in \mathbb {R} ^{n}}\min _{\overset {x\in \mathbb {R} ^{n},x\neq \mathbf {0} }{x\bot v_{1},v_{2},\ldots ,v_{n-k}}}{\frac {x^{T}Ax}{x^{T}x}}\\&=\min _{v_{1},v_{2},\ldots ,v_{k-1}\in \mathbb {R} ^{n}}\max _{\overset {x\in \mathbb {R} ^{n},x\neq \mathbf {0} }{x\bot v_{1},v_{2},\ldots ,v_{k-1}}}{\frac {x^{T}Ax}{x^{T}x}}.\end{aligned}}}

### Graph Laplacian

Let ${\displaystyle G(V,E)}$ be a ${\displaystyle d}$-regular graph of ${\displaystyle n}$ vertices and let ${\displaystyle A}$ be its adjacency matrix. We define ${\displaystyle L=dI-A}$ to be the Laplacian of the graph ${\displaystyle G}$. Take ${\displaystyle x\in \mathbb {R} ^{V}}$ as a distribution over vertices, its Laplacian quadratic form ${\displaystyle x^{T}Lx}$ measures the "smoothness" of ${\displaystyle x}$ over the graph topology, just as what the Laplacian operator does to the differentiable functions.

 Laplacian Property For any vector ${\displaystyle x\in \mathbb {R} ^{n}}$, it holds that ${\displaystyle x^{T}Lx=\sum _{uv\in E}(x_{u}-x_{v})^{2}}$.
Proof.
 {\displaystyle {\begin{aligned}x^{T}Lx&=\sum _{u,v\in V}x_{u}(dI-A)_{uv}x_{v}\\&=\sum _{u}\left(dx_{u}^{2}-\sum _{uv\in E}x_{u}x_{v}\right)\\&=\sum _{u\in V}\sum _{uv\in E}(x_{u}^{2}-x_{u}x_{v}).\end{aligned}}} On the other hand, {\displaystyle {\begin{aligned}\sum _{uv\in E}(x_{u}-x_{v})^{2}&=\sum _{uv\in E}\left(x_{u}^{2}-2x_{u}x_{v}+x_{v}^{2}\right)\\&=\sum _{uv\in E}\left((x_{u}^{2}-x_{u}x_{v})+(x_{v}^{2}-x_{v}x_{u})\right)\\&=\sum _{u\in V}\sum _{uv\in E}(x_{u}^{2}-x_{u}x_{v}).\end{aligned}}}
${\displaystyle \square }$

Applying the Rayleigh-Ritz theorem to the Laplacian matrix of the graph, we have the following "variational characterization" of the spectral gap ${\displaystyle d-\lambda _{2}}$.

 Theorem (Variational Characterization) Let ${\displaystyle G(V,E)}$ be a ${\displaystyle d}$-regular graph of ${\displaystyle n}$ vertices. Suppose that its adjacency matrix is ${\displaystyle A}$, whose eigenvalues are ${\displaystyle \lambda _{1}\geq \lambda _{2}\geq \cdots \geq \lambda _{n}}$. Let ${\displaystyle L=dI-A}$ be the Laplacian matrix. Then {\displaystyle {\begin{aligned}d-\lambda _{2}&=\min _{x\bot {\boldsymbol {1}}}{\frac {x^{T}Lx}{x^{T}x}}=\min _{x\bot {\boldsymbol {1}}}{\frac {\sum _{uv\in E}(x_{u}-x_{v})^{2}}{\sum _{v\in V}x_{v}^{2}}}.\end{aligned}}}
Proof.
 For ${\displaystyle d}$-regular graph, we know that ${\displaystyle \lambda _{1}=d}$ and ${\displaystyle {\boldsymbol {1}}A=d{\boldsymbol {1}}}$, thus ${\displaystyle {\boldsymbol {1}}}$ is the eigenvector of ${\displaystyle \lambda _{1}}$. Due to Rayleigh-Ritz Theorem, it holds that ${\displaystyle \lambda _{2}=\max _{x\bot {\boldsymbol {1}}}{\frac {x^{T}Ax}{x^{T}x}}}$. Then {\displaystyle {\begin{aligned}\min _{x\bot {\boldsymbol {1}}}{\frac {x^{T}Lx}{x^{T}x}}&=\min _{x\bot {\boldsymbol {1}}}{\frac {x^{T}(dI-A)x}{x^{T}x}}\\&=\min _{x\bot {\boldsymbol {1}}}{\frac {dx^{T}x-x^{T}Ax}{x^{T}x}}\\&=\min _{x\bot {\boldsymbol {1}}}\left(d-{\frac {x^{T}Ax}{x^{T}x}}\right)\\&=d-\max _{x\bot {\boldsymbol {1}}}{\frac {x^{T}Ax}{x^{T}x}}\\&=d-\lambda _{2}.\end{aligned}}} We know it holds for the graph Laplacian that ${\displaystyle x^{T}Lx=\sum _{uv\in E}(x_{u}-x_{v})^{2}}$. So the variational characterization of the second eigenvalue of graph is proved.
${\displaystyle \square }$

### Proof of Cheeger's Inequality

We will first give an informal explanation why Cheeger's inequality holds.

Recall that the expansion is defined as

${\displaystyle \phi (G)=\min _{\overset {S\subset V}{|S|\leq {\frac {n}{2}}}}{\frac {|\partial S|}{|S|}}.}$

Let ${\displaystyle \chi _{S}}$ be the characteristic vector of the set ${\displaystyle S}$ such that

${\displaystyle \chi _{S}(v)={\begin{cases}1&v\in S,\\0&v\not \in S.\end{cases}}}$

It is easy to see that

${\displaystyle {\frac {\sum _{uv\in E}(\chi _{S}(u)-\chi _{S}(v))^{2}}{\sum _{v\in V}\chi _{S}(v)^{2}}}={\frac {|\partial S|}{|S|}}.}$

Thus, the expansion can be expressed algebraically as

${\displaystyle \phi (G)=\min _{\overset {S\subset V}{|S|\leq {\frac {n}{2}}}}{\frac {\sum _{uv\in E}(\chi _{S}(u)-\chi _{S}(v))^{2}}{\sum _{v\in V}\chi _{S}(v)^{2}}}=\min _{\overset {x\in \{0,1\}^{n}}{\|x\|_{1}\leq {\frac {n}{2}}}}{\frac {\sum _{uv\in E}(x_{u}-x_{v})^{2}}{\sum _{v\in V}x_{v}^{2}}}.}$

On the other hand, due to the variational characterization of the spectral gap, we have

${\displaystyle d-\lambda _{2}=\min _{x\bot {\boldsymbol {1}}}{\frac {\sum _{uv\in E}(x_{u}-x_{v})^{2}}{\sum _{v\in V}x_{v}^{2}}}.}$

We can easily observe the similarity between the two formulas. Both the expansion ration ${\displaystyle \phi (G)}$ and the spectral gap ${\displaystyle d-\lambda _{2}}$ can be characterized by optimizations of the same objective function ${\displaystyle {\frac {\sum _{uv\in E}(x_{u}-x_{v})^{2}}{\sum _{v\in V}x_{v}^{2}}}}$ over different domains (for the spectral gap, the optimization is over all ${\displaystyle x\bot {\boldsymbol {1}}}$; and for the expansion ratio, it is over all such vectors ${\displaystyle x\in \{0,1\}^{n}}$ with at most ${\displaystyle n/2}$ many 1-entries).

Notations

Throughout the proof, we assume that ${\displaystyle G(V,E)}$ is the ${\displaystyle d}$-regular graph of ${\displaystyle n}$ vertices, ${\displaystyle A}$ is the adjacency matrix, whose eigenvalues are ${\displaystyle \lambda _{1}\geq \lambda _{2}\geq \cdots \geq \lambda _{n}}$, and ${\displaystyle L=(dI-A)}$ is the graph Laplacian.

#### Large spectral gap implies high expansion

 Cheeger's inequality (lower bound) ${\displaystyle \phi (G)\geq {\frac {d-\lambda _{2}}{2}}.}$
Proof.
 Let ${\displaystyle S^{*}\subset V}$, ${\displaystyle |S^{*}|\leq {\frac {n}{2}}}$, be the vertex set achieving the optimal expansion ratio ${\displaystyle \phi (G)=\min _{\overset {S\subset V}{|S|\leq {\frac {n}{2}}}}{\frac {|\partial S|}{|S|}}={\frac {|\partial S^{*}|}{|S^{*}|}}}$, and ${\displaystyle x\in \mathbb {R} ^{n}}$ be a vector defined as ${\displaystyle x_{v}={\begin{cases}1/|S^{*}|&{\text{if }}v\in S^{*},\\-1/\left|{\overline {S^{*}}}\right|&{\text{if }}v\in {\overline {S^{*}}}.\end{cases}}}$ Clearly, ${\displaystyle x\cdot {\boldsymbol {1}}=\sum _{v\in S^{*}}{\frac {1}{|S^{*}|}}-\sum _{v\in {\overline {S^{*}}}}{\frac {1}{\left|{\overline {S^{*}}}\right|}}=0}$, thus ${\displaystyle x\bot {\boldsymbol {1}}}$. Due to the variational characterization of the second eigenvalue, {\displaystyle {\begin{aligned}d-\lambda _{2}&\leq {\frac {\sum _{uv\in E}(x_{u}-x_{v})^{2}}{\sum _{v\in V}x_{v}^{2}}}\\&={\frac {\sum _{u\in S^{*},v\in {\overline {S^{*}}},uv\in E}\left(1/|S^{*}|+1/|{\overline {S^{*}}}|\right)^{2}}{1/|S^{*}|+1/|{\overline {S^{*}}}|}}\\&=\left({\frac {1}{|S^{*}|}}+{\frac {1}{\left|{\overline {S^{*}}}\right|}}\right)\cdot |\partial S^{*}|\\&\leq {\frac {2|\partial S^{*}|}{|S^{*}|}}&({\text{since }}|S^{*}|\leq {\frac {n}{2}})\\&=2\phi (G).\end{aligned}}}
${\displaystyle \square }$

#### High expansion implies large spectral gap

We next prove the upper bound direction of the Cheeger's inequality:

 Cheeger's inequality (upper bound) ${\displaystyle \phi (G)\leq {\sqrt {2d(d-\lambda _{2})}}.}$

This direction is harder than the lower bound direction. But it is mathematically more interesting and also more useful to us for analyzing the mixing time of random walks.

We prove the following equivalent inequality:

${\displaystyle {\frac {\phi ^{2}}{2d}}\leq d-\lambda _{2}.}$

Let ${\displaystyle x}$ satisfy that

• ${\displaystyle Ax=\lambda _{2}x}$, i.e., it is a eigenvector for ${\displaystyle \lambda _{2}}$;
• ${\displaystyle |\{v\in V\mid x_{v}>0\}|\leq {\frac {n}{2}}}$, i.e., ${\displaystyle x}$ has at most ${\displaystyle n/2}$ positive entries. (We can always choose ${\displaystyle x}$ to be ${\displaystyle -x}$ if this is not satisfied.)

And let nonnegative vector ${\displaystyle y}$ be defined as

${\displaystyle y_{v}={\begin{cases}x_{v}&x_{v}>0,\\0&{\text{otherwise.}}\end{cases}}}$

We then prove the following inequalities:

1. ${\displaystyle {\frac {y^{T}Ly}{y^{T}y}}\leq d-\lambda _{2}}$;
2. ${\displaystyle {\frac {\phi ^{2}}{2d}}\leq {\frac {y^{T}Ly}{y^{T}y}}}$.

The theorem is then a simple consequence by combining these two inequalities.

We prove the first inequality:

 Lemma ${\displaystyle {\frac {y^{T}Ly}{y^{T}y}}\leq d-\lambda _{2}}$.
Proof.
 If ${\displaystyle x_{u}\geq 0}$, then {\displaystyle {\begin{aligned}(Ly)_{u}&=((dI-A)y)_{u}=dy_{u}-\sum _{v}A(u,v)y_{v}=dx_{u}-\sum _{v:x_{v}\geq 0}A(u,v)x_{v}\\&\leq dx_{u}-\sum _{v}A(u,v)x_{v}=((dI-A)x)_{u}=(d-\lambda _{2})x_{u}.\end{aligned}}} Then {\displaystyle {\begin{aligned}y^{T}Ly&=\sum _{u}y_{u}(Ly)_{u}=\sum _{u:x_{u}\geq 0}y_{u}(Ly)_{u}=\sum _{u:x_{u}\geq 0}x_{u}(Ly)_{u}\\&\leq (d-\lambda _{2})\sum _{u:x_{u}\geq 0}x_{u}^{2}=(d-\lambda _{2})\sum _{u}y_{u}^{2}=(d-\lambda _{2})y^{T}y,\end{aligned}}} which proves the lemma.
${\displaystyle \square }$

We then prove the second inequality:

 Lemma ${\displaystyle {\frac {\phi ^{2}}{2d}}\leq {\frac {y^{T}Ly}{y^{T}y}}}$.
Proof.

To prove this, we introduce a new quantity ${\displaystyle {\frac {\sum _{uv\in E}|y_{u}^{2}-y_{v}^{2}|}{y^{T}y}}}$ and shows that

${\displaystyle \phi \leq {\frac {\sum _{uv\in E}|y_{u}^{2}-y_{v}^{2}|}{y^{T}y}}\leq {\sqrt {2d}}\cdot {\sqrt {\frac {y^{T}Ly}{y^{T}y}}}}$.

This will give us the desired inequality ${\displaystyle {\frac {\phi ^{2}}{2d}}\leq {\frac {y^{T}Ly}{y^{T}y}}}$.

 Lemma ${\displaystyle {\frac {\sum _{uv\in E}|y_{u}^{2}-y_{v}^{2}|}{y^{T}y}}\leq {\sqrt {2d}}\cdot {\sqrt {\frac {y^{T}Ly}{y^{T}y}}}}$.
Proof.
 By the Cauchy-Schwarz Inequality, {\displaystyle {\begin{aligned}\sum _{uv\in E}|y_{u}^{2}-y_{v}^{2}|&=\sum _{uv\in E}|y_{u}-y_{v}||y_{u}+y_{v}|\\&\leq {\sqrt {\sum _{uv\in E}(y_{u}-y_{v})^{2}}}\cdot {\sqrt {\sum _{uv\in E}(y_{u}+y_{v})^{2}}}.\end{aligned}}} By the Laplacian property, the first term ${\displaystyle {\sqrt {\sum _{uv\in E}(y_{u}-y_{v})^{2}}}={\sqrt {y^{T}Ly}}}$. By the Inequality of Arithmetic and Geometric Means, the second term ${\displaystyle {\sqrt {\sum _{uv\in E}(y_{u}+y_{v})^{2}}}\leq {\sqrt {2\sum _{uv\in E}(y_{u}^{2}+y_{v}^{2})}}={\sqrt {2d\sum _{u\in V}y_{u}^{2}}}={\sqrt {2dy^{T}y}}.}$ Combining them together, we have ${\displaystyle \sum _{uv\in E}|y_{u}^{2}-y_{v}^{2}|\leq {\sqrt {2d}}\cdot {\sqrt {y^{T}Ly}}\cdot {\sqrt {y^{T}y}}}$.
${\displaystyle \square }$
 Lemma ${\displaystyle \phi \leq {\frac {\sum _{uv\in E}|y_{u}^{2}-y_{v}^{2}|}{y^{T}y}}}$.
Proof.
 Suppose that ${\displaystyle y}$ has ${\displaystyle t}$ nonzero entries. We know that ${\displaystyle t\leq n/2}$ due to the definition of ${\displaystyle y}$. We enumerate the vertices ${\displaystyle u_{1},u_{2},\ldots ,u_{n}\in V}$ such that ${\displaystyle y_{u_{1}}\geq y_{u_{2}}\geq \cdots \geq y_{u_{t}}>y_{u_{t+1}}=\cdots =y_{u_{n}}=0}$. Then {\displaystyle {\begin{aligned}\sum _{uv\in E}|y_{u}^{2}-y_{v}^{2}|&=\sum _{u_{i}u_{j}\in E \atop ii}A(u_{i},u_{j})\sum _{k=i}^{j-1}(y_{u_{k}}^{2}-y_{u_{k+1}}^{2})=\sum _{i=1}^{n}\sum _{j>i}\sum _{k=i}^{j-1}A(u_{i},u_{j})(y_{u_{k}}^{2}-y_{u_{k+1}}^{2}).\end{aligned}}} We have the following universal equation for sums: {\displaystyle {\begin{aligned}\sum _{i=1}^{n}\sum _{j>i}\sum _{k=i}^{j-1}A(u_{i},u_{j})(y_{u_{k}}^{2}-y_{u_{k+1}}^{2})&=\sum _{k=1}^{n}\sum _{i\leq k}\sum _{j>k}A(u_{i},u_{j})(y_{u_{k}}^{2}-y_{u_{k+1}}^{2})\\&=\sum _{k=1}^{t}(y_{u_{k}}^{2}-y_{u_{k+1}}^{2})\sum _{i\leq k}\sum _{j>k}A(u_{i},u_{j})\end{aligned}}} Notice that ${\displaystyle \sum _{i\leq k}\sum _{j>k}A(u_{i},u_{j})=|\partial \{u_{1},\ldots ,u_{k}\}|}$, which is at most ${\displaystyle \phi k}$ since ${\displaystyle k\leq t\leq n/2}$. Therefore, combining these together, we have {\displaystyle {\begin{aligned}\sum _{uv\in E}|y_{u}^{2}-y_{v}^{2}|&=\sum _{k=1}^{t}(y_{u_{k}}^{2}-y_{u_{k+1}}^{2})\sum _{i\leq k}\sum _{j>k}A(u_{i},u_{j})\\&=\sum _{k=1}^{t}(y_{u_{k}}^{2}-y_{u_{k+1}}^{2})|\partial \{u_{1},\ldots ,u_{k}\}|\\&\leq \phi \sum _{k=1}^{t}(y_{u_{k}}^{2}-y_{u_{k+1}}^{2})k\\&=\phi \sum _{k=1}^{t}y_{u_{k}}^{2}\\&=\phi y^{T}y.\end{aligned}}}
${\displaystyle \square }$
${\displaystyle \square }$

# Mixing Time

The mixing time of a Markov chain gives the rate at which a Markov chain converges to the stationary distribution. To rigorously define this notion, we need a way of measuring the closeness between two distributions.

## Total Variation Distance

In probability theory, the total variation distance measures the difference between two probability distributions.

 Definition (total variation distance) Let ${\displaystyle p}$ and ${\displaystyle q}$ be two probability distributions over the same finite state space ${\displaystyle \Omega }$, the total variation distance between ${\displaystyle p}$ and ${\displaystyle q}$ is defined as ${\displaystyle \|p-q\|_{TV}={\frac {1}{2}}\sum _{x\in \Omega }|p(x)-q(x)|={\frac {1}{2}}\|p-q\|_{1}}$, where ${\displaystyle \|\cdot \|_{1}}$ is the ${\displaystyle \ell _{1}}$-norm of vectors.

It can be verified (left as an exercise) that

${\displaystyle \max _{A\subset \Omega }|p(A)-q(A)|={\frac {1}{2}}\sum _{x\in \Omega }|p(x)-q(x)|}$,

thus the total variation distance can be equivalently defined as

${\displaystyle \|p-q\|_{TV}=\max _{A\subset \Omega }|p(A)-q(A)|}$.

So the total variation distance between two distributions gives an upper bound on the difference between the probabilities of the same event according to the two distributions.

## Mixing Time

 Definition (mixing time) Let ${\displaystyle \pi }$ be the stationary of the chain, and ${\displaystyle p_{x}^{(t)}}$ be the distribution after ${\displaystyle t}$ steps when the initial state is ${\displaystyle x}$. ${\displaystyle \Delta _{x}(t)=\|p_{x}^{(t)}-\pi \|_{TV}}$ is the distance to stationary distribution ${\displaystyle \pi }$ after ${\displaystyle t}$ steps, started at state ${\displaystyle x}$. ${\displaystyle \Delta (t)=\max _{x\in \Omega }\Delta _{x}(t)}$ is the maximum distance to stationary distribution ${\displaystyle \pi }$ after ${\displaystyle t}$ steps. ${\displaystyle \tau _{x}(\epsilon )=\min\{t\mid \Delta _{x}(t)\leq \epsilon \}}$ is the time until the total variation distance to the stationary distribution, started at the initial state ${\displaystyle x}$, reaches ${\displaystyle \epsilon }$. ${\displaystyle \tau (\epsilon )=\max _{x\in \Omega }\tau _{x}(\epsilon )}$ is the time until the total variation distance to the stationary distribution, started at the worst possible initial state, reaches ${\displaystyle \epsilon }$.

We note that ${\displaystyle \Delta _{x}(t)}$ is monotonically non-increasing in ${\displaystyle t}$. So the definition of ${\displaystyle \tau _{x}(\epsilon )}$ makes sense, and is actually the inverse of ${\displaystyle \Delta _{x}(t)}$.

 Definition (mixing time) The mixing time ${\displaystyle \tau _{\mathrm {mix} }}$ of a Markov chain is ${\displaystyle \tau (1/2\mathrm {e} )}$.

The mixing time is the time until the total variation distance to the stationary distribution, starting from the worst possible initial state ${\displaystyle x\in \Omega }$, reaches ${\displaystyle {\frac {1}{2\mathrm {e} }}}$. The value ${\displaystyle {\frac {1}{2\mathrm {e} }}}$ is chosen just for the convenience of calculation. The next proposition says that ${\displaystyle \tau (\epsilon )}$ with general ${\displaystyle \epsilon }$ can be estimated from ${\displaystyle \tau _{\mathrm {mix} }}$.

 Proposition ${\displaystyle \Delta (k\cdot \tau _{\mathrm {mix} })\leq \mathrm {e} ^{-k}}$ for any integer ${\displaystyle k\geq 1}$. ${\displaystyle \tau (\epsilon )\leq \tau _{\mathrm {mix} }\cdot \left\lceil \ln {\frac {1}{\epsilon }}\right\rceil }$.

So the distance to stationary distribution ${\displaystyle \Delta (t)}$ decays exponentially in multiplications of ${\displaystyle \tau _{\mathrm {mix} }}$.

Both the formal proofs of the monotonicity of ${\displaystyle \Delta _{x}(t)}$ and the above proposition uses the coupling technique and is postponed to next section.

# Spectral approach for symmetric chain

We consider the symmetric Markov chains defined on the state space ${\displaystyle \Omega }$, where the transition matrix ${\displaystyle P}$ is symmetric.

We have the following powerful spectral theorem for symmetric matrices.

 Theorem (Spectral theorem) Let ${\displaystyle S}$ be a symmetric ${\displaystyle n\times n}$ matrix, whose eigenvalues are ${\displaystyle \lambda _{1}\geq \lambda _{2}\geq \cdots \geq \lambda _{n}}$. There exist eigenvectors ${\displaystyle v_{1},v_{2},\ldots ,v_{n}}$ such that ${\displaystyle v_{i}S=\lambda _{i}v_{i}}$ for all ${\displaystyle i=1,\ldots ,n}$ and ${\displaystyle v_{1},v_{2},\ldots ,v_{n}}$ are orthonormal.

A set of orthonormal vectors ${\displaystyle v_{1},v_{2},\ldots ,v_{n}}$ satisfy that

• for any ${\displaystyle i\neq j}$, ${\displaystyle v_{i}}$ is orthogonal to ${\displaystyle v_{j}}$, which means that ${\displaystyle v_{i}^{T}v_{j}=0}$, denoted ${\displaystyle v_{i}\bot v_{j}}$;
• all ${\displaystyle v_{i}}$ have unit length, i.e., ${\displaystyle \|v_{i}\|_{2}=1}$.

Since the eigenvectors ${\displaystyle v_{1},v_{2},\ldots ,v_{n}}$ are orthonormal, we can use them as orthogonal basis, so that any vector ${\displaystyle x\in \mathbb {R} ^{n}}$ can be expressed as ${\displaystyle x=\sum _{i=1}^{n}c_{i}v_{i}}$ where ${\displaystyle c_{i}=q^{T}v_{i}}$, therefore

${\displaystyle xS=\sum _{i=1}^{n}c_{i}v_{i}S=\sum _{i=1}^{n}c_{i}\lambda _{i}v_{i}.}$

So multiplying by ${\displaystyle S}$ corresponds to multiplying the length of ${\displaystyle x}$ along the direction of every eigenvector by a factor of the corresponding eigenvalue.

Back to the symmetric Markov chain. Let ${\displaystyle \Omega }$ be a finite state space of size ${\displaystyle N}$, and ${\displaystyle P}$ be a symmetric transition matrix, whose eigenvalues are ${\displaystyle \lambda _{1}\geq \lambda _{2}\geq \ldots \geq \lambda _{N}}$. The followings hold for a symmetric transition matrix ${\displaystyle P}$:

• Due to the spectral theorem, there exist orthonormal eigenvectors ${\displaystyle v_{1},v_{2},\ldots ,v_{N}}$ such that ${\displaystyle v_{i}P=\lambda _{i}v_{i}}$ for ${\displaystyle i=1,2,\ldots ,N}$ and any distribution ${\displaystyle q}$ over ${\displaystyle \Omega }$ can be expressed as ${\displaystyle q=\sum _{i=1}^{N}c_{i}v_{i}}$ where ${\displaystyle c_{i}=q^{T}v_{i}}$.
• A symmetric ${\displaystyle P}$ must be double stochastic, thus the stationary distribution ${\displaystyle \pi }$ is the uniform distribution.

Recall that due to Perron-Frobenius theorem, ${\displaystyle \lambda _{1}=1}$. And ${\displaystyle {\boldsymbol {1}}P={\boldsymbol {1}}}$ since ${\displaystyle P}$ is double stochastic, thus ${\displaystyle v_{1}={\frac {\boldsymbol {1}}{\|{\boldsymbol {1}}\|_{2}}}=\left({\frac {1}{\sqrt {N}}},\ldots ,{\frac {1}{\sqrt {N}}}\right)}$.

When ${\displaystyle q}$ is a distribution, i.e., ${\displaystyle q}$ is a nonnegative vector and ${\displaystyle \|q\|_{1}=1}$, it holds that ${\displaystyle c_{1}=q^{T}v_{1}={\frac {1}{\sqrt {N}}}}$ and ${\displaystyle c_{1}v_{1}=\left({\frac {1}{N}},\ldots ,{\frac {1}{N}}\right)=\pi }$, thus

${\displaystyle q=\sum _{i=1}^{N}c_{i}v_{i}=\pi +\sum _{i=2}^{N}c_{i}v_{i},}$

and the distribution at time ${\displaystyle t}$ when the initial distribution is ${\displaystyle q}$, is given by

${\displaystyle qP^{t}=\pi P^{t}+\sum _{i=2}^{N}c_{i}v_{i}P^{t}=\pi +\sum _{i=2}^{N}c_{i}\lambda _{i}^{t}v_{i}.}$

It is easy to see that this distribution converges to ${\displaystyle \pi }$ when the absolute values of ${\displaystyle \lambda _{2},\ldots ,\lambda _{N}}$ are all less than 1. And the rate at which it converges to ${\displaystyle \pi }$, namely the mixing rate, is determined by the quantity ${\displaystyle \lambda _{\max }=\max\{|\lambda _{2}|,|\lambda _{N}|\}\,}$, which is the largest absolute eigenvalues other than ${\displaystyle \lambda _{1}}$.

 Theorem Let ${\displaystyle P}$ be the transition matrix for a symmetric Markov chain on state space ${\displaystyle \Omega }$ where ${\displaystyle |\Omega |=N}$. Let ${\displaystyle \lambda _{1}\geq \lambda _{2}\geq \cdots \geq \lambda _{N}}$ be the spectrum of ${\displaystyle P}$ and ${\displaystyle \lambda _{\max }=\max\{|\lambda _{2}|,|\lambda _{N}|\}\,}$. The mixing rate of the Markov chain is ${\displaystyle \tau (\epsilon )\leq {\frac {{\frac {1}{2}}\ln N+\ln {\frac {1}{2\epsilon }}}{1-\lambda _{\text{max}}}}}$.
Proof.
 As analysed above, if ${\displaystyle P}$ is symmetric, it has orthonormal eigenvectors ${\displaystyle v_{1},\ldots ,v_{N}}$ such that any distribution ${\displaystyle q}$ over ${\displaystyle \Omega }$ can be expressed as ${\displaystyle q=\sum _{i=1}^{N}c_{i}v_{i}=\pi +\sum _{i=2}^{N}c_{i}v_{i}}$ with ${\displaystyle c_{i}=q^{T}v_{i}}$, and ${\displaystyle qP^{t}=\pi +\sum _{i=2}^{N}c_{i}\lambda _{i}^{t}v_{i}.}$ Thus, {\displaystyle {\begin{aligned}\|qP^{t}-\pi \|_{1}&=\left\|\sum _{i=2}^{N}c_{i}\lambda _{i}^{t}v_{i}\right\|_{1}\\&\leq {\sqrt {N}}\left\|\sum _{i=2}^{N}c_{i}\lambda _{i}^{t}v_{i}\right\|_{2}&\quad {\text{(Cauchy-Schwarz)}}\\&={\sqrt {N}}{\sqrt {\sum _{i=2}^{N}c_{i}^{2}\lambda _{i}^{2t}}}\\&\leq {\sqrt {N}}\lambda _{\max }^{t}{\sqrt {\sum _{i=2}^{N}c_{i}^{2}}}\\&={\sqrt {N}}\lambda _{\max }^{t}\|q\|_{2}\\&\leq {\sqrt {N}}\lambda _{\max }^{t}.\end{aligned}}} The last inequality is due to a universal relation ${\displaystyle \|q\|_{2}\leq \|q\|_{1}}$ and the fact that ${\displaystyle q}$ is a distribution. Then for any ${\displaystyle x\in \Omega }$, denoted by ${\displaystyle {\boldsymbol {1}}_{x}}$ the indicator vector for ${\displaystyle x}$ such that ${\displaystyle {\boldsymbol {1}}_{x}(x)=1}$ and ${\displaystyle {\boldsymbol {1}}_{x}(y)=0}$ for ${\displaystyle y\neq x}$, we have {\displaystyle {\begin{aligned}\Delta _{x}(t)&=\left\|{\boldsymbol {1}}_{x}P^{t}-\pi \right\|_{TV}={\frac {1}{2}}\left\|{\boldsymbol {1}}_{x}P^{t}-\pi \right\|_{1}\\&\leq {\frac {\sqrt {N}}{2}}\lambda _{\max }^{t}\leq {\frac {\sqrt {N}}{2}}\mathrm {e} ^{-t(1-\lambda _{\max })}.\end{aligned}}} Therefore, we have ${\displaystyle \tau _{x}(\epsilon )=\min\{t\mid \Delta _{x}(t)\leq \epsilon \}\leq {\frac {{\frac {1}{2}}\ln N+\ln {\frac {1}{2\epsilon }}}{1-\lambda _{\max }}}}$ for any ${\displaystyle x\in \Omega }$, thus the bound holds for ${\displaystyle \tau (\epsilon )=\max _{x}\tau _{x}(\epsilon )}$.
${\displaystyle \square }$

## Rapid mixing of expander walk

Let ${\displaystyle G(V,E)}$ be a ${\displaystyle d}$-regular graph on ${\displaystyle n}$ vertices. Let ${\displaystyle A}$ be its adjacency matrix. The transition matrix of the lazy random walk on ${\displaystyle G}$ is given by ${\displaystyle P={\frac {1}{2}}\left(I+{\frac {1}{d}}A\right)}$. Specifically,

${\displaystyle P(u,v)={\begin{cases}{\frac {1}{2}}&{\text{if }}u=v,\\{\frac {1}{2d}}&{\text{if }}uv\in E,\\0&{\text{otherwise.}}\end{cases}}}$

Obviously ${\displaystyle P}$ is symmetric.

Let ${\displaystyle \lambda _{1}\geq \lambda _{2}\geq \cdots \geq \lambda _{n}}$ be the eigenvalues of ${\displaystyle A}$, and ${\displaystyle \nu _{1}\geq \nu _{2}\geq \cdots \geq \nu _{n}}$ be the eigenvalues of ${\displaystyle P}$. It is easy to verify that

${\displaystyle \nu _{i}={\frac {1}{2}}\left(1+{\frac {\lambda _{i}}{d}}\right)}$.

We know that ${\displaystyle -d\leq \lambda _{i}\leq d}$, thus ${\displaystyle 0\leq \nu _{i}\leq 1}$. Therefore, ${\displaystyle \nu _{\max }=\max\{|\nu _{2}|,|\nu _{n}|\}=\nu _{2}\,}$. Due to the above analysis of symmetric Markov chain,

${\displaystyle \tau (\epsilon )\leq {\frac {{\frac {1}{2}}(\ln n+\ln {\frac {1}{2\epsilon }})}{1-\nu _{\max }}}={\frac {{\frac {1}{2}}(\ln n+\ln {\frac {1}{2\epsilon }})}{1-\nu _{2}}}={\frac {d(\ln n+\ln {\frac {1}{2\epsilon }})}{d-\lambda _{2}}}}$.

Thus we prove the following theorem for lazy random walk on ${\displaystyle d}$-regular graphs.

 Theorem Let ${\displaystyle G(V,E)}$ be a ${\displaystyle d}$-regular graph on ${\displaystyle n}$ vertices, with spectrum ${\displaystyle \lambda _{1}\geq \lambda _{2}\geq \cdots \geq \lambda _{n}}$. The mixing rate of lazy random walk on ${\displaystyle G}$ is ${\displaystyle \tau (\epsilon )\leq {\frac {d(\ln n+\ln {\frac {1}{2\epsilon }})}{d-\lambda _{2}}}}$.

Due to Cheeger's inequality, the spectral gap is bounded by expansion ratio as

${\displaystyle d-\lambda _{2}\geq {\frac {\phi ^{2}}{2d}},}$

where ${\displaystyle \phi =\phi (G)}$ is the expansion ratio of the graph ${\displaystyle G}$. Therefore, we have the following corollary which bounds the mixing time by graph expansion.

 Corollary Let ${\displaystyle G(V,E)}$ be a ${\displaystyle d}$-regular graph on ${\displaystyle n}$ vertices, whose expansion ratio is ${\displaystyle \phi =\phi (G)}$. The mixing rate of lazy random walk on ${\displaystyle G}$ is ${\displaystyle \tau (\epsilon )\leq {\frac {2d^{2}(\ln n+\ln {\frac {1}{2\epsilon }})}{\phi ^{2}}}}$. In particular, the mixing time is ${\displaystyle \tau _{\text{mix}}=\tau (1/2\mathrm {e} )\leq {\frac {2d^{2}(\ln n+2)}{\phi ^{2}}}}$.

For expander graphs, both ${\displaystyle d}$ and ${\displaystyle \phi }$ are constants. The mixing time of lazy random walk is ${\displaystyle \tau _{\text{mix}}=O(\ln n)}$ so the random walk is rapidly mixing.