高级算法 (Fall 2016)/Min-Cut and Max-Cut
under construction
Graph Cut
Let [math]\displaystyle{ G(V, E) }[/math] be an undirected graph. A subset [math]\displaystyle{ C\subseteq E }[/math] of edges is a cut of graph [math]\displaystyle{ G }[/math] if [math]\displaystyle{ G }[/math] becomes disconnected after deleting all edges in [math]\displaystyle{ C }[/math].
More formally, a pair of disjoint subsets [math]\displaystyle{ S,T\subseteq V }[/math] of vertices is called a bipartition of [math]\displaystyle{ V }[/math] if [math]\displaystyle{ S }[/math] and [math]\displaystyle{ T }[/math] are not empty and [math]\displaystyle{ S\cup T=V }[/math]. Given a bipartition [math]\displaystyle{ \{S,T\} }[/math] of [math]\displaystyle{ V }[/math], we denote by
- [math]\displaystyle{ E(S,T)=\{uv\in E\mid u\in S, v\in T\} }[/math]
the set of "crossing edges" with one endpoint in each of [math]\displaystyle{ S }[/math] and [math]\displaystyle{ T }[/math].
Then every cut [math]\displaystyle{ C\subseteq E }[/math] in [math]\displaystyle{ G }[/math] corresponds to a
- [math]\displaystyle{ C=E(S,T) }[/math],
where [math]\displaystyle{ \{S,T\} }[/math] is a bipartition of [math]\displaystyle{ V }[/math].
Given a graph [math]\displaystyle{ G }[/math], there might be many cuts in [math]\displaystyle{ G }[/math]. We are interested in looking for the minimum or maximum cut:
- Min-cut: the bipartition [math]\displaystyle{ \{S,T\} }[/math] of [math]\displaystyle{ V }[/math] that minimizes [math]\displaystyle{ |E(S,\overline{S})| }[/math].
- Max-cut: the bipartition [math]\displaystyle{ \{S,T\} }[/math] of [math]\displaystyle{ V }[/math] that maximizes [math]\displaystyle{ |E(S,\overline{S})| }[/math].
Min-Cut
The min-cut problem, also called the global minimum cut problem, is defined as follows.
Min-cut problem - Input: an undirected graph [math]\displaystyle{ G(V,E) }[/math];
- Output: a cut [math]\displaystyle{ C }[/math] in [math]\displaystyle{ G }[/math] with the smallest size [math]\displaystyle{ |C| }[/math].
Equivalently, the problem asks to find a non-empty subset [math]\displaystyle{ S\subset V }[/math] of vertices with the smallest [math]\displaystyle{ |E(S,\overline{S})| }[/math].
We consider the problem in a slightly more generalized setting, where the input graphs [math]\displaystyle{ G }[/math] can be multi-graphs, meaning that there could be multiple edges between two vertices [math]\displaystyle{ u }[/math] and [math]\displaystyle{ v }[/math]. We call such edges the parallel edges. The cuts in multi-graphs are defined in the same way as before, and the cost of a cut [math]\displaystyle{ C }[/math] is given by the total number of edges (including parallel edges) in [math]\displaystyle{ C }[/math]. Equivalently, one may think of multi-graphs as graphs with integer edge weights, and the cost of a cut [math]\displaystyle{ C }[/math] is the total weights of all edges in [math]\displaystyle{ C }[/math].
A canonical deterministic algorithm for this problem is through the max-flow min-cut theorem. The max-flow algorithm finds us a minimum [math]\displaystyle{ s }[/math]-[math]\displaystyle{ t }[/math] cut, which disconnects a source [math]\displaystyle{ s\in V }[/math] from a sink [math]\displaystyle{ t\in V }[/math], both specified as part of the input. A global min cut can be found by exhaustively finding the minimum [math]\displaystyle{ s }[/math]-[math]\displaystyle{ t }[/math] cut for an arbitrarily fixed source [math]\displaystyle{ s }[/math] and all possible sink [math]\displaystyle{ t\neq s }[/math]. This takes [math]\displaystyle{ (n-1)\times }[/math]max-flow time.
The fastest known deterministic algorithm for the minimum cut problem on multi-graphs is the Stoer–Wagner algorithm, which achieves an [math]\displaystyle{ O(mn+n^2\log n) }[/math] time complexity.
If we restrict the input to be simple graphs (meaning there is no parallel edges) with no edge weight, there are better algorithms. The most recent one was published in STOC 2015, achieving a near-linear (in the number of edges) time complexity.
Karger's Contraction algorithm
We will introduce a very simple and elegant algorithm discovered by David Karger.
Let [math]\displaystyle{ G(V, E) }[/math] be a multi-graph, which allows parallel edges between two distinct vertices [math]\displaystyle{ u }[/math] and [math]\displaystyle{ v }[/math] but does not allow any self-loop, i.e. an edge connect a vertex to itself. Such a multi-graph can be represented as data structures like adjacency matrix [math]\displaystyle{ A }[/math], where [math]\displaystyle{ A }[/math] is symmetric (undirected graph) with zero diagonal, and each entry [math]\displaystyle{ A(u,v) }[/math] is a nonnegative integer giving the number of edges between vertices [math]\displaystyle{ u }[/math] and [math]\displaystyle{ v }[/math].
We define an operation on multi-graphs called contraction: For a multigraph [math]\displaystyle{ G(V, E) }[/math], for any edge [math]\displaystyle{ uv\in E }[/math], let [math]\displaystyle{ contract(G,uv) }[/math] be a new multigraph obtained by:
- replacing the vertices [math]\displaystyle{ u }[/math] and [math]\displaystyle{ v }[/math] by a new vertex [math]\displaystyle{ x\not\in V }[/math];
- for each [math]\displaystyle{ w\not\in\{u,v\} }[/math] replacing any edge [math]\displaystyle{ uw }[/math] or [math]\displaystyle{ vw }[/math] by the edge [math]\displaystyle{ xw }[/math];
- removing all parallel edges between [math]\displaystyle{ u }[/math] and [math]\displaystyle{ v }[/math] in [math]\displaystyle{ E }[/math];
- the rest of the graph remains unchanged.
To conclude, the [math]\displaystyle{ contract(G,uv) }[/math] operation merges the two vertices [math]\displaystyle{ u }[/math] and [math]\displaystyle{ v }[/math] into a new vertex which maintains the old neighborhoods of both [math]\displaystyle{ u }[/math] and [math]\displaystyle{ v }[/math] except for that all the parallel edges between [math]\displaystyle{ u }[/math] and [math]\displaystyle{ v }[/math] are removed.
Perhaps a better way to look at contraction is to interpret it as union of equivalent classes of vertices. Initially every vertex is in a dinstinct equivalent class. Upon call a [math]\displaystyle{ contract(G,uv) }[/math], the two equivalent classes corresponding to [math]\displaystyle{ u }[/math] and [math]\displaystyle{ v }[/math] are unioned together, and only those edges crossing between different equivalent classes are counted as valid edges in the graph.
RandomContract (Karger 1993) - while [math]\displaystyle{ |V|\gt 2 }[/math] do
- choose an edge [math]\displaystyle{ uv\in E }[/math] uniformly at random;
- [math]\displaystyle{ G=contract(G,uv) }[/math];
- return [math]\displaystyle{ C=E }[/math] (the parallel edges between the only remaining vertices in [math]\displaystyle{ V }[/math]);
- while [math]\displaystyle{ |V|\gt 2 }[/math] do
A multi-graph can be maintained by appropriate data strucrtures such that each contraction takes [math]\displaystyle{ O(n) }[/math] time, where [math]\displaystyle{ n }[/math] is the number of vertices, so the algorithm terminates in time [math]\displaystyle{ O(n^2) }[/math]. We leave this as an exercise.
Analysis of accuracy
For convenience, we assume that each edge has a unique "identity" [math]\displaystyle{ e }[/math]. And when an edge [math]\displaystyle{ uv\in E }[/math] is contracted to new vertex [math]\displaystyle{ x }[/math], and each adjacent edge [math]\displaystyle{ uw }[/math] of [math]\displaystyle{ u }[/math] (or adjacent edge [math]\displaystyle{ vw }[/math] of [math]\displaystyle{ v }[/math]) is replaced by [math]\displaystyle{ xw }[/math], the identity [math]\displaystyle{ e }[/math] of the edge [math]\displaystyle{ uw }[/math] (or [math]\displaystyle{ vw }[/math]) is transfered to the new edge [math]\displaystyle{ xw }[/math] replacing it. When referring a cut [math]\displaystyle{ C }[/math], we consider [math]\displaystyle{ C }[/math] as a set of edge identities [math]\displaystyle{ e }[/math], so that a cut [math]\displaystyle{ C }[/math] is changed by the algorithm only if some of its edges are removed during contraction.
We first prove some lemma.
Lemma 1 - If [math]\displaystyle{ C }[/math] is a cut in a multi-graph [math]\displaystyle{ G }[/math] and [math]\displaystyle{ e\not\in C }[/math], then [math]\displaystyle{ C }[/math] is still a cut in [math]\displaystyle{ G'=contract(G,e) }[/math].
Proof. It is easy to verify that [math]\displaystyle{ C }[/math] is a cut in [math]\displaystyle{ G'=contract(G,e) }[/math] if none of its edges is lost during the contraction. Since [math]\displaystyle{ C }[/math] is a cut in [math]\displaystyle{ G(V,E) }[/math], there exists a nonempty vertex set [math]\displaystyle{ S\subset V }[/math] and its complement [math]\displaystyle{ \bar{S}=V\setminus S }[/math] such that [math]\displaystyle{ C=\{uv\mid u\in S, v\in\bar{S}\} }[/math]. And if [math]\displaystyle{ e\not\in C }[/math], it must hold that either [math]\displaystyle{ e\in G[S] }[/math] or [math]\displaystyle{ e\in G[\bar{S}] }[/math] where [math]\displaystyle{ G[S] }[/math] and [math]\displaystyle{ G[\bar{S}] }[/math] are the subgraphs induced by [math]\displaystyle{ S }[/math] and [math]\displaystyle{ \bar{S} }[/math] respectively. In both cases none of edges in [math]\displaystyle{ C }[/math] is removed in [math]\displaystyle{ G'=contract(G,e) }[/math].
- [math]\displaystyle{ \square }[/math]
Lemma 2 - The size of min-cut in [math]\displaystyle{ G'=contract(G,e) }[/math] is at least as large as the size of min-cut in [math]\displaystyle{ G }[/math], i.e. contraction never reduces the size of min-cut.
Proof. - Note that every cut in the contracted graph [math]\displaystyle{ G' }[/math] is also a cut in the original graph [math]\displaystyle{ G }[/math].
- [math]\displaystyle{ \square }[/math]
Lemma 3 - If [math]\displaystyle{ C }[/math] is a min-cut in a multi-graph [math]\displaystyle{ G(V,E) }[/math], then [math]\displaystyle{ |E|\ge \frac{|V||C|}{2} }[/math].
Proof. - It must hold that the degree of each vertex [math]\displaystyle{ v\in V }[/math] is at least [math]\displaystyle{ |C| }[/math], or otherwise the set of adjacent edges of [math]\displaystyle{ v }[/math] forms a cut which separates [math]\displaystyle{ v }[/math] from the rest of [math]\displaystyle{ V }[/math] and has size less than [math]\displaystyle{ |C| }[/math], contradicting the assumption that [math]\displaystyle{ |C| }[/math] is a min-cut. And the bound [math]\displaystyle{ |E|\ge \frac{|V||C|}{2} }[/math] follows directly from the fact that every vertex in [math]\displaystyle{ G }[/math] has degree at least [math]\displaystyle{ |C| }[/math].
- [math]\displaystyle{ \square }[/math]
For a multigraph [math]\displaystyle{ G(V, E) }[/math], fixed a minimum cut [math]\displaystyle{ C }[/math] (there might be more than one minimum cuts), we analyze the probability that [math]\displaystyle{ C }[/math] is returned by the above algorithm.
Initially [math]\displaystyle{ |V|=n }[/math]. We say that the min-cut [math]\displaystyle{ C }[/math] "survives" a random contraction if none of the edges in [math]\displaystyle{ C }[/math] is chosen to be contracted. After [math]\displaystyle{ (i-1) }[/math] contractions, denote the current multigraph as [math]\displaystyle{ G_i(V_i, E_i) }[/math]. Supposed that [math]\displaystyle{ C }[/math] survives the first [math]\displaystyle{ (i-1) }[/math] contractions, according to Lemma 1 and 2, [math]\displaystyle{ C }[/math] must be a minimum cut in the current multi-graph [math]\displaystyle{ G_i }[/math]. Then due to Lemma 3, the current edge number is [math]\displaystyle{ |E_i|\ge |V_i||C|/2 }[/math]. Uniformly choosing an edge [math]\displaystyle{ e\in E_i }[/math] to contract, the probability that the [math]\displaystyle{ i }[/math]th contraction contracts an edge in [math]\displaystyle{ C }[/math] is given by:
- [math]\displaystyle{ \begin{align}\Pr_{e\in E_i}[e\in C] &= \frac{|C|}{|E_i|} &\le |C|\cdot\frac{2}{|V_i||C|} &= \frac{2}{|V_i|}.\end{align} }[/math]
Therefore, conditioning on that [math]\displaystyle{ C }[/math] survives the first [math]\displaystyle{ (i-1) }[/math] contractions, the probability that [math]\displaystyle{ C }[/math] survives the [math]\displaystyle{ i }[/math]th contraction is at least [math]\displaystyle{ 1-2/|V_i| }[/math]. Note that [math]\displaystyle{ |V_i|=n-i+1 }[/math], because each contraction decrease the vertex number by 1.
The probability that no edge in the minimum cut [math]\displaystyle{ C }[/math] is ever contracted is:
- [math]\displaystyle{ \begin{align} &\quad\,\prod_{i=1}^{n-2}\Pr[\,C\mbox{ survives all }(n-2)\mbox{ contractions }]\\ &= \prod_{i=1}^{n-2}\Pr[\,C\mbox{ survives the }i\mbox{-th contraction}\mid C\mbox{ survives the first }(i-1)\mbox{-th contractions}]\\ &\ge \prod_{i=1}^{n-2}\left(1-\frac{2}{|V_i|}\right) \\ &= \prod_{i=1}^{n-2}\left(1-\frac{2}{n-i+1}\right)\\ &= \prod_{k=3}^{n}\frac{k-2}{k}\\ &= \frac{2}{n(n-1)}. \end{align} }[/math]
This gives the following theorem.
Theorem - For any multigraph with [math]\displaystyle{ n }[/math] vertices, the RandomContract algorithm returns a minimum cut with probability at least [math]\displaystyle{ \frac{2}{n(n-1)} }[/math].
Run RandomContract independently for [math]\displaystyle{ n(n-1)/2 }[/math] times and return the smallest cut returned. The probability that a minimum cut is found is at least:
- [math]\displaystyle{ \begin{align} 1-\Pr[\mbox{failed every time}] &= 1-\Pr[{RandomContract}\mbox{ fails}]^{n(n-1)/2} \\ &\ge 1- \left(1-\frac{2}{n(n-1)}\right)^{n(n-1)/2} \\ &\ge 1-\frac{1}{e}. \end{align} }[/math]
A constant probability!
A Corollary by the Probabilistic Method
Karger's algorithm and its analysis implies the following combinatorial theorem regarding the number of distinct minimum cuts in a graph.
Corollary - For any graph [math]\displaystyle{ G(V,E) }[/math] of [math]\displaystyle{ n }[/math] vertices, the number of distinct minimum cuts in [math]\displaystyle{ G }[/math] is at most [math]\displaystyle{ \frac{n(n-2)}{2} }[/math].
Proof. For each minimum cut [math]\displaystyle{ C }[/math] in [math]\displaystyle{ G }[/math], we define [math]\displaystyle{ \mathcal{E}_C }[/math] to be the event that RandomContract returns [math]\displaystyle{ C }[/math]. Due to the analysis of RandomContract, [math]\displaystyle{ \Pr[\mathcal{E}_C]\ge \frac{2}{n(n-1)} }[/math]. The events [math]\displaystyle{ \mathcal{E}_C }[/math] are mutually disjoint for distinct [math]\displaystyle{ C }[/math] and the event that RandomContract returns a min-cut is the disjoint union of [math]\displaystyle{ \mathcal{E}_C }[/math] over all min-cut [math]\displaystyle{ C }[/math]. Therefore,
- [math]\displaystyle{ \begin{align} &\Pr[\mbox{ RandomContract returns a min-cut}]\\ = &\sum_{\mbox{min-cut }C\mbox{ in }G}\Pr[\mathcal{E}_C]\\ \ge &\sum_{\mbox{min-cut }C\mbox{ in }G}\frac{2}{n(n-1)}, \end{align} }[/math]
which must be no greater than 1 for a well-defined probability space. This means the total number of min-cut in [math]\displaystyle{ G }[/math] must be no greater than [math]\displaystyle{ \frac{n(n-1)}{2} }[/math].
- [math]\displaystyle{ \square }[/math]
Note that the statement of this theorem has no randomness at all, however the proof involves a randomized algorithm. This is an example of the probabilistic method.
Fast Min-Cut
In the analysis of RandomContract, we have the following observation:
- The probability of success is only getting worse when the graph becomes small.
This motivates us to consider the following alternation to the algorithm: first using random contractions to reduce the number of vertices to a moderately small number, and then recursively finding a min-cut in this smaller instance. This seems just a restatement of exactly what we have been doing. Inspired by the idea of boosting the accuracy via independent repetition, here we apply the recursion on two smaller instances generated independently.
The algorithm obtained in this way is called FastCut. We first define a procedure to randomly contract edges until there are [math]\displaystyle{ t }[/math] number of vertices left.
RandomContract[math]\displaystyle{ (G, t) }[/math] - while [math]\displaystyle{ |V|\gt t }[/math] do
- choose an edge [math]\displaystyle{ uv\in E }[/math] uniformly at random;
- [math]\displaystyle{ G=contract(G,uv) }[/math];
- return [math]\displaystyle{ G }[/math];
- while [math]\displaystyle{ |V|\gt t }[/math] do
The FastCut algorithm is recursively defined as follows.
FastCut[math]\displaystyle{ (G) }[/math] - if [math]\displaystyle{ |V|\le 6 }[/math] then return a mincut by brute force;
- else let [math]\displaystyle{ t=\left\lceil1+|V|/\sqrt{2}\right\rceil }[/math];
- [math]\displaystyle{ G_1=RandomContract(G,t) }[/math];
- [math]\displaystyle{ G_2=RandomContract(G,t) }[/math];
- return the smaller one of [math]\displaystyle{ FastCut(G_1) }[/math] and [math]\displaystyle{ FastCut(G_2) }[/math];
As before, all [math]\displaystyle{ G }[/math] are multigraphs.
Let [math]\displaystyle{ C }[/math] be a min-cut in the original multigraph [math]\displaystyle{ G }[/math]. By the same analysis as in the case of RandomContract, we have
- [math]\displaystyle{ \begin{align} &\Pr[C\text{ survives all contractions in }RandomContract(G,t)]\\ = &\prod_{i=1}^{n-t}\Pr[C\text{ survives the }i\text{-th contraction}\mid C\text{ survives the first }(i-1)\text{-th contractions}]\\ \ge &\prod_{i=1}^{n-t}\left(1-\frac{2}{n-i+1}\right)\\ = &\prod_{k=t+1}^{n}\frac{k-2}{k}\\ = &\frac{t(t-1)}{n(n-1)}. \end{align} }[/math]
When [math]\displaystyle{ t=\left\lceil1+n/\sqrt{2}\right\rceil }[/math], this probability is at least [math]\displaystyle{ 1/2 }[/math].
We use [math]\displaystyle{ p(n) }[/math] to denote the probability that [math]\displaystyle{ C }[/math] is returned by [math]\displaystyle{ FastCut(G) }[/math], where [math]\displaystyle{ G }[/math] is a multigraph of [math]\displaystyle{ n }[/math] vertices. We then have the following recursion for [math]\displaystyle{ p(n) }[/math].
- [math]\displaystyle{ \begin{align} p(n) &= \Pr[C\text{ is returned by }\textit{FastCut}(G)]\\ &= 1-\left(1-\Pr[C\text{ survives in }G_1\wedge C=\textit{FastCut}(G_1)]\right)^2\\ &= 1-\left(1-\Pr[C\text{ survives in }G_1]\Pr[C=\textit{FastCut}(G_1)\mid C\text{ survives in }G_1]\right)^2\\ &\ge 1-\left(1-\frac{1}{2}p\left(\left\lceil1+n/\sqrt{2}\right\rceil\right)\right)^2, \end{align} }[/math]
where the last inequality is due to the fact that [math]\displaystyle{ \Pr[C\text{ survives all contractions in }RandomContract(G,t)]\ge1/2 }[/math] and our previous discussions in the analysis of RandomContract that if the min-cut [math]\displaystyle{ C }[/math] survives all first [math]\displaystyle{ (n-t) }[/math] contractions then [math]\displaystyle{ C }[/math] must be a min-cut in the remaining multigraph.
The base case is that [math]\displaystyle{ p(n)=1 }[/math] for [math]\displaystyle{ n\le 6 }[/math]. Solving this recursion of [math]\displaystyle{ p(n) }[/math] (or proving by induction) gives us that
- [math]\displaystyle{ p(n)=\Omega\left(\frac{1}{\log n}\right). }[/math]
Recall that we can implement an edge contraction in [math]\displaystyle{ O(n) }[/math] time, thus it is easy to verify the following recursion of time complexity:
- [math]\displaystyle{ T(n)=2T\left(\left\lceil1+n/\sqrt{2}\right\rceil\right)+O(n^2), }[/math]
where [math]\displaystyle{ T(n) }[/math] denotes the running time of [math]\displaystyle{ FastCut(G) }[/math] on a multigraph [math]\displaystyle{ G }[/math] of [math]\displaystyle{ n }[/math] vertices.
Solving the recursion of [math]\displaystyle{ T(n) }[/math] with the base case [math]\displaystyle{ T(n)=O(1) }[/math] for [math]\displaystyle{ n\le 6 }[/math], we have [math]\displaystyle{ T(n)=O(n^2\log n) }[/math].
Therefore, for a multigraph [math]\displaystyle{ G }[/math] of [math]\displaystyle{ n }[/math] vertices, the algorithm [math]\displaystyle{ FastCut(G) }[/math] returns a min-cut in [math]\displaystyle{ G }[/math] with probability [math]\displaystyle{ \Omega\left(\frac{1}{\log n}\right) }[/math] in time [math]\displaystyle{ O(n^2\log n) }[/math]. Repeat this independently for [math]\displaystyle{ O(log n) }[/math] times, we have an algorithm which runs in time [math]\displaystyle{ O(n^2\log^2n) }[/math] and returns a min-cut with probability [math]\displaystyle{ 1-O(1/n) }[/math], a high probability.