# 高级算法 (Fall 2017)/Problem Set 1

每道题目的解答都要有完整的解题过程。中英文不限。


## Problem 1

Recall that in class we show by the probabilistic method how to deduce a ${\displaystyle {\frac {n(n-1)}{2}}}$ upper bound on the number of distinct min-cuts in any multigraph ${\displaystyle G}$ with ${\displaystyle n}$ vertices from the ${\displaystyle {\frac {2}{n(n-1)}}}$ lower bound for success probability of Karger's min-cut algorithm.

Also recall that the ${\displaystyle FastCut}$ algorithm taught in class guarantees to return a min-cut with probability at least ${\displaystyle \Omega (1/\log n)}$. Does this imply a much tighter ${\displaystyle O(\log n)}$ upper bound on the number of distinct min-cuts in any multigraph ${\displaystyle G}$ with ${\displaystyle n}$ vertices? Prove your improved upper bound if your answer is "yes", and give a satisfactory explanation if your answer is "no".

## Problem 2

Two rooted trees ${\displaystyle T_{1}}$ and ${\displaystyle T_{2}}$ are said to be isomorphic if there exists a bijection ${\displaystyle \phi }$ that maps vertices of ${\displaystyle T_{1}}$ to those of ${\displaystyle T_{2}}$ satisfying the following condition: for each internal vertex ${\displaystyle v}$ of ${\displaystyle T_{1}}$ with children ${\displaystyle u_{1},u_{2},\ldots ,u_{k}}$, the set of children of vertex ${\displaystyle \phi (v)}$ in ${\displaystyle T_{2}}$ is precisely ${\displaystyle \{\phi (u_{1}),\phi (u_{2}),\ldots ,\phi (u_{k})\}}$, no ordering among children assumed.

Give an efficient randomized algorithm with bounded one-sided error (false positive), for testing isomorphism between rooted trees with ${\displaystyle n}$ vertices. Analyze your algorithm.

## Problem 3

Suppose ${\displaystyle x_{1},x_{2},\ldots ,x_{n}}$ is an unsorted list of ${\displaystyle n}$ distinct numbers. We sample (with replacement) ${\displaystyle t}$ items uniformly at random from the list, and denote them as ${\displaystyle Y_{1},Y_{2},\ldots ,Y_{t}}$. Obviously ${\displaystyle \{Y_{1},Y_{2},\ldots ,Y_{t}\}\subseteq \{x_{1},x_{2},\ldots ,x_{n}\}}$.

Describe a strategy of choosing an ${\displaystyle x}$ from the sampled set ${\displaystyle \{Y_{1},Y_{2},\ldots ,Y_{t}\}}$ such that ${\displaystyle \mathrm {rank} (x)}$ is approximately ${\displaystyle k}$. Here ${\displaystyle \mathrm {rank} (x)}$ denotes the rank of ${\displaystyle x}$ in the original list ${\displaystyle \{x_{1},x_{2},\ldots ,x_{n}\}}$: The rank of the largest number among ${\displaystyle x_{1},x_{2},\ldots ,x_{n}}$ is 1; the rank of the second largest number among ${\displaystyle x_{1},x_{2},\ldots ,x_{n}}$ is 2, and so on. Choose your ${\displaystyle t}$ as small as possible (in big-O notation) so that with probability at least ${\displaystyle 1-\delta }$, your strategy returns an ${\displaystyle x}$ such that ${\displaystyle (1-\epsilon )k\leq \mathrm {rank} (x)\leq (1+\epsilon )k}$.

## Problem 4

In Balls-and-Bins model, we throw ${\displaystyle m}$ balls independently and uniformly at random into ${\displaystyle n}$ bins. We know that the maximum load is ${\displaystyle \Theta \left({\frac {\log n}{\log \log n}}\right)}$ with high probability when ${\displaystyle m=\Theta (n)}$. The two-choice paradigm is another way to throw ${\displaystyle m}$ balls into ${\displaystyle n}$ bins: each ball is thrown into the least loaded of two bins chosen independently and uniformly at random(it could be the case that the two chosen bins are exactly the same, and then the ball will be thrown into that bin), and breaks the tie arbitrarily. When ${\displaystyle m=\Theta (n)}$, the maximum load of two-choice paradigm is known to be ${\displaystyle \Theta (\log \log n)}$ with high probability, which is exponentially less than the maxim load when there is only one random choice. This phenomenon is called the power of two choices.

Here are the questions:

• Consider the following paradigm: we throw ${\displaystyle n}$ balls into ${\displaystyle n}$ bins. The first ${\displaystyle {\frac {n}{2}}}$ balls are thrown into bins independently and uniformly at random. The remaining ${\displaystyle {\frac {n}{2}}}$ balls are thrown into bins using the two-choice paradigm. What is the maximum load with high probability? You need to give an asymptotically tight bound (in the form of ${\displaystyle \Theta (\cdot )}$).
• Replace the above paradigm to the following: the first ${\displaystyle {\frac {n}{2}}}$ balls are thrown into bins using the two-choice paradigm while the remaining ${\displaystyle {\frac {n}{2}}}$ balls are thrown into bins independently and uniformly at random. What is the maximum load with high probability in this case? You need to give an asymptotically tight bound.
• Replace the above paradigm to the following: assume all ${\displaystyle n}$ balls are thrown in a sequence. For every ${\displaystyle 1\leq i\leq n}$, if ${\displaystyle i}$ is odd, we throw ${\displaystyle i}$-th ball into bins independently and uniformly at random, otherwise, we throw it into bins using the two-choice paradigm. What is the maximum load with high probability in this case? You need to give an asymptotically tight bound.

## Problem 5

Let ${\displaystyle X}$ be a real-valued random variable with finite ${\displaystyle \mathbb {E} [X]}$ and finite ${\displaystyle \mathbb {E} \left[\mathrm {e} ^{\lambda X}\right]}$ for all ${\displaystyle \lambda \geq 0}$. We define the log-moment-generating function as

${\displaystyle \Psi _{X}(\lambda ):=\ln \mathbb {E} [\mathrm {e} ^{\lambda X}]\quad {\text{ for all }}\lambda \geq 0}$,

and its dual function:

${\displaystyle \Psi _{X}^{*}(t):=\sup _{\lambda \geq 0}(\lambda t-\Psi _{X}(\lambda ))}$.

Assume that ${\displaystyle X}$ is NOT almost surely constant. Then due to the convexity of ${\displaystyle \mathrm {e} ^{\lambda X}}$ with respect to ${\displaystyle \lambda }$, the function ${\displaystyle \Psi _{X}(\lambda )}$ is strictly convex over ${\displaystyle \lambda \geq 0}$.

• Prove the following Chernoff bound:
${\displaystyle \Pr[X\geq t]\leq \exp(-\Psi _{X}^{*}(t))}$.
In particular if ${\displaystyle \Psi _{X}(\lambda )}$ is continuously differentiable, prove that the supreme in ${\displaystyle \Psi _{X}^{*}(t)}$ is achieved at the unique ${\displaystyle \lambda \geq 0}$ satisfying
${\displaystyle \Psi _{X}'(\lambda )=t}$
where ${\displaystyle \Psi _{X}'(\lambda )}$ denotes the derivative of ${\displaystyle \Psi _{X}(\lambda )}$ with respect to ${\displaystyle \lambda }$.
• Normal random variables. Let ${\displaystyle X\sim \mathrm {N} (\mu ,\sigma )}$ be a Gaussian random variable with mean ${\displaystyle \mu }$ and standard deviation ${\displaystyle \sigma }$. What are the ${\displaystyle \Psi _{X}(\lambda )}$ and ${\displaystyle \Psi _{X}^{*}(t)}$? And give a tail inequality to upper bound the probability ${\displaystyle \Pr[X\geq t]}$.
• Poisson random variables. Let ${\displaystyle X\sim \mathrm {Pois} (\nu )}$ be a Poisson random variable with parameter ${\displaystyle \nu }$, that is, ${\displaystyle \Pr[X=k]=\mathrm {e} ^{-\nu }\nu ^{k}/k!}$ for all ${\displaystyle k=0,1,2,\ldots }$. What are the ${\displaystyle \Psi _{X}(\lambda )}$ and ${\displaystyle \Psi _{X}^{*}(t)}$? And give a tail inequality to upper bound the probability ${\displaystyle \Pr[X\geq t]}$.
• Bernoulli random variables. Let ${\displaystyle X\in \{0,1\}}$ be a single Bernoulli trial with probability of success ${\displaystyle p}$, that is, ${\displaystyle \Pr[X=1]=1-\Pr[X=0]=p}$. Show that for any ${\displaystyle t\in (p,1)}$, we have ${\displaystyle \Psi _{X}^{*}(t)=D(Y\|X)}$ where ${\displaystyle Y\in \{0,1\}}$ is a Bernoulli random variable with parameter ${\displaystyle t}$ and ${\displaystyle D(Y\|X)=(1-t)\ln {\frac {1-t}{1-p}}+t\ln {\frac {t}{p}}}$ is the Kullback-Leibler divergence between ${\displaystyle Y}$ and ${\displaystyle X}$.
• Sum of independent random variables. Let ${\displaystyle X=\sum _{i=1}^{n}X_{i}}$ be the sum of ${\displaystyle n}$ independently and identically distributed random variables ${\displaystyle X_{1},X_{2},\ldots ,X_{n}}$. Show that ${\displaystyle \Psi _{X}(\lambda )=\sum _{i=1}^{n}\Psi _{X_{i}}(\lambda )}$ and ${\displaystyle \Psi _{X}^{*}(t)=n\Psi _{X_{i}}^{*}({\frac {t}{n}})}$. Also for binomial random variable ${\displaystyle X\sim \mathrm {Bin} (n,p)}$, give an upper bound to the tail inequality ${\displaystyle \Pr[X\geq t]}$ in terms of KL-divergence.
Give an upper bound to ${\displaystyle \Pr[X\geq t]}$ when every ${\displaystyle X_{i}}$ follows the geometric distribution with a probability ${\displaystyle p}$ of success.

## Problem 6

A boolean code is a mapping ${\displaystyle C:\{0,1\}^{k}\rightarrow \{0,1\}^{n}}$. Each ${\displaystyle x\in \{0,1\}^{k}}$ is called a message and ${\displaystyle y=C(x)}$ is called a codeword. The code rate ${\displaystyle r}$ of a code ${\displaystyle C}$ is ${\displaystyle r={\frac {k}{n}}}$. A boolean code ${\displaystyle C:\{0,1\}^{k}\rightarrow \{0,1\}^{n}}$ is a linear code if it is a linear transformation, i.e. there is a matrix ${\displaystyle A\in \{0,1\}^{n\times k}}$ such that ${\displaystyle C(x)=Ax}$ for any ${\displaystyle x\in \{0,1\}^{k}}$, where the additions and multiplications are defined over the finite field of order two, ${\displaystyle (\{0,1\},+_{\bmod {2}},\times _{\bmod {2}})}$.

The distance between two codeword ${\displaystyle y_{1}}$ and ${\displaystyle y_{2}}$, denoted by ${\displaystyle d(y_{1},y_{2})}$, is defined as the Hamming distance between them. Formally, ${\displaystyle d(y_{1},y_{2})=\|y_{1}-y_{2}\|_{1}=\sum _{i=1}^{n}|y_{1}(i)-y_{2}(i)|}$. The distance of a code ${\displaystyle C}$ is the minimum distance between any two codewords. Formally, ${\displaystyle d=\min _{x_{1},x_{2}\in \{0,1\}^{k} \atop x_{1}\neq x_{2}}d(C(x_{1}),C(x_{2}))}$.

Usually we want to make both the code rate ${\displaystyle r}$ and the code distance ${\displaystyle d}$ as large as possible, because a larger rate means that the amount of actual message per transmitted bit is high, and a larger distance allows for more error correction and detection.

• Use the probabilistic method to prove that there exists a boolean code ${\displaystyle C:\{0,1\}^{k}\rightarrow \{0,1\}^{n}}$ of code rate ${\displaystyle r}$ and distance ${\displaystyle \left({\frac {1}{2}}-\Theta \left({\sqrt {r}}\right)\right)n}$. Try to optimize the constant in ${\displaystyle \Theta (\cdot )}$.
• Prove a similar result for linear boolean codes.

## Bonus problem

Let ${\displaystyle X}$ be a centralized random variable (${\displaystyle \mathbb {E} [X]=0}$) with finite ${\displaystyle \mathbb {E} \left[\mathrm {e} ^{\lambda |X|}\right]}$ for ${\displaystyle \lambda >0}$. We have the following two kinds of tail inequalities.

Chernoff bound:
${\displaystyle \Pr[|X|\geq t]\leq \inf _{\lambda \geq 0}{\frac {\mathbb {E} \left[\mathrm {e} ^{\lambda |X|}\right]}{\mathrm {e} ^{\lambda t}}}}$.
${\displaystyle k}$-th moment bound:
${\displaystyle \Pr[|X|\geq t]\leq {\frac {\mathbb {E} \left[|X|^{k}\right]}{t^{k}}}}$.
• Use the probabilistic method to show that for any ${\displaystyle t>0}$, there exists a choice of ${\displaystyle k}$ such that the ${\displaystyle k}$-th moment bound is strictly stronger than the Chernoff bound.
• Why would we still prefer the Chernoff bound to the seemingly stronger ${\displaystyle k}$-th moment method?