Randomized Algorithms (Spring 2010)/Tail inequalities

Select the Median

The selection problem is the problem of finding the [math]\displaystyle{ k }[/math]th smallest element in a set [math]\displaystyle{ S }[/math]. A typical case of selection problem is finding the median, the [math]\displaystyle{ (\lceil n/2\rceil) }[/math]th element in the sorted order of [math]\displaystyle{ S }[/math].

The median can be found in [math]\displaystyle{ O(n\log n) }[/math] time by sorting. There is a linear-time deterministic algorithm, "median of medians" algorithm, which is very sophisticated. Here we introduce a much simpler randomized algorithm which also runs in linear time. The idea of this randomized algorithm is by sampling.

Chernoff Bound

Theorem (Chernoff bound): Let [math]\displaystyle{ X_1, X_2, \ldots, X_n }[/math] be independent Poisson trials such that [math]\displaystyle{ \mathbf{E}[X_i]=\Pr[X_i=1]=p_i }[/math]. Let [math]\displaystyle{ X=\sum_{i=1}^n X_i }[/math] and [math]\displaystyle{ \mu=\mathbf{E}[X] }[/math]. Then for any [math]\displaystyle{ \delta\gt 0 }[/math], [math]\displaystyle{ \Pr[X\ge (1+\delta)\mu]\lt \left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}. }[/math]

Corollary: Let [math]\displaystyle{ X_1, X_2, \ldots, X_n }[/math] be independent Poisson trials, [math]\displaystyle{ X=\sum_{i=1}^n X_i }[/math] and [math]\displaystyle{ \mu=\mathbf{E}[X] }[/math]. Then 1. for [math]\displaystyle{ 0\lt \delta\le 1 }[/math], [math]\displaystyle{ \Pr[X\ge (1+\delta)\mu]\lt e^{-\mu\delta^2/3}; }[/math] 2. for [math]\displaystyle{ R\ge 6\mu }[/math], [math]\displaystyle{ \Pr[X\ge R]\le 2^{-R}. }[/math]