概率论与数理统计 (Spring 2023)/Problem Set 3

• 每道题目的解答都要有完整的解题过程，中英文不限。

• 我们推荐大家使用LaTeX, markdown等对作业进行排版。

Assumption throughout Problem Set 3

Without further notice, we are working on probability space [math]\displaystyle{ (\Omega,\mathcal{F},\mathbf{Pr}) }[/math].

Without further notice, we assume that the expectation of random variables are well-defined.

The term [math]\displaystyle{ \log }[/math] used in this context refers to the natural logarithm.

Problem 1 (Warm-up Problems)

• [Variance (I)] Let [math]\displaystyle{ X_1,X_2,\cdots, X_n }[/math] be pairwise independent random variables. Show that [math]\displaystyle{ \textbf{Var}\left[\sum_{i=1}^n X_i\right] =\sum_{i=1}^n \textbf{Var} [X_i] }[/math].
• [Variance (II)] Each member of a group of [math]\displaystyle{ n }[/math] players rolls a (fair) die. For any pair of players who throw the same number, the group scores [math]\displaystyle{ 1 }[/math] point. Find the mean and variance of the total score of the group.
• [Variance (III)] An urn contains [math]\displaystyle{ n }[/math] balls numbered [math]\displaystyle{ 1, 2, \ldots, n }[/math]. We select [math]\displaystyle{ k }[/math] balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
• [Variance (IV)] Let [math]\displaystyle{ N }[/math] be an integer-valued, positive random variable and let [math]\displaystyle{ \{X_i\}_{i=1}^{\infty} }[/math] be indepedently identically distributed random variables that are independent of [math]\displaystyle{ N }[/math], too. Precisely, for any finite subset [math]\displaystyle{ I \subseteq\mathbb{N}_+ }[/math], [math]\displaystyle{ \{X_i\}_{i \in I} }[/math] and [math]\displaystyle{ N }[/math] are mutually independent. Let [math]\displaystyle{ X = \sum_{i=1}^N X_i }[/math], show that [math]\displaystyle{ \textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N] }[/math].
• [Moments (I)] Find an example of a random variable with finite [math]\displaystyle{ j }[/math]-th moments for [math]\displaystyle{ 1 \leq j \leq k }[/math] but an unbounded [math]\displaystyle{ (k + 1) }[/math]-th moment. Give a clear argument showing that your choice has these properties.
• [Moments (II)] Let [math]\displaystyle{ X\sim \text{Geo}(p) }[/math] for some [math]\displaystyle{ p \in (0,1) }[/math]. Find [math]\displaystyle{ \mathbb{E}[X^3] }[/math] and [math]\displaystyle{ \mathbb{E}[X^4] }[/math].
• [Moments (III)] Let [math]\displaystyle{ X\sim \text{Pois}(\lambda) }[/math] for some [math]\displaystyle{ \lambda \gt 0 }[/math]. Find [math]\displaystyle{ \mathbb{E}[X^3] }[/math] and [math]\displaystyle{ \mathbb{E}[X^4] }[/math].
• [Covariance and correlation (I)] Let [math]\displaystyle{ X }[/math] and [math]\displaystyle{ Y }[/math] be discrete random variables with correlation [math]\displaystyle{ \rho }[/math]. Show that [math]\displaystyle{ |\rho|\leq 1 }[/math].
• [Covariance and correlation (II)] Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean [math]\displaystyle{ 0 }[/math], variance [math]\displaystyle{ 1 }[/math], and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
• [Covariance and correlation (III)] Construct two random variables [math]X[/math] and [math]Y[/math] such that their covariance [math]\textbf{Cov}(X,Y) = 0[/math] but [math]X[/math] and [math]Y[/math] are not independent. You should prove your construction is true.

Problem 2 (Inequalities)

• [Reverse Markov's inequality] Let [math]\displaystyle{ X }[/math] be a discrete random variable with bounded range [math]\displaystyle{ 0 \le X \le U }[/math] for some [math]\displaystyle{ U \gt 0 }[/math]. Show that [math]\displaystyle{ \mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a} }[/math] for any [math]\displaystyle{ 0 \lt a \lt U }[/math].
• [Markov's inequality] Let [math]\displaystyle{ X }[/math] be a discrete random variable. Show that for all [math]\displaystyle{ \beta \geq 0 }[/math] and all [math]\displaystyle{ x \gt 0 }[/math], [math]\displaystyle{ \mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x} }[/math].
• [Cantelli's inequality] Let [math]\displaystyle{ X }[/math] be a discrete random variable with mean [math]\displaystyle{ 0 }[/math] and variance [math]\displaystyle{ \sigma^2 }[/math]. Prove that for any [math]\displaystyle{ \lambda \gt 0 }[/math], [math]\displaystyle{ \mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2} }[/math]. (Hint: You may first show that [math]\displaystyle{ \mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2} }[/math] for all [math]\displaystyle{ u \gt 0 }[/math].)
• [The weak law of large numbers] Let [math]\displaystyle{ X_1,X_2,\cdots, X_n }[/math] be independent and identically distributed random variables with mean [math]\displaystyle{ \mu }[/math] and finite variance, use Chebyshev's inequality to show that for any constant [math]\displaystyle{ \epsilon\gt 0 }[/math] we have [math]\displaystyle{ \lim_{n\rightarrow \infty} \mathbf{Pr}\left( \left|\frac{X_1 + X_2 + \cdots + X_n}{n} - \mu\right| \gt \epsilon\right) = 0 }[/math].
• [Median trick] Suppose we want to estimate the value of [math]\displaystyle{ Z }[/math]. Let [math]\displaystyle{ \mathcal{A} }[/math] be a randomized algorithm that outputs [math]\displaystyle{ \widehat{Z} }[/math] satisfying [math]\displaystyle{ \mathbf{Pr}[(1-\epsilon) Z \leq \widehat{Z} \leq (1+\epsilon)Z]\geq \frac{3}{4} }[/math] for some fixed parameter [math]\displaystyle{ \epsilon \gt 0 }[/math]. We run [math]\displaystyle{ \mathcal{A} }[/math] independently for [math]\displaystyle{ 2n-1 }[/math] times, and obtain the outputs [math]\displaystyle{ \widehat{Z}_1, \widehat{Z}_2, \cdots, \widehat{Z}_{2n-1} }[/math]. Let [math]\displaystyle{ X }[/math] be the median (中位数) of [math]\displaystyle{ \widehat{Z}_1, \widehat{Z}_2, \cdots, \widehat{Z}_{2n-1} }[/math]. Use Chebyshev's inequality to show that [math]\displaystyle{ \mathbf{Pr}[(1-\epsilon) Z \leq X \leq (1+\epsilon)Z] = 1 - O(1/n) }[/math]. (Remark: The bound can be drastically improved with Chernoff bound).

Problem 3 (Probability meets graph theory)

• [Common neighbor] Let [math]\displaystyle{ p \in (0,1) }[/math] be a constant. Show that with a probability approaching to [math]\displaystyle{ 1 }[/math] (as [math]\displaystyle{ n }[/math] tends to infinity) the Erdős–Rényi random graph [math]\displaystyle{ \mathbf{G}(n,p) }[/math] has the property that every pair of its vertices has a common neighbor. (Hint: You may use Markov's inequality.)
• [Isolated vertices] Prove that [math]\displaystyle{ p = \log n/n }[/math] is the threshold probability for the disappearance of isolated vertices. Formally, you are required to show that
  1. with a probability approaching to [math]\displaystyle{ 1 }[/math] (as [math]\displaystyle{ n }[/math] tends to infinity) the Erdős–Rényi random graph [math]\displaystyle{ \mathbf{G} = \mathbf{G}(n,p) }[/math] has the property that [math]\displaystyle{ \mathbf{G} }[/math] has no isolated vertices when [math]\displaystyle{ p = \omega(\log n/n) }[/math];
  2. with a probability approaching to [math]\displaystyle{ 0 }[/math] (as [math]\displaystyle{ n }[/math] tends to infinity) the Erdős–Rényi random graph [math]\displaystyle{ \mathbf{G} = \mathbf{G}(n,p) }[/math] has the property that [math]\displaystyle{ \mathbf{G} }[/math] has no isolated vertices when [math]\displaystyle{ p = o(\log n/n) }[/math].