TCS Wiki - User contributions [en]

概率论与数理统计 (Spring 2026)

2026-04-23T09:47:02Z

Yqzhu: /* Assignments */

{{Infobox
|name = Infobox
|bodystyle =
|title = '''概率论与数理统计''' 
'''Probability Theory''' & '''Mathematical Statistics'''
|titlestyle =

|image =
|imagestyle =
|caption =
|captionstyle =
|headerstyle = background:#ccf;
|labelstyle = background:#ddf;
|datastyle =

|header1 =Instructor
|label1 =
|data1 =
|header2 =
|label2 =
|data2 = '''尹一通'''
|header3 =
|label3 = Email
|data3 = yinyt@nju.edu.cn
|header4 =
|label4 = office
|data4 = 计算机系 804
|header5 =
|label5 =
|data5 = '''刘景铖'''
|header6 =
|label6 = Email
|data6 = liu@nju.edu.cn
|header7 =
|label7 = office
|data7 = 计算机系 516
|header8 = Class
|label8 =
|data8 =
|header9 =
|label9 = Class meeting
|data9 = Wednesday, 9am-12am 
仙Ⅱ-212
|header10=
|label10 = Office hour
|data10 = TBA 计算机系 804（尹一通） 计算机系 516（刘景铖）
|header11= Textbook
|label11 =
|data11 =
|header12=
|label12 =
|data12 = [[File:概率导论.jpeg|border|100px]]
|header13=
|label13 =
|data13 = '''概率导论'''（第2版·修订版） Dimitri P. Bertsekas and John N. Tsitsiklis 郑忠国童行伟译；人民邮电出版社 (2022)
|header14=
|label14 =
|data14 = [[File:Grimmett_probability.jpg|border|100px]]
|header15=
|label15 =
|data15 = '''Probability and Random Processes''' (4E) Geoffrey Grimmett and David Stirzaker Oxford University Press (2020)
|header16=
|label16 =
|data16 = [[File:Probability_and_Computing_2ed.jpg|border|100px]]
|header17=
|label17 =
|data17 = '''Probability and Computing''' (2E) Michael Mitzenmacher and Eli Upfal Cambridge University Press (2017)
|belowstyle = background:#ddf;
|below =
}}

This is the webpage for the ''Probability Theory and Mathematical Statistics'' (概率论与数理统计) class of Spring 2026. Students who take this class should check this page periodically for content updates and new announcements.

= Announcement =
* TBA

= Course info =
* '''Instructor ''':
:* [http://tcs.nju.edu.cn/yinyt/ 尹一通]：[mailto:yinyt@nju.edu.cn <yinyt@nju.edu.cn>]，计算机系 804
:* [https://liuexp.github.io 刘景铖]：[mailto:liu@nju.edu.cn <liu@nju.edu.cn>]，计算机系 516
* '''Teaching assistant''':
** 鞠哲：[mailto:juzhe@smail.nju.edu.cn <juzhe@smail.nju.edu.cn>]，计算机系 426
** 祝永祺：[mailto:652025330045@smail.nju.edu.cn <652025330045@smail.nju.edu.cn>]，计算机系 426
* '''Class meeting''':
** 周三：9am-12am，仙Ⅱ-212
* '''Office hour''':
:* TBA, 计算机系 804（尹一通）
:* TBA, 计算机系 516（刘景铖）
:* '''QQ群''': 1090092561（申请加入需提供姓名、院系、学号）

= Syllabus =
课程内容分为三大部分：
* '''经典概率论'''：概率空间、随机变量及其数字特征、多维与连续随机变量、极限定理等内容
* '''概率与计算'''：测度集中现象 (concentration of measure)、概率法 (the probabilistic method)、离散随机过程的相关专题
* '''数理统计'''：参数估计、假设检验、贝叶斯估计、线性回归等统计推断等概念

对于第一和第二部分，要求清楚掌握基本概念，深刻理解关键的现象与规律以及背后的原理，并可以灵活运用所学方法求解相关问题。对于第三部分，要求熟悉数理统计的若干基本概念，以及典型的统计模型和统计推断问题。

经过本课程的训练，力求使学生能够熟悉掌握概率的语言，并会利用概率思维来理解客观世界并对其建模，以及驾驭概率的数学工具来分析和求解专业问题。

=== 教材与参考书 Course Materials ===
* '''[BT]''' 概率导论（第2版·修订版），[美]伯特瑟卡斯（Dimitri P.Bertsekas）[美]齐齐克利斯（John N.Tsitsiklis）著，郑忠国童行伟译，人民邮电出版社（2022）。
* '''[GS]''' ''Probability and Random Processes'', by Geoffrey Grimmett and David Stirzaker; Oxford University Press; 4th edition (2020).
* '''[MU]''' ''Probability and Computing: Randomization and Probabilistic Techniques in Algorithms and Data Analysis'', by Michael Mitzenmacher, Eli Upfal; Cambridge University Press; 2nd edition (2017).

=== 成绩 Grading Policy ===
* 课程成绩：本课程将会有若干次作业和一次期末考试。最终成绩将由平时作业成绩和期末考试成绩综合得出。
* 迟交：如果有特殊的理由，无法按时完成作业，请提前联系授课老师，给出正当理由。否则迟交的作业将不被接受。

=== 学术诚信 Academic Integrity ===
学术诚信是所有从事学术活动的学生和学者最基本的职业道德底线，本课程将不遗余力的维护学术诚信规范，违反这一底线的行为将不会被容忍。

作业完成的原则：署你名字的工作必须是你个人的贡献。在完成作业的过程中，允许讨论，前提是讨论的所有参与者均处于同等完成度。但关键想法的执行、以及作业文本的写作必须独立完成，并在作业中致谢（acknowledge）所有参与讨论的人。符合规则的讨论与致谢将不会影响得分。不允许其他任何形式的合作——尤其是与已经完成作业的同学“讨论”。

本课程将对剽窃行为采取零容忍的态度。在完成作业过程中，对他人工作（出版物、互联网资料、其他人的作业等）直接的文本抄袭和对关键思想、关键元素的抄袭，按照 [http://www.acm.org/publications/policies/plagiarism_policy ACM Policy on Plagiarism]的解释，都将视为剽窃。剽窃者成绩将被取消。如果发现互相抄袭行为， 抄袭和被抄袭双方的成绩都将被取消。因此请主动防止自己的作业被他人抄袭。

学术诚信影响学生个人的品行，也关乎整个教育系统的正常运转。为了一点分数而做出学术不端的行为，不仅使自己沦为一个欺骗者，也使他人的诚实努力失去意义。让我们一起努力维护一个诚信的环境。

= Assignments =
*[[概率论与数理统计 (Spring 2026)/Problem Set 1|Problem Set 1]] 请在 2026/4/1 上课之前(9am UTC+8)提交到 [mailto:pr2026_nju@163.com pr2026_nju@163.com] (文件名为'学号_姓名_A1.pdf').
** [[概率论与数理统计 (Spring 2026)/第一次作业提交名单|第一次作业提交名单]]

*[[概率论与数理统计 (Spring 2026)/Problem Set 2|Problem Set 2]] 请在 2026/4/22 上课之前(9am UTC+8)提交到 [mailto:pr2026_nju@163.com pr2026_nju@163.com] (文件名为'学号_姓名_A2.pdf').
** [[概率论与数理统计 (Spring 2026)/第二次作业提交名单|第二次作业提交名单]]

*[[概率论与数理统计 (Spring 2026)/Problem Set 3|Problem Set 3]] 请在 2026/5/13 上课之前(9am UTC+8)提交到 [mailto:pr2026_nju@163.com pr2026_nju@163.com] (文件名为'学号_姓名_A3.pdf').

= Lectures =
# [http://tcs.nju.edu.cn/slides/prob2026/Intro.pdf 课程简介]
# [http://tcs.nju.edu.cn/slides/prob2026/ProbSpace.pdf 概率空间]
#* 阅读：'''[BT] 第1章''' 或 '''[GS] Chapter 1'''
#* [[概率论与数理统计 (Spring 2026)/Entropy and volume of Hamming balls|Entropy and volume of Hamming balls]]
#* [[概率论与数理统计 (Spring 2026)/Karger's min-cut algorithm| Karger's min-cut algorithm]]
# [http://tcs.nju.edu.cn/slides/prob2026/RandVar.pdf 随机变量]
#* 阅读：'''[BT] 第2章''' 或 '''[GS] Chapter 2, Sections 3.1~3.5, 3.7'''
#* 阅读：'''[MU] Chapter 2'''
#* [[概率论与数理统计 (Spring 2026)/Average-case analysis of QuickSort|Average-case analysis of '''''QuickSort''''']]
# [http://tcs.nju.edu.cn/slides/prob2026/Deviation.pdf 矩与偏差]
#* 阅读：'''[MU] Chapter 3'''
#* 阅读：'''[BT] 章节 2.4, 4.2, 4.3, 5.1''' 或 '''[GS] Sections 3.3, 3.6, 7.3'''
#* [[概率论与数理统计 (Spring 2026)/Threshold of k-clique in random graph|Threshold of <math>k</math>-clique in random graph]]
#* [[概率论与数理统计 (Spring 2026)/Weierstrass Approximation Theorem|Weierstrass approximation]]

= Concepts =
* [https://plato.stanford.edu/entries/probability-interpret/ Interpretations of probability]
* [https://en.wikipedia.org/wiki/History_of_probability History of probability]
* Example problems:
** [https://dornsifecms.usc.edu/assets/sites/520/docs/VonNeumann-ams12p36-38.pdf von Neumann's Bernoulli factory] and other [https://peteroupc.github.io/bernoulli.html Bernoulli factory algorithms]
** [https://en.wikipedia.org/wiki/Boy_or_Girl_paradox Boy or Girl paradox]
** [https://en.wikipedia.org/wiki/Monty_Hall_problem Monty Hall problem]
** [https://en.wikipedia.org/wiki/Bertrand_paradox_(probability) Bertrand paradox]
** [https://en.wikipedia.org/wiki/Hard_spheres Hard spheres model] and [https://en.wikipedia.org/wiki/Ising_model Ising model]
** [https://en.wikipedia.org/wiki/PageRank ''PageRank''] and stationary [https://en.wikipedia.org/wiki/Random_walk random walk]
** [https://en.wikipedia.org/wiki/Diffusion_process Diffusion process] and [https://en.wikipedia.org/wiki/Diffusion_model diffusion model]
*[https://en.wikipedia.org/wiki/Probability_space Probability space]
** [https://en.wikipedia.org/wiki/Sample_space Sample space]
** [https://en.wikipedia.org/wiki/Event_(probability_theory) Event] and [https://en.wikipedia.org/wiki/Σ-algebra <math>\sigma</math>-algebra]
** Kolmogorov's [https://en.wikipedia.org/wiki/Probability_axioms axioms of probability]
* [https://en.wikipedia.org/wiki/Discrete_uniform_distribution Classical] and [https://en.wikipedia.org/wiki/Geometric_probability goemetric probability]
* [https://en.wikipedia.org/wiki/Boole%27s_inequality Union bound]
** [https://en.wikipedia.org/wiki/Inclusion%E2%80%93exclusion_principle Inclusion-Exclusion principle]
** [https://en.wikipedia.org/wiki/Boole%27s_inequality#Bonferroni_inequalities Bonferroni inequalities]
* [https://en.wikipedia.org/wiki/Conditional_probability Conditional probability]
** [https://en.wikipedia.org/wiki/Chain_rule_(probability) Chain rule]
** [https://en.wikipedia.org/wiki/Law_of_total_probability Law of total probability]
** [https://en.wikipedia.org/wiki/Bayes%27_theorem Bayes' law]
* [https://en.wikipedia.org/wiki/Independence_(probability_theory) Independence]
** [https://en.wikipedia.org/wiki/Pairwise_independence Pairwise independence]
* [https://en.wikipedia.org/wiki/Random_variable Random variable]
** [https://en.wikipedia.org/wiki/Cumulative_distribution_function Cumulative distribution function]
** [https://en.wikipedia.org/wiki/Probability_mass_function Probability mass function]
** [https://en.wikipedia.org/wiki/Probability_density_function Probability density function]
* [https://en.wikipedia.org/wiki/Multivariate_random_variable Random vector]
** [https://en.wikipedia.org/wiki/Joint_probability_distribution Joint probability distribution]
** [https://en.wikipedia.org/wiki/Conditional_probability_distribution Conditional probability distribution]
** [https://en.wikipedia.org/wiki/Marginal_distribution Marginal distribution]
* Some '''discrete''' probability distributions
** [https://en.wikipedia.org/wiki/Bernoulli_trial Bernoulli trial] and [https://en.wikipedia.org/wiki/Bernoulli_distribution Bernoulli distribution]
** [https://en.wikipedia.org/wiki/Discrete_uniform_distribution Discrete uniform distribution]
** [https://en.wikipedia.org/wiki/Binomial_distribution Binomial distribution]
** [https://en.wikipedia.org/wiki/Geometric_distribution Geometric distribution]
** [https://en.wikipedia.org/wiki/Negative_binomial_distribution Negative binomial distribution]
** [https://en.wikipedia.org/wiki/Hypergeometric_distribution Hypergeometric distribution]
** [https://en.wikipedia.org/wiki/Poisson_distribution Poisson distribution]
** and [https://en.wikipedia.org/wiki/List_of_probability_distributions#Discrete_distributions others]
* Balls into bins model
** [https://en.wikipedia.org/wiki/Multinomial_distribution Multinomial distribution]
** [https://en.wikipedia.org/wiki/Birthday_problem Birthday problem]
** [https://en.wikipedia.org/wiki/Coupon_collector%27s_problem Coupon collector]
** [https://en.wikipedia.org/wiki/Balls_into_bins_problem Occupancy problem]
* Random graphs
** [https://en.wikipedia.org/wiki/Erd%C5%91s%E2%80%93R%C3%A9nyi_model Erdős–Rényi random graph model]
** [https://en.wikipedia.org/wiki/Galton%E2%80%93Watson_process Galton–Watson branching process]
* [https://en.wikipedia.org/wiki/Expected_value Expectation]
** [https://en.wikipedia.org/wiki/Law_of_the_unconscious_statistician Law of the unconscious statistician, ''LOTUS'']
** [https://dlsun.github.io/probability/linearity.html Linearity of expectation]
** [https://en.wikipedia.org/wiki/Conditional_expectation Conditional expectation]
** [https://en.wikipedia.org/wiki/Law_of_total_expectation Law of total expectation]

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T11:03:32Z

Yqzhu: /* Problem 4 (Chernoff bound vs k-th moment method) */

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
<li>
[Chebyshev's inequality] Fix <math>0 . Construct a random variable <math>X</math> with <math>\mathbb{E}[X^2] = b^2</math> for which <math>\mathbf{Pr}(|X| \ge a) = b^2/a^2</math>.
</li>
</li>
<li>
[Chernoff bound] Let <math>X_1,...,X_n</math> be independent Poisson trials. Let <math>X=\sum \limits_{i=1}^n X_i</math> and <math>\mu=\mathbb{E}[X]</math>. Prove that for any <math>\delta>0</math>,
::<center><math>\mathbf{Pr}[X\ge (1+\delta)\mu]\le\left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}.</math></center>

</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

== Problem 4 (Chernoff bound vs k-th moment method) ==
<ul>

Let <math>X</math> be a random variable such that the moment generating function <math>\mathbf{E}[e^{t|X|}]</math> is finite for some <math>t > 0</math>.

We can use the following two kinds of tail inequalities for <math>X</math>.

 
'''Chernoff Bound:'''
<center><math>\mathbf{Pr}[|X| \ge \delta] \le \min_{t \ge 0} \frac{\mathbf{E}[e^{t|X|}]}{e^{t\delta}}.</math></center>

'''kth-Moment Bound:'''
<center><math>\mathbf{Pr}[|X| \ge \delta] \le \frac{\mathbf{E}[|X|^k]}{\delta^k}.</math></center>

(a) Prove that for each <math>\delta>0</math>, there is a choice of <math>k \in \mathbb{Z}_{\ge 0}</math> such that the <math>k</math>-th moment bound is at least as strong as the Chernoff bound.

(Hint: Consider the Taylor expansion of the moment generating function.)
 
(b) Why do we still prefer to use the Chernoff bound rather than the <math>k</math>-th moment bound in algorithmic analysis?

</ul>

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T11:01:35Z

Yqzhu: /* Problem 4 (Chernoff bound vs k-th moment method) */

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
<li>
[Chebyshev's inequality] Fix <math>0 . Construct a random variable <math>X</math> with <math>\mathbb{E}[X^2] = b^2</math> for which <math>\mathbf{Pr}(|X| \ge a) = b^2/a^2</math>.
</li>
</li>
<li>
[Chernoff bound] Let <math>X_1,...,X_n</math> be independent Poisson trials. Let <math>X=\sum \limits_{i=1}^n X_i</math> and <math>\mu=\mathbb{E}[X]</math>. Prove that for any <math>\delta>0</math>,
::<center><math>\mathbf{Pr}[X\ge (1+\delta)\mu]\le\left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}.</math></center>

</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

== Problem 4 (Chernoff bound vs k-th moment method) ==
<ul>

Let <math>X</math> be a random variable such that the moment generating function <math>\mathbf{E}[e^{t|X|}]</math> is finite for some <math>t > 0</math>.

We can use the following two kinds of tail inequalities for <math>X</math>.

 
'''Chernoff Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \min_{t \ge 0} \frac{\mathbf{E}[e^{t|X|}]}{e^{t\delta}}.</math>

'''kth-Moment Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \frac{\mathbf{E}[|X|^k]}{\delta^k}.</math>

(a) Prove that for each <math>\delta>0</math>, there is a choice of <math>k \in \mathbb{Z}^{+}</math> such that the <math>k</math>-th moment bound is at least as strong as the Chernoff bound.

(Hint: Consider the Taylor expansion of the moment generating function.)
 
(b) Why do we still prefer to use the Chernoff bound rather than the <math>k</math>-th moment bound in algorithmic analysis?

</ul>

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T10:58:56Z

Yqzhu:

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
<li>
[Chebyshev's inequality] Fix <math>0 . Construct a random variable <math>X</math> with <math>\mathbb{E}[X^2] = b^2</math> for which <math>\mathbf{Pr}(|X| \ge a) = b^2/a^2</math>.
</li>
</li>
<li>
[Chernoff bound] Let <math>X_1,...,X_n</math> be independent Poisson trials. Let <math>X=\sum \limits_{i=1}^n X_i</math> and <math>\mu=\mathbb{E}[X]</math>. Prove that for any <math>\delta>0</math>,
::<center><math>\mathbf{Pr}[X\ge (1+\delta)\mu]\le\left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}.</math></center>

</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

== Problem 4 (Chernoff bound vs k-th moment method) ==
<ul>

Let <math>X</math> be a random variable such that the moment generating function <math>\mathbf{E}[e^{t|X|}]</math> is finite for some <math>t > 0</math>.

We can use the following two kinds of tail inequalities for <math>X</math>.

 
'''Chernoff Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \min_{t \ge 0} \frac{\mathbf{E}[e^{t|X|}]}{e^{t\delta}}.</math>

'''kth-Moment Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \frac{\mathbf{E}[|X|^k]}{\delta^k}.</math>

(a) Prove that for each <math>\delta>0</math>, there is a choice of <math>k \in \mathbb{Z}_{\ge 0}</math> such that the <math>k</math>-th moment bound is at least as strong as the Chernoff bound.

(Hint: Consider the Taylor expansion of the moment generating function.)
 
(b) Why do we still prefer to use the Chernoff bound rather than the <math>k</math>-th moment bound in algorithmic analysis?

</ul>

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T10:54:54Z

Yqzhu: /* Problem 4 (Chernoff bound vs k-th moment method) */

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
<li>
[Chebyshev's inequality] Fix <math>0 . Construct a random variable <math>X</math> with <math>\mathbb{E}[X^2] = b^2</math> for which <math>\mathbf{Pr}(|X| \ge a) = b^2/a^2</math>.
</li>
</li>
<li>
[Chernoff bound] Let <math>X_1,...,X_n</math> be independent Poisson trials. Let <math>X=\sum \limits_{i=1}^n X_i</math> and <math>\mu=\mathbb{E}[X]</math>. Prove that for any <math>\delta>0</math>,
::<center><math>\mathbf{Pr}[X\ge (1+\delta)\mu]\le\left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}.</math></center>

</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

== Problem 4 (Chernoff bound vs k-th moment method) ==
<ul>

Let <math>X</math> be a random variable such that the moment generating function <math>\mathbf{E}[e^{t|X|}]</math> is finite for some <math>t > 0</math>.

We can use the following two kinds of tail inequalities for <math>X</math>.

 
'''Chernoff Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \min_{t \ge 0} \frac{\mathbf{E}[e^{t|X|}]}{e^{t\delta}}.</math>

'''kth-Moment Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \frac{\mathbf{E}[|X|^k]}{\delta^k}.</math>

(a) Prove that for each <math>\delta>0</math>, there is a choice of <math>k \in \mathbb{Z}_{\ge 0}</math> such that the <math>k</math>-th moment bound is stronger than the Chernoff bound.

(Hint: Consider the Taylor expansion of the moment generating function.)
 
(b) Why do we still prefer to use the Chernoff bound rather than the <math>k</math>-th moment bound in algorithmic analysis?

</ul>

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T10:51:59Z

Yqzhu: /* Problem 4 (Chernoff bound vs k-th moment method) */

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
<li>
[Chebyshev's inequality] Fix <math>0 . Construct a random variable <math>X</math> with <math>\mathbb{E}[X^2] = b^2</math> for which <math>\mathbf{Pr}(|X| \ge a) = b^2/a^2</math>.
</li>
</li>
<li>
[Chernoff bound] Let <math>X_1,...,X_n</math> be independent Poisson trials. Let <math>X=\sum \limits_{i=1}^n X_i</math> and <math>\mu=\mathbb{E}[X]</math>. Prove that for any <math>\delta>0</math>,
::<center><math>\mathbf{Pr}[X\ge (1+\delta)\mu]\le\left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}.</math></center>

</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

== Problem 4 (Chernoff bound vs k-th moment method) ==
<ul>

Let <math>X</math> be a random variable such that the moment generating function <math>\mathbf{E}[e^{t|X|}]</math> is finite for some <math>t > 0</math>.

We can use the following two kinds of tail inequalities for <math>X</math>.

 
'''Chernoff Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \min_{t \ge 0} \frac{\mathbf{E}[e^{t|X|}]}{e^{t\delta}}.</math>

'''kth-Moment Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \frac{\mathbf{E}[|X|^k]}{\delta^k}.</math>

(a) Prove that for each <math>\delta>0</math> and every fixed admissible <math>t</math>, there is a choice of <math>k \in \mathbb{Z}_{\ge 0}</math> such that the <math>k</math>-th moment bound is stronger than the Chernoff bound obtained from this fixed <math>t</math>.

(Hint: Consider the Taylor expansion of the moment generating function.)
 
(b) Why do we still prefer to use the Chernoff bound rather than the <math>k</math>-th moment bound in algorithmic analysis?

</ul>

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T10:51:20Z

Yqzhu: /* Problem 4 (Chernoff bound vs k-th moment method) */

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
<li>
[Chebyshev's inequality] Fix <math>0 . Construct a random variable <math>X</math> with <math>\mathbb{E}[X^2] = b^2</math> for which <math>\mathbf{Pr}(|X| \ge a) = b^2/a^2</math>.
</li>
</li>
<li>
[Chernoff bound] Let <math>X_1,...,X_n</math> be independent Poisson trials. Let <math>X=\sum \limits_{i=1}^n X_i</math> and <math>\mu=\mathbb{E}[X]</math>. Prove that for any <math>\delta>0</math>,
::<center><math>\mathbf{Pr}[X\ge (1+\delta)\mu]\le\left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}.</math></center>

</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

== Problem 4 (Chernoff bound vs k-th moment method) ==
<ul>

Let <math>X</math> be a random variable such that the moment generating function <math>\mathbf{E}[e^{t|X|}]</math> is finite for some <math>t > 0</math>.

We can use the following two kinds of tail inequalities for <math>X</math>.

 
'''Chernoff Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \min_{t \ge 0} \frac{\mathbf{E}[e^{t|X|}]}{e^{t\delta}}.</math>

'''kth-Moment Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \frac{\mathbf{E}[|X|^k]}{\delta^k}.</math>

(a) Prove that for each <math>\delta>0</math> and every fixed admissible <math>t</math>, there is a choice of <math>k \in \mathbb{Z}_{\ge 0}</math> such that the <math>k</math>-th moment bound is stronger than the Chernoff bound from this fixed <math>t</math>.

(Hint: Consider the Taylor expansion of the moment generating function.)
 
(b) Why do we still prefer to use the Chernoff bound rather than the <math>k</math>-th moment bound in algorithmic analysis?

</ul>

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T10:48:36Z

Yqzhu: /* Problem 4 (Chernoff bound vs k-th moment method) */

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
<li>
[Chebyshev's inequality] Fix <math>0 . Construct a random variable <math>X</math> with <math>\mathbb{E}[X^2] = b^2</math> for which <math>\mathbf{Pr}(|X| \ge a) = b^2/a^2</math>.
</li>
</li>
<li>
[Chernoff bound] Let <math>X_1,...,X_n</math> be independent Poisson trials. Let <math>X=\sum \limits_{i=1}^n X_i</math> and <math>\mu=\mathbb{E}[X]</math>. Prove that for any <math>\delta>0</math>,
::<center><math>\mathbf{Pr}[X\ge (1+\delta)\mu]\le\left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}.</math></center>

</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

== Problem 4 (Chernoff bound vs k-th moment method) ==
<ul>

Let <math>X</math> be a random variable such that the moment generating function <math>\mathbf{E}[e^{t|X|}]</math> is finite for some <math>t > 0</math>.

We can use the following two kinds of tail inequalities for <math>X</math>.

 
'''Chernoff Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \min_{t \ge 0} \frac{\mathbf{E}[e^{t|X|}]}{e^{t\delta}}.</math>

'''kth-Moment Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \frac{\mathbf{E}[|X|^k]}{\delta^k}.</math>

(a) Prove that for each <math>\delta>0</math> and every fixed admissible <math>t</math>, there is a choice of <math>k \in \mathbb{Z}^+</math> such that the <math>k</math>-th moment bound is stronger than the Chernoff bound.

(Hint: Consider the Taylor expansion of the moment generating function.)
 
(b) Why do we still prefer to use the Chernoff bound rather than the <math>k</math>-th moment bound in algorithmic analysis?

</ul>

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T10:45:54Z

Yqzhu: /* Problem 4 (Chernoff bound vs k-th moment method) */

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
<li>
[Chebyshev's inequality] Fix <math>0 . Construct a random variable <math>X</math> with <math>\mathbb{E}[X^2] = b^2</math> for which <math>\mathbf{Pr}(|X| \ge a) = b^2/a^2</math>.
</li>
</li>
<li>
[Chernoff bound] Let <math>X_1,...,X_n</math> be independent Poisson trials. Let <math>X=\sum \limits_{i=1}^n X_i</math> and <math>\mu=\mathbb{E}[X]</math>. Prove that for any <math>\delta>0</math>,
::<center><math>\mathbf{Pr}[X\ge (1+\delta)\mu]\le\left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}.</math></center>

</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

== Problem 4 (Chernoff bound vs k-th moment method) ==
<ul>

Let <math>X</math> be a random variable such that the moment generating function <math>\mathbf{E}[e^{t|X|}]</math> is finite for some <math>t > 0</math>.

We can use the following two kinds of tail inequalities for <math>X</math>.

 
'''Chernoff Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \min_{t \ge 0} \frac{\mathbf{E}[e^{t|X|}]}{e^{t\delta}}.</math>

'''kth-Moment Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \frac{\mathbf{E}[|X|^k]}{\delta^k}.</math>

(a) Prove that for each <math>\delta>0</math>, there is a choice of <math>k \in \mathbb{Z}^+</math> such that the <math>k</math>-th moment bound is stronger than the Chernoff bound.

(Hint: Consider the Taylor expansion of the moment generating function.)
 
(b) Why do we still prefer to use the Chernoff bound rather than the <math>k</math>-th moment bound in algorithmic analysis?

</ul>

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T10:37:06Z

Yqzhu: /* Problem 4 (Chernoff bound vs k-th moment method) */

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
<li>
[Chebyshev's inequality] Fix <math>0 . Construct a random variable <math>X</math> with <math>\mathbb{E}[X^2] = b^2</math> for which <math>\mathbf{Pr}(|X| \ge a) = b^2/a^2</math>.
</li>
</li>
<li>
[Chernoff bound] Let <math>X_1,...,X_n</math> be independent Poisson trials. Let <math>X=\sum \limits_{i=1}^n X_i</math> and <math>\mu=\mathbb{E}[X]</math>. Prove that for any <math>\delta>0</math>,
::<center><math>\mathbf{Pr}[X\ge (1+\delta)\mu]\le\left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}.</math></center>

</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

== Problem 4 (Chernoff bound vs k-th moment method) ==
<ul>

Let <math>X</math> be a random variable such that the moment generating function <math>\mathbf{E}[e^{t|X|}]</math> is finite for some <math>t > 0</math>.

We can use the following two kinds of tail inequalities for <math>X</math>.

 
'''Chernoff Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \min_{t \ge 0} \frac{\mathbf{E}[e^{t|X|}]}{e^{t\delta}}.</math>

'''kth-Moment Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \frac{\mathbf{E}[|X|^k]}{\delta^k}.</math>

(a) Prove that for each <math>\delta>0</math>, there is a choice of <math>k</math> such that the <math>k</math>-th moment bound is stronger than the Chernoff bound.

(Hint: Consider the Taylor expansion of the moment generating function.)
 
(b) Why do we still prefer to use the Chernoff bound rather than the <math>k</math>-th moment bound in algorithmic analysis?

</ul>

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T10:34:25Z

Yqzhu: /* Problem 4 (Chernoff bound vs k-th moment method) */

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
<li>
[Chebyshev's inequality] Fix <math>0 . Construct a random variable <math>X</math> with <math>\mathbb{E}[X^2] = b^2</math> for which <math>\mathbf{Pr}(|X| \ge a) = b^2/a^2</math>.
</li>
</li>
<li>
[Chernoff bound] Let <math>X_1,...,X_n</math> be independent Poisson trials. Let <math>X=\sum \limits_{i=1}^n X_i</math> and <math>\mu=\mathbb{E}[X]</math>. Prove that for any <math>\delta>0</math>,
::<center><math>\mathbf{Pr}[X\ge (1+\delta)\mu]\le\left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}.</math></center>

</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

== Problem 4 (Chernoff bound vs k-th moment method) ==
<ul>

Let <math>X</math> be a random variable such that the moment generating function <math>\mathbf{E}[\exp(t|X|)]</math> is finite for some <math>t > 0</math>.

We can use the following two kinds of tail inequalities for <math>X</math>.

 
'''Chernoff Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \min_{t \ge 0} \frac{\mathbf{E}[e^{t|X|}]}{e^{t\delta}}.</math>

'''kth-Moment Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \frac{\mathbf{E}[|X|^k]}{\delta^k}.</math>

(a) Prove that for each <math>\delta>0</math>, there is a choice of <math>k</math> such that the <math>k</math>-th moment bound is stronger than the Chernoff bound.

(Hint: Consider the Taylor expansion of the moment generating function.)
 
(b) Why do we still prefer to use the Chernoff bound rather than the <math>k</math>-th moment bound in algorithmic analysis?

</ul>

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T10:30:53Z

Yqzhu: /* Problem 4 (k-th moment method vs Chernoff bound) */

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
<li>
[Chebyshev's inequality] Fix <math>0 . Construct a random variable <math>X</math> with <math>\mathbb{E}[X^2] = b^2</math> for which <math>\mathbf{Pr}(|X| \ge a) = b^2/a^2</math>.
</li>
</li>
<li>
[Chernoff bound] Let <math>X_1,...,X_n</math> be independent Poisson trials. Let <math>X=\sum \limits_{i=1}^n X_i</math> and <math>\mu=\mathbb{E}[X]</math>. Prove that for any <math>\delta>0</math>,
::<center><math>\mathbf{Pr}[X\ge (1+\delta)\mu]\le\left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}.</math></center>

</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

== Problem 4 (Chernoff bound vs k-th moment method) ==
<ul>

Let <math>X</math> be a random variable such that the moment generating function <math>\mathbf{E}[\exp(t|X|)]</math> is finite for some <math>t > 0</math>.

We can use the following two kinds of tail inequalities for <math>X</math>.

 
'''Chernoff Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \min_{t \ge 0} \frac{\mathbf{E}[e^{t|X|}]}{e^{t\delta}}.</math>

'''kth-Moment Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \frac{\mathbf{E}[|X|^k]}{\delta^k}.</math>

(a) Prove that for each <math>\delta</math>, there is a choice of <math>k</math> such that the <math>k</math>-th moment bound is stronger than the Chernoff bound.

(Hint: Consider the Taylor expansion of the moment generating function.)
 
(b) Why do we still prefer to use the Chernoff bound rather than the <math>k</math>-th moment bound in algorithmic analysis?

</ul>

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T10:30:31Z

Yqzhu: /* Problem 4(k-th moment method vs Chernoff bound) */

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
<li>
[Chebyshev's inequality] Fix <math>0 . Construct a random variable <math>X</math> with <math>\mathbb{E}[X^2] = b^2</math> for which <math>\mathbf{Pr}(|X| \ge a) = b^2/a^2</math>.
</li>
</li>
<li>
[Chernoff bound] Let <math>X_1,...,X_n</math> be independent Poisson trials. Let <math>X=\sum \limits_{i=1}^n X_i</math> and <math>\mu=\mathbb{E}[X]</math>. Prove that for any <math>\delta>0</math>,
::<center><math>\mathbf{Pr}[X\ge (1+\delta)\mu]\le\left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}.</math></center>

</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

== Problem 4 (k-th moment method vs Chernoff bound) ==
<ul>

Let <math>X</math> be a random variable such that the moment generating function <math>\mathbf{E}[\exp(t|X|)]</math> is finite for some <math>t > 0</math>.

We can use the following two kinds of tail inequalities for <math>X</math>.

 
'''Chernoff Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \min_{t \ge 0} \frac{\mathbf{E}[e^{t|X|}]}{e^{t\delta}}.</math>

'''kth-Moment Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \frac{\mathbf{E}[|X|^k]}{\delta^k}.</math>

(a) Prove that for each <math>\delta</math>, there is a choice of <math>k</math> such that the <math>k</math>-th moment bound is stronger than the Chernoff bound.

(Hint: Consider the Taylor expansion of the moment generating function.)
 
(b) Why do we still prefer to use the Chernoff bound rather than the <math>k</math>-th moment bound in algorithmic analysis?

</ul>

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T10:29:40Z

Yqzhu: /* Problem 4(k-th moment method vs Chernoff bound) */

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
<li>
[Chebyshev's inequality] Fix <math>0 . Construct a random variable <math>X</math> with <math>\mathbb{E}[X^2] = b^2</math> for which <math>\mathbf{Pr}(|X| \ge a) = b^2/a^2</math>.
</li>
</li>
<li>
[Chernoff bound] Let <math>X_1,...,X_n</math> be independent Poisson trials. Let <math>X=\sum \limits_{i=1}^n X_i</math> and <math>\mu=\mathbb{E}[X]</math>. Prove that for any <math>\delta>0</math>,
::<center><math>\mathbf{Pr}[X\ge (1+\delta)\mu]\le\left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}.</math></center>

</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

== Problem 4(k-th moment method vs Chernoff bound) ==
<ul>

Let <math>X</math> be a random variable such that the moment generating function <math>\mathbf{E}[\exp(t|X|)]</math> is finite for some <math>t > 0</math>.

We can use the following two kinds of tail inequalities for <math>X</math>.

 
'''Chernoff Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \min_{t \ge 0} \frac{\mathbf{E}[e^{t|X|}]}{e^{t\delta}}.</math>

'''kth-Moment Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \frac{\mathbf{E}[|X|^k]}{\delta^k}.</math>

(a) Prove that for each <math>\delta</math>, there is a choice of <math>k</math> such that the <math>k</math>-th moment bound is stronger than the Chernoff bound.

(Hint: Consider the Taylor expansion of the moment generating function.)
 
(b) Why do we still prefer to use the Chernoff bound rather than the <math>k</math>-th moment bound in algorithmic analysis?

</ul>

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T10:29:28Z

Yqzhu: /* Problem 4(k-th moment method vs Chernoff bound) */

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
<li>
[Chebyshev's inequality] Fix <math>0 . Construct a random variable <math>X</math> with <math>\mathbb{E}[X^2] = b^2</math> for which <math>\mathbf{Pr}(|X| \ge a) = b^2/a^2</math>.
</li>
</li>
<li>
[Chernoff bound] Let <math>X_1,...,X_n</math> be independent Poisson trials. Let <math>X=\sum \limits_{i=1}^n X_i</math> and <math>\mu=\mathbb{E}[X]</math>. Prove that for any <math>\delta>0</math>,
::<center><math>\mathbf{Pr}[X\ge (1+\delta)\mu]\le\left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}.</math></center>

</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

== Problem 4(k-th moment method vs Chernoff bound) ==
<ul>

Let <math>X</math> be a random variable such that the moment generating function <math>\mathbf{E}[\exp(t|X|)]</math> is finite for some <math>t > 0</math>.

We can use the following two kinds of tail inequalities for <math>X</math>.

 
'''Chernoff Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \min_{t \ge 0} \frac{\mathbf{E}[e^{t|X|}]}{e^{t\delta}}.</math>

'''kth-Moment Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \frac{\mathbf{E}[|X|^k]}{\delta^k}.</math>

(a) Prove that for each <math>\delta</math>, there is a choice of <math>k</math> such that the <math>k</math>-th moment bound is stronger than the Chernoff bound.

(Hint: Consider the Taylor expansion of the moment generating function.)

(b) Why do we still prefer to use the Chernoff bound rather than the <math>k</math>-th moment bound in algorithmic analysis?

</ul>

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T10:28:52Z

Yqzhu: /* Problem 4(k-th moment method vs Chernoff bound) */

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
<li>
[Chebyshev's inequality] Fix <math>0 . Construct a random variable <math>X</math> with <math>\mathbb{E}[X^2] = b^2</math> for which <math>\mathbf{Pr}(|X| \ge a) = b^2/a^2</math>.
</li>
</li>
<li>
[Chernoff bound] Let <math>X_1,...,X_n</math> be independent Poisson trials. Let <math>X=\sum \limits_{i=1}^n X_i</math> and <math>\mu=\mathbb{E}[X]</math>. Prove that for any <math>\delta>0</math>,
::<center><math>\mathbf{Pr}[X\ge (1+\delta)\mu]\le\left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}.</math></center>

</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

== Problem 4(k-th moment method vs Chernoff bound) ==
<ul>

Let <math>X</math> be a random variable such that the moment generating function <math>\mathbf{E}[\exp(t|X|)]</math> is finite for some <math>t > 0</math>.

We can use the following two kinds of tail inequalities for <math>X</math>.

 
'''Chernoff Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \min_{t \ge 0} \frac{\mathbf{E}[e^{t|X|}]}{e^{t\delta}}.</math>

'''kth-Moment Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \frac{\mathbf{E}[|X|^k]}{\delta^k}.</math>

(a) Prove that for each <math>\delta</math>, there is a choice of <math>k</math> such that the <math>k</math>-th moment bound is stronger than the Chernoff bound.

(Hint: Consider the Taylor expansion of the moment generating function.)

 

(b) Why do we still prefer to use the Chernoff bound rather than the <math>k</math>-th moment bound in algorithmic analysis?

</ul>

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T10:27:10Z

Yqzhu: /* Problem 4(k-th moment method vs Chernoff bound) */

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
<li>
[Chebyshev's inequality] Fix <math>0 . Construct a random variable <math>X</math> with <math>\mathbb{E}[X^2] = b^2</math> for which <math>\mathbf{Pr}(|X| \ge a) = b^2/a^2</math>.
</li>
</li>
<li>
[Chernoff bound] Let <math>X_1,...,X_n</math> be independent Poisson trials. Let <math>X=\sum \limits_{i=1}^n X_i</math> and <math>\mu=\mathbb{E}[X]</math>. Prove that for any <math>\delta>0</math>,
::<center><math>\mathbf{Pr}[X\ge (1+\delta)\mu]\le\left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}.</math></center>

</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

== Problem 4(k-th moment method vs Chernoff bound) ==
<ul>

Let <math>X</math> be a random variable such that the moment generating function <math>\mathbf{E}[\exp(t|X|)]</math> is finite for some <math>t > 0</math>.

We can use the following two kinds of tail inequalities for <math>X</math>.

 
'''Chernoff Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \min_{t \ge 0} \frac{\mathbf{E}[e^{t|X|}]}{e^{t\delta}}.</math>

'''kth-Moment Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \frac{\mathbf{E}[|X|^k]}{\delta^k}.</math>

(a) Prove that for every <math>\delta</math>, there is a choice of <math>k</math> such that the <math>k</math>-th moment bound is stronger than the Chernoff bound.

(Hint: Consider the Taylor expansion of the moment generating function.)

 

(b) Why do we still prefer to use the Chernoff bound rather than the <math>k</math>-th moment bound in algorithmic analysis?

</ul>

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T10:27:00Z

Yqzhu:

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
<li>
[Chebyshev's inequality] Fix <math>0 . Construct a random variable <math>X</math> with <math>\mathbb{E}[X^2] = b^2</math> for which <math>\mathbf{Pr}(|X| \ge a) = b^2/a^2</math>.
</li>
</li>
<li>
[Chernoff bound] Let <math>X_1,...,X_n</math> be independent Poisson trials. Let <math>X=\sum \limits_{i=1}^n X_i</math> and <math>\mu=\mathbb{E}[X]</math>. Prove that for any <math>\delta>0</math>,
::<center><math>\mathbf{Pr}[X\ge (1+\delta)\mu]\le\left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}.</math></center>

</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

== Problem 4(k-th moment method vs Chernoff bound) ==
<ul>

Let <math>X</math> be a random variable such that the moment generating function <math>\mathbb{E}[\exp(t|X|)]</math> is finite for some <math>t > 0</math>.

We can use the following two kinds of tail inequalities for <math>X</math>.

 
'''Chernoff Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \min_{t \ge 0} \frac{\mathbf{E}[e^{t|X|}]}{e^{t\delta}}.</math>

'''kth-Moment Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \frac{\mathbf{E}[|X|^k]}{\delta^k}.</math>

(a) Prove that for every <math>\delta</math>, there is a choice of <math>k</math> such that the <math>k</math>-th moment bound is stronger than the Chernoff bound.

(Hint: Consider the Taylor expansion of the moment generating function.)

 

(b) Why do we still prefer to use the Chernoff bound rather than the <math>k</math>-th moment bound in algorithmic analysis?

</ul>

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T10:26:37Z

Yqzhu:

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
<li>
[Chebyshev's inequality] Fix <math>0 . Construct a random variable <math>X</math> with <math>\mathbb{E}[X^2] = b^2</math> for which <math>\mathbf{Pr}(|X| \ge a) = b^2/a^2</math>.
</li>
</li>
<li>
[Chernoff bound] Let <math>X_1,...,X_n</math> be independent Poisson trials. Let <math>X=\sum \limits_{i=1}^n X_i</math> and <math>\mu=\mathbb{E}[X]</math>. Prove that for any <math>\delta>0</math>,
::<center><math>\mathbf{Pr}[X\ge (1+\delta)\mu]\le\left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}.</math></center>

</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

== Problem 4(k-th moment method vs Chernoff bound) ==
<ul>

Let <math>X</math> be a random variable such that the moment generating function <math>\mathbb{E}[\exp(t|X|)]</math> is finite for some <math>t > 0</math>.

We can use the following two kinds of tail inequalities for <math>X</math>.

 
'''Chernoff Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \min_{t \ge 0} \frac{\mathbb{E}[e^{t|X|}]}{e^{t\delta}}.</math>

'''kth-Moment Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \frac{\mathbb{E}[|X|^k]}{\delta^k}.</math>

(a) Prove that for every <math>\delta</math>, there is a choice of <math>k</math> such that the <math>k</math>-th moment bound is stronger than the Chernoff bound.

(Hint: Consider the Taylor expansion of the moment generating function.)

 

(b) Why do we still prefer to use the Chernoff bound rather than the <math>k</math>-th moment bound in algorithmic analysis?

</ul>

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T10:25:59Z

Yqzhu:

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
<li>
[Chebyshev's inequality] Fix <math>0 . Construct a random variable <math>X</math> with <math>\mathbb{E}[X^2] = b^2</math> for which <math>\mathbf{Pr}(|X| \ge a) = b^2/a^2</math>.
</li>
</li>
<li>
[Chernoff bound] Let <math>X_1,...,X_n</math> be independent Poisson trials. Let <math>X=\sum \limits_{i=1}^n X_i</math> and <math>\mu=\mathbb{E}[X]</math>. Prove that for any <math>\delta>0</math>,
::<center><math>\mathbf{Pr}[X\ge (1+\delta)\mu]\le\left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}.</math></center>

</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

== Problem 4(k-th moment method vs Chernoff bound) ==
<ul>

Let <math>X</math> be a random variable such that the moment generating function <math>\mathbb{E}[\exp(t|X|)]</math> is finite for some <math>t > 0</math>.

We can use the following two kinds of tail inequalities for <math>X</math>.

 
'''Chernoff Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \min_{t \ge 0} \frac{\mathbf{E}[e^{t|X|}]}{e^{t\delta}}.</math>

'''kth-Moment Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge \delta] \le \frac{\mathbf{E}[|X|^k]}{\delta^k}.</math>

(a) Prove that for every <math>\delta</math>, there is a choice of <math>k</math> such that the <math>k</math>-th moment bound is stronger than the Chernoff bound.

(Hint: Consider the Taylor expansion of the moment generating function.)

 

(b) Why do we still prefer to use the Chernoff bound rather than the <math>k</math>-th moment bound in algorithmic analysis?

</ul>

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T10:24:18Z

Yqzhu:

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
<li>
[Chebyshev's inequality] Fix <math>0 . Construct a random variable <math>X</math> with <math>\mathbb{E}[X^2] = b^2</math> for which <math>\mathbf{Pr}(|X| \ge a) = b^2/a^2</math>.
</li>
</li>
<li>
[Chernoff bound] Let <math>X_1,...,X_n</math> be independent Poisson trials. Let <math>X=\sum \limits_{i=1}^n X_i</math> and <math>\mu=\mathbb{E}[X]</math>. Prove that for any <math>\delta>0</math>,
::<center><math>\mathbf{Pr}[X\ge (1+\delta)\mu]\le\left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}.</math></center>

</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

== Problem 4(K-th moment method vs Chernoff bound) ==
<ul>

Let <math>X</math> be a random variable such that the moment generating function <math>\mathbb{E}[\exp(t|X|)]</math> is finite for some <math>t > 0</math>.

We can use the following two kinds of tail inequalities for <math>X</math>.

 
'''Chernoff Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge a] \le \min_{t>0} \frac{\mathbf{E}[e^{t|X|}]}{e^{ta}}.</math>

'''kth-Moment Bound:'''
:<center><math>\mathbf{Pr}[|X| \ge a] \le \frac{\mathbf{E}[|X|^k]}{a^k}.</math>

(a) Prove that for every <math>\delta</math>, there is a choice of <math>k</math> such that the <math>k</math>-th moment bound is stronger than the Chernoff bound.

(Hint: Consider the Taylor expansion of the moment generating function and apply the probabilistic method.)

 

(b) Why do we still prefer to use the Chernoff bound rather than the (seemingly stronger) <math>k</math>-th moment bound in algorithmic analysis?

</ul>

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T10:24:06Z

Yqzhu:

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
<li>
[Chebyshev's inequality] Fix <math>0 . Construct a random variable <math>X</math> with <math>\mathbb{E}[X^2] = b^2</math> for which <math>\mathbf{Pr}(|X| \ge a) = b^2/a^2</math>.
</li>
</li>
<li>
[Chernoff bound] Let <math>X_1,...,X_n</math> be independent Poisson trials. Let <math>X=\sum \limits_{i=1}^n X_i</math> and <math>\mu=\mathbb{E}[X]</math>. Prove that for any <math>\delta>0</math>,
::<center><math>\mathbf{Pr}[X\ge (1+\delta)\mu]\le\left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}.</math></center>

</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

== Problem 4(K-th moment method vs Chernoff bound) ==
<ul>

Let <math>X</math> be a random variable such that the moment generating function <math>\mathbb{E}[\exp(t|X|)]</math> is finite for some <math>t > 0</math>.

We can use the following two kinds of tail inequalities for <math>X</math>.

 
'''Chernoff Bound:'''
:<center><math>\mathbb{Pr}[|X| \ge a] \le \min_{t>0} \frac{\mathbf{E}[e^{t|X|}]}{e^{ta}}.</math>

'''kth-Moment Bound:'''
:<center><math>\mathbb{Pr}[|X| \ge a] \le \frac{\mathbf{E}[|X|^k]}{a^k}.</math>

(a) Prove that for every <math>\delta</math>, there is a choice of <math>k</math> such that the <math>k</math>-th moment bound is stronger than the Chernoff bound.

(Hint: Consider the Taylor expansion of the moment generating function and apply the probabilistic method.)

 

(b) Why do we still prefer to use the Chernoff bound rather than the (seemingly stronger) <math>k</math>-th moment bound in algorithmic analysis?

</ul>

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T10:23:46Z

Yqzhu:

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
<li>
[Chebyshev's inequality] Fix <math>0 . Construct a random variable <math>X</math> with <math>\mathbb{E}[X^2] = b^2</math> for which <math>\mathbf{Pr}(|X| \ge a) = b^2/a^2</math>.
</li>
</li>
<li>
[Chernoff bound] Let <math>X_1,...,X_n</math> be independent Poisson trials. Let <math>X=\sum \limits_{i=1}^n X_i</math> and <math>\mu=\mathbb{E}[X]</math>. Prove that for any <math>\delta>0</math>,
::<center><math>\mathbf{Pr}[X\ge (1+\delta)\mu]\le\left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}.</math></center>

</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

== Problem 4(K-th moment method vs Chernoff bound) ==
<ul>

Let <math>X</math> be a random variable such that the moment generating function <math>\mathbb{E}[\exp(t|X|)]</math> is finite for some <math>t > 0</math>.

We can use the following two kinds of tail inequalities for <math>X</math>.

 
'''Chernoff Bound:'''
:<center><math>\Pr[|X| \ge a] \le \min_{t>0} \frac{\mathbf{E}[e^{t|X|}]}{e^{ta}}.</math>

'''kth-Moment Bound:'''
:<center><math>\Pr[|X| \ge a] \le \frac{\mathbf{E}[|X|^k]}{a^k}.</math>

(a) Prove that for every <math>\delta</math>, there is a choice of <math>k</math> such that the <math>k</math>-th moment bound is stronger than the Chernoff bound.

(Hint: Consider the Taylor expansion of the moment generating function and apply the probabilistic method.)

 

(b) Why do we still prefer to use the Chernoff bound rather than the (seemingly stronger) <math>k</math>-th moment bound in algorithmic analysis?

</ul>

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T10:23:32Z

Yqzhu:

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
<li>
[Chebyshev's inequality] Fix <math>0 . Construct a random variable <math>X</math> with <math>\mathbb{E}[X^2] = b^2</math> for which <math>\mathbf{Pr}(|X| \ge a) = b^2/a^2</math>.
</li>
</li>
<li>
[Chernoff bound] Let <math>X_1,...,X_n</math> be independent Poisson trials. Let <math>X=\sum \limits_{i=1}^n X_i</math> and <math>\mu=\mathbb{E}[X]</math>. Prove that for any <math>\delta>0</math>,
::<center><math>\mathbf{Pr}[X\ge (1+\delta)\mu]\le\left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}.</math></center>

</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

== Problem 4(K-th moment method vs Chernoff bound) ==
<ul>

Let <math>X</math> be a random variable such that the moment generating function <math>\mathbb{E}[\exp(t|X|)]</math> is finite for some <math>t > 0</math>.

We can use the following two kinds of tail inequalities for <math>X</math>.

 
'''Chernoff Bound:'''
:<center><math>\Pr[|X| \ge a] \le \min_{t>0} \frac{\mathbf{E}[e^{t|X|}]}{e^{ta}}.</math>
<\center>

'''kth-Moment Bound:'''
:<center><math>\Pr[|X| \ge a] \le \frac{\mathbf{E}[|X|^k]}{a^k}.</math>
<\center>

(a) Prove that for every <math>\delta</math>, there is a choice of <math>k</math> such that the <math>k</math>-th moment bound is stronger than the Chernoff bound.

(Hint: Consider the Taylor expansion of the moment generating function and apply the probabilistic method.)

 

(b) Why do we still prefer to use the Chernoff bound rather than the (seemingly stronger) <math>k</math>-th moment bound in algorithmic analysis?

</ul>

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T10:23:08Z

Yqzhu:

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
<li>
[Chebyshev's inequality] Fix <math>0 . Construct a random variable <math>X</math> with <math>\mathbb{E}[X^2] = b^2</math> for which <math>\mathbf{Pr}(|X| \ge a) = b^2/a^2</math>.
</li>
</li>
<li>
[Chernoff bound] Let <math>X_1,...,X_n</math> be independent Poisson trials. Let <math>X=\sum \limits_{i=1}^n X_i</math> and <math>\mu=\mathbb{E}[X]</math>. Prove that for any <math>\delta>0</math>,
::<center><math>\mathbf{Pr}[X\ge (1+\delta)\mu]\le\left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}.</math></center>

</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

== Problem 4(K-th moment method vs Chernoff bound) ==
<ul>

Let <math>X</math> be a random variable such that the moment generating function <math>\mathbb{E}[\exp(t|X|)]</math> is finite for some <math>t > 0</math>.

We can use the following two kinds of tail inequalities for <math>X</math>.

 
'''Chernoff Bound:'''
:<center><math>\Pr[|X| \ge a] \le \min_{t>0} \frac{\mathbf{E}[e^{t|X|}]}{e^{ta}}.</math><\center>

'''kth-Moment Bound:'''
:<center><math>\Pr[|X| \ge a] \le \frac{\mathbf{E}[|X|^k]}{a^k}.</math><\center>

(a) Prove that for every <math>\delta</math>, there is a choice of <math>k</math> such that the <math>k</math>-th moment bound is stronger than the Chernoff bound.

(Hint: Consider the Taylor expansion of the moment generating function and apply the probabilistic method.)

 

(b) Why do we still prefer to use the Chernoff bound rather than the (seemingly stronger) <math>k</math>-th moment bound in algorithmic analysis?

</ul>

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T10:22:15Z

Yqzhu:

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
<li>
[Chebyshev's inequality] Fix <math>0 . Construct a random variable <math>X</math> with <math>\mathbb{E}[X^2] = b^2</math> for which <math>\mathbf{Pr}(|X| \ge a) = b^2/a^2</math>.
</li>
</li>
<li>
[Chernoff bound] Let <math>X_1,...,X_n</math> be independent Poisson trials. Let <math>X=\sum \limits_{i=1}^n X_i</math> and <math>\mu=\mathbb{E}[X]</math>. Prove that for any <math>\delta>0</math>,
::<center><math>\mathbf{Pr}[X\ge (1+\delta)\mu]\le\left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}.</math></center>

</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

== Problem 4(K-th moment method vs Chernoff bound) ==
<ul>

Let <math>X</math> be a random variable such that the moment generating function <math>\mathbb{E}[\exp(t|X|)]</math> is finite for some <math>t > 0</math>.

We can use the following two kinds of tail inequalities for <math>X</math>.

 
'''Chernoff Bound:'''
:<math>\Pr[|X| \ge a] \le \min_{t>0} \frac{\mathbf{E}[e^{t|X|}]}{e^{ta}}</math>

'''kth-Moment Bound:'''
:<math>\Pr[|X| \ge a] \le \frac{\mathbf{E}[|X|^k]}{a^k}</math>

(a) Prove that for every <math>\delta</math>, there is a choice of <math>k</math> such that the <math>k</math>-th moment bound is stronger than the Chernoff bound.

(Hint: Consider the Taylor expansion of the moment generating function and apply the probabilistic method.)

 

(b) Why do we still prefer to use the Chernoff bound rather than the (seemingly stronger) <math>k</math>-th moment bound in algorithmic analysis?

</ul>

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T10:18:32Z

Yqzhu:

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
<li>
[Chebyshev's inequality] Fix <math>0 . Construct a random variable <math>X</math> with <math>\mathbb{E}[X^2] = b^2</math> for which <math>\mathbf{Pr}(|X| \ge a) = b^2/a^2</math>.
</li>
</li>
<li>
[Chernoff bound] Let <math>X_1,...,X_n</math> be independent Poisson trials. Let <math>X=\sum \limits_{i=1}^n X_i</math> and <math>\mu=\mathbb{E}[X]</math>. Prove that for any <math>\delta>0</math>,
::<center><math>\mathbf{Pr}[X\ge (1+\delta)\mu]\le\left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}.</math></center>

</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

== Problem 4(K-th moment method vs Chernoff bound) ==
<ul>

Let <math>X</math> be a random variable such that the moment generating function <math>E[\exp(t|X|)]</math> is finite for some <math>t > 0</math>.

We can use two tail inequalities on <math>X</math>: the Chernoff bound and the <math>k</math>-th moment bound.

 

(a) Prove that for every <math>\delta</math>, there is a choice of <math>k</math> such that the <math>k</math>-th moment bound is stronger than the Chernoff bound.

(Hint: Consider the Taylor expansion of the moment generating function and apply the probabilistic method.)

 

(b) Why do we still prefer to use the Chernoff bound rather than the (seemingly stronger) <math>k</math>-th moment bound in algorithmic analysis?

</ul>

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T10:17:01Z

Yqzhu:

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
<li>
[Chebyshev's inequality] Fix <math>0 . Construct a random variable <math>X</math> with <math>\mathbb{E}[X^2] = b^2</math> for which <math>\mathbf{Pr}(|X| \ge a) = b^2/a^2</math>.
</li>
</li>
<li>
[Chernoff bound] Let <math>X_1,...,X_n</math> be independent Poisson trials. Let <math>X=\sum \limits_{i=1}^n X_i</math> and <math>\mu=\mathbb{E}[X]</math>. Prove that for any <math>\delta>0</math>,
::<center><math>\mathbf{Pr}[X\ge (1+\delta)\mu]\le\left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}.</math></center>

</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

== Problem 4(K-th moment method vs Chernoff bound) ==

<ul>

Let <math>X</math> be a random variable such that the moment generating function <math>E[\exp(t|X|)]</math> is finite for some <math>t > 0</math>. We can use two tail inequalities on <math>X</math>: the Chernoff bound and the <math>k</math>-th moment bound.

(a) Prove that for every <math>\delta</math>, there is a choice of <math>k</math> such that the <math>k</math>-th moment bound is stronger than the Chernoff bound. (Hint: Consider the Taylor expansion of the moment generating function and apply the probabilistic method.)

(b) Why do we still prefer to use the Chernoff bound rather than the (seemingly stronger) <math>k</math>-th moment bound in algorithmic analysis?

</ul>

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T10:16:37Z

Yqzhu:

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
<li>
[Chebyshev's inequality] Fix <math>0 . Construct a random variable <math>X</math> with <math>\mathbb{E}[X^2] = b^2</math> for which <math>\mathbf{Pr}(|X| \ge a) = b^2/a^2</math>.
</li>
</li>
<li>
[Chernoff bound] Let <math>X_1,...,X_n</math> be independent Poisson trials. Let <math>X=\sum \limits_{i=1}^n X_i</math> and <math>\mu=\mathbb{E}[X]</math>. Prove that for any <math>\delta>0</math>,
::<center><math>\mathbf{Pr}[X\ge (1+\delta)\mu]\le\left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}.</math></center>

</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

== Problem 4(K-th moment method vs Chernoff bound) ==

<ul>

Let <math>X</math> be a random variable such that the moment generating function <math>E[\exp(t|X|)]</math> is finite for some <math>t > 0</math>. We can use two tail inequalities on <math>X</math>: the Chernoff bound and the <math>k</math>-th moment bound.

(a) Prove that for every <math>\delta</math>, there is a choice of <math>k</math> such that the <math>k</math>-th moment bound is stronger than the Chernoff bound. (Hint: Consider the Taylor expansion of the moment generating function and apply the probabilistic method.)

(b) Why do we still prefer to use the Chernoff bound rather than the (seemingly stronger) <math>k</math>-th moment bound in algorithmic analysis?

</ul>

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T03:20:54Z

Yqzhu:

概率论与数理统计 (Spring 2026)

2026-04-21T03:17:20Z

Yqzhu: /* Assignments */

{{Infobox
|name = Infobox
|bodystyle =
|title = '''概率论与数理统计''' 
'''Probability Theory''' & '''Mathematical Statistics'''
|titlestyle =

|image =
|imagestyle =
|caption =
|captionstyle =
|headerstyle = background:#ccf;
|labelstyle = background:#ddf;
|datastyle =

|header1 =Instructor
|label1 =
|data1 =
|header2 =
|label2 =
|data2 = '''尹一通'''
|header3 =
|label3 = Email
|data3 = yinyt@nju.edu.cn
|header4 =
|label4 = office
|data4 = 计算机系 804
|header5 =
|label5 =
|data5 = '''刘景铖'''
|header6 =
|label6 = Email
|data6 = liu@nju.edu.cn
|header7 =
|label7 = office
|data7 = 计算机系 516
|header8 = Class
|label8 =
|data8 =
|header9 =
|label9 = Class meeting
|data9 = Wednesday, 9am-12am 
仙Ⅱ-212
|header10=
|label10 = Office hour
|data10 = TBA 计算机系 804（尹一通） 计算机系 516（刘景铖）
|header11= Textbook
|label11 =
|data11 =
|header12=
|label12 =
|data12 = [[File:概率导论.jpeg|border|100px]]
|header13=
|label13 =
|data13 = '''概率导论'''（第2版·修订版） Dimitri P. Bertsekas and John N. Tsitsiklis 郑忠国童行伟译；人民邮电出版社 (2022)
|header14=
|label14 =
|data14 = [[File:Grimmett_probability.jpg|border|100px]]
|header15=
|label15 =
|data15 = '''Probability and Random Processes''' (4E) Geoffrey Grimmett and David Stirzaker Oxford University Press (2020)
|header16=
|label16 =
|data16 = [[File:Probability_and_Computing_2ed.jpg|border|100px]]
|header17=
|label17 =
|data17 = '''Probability and Computing''' (2E) Michael Mitzenmacher and Eli Upfal Cambridge University Press (2017)
|belowstyle = background:#ddf;
|below =
}}

This is the webpage for the ''Probability Theory and Mathematical Statistics'' (概率论与数理统计) class of Spring 2026. Students who take this class should check this page periodically for content updates and new announcements.

= Announcement =
* TBA

= Course info =
* '''Instructor ''':
:* [http://tcs.nju.edu.cn/yinyt/ 尹一通]：[mailto:yinyt@nju.edu.cn <yinyt@nju.edu.cn>]，计算机系 804
:* [https://liuexp.github.io 刘景铖]：[mailto:liu@nju.edu.cn <liu@nju.edu.cn>]，计算机系 516
* '''Teaching assistant''':
** 鞠哲：[mailto:juzhe@smail.nju.edu.cn <juzhe@smail.nju.edu.cn>]，计算机系 426
** 祝永祺：[mailto:652025330045@smail.nju.edu.cn <652025330045@smail.nju.edu.cn>]，计算机系 426
* '''Class meeting''':
** 周三：9am-12am，仙Ⅱ-212
* '''Office hour''':
:* TBA, 计算机系 804（尹一通）
:* TBA, 计算机系 516（刘景铖）
:* '''QQ群''': 1090092561（申请加入需提供姓名、院系、学号）

= Syllabus =
课程内容分为三大部分：
* '''经典概率论'''：概率空间、随机变量及其数字特征、多维与连续随机变量、极限定理等内容
* '''概率与计算'''：测度集中现象 (concentration of measure)、概率法 (the probabilistic method)、离散随机过程的相关专题
* '''数理统计'''：参数估计、假设检验、贝叶斯估计、线性回归等统计推断等概念

对于第一和第二部分，要求清楚掌握基本概念，深刻理解关键的现象与规律以及背后的原理，并可以灵活运用所学方法求解相关问题。对于第三部分，要求熟悉数理统计的若干基本概念，以及典型的统计模型和统计推断问题。

经过本课程的训练，力求使学生能够熟悉掌握概率的语言，并会利用概率思维来理解客观世界并对其建模，以及驾驭概率的数学工具来分析和求解专业问题。

=== 教材与参考书 Course Materials ===
* '''[BT]''' 概率导论（第2版·修订版），[美]伯特瑟卡斯（Dimitri P.Bertsekas）[美]齐齐克利斯（John N.Tsitsiklis）著，郑忠国童行伟译，人民邮电出版社（2022）。
* '''[GS]''' ''Probability and Random Processes'', by Geoffrey Grimmett and David Stirzaker; Oxford University Press; 4th edition (2020).
* '''[MU]''' ''Probability and Computing: Randomization and Probabilistic Techniques in Algorithms and Data Analysis'', by Michael Mitzenmacher, Eli Upfal; Cambridge University Press; 2nd edition (2017).

=== 成绩 Grading Policy ===
* 课程成绩：本课程将会有若干次作业和一次期末考试。最终成绩将由平时作业成绩和期末考试成绩综合得出。
* 迟交：如果有特殊的理由，无法按时完成作业，请提前联系授课老师，给出正当理由。否则迟交的作业将不被接受。

=== 学术诚信 Academic Integrity ===
学术诚信是所有从事学术活动的学生和学者最基本的职业道德底线，本课程将不遗余力的维护学术诚信规范，违反这一底线的行为将不会被容忍。

作业完成的原则：署你名字的工作必须是你个人的贡献。在完成作业的过程中，允许讨论，前提是讨论的所有参与者均处于同等完成度。但关键想法的执行、以及作业文本的写作必须独立完成，并在作业中致谢（acknowledge）所有参与讨论的人。符合规则的讨论与致谢将不会影响得分。不允许其他任何形式的合作——尤其是与已经完成作业的同学“讨论”。

本课程将对剽窃行为采取零容忍的态度。在完成作业过程中，对他人工作（出版物、互联网资料、其他人的作业等）直接的文本抄袭和对关键思想、关键元素的抄袭，按照 [http://www.acm.org/publications/policies/plagiarism_policy ACM Policy on Plagiarism]的解释，都将视为剽窃。剽窃者成绩将被取消。如果发现互相抄袭行为， 抄袭和被抄袭双方的成绩都将被取消。因此请主动防止自己的作业被他人抄袭。

学术诚信影响学生个人的品行，也关乎整个教育系统的正常运转。为了一点分数而做出学术不端的行为，不仅使自己沦为一个欺骗者，也使他人的诚实努力失去意义。让我们一起努力维护一个诚信的环境。

= Assignments =
*[[概率论与数理统计 (Spring 2026)/Problem Set 1|Problem Set 1]] 请在 2026/4/1 上课之前(9am UTC+8)提交到 [mailto:pr2026_nju@163.com pr2026_nju@163.com] (文件名为'学号_姓名_A1.pdf').
** [[概率论与数理统计 (Spring 2026)/第一次作业提交名单|第一次作业提交名单]]

*[[概率论与数理统计 (Spring 2026)/Problem Set 2|Problem Set 2]] 请在 2026/4/22 上课之前(9am UTC+8)提交到 [mailto:pr2026_nju@163.com pr2026_nju@163.com] (文件名为'学号_姓名_A2.pdf').

*[[概率论与数理统计 (Spring 2026)/Problem Set 3|Problem Set 3]] 请在 TBA 上课之前(9am UTC+8)提交到 [mailto:pr2026_nju@163.com pr2026_nju@163.com] (文件名为'学号_姓名_A3.pdf').

= Lectures =
# [http://tcs.nju.edu.cn/slides/prob2026/Intro.pdf 课程简介]
# [http://tcs.nju.edu.cn/slides/prob2026/ProbSpace.pdf 概率空间]
#* 阅读：'''[BT] 第1章''' 或 '''[GS] Chapter 1'''
#* [[概率论与数理统计 (Spring 2026)/Entropy and volume of Hamming balls|Entropy and volume of Hamming balls]]
#* [[概率论与数理统计 (Spring 2026)/Karger's min-cut algorithm| Karger's min-cut algorithm]]
# [http://tcs.nju.edu.cn/slides/prob2026/RandVar.pdf 随机变量]
#* 阅读：'''[BT] 第2章''' 或 '''[GS] Chapter 2, Sections 3.1~3.5, 3.7'''
#* 阅读：'''[MU] Chapter 2'''
#* [[概率论与数理统计 (Spring 2026)/Average-case analysis of QuickSort|Average-case analysis of '''''QuickSort''''']]
# [http://tcs.nju.edu.cn/slides/prob2026/Deviation.pdf 矩与偏差]
#* 阅读：'''[MU] Chapter 3'''
#* 阅读：'''[BT] 章节 2.4, 4.2, 4.3, 5.1''' 或 '''[GS] Sections 3.3, 3.6, 7.3'''
#* [[概率论与数理统计 (Spring 2026)/Threshold of k-clique in random graph|Threshold of <math>k</math>-clique in random graph]]
#* [[概率论与数理统计 (Spring 2026)/Weierstrass Approximation Theorem|Weierstrass approximation]]

= Concepts =
* [https://plato.stanford.edu/entries/probability-interpret/ Interpretations of probability]
* [https://en.wikipedia.org/wiki/History_of_probability History of probability]
* Example problems:
** [https://dornsifecms.usc.edu/assets/sites/520/docs/VonNeumann-ams12p36-38.pdf von Neumann's Bernoulli factory] and other [https://peteroupc.github.io/bernoulli.html Bernoulli factory algorithms]
** [https://en.wikipedia.org/wiki/Boy_or_Girl_paradox Boy or Girl paradox]
** [https://en.wikipedia.org/wiki/Monty_Hall_problem Monty Hall problem]
** [https://en.wikipedia.org/wiki/Bertrand_paradox_(probability) Bertrand paradox]
** [https://en.wikipedia.org/wiki/Hard_spheres Hard spheres model] and [https://en.wikipedia.org/wiki/Ising_model Ising model]
** [https://en.wikipedia.org/wiki/PageRank ''PageRank''] and stationary [https://en.wikipedia.org/wiki/Random_walk random walk]
** [https://en.wikipedia.org/wiki/Diffusion_process Diffusion process] and [https://en.wikipedia.org/wiki/Diffusion_model diffusion model]
*[https://en.wikipedia.org/wiki/Probability_space Probability space]
** [https://en.wikipedia.org/wiki/Sample_space Sample space]
** [https://en.wikipedia.org/wiki/Event_(probability_theory) Event] and [https://en.wikipedia.org/wiki/Σ-algebra <math>\sigma</math>-algebra]
** Kolmogorov's [https://en.wikipedia.org/wiki/Probability_axioms axioms of probability]
* [https://en.wikipedia.org/wiki/Discrete_uniform_distribution Classical] and [https://en.wikipedia.org/wiki/Geometric_probability goemetric probability]
* [https://en.wikipedia.org/wiki/Boole%27s_inequality Union bound]
** [https://en.wikipedia.org/wiki/Inclusion%E2%80%93exclusion_principle Inclusion-Exclusion principle]
** [https://en.wikipedia.org/wiki/Boole%27s_inequality#Bonferroni_inequalities Bonferroni inequalities]
* [https://en.wikipedia.org/wiki/Conditional_probability Conditional probability]
** [https://en.wikipedia.org/wiki/Chain_rule_(probability) Chain rule]
** [https://en.wikipedia.org/wiki/Law_of_total_probability Law of total probability]
** [https://en.wikipedia.org/wiki/Bayes%27_theorem Bayes' law]
* [https://en.wikipedia.org/wiki/Independence_(probability_theory) Independence]
** [https://en.wikipedia.org/wiki/Pairwise_independence Pairwise independence]
* [https://en.wikipedia.org/wiki/Random_variable Random variable]
** [https://en.wikipedia.org/wiki/Cumulative_distribution_function Cumulative distribution function]
** [https://en.wikipedia.org/wiki/Probability_mass_function Probability mass function]
** [https://en.wikipedia.org/wiki/Probability_density_function Probability density function]
* [https://en.wikipedia.org/wiki/Multivariate_random_variable Random vector]
** [https://en.wikipedia.org/wiki/Joint_probability_distribution Joint probability distribution]
** [https://en.wikipedia.org/wiki/Conditional_probability_distribution Conditional probability distribution]
** [https://en.wikipedia.org/wiki/Marginal_distribution Marginal distribution]
* Some '''discrete''' probability distributions
** [https://en.wikipedia.org/wiki/Bernoulli_trial Bernoulli trial] and [https://en.wikipedia.org/wiki/Bernoulli_distribution Bernoulli distribution]
** [https://en.wikipedia.org/wiki/Discrete_uniform_distribution Discrete uniform distribution]
** [https://en.wikipedia.org/wiki/Binomial_distribution Binomial distribution]
** [https://en.wikipedia.org/wiki/Geometric_distribution Geometric distribution]
** [https://en.wikipedia.org/wiki/Negative_binomial_distribution Negative binomial distribution]
** [https://en.wikipedia.org/wiki/Hypergeometric_distribution Hypergeometric distribution]
** [https://en.wikipedia.org/wiki/Poisson_distribution Poisson distribution]
** and [https://en.wikipedia.org/wiki/List_of_probability_distributions#Discrete_distributions others]
* Balls into bins model
** [https://en.wikipedia.org/wiki/Multinomial_distribution Multinomial distribution]
** [https://en.wikipedia.org/wiki/Birthday_problem Birthday problem]
** [https://en.wikipedia.org/wiki/Coupon_collector%27s_problem Coupon collector]
** [https://en.wikipedia.org/wiki/Balls_into_bins_problem Occupancy problem]
* Random graphs
** [https://en.wikipedia.org/wiki/Erd%C5%91s%E2%80%93R%C3%A9nyi_model Erdős–Rényi random graph model]
** [https://en.wikipedia.org/wiki/Galton%E2%80%93Watson_process Galton–Watson branching process]
* [https://en.wikipedia.org/wiki/Expected_value Expectation]
** [https://en.wikipedia.org/wiki/Law_of_the_unconscious_statistician Law of the unconscious statistician, ''LOTUS'']
** [https://dlsun.github.io/probability/linearity.html Linearity of expectation]
** [https://en.wikipedia.org/wiki/Conditional_expectation Conditional expectation]
** [https://en.wikipedia.org/wiki/Law_of_total_expectation Law of total expectation]

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T03:10:42Z

Yqzhu: /* Problem 2 (Inequalities) */

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
<li>
[Chebyshev's inequality] Fix <math>0 . Construct a random variable <math>X</math> with <math>\mathbb{E}[X^2] = b^2</math> for which <math>\mathbf{Pr}(|X| \ge a) = b^2/a^2</math>.
</li>
</li>
<li>
[Chernoff Bound] Let <math>X_1,...,X_n</math> be independent Poisson trials. Let <math>X=\sum \limits_{i=1}^n X_i</math> and <math>\mu=\mathbb{E}[X]</math>. Prove that for any <math>\delta>0</math>,
::<center><math>\mathbf{Pr}[X\ge (1+\delta)\mu]\le\left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}.</math></center>

</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T03:08:37Z

Yqzhu: /* Problem 2 (Inequalities) */

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
<li>
[Chebyshev's inequality] Fix <math>0 . Construct a random variable <math>X</math> with <math>\mathbb{E}[X^2] = b^2</math> for which <math>\mathbf{Pr}(|X| \ge a) = b^2/a^2</math>.
</li>
</li>
<li>
[Chernoff Bound] Let <math>X_1,...,X_n</math> be independent Poisson trials. Let <math>X=\sum \limits_{i=1}^n X_i</math> and <math>\mu=\mathbb{E}[X]</math>. Prove that for any <math>\delta>0</math>,
::<center><math>\Pr[X\ge (1+\delta)\mu]\le\left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}.</math></center>

</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T03:08:16Z

Yqzhu: /* Problem 2 (Inequalities) */

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
<li>
[Chebyshev's inequality] Fix <math>0 . Construct a random variable <math>X</math> with <math>\mathbb{E}[X^2] = b^2</math> for which <math>\mathbf{Pr}(|X| \ge a) = b^2/a^2</math>.
</li>
</li>
<li>
[Chernoff Bound] Let <math>X_1,...,X_n</math> be independent Poisson trials. Let <math>X=\sum \limits_{i=1}^n X_i</math> and <math>\mu=\mathbb{E}[X]</math>. Prove that for any <math>\delta>0</math>,
::<math><center>\Pr[X\ge (1+\delta)\mu]\le\left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}.</math></center>

</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T03:05:17Z

Yqzhu: /* Problem 2 (Inequalities) */

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
<li>
[Chebyshev's inequality] Fix <math>0 . Construct a random variable <math>X</math> with <math>\mathbb{E}[X^2] = b^2</math> for which <math>\mathbf{Pr}(|X| \ge a) = b^2/a^2</math>.
</li>
</li>
<li>
[Chernoff Bound] Let <math>X_1,...,X_n</math> be independent Poisson trials such that <math>\mathbf{Pr}(X_i)=p_i</math>. Let <math>X=\sum \limits_{i=1}^n X_i</math> and <math>\mu=\mathbb{E}[X]</math>. Then for any <math>\delta>0</math>,
::<math>\Pr[X\ge (1+\delta)\mu]\le\left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}.</math>

</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T03:04:53Z

Yqzhu: /* Problem 2 (Inequalities) */

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
<li>
[Chebyshev's inequality] Fix <math>0 . Construct a random variable <math>X</math> with <math>\mathbb{E}[X^2] = b^2</math> for which <math>\mathbf{Pr}(|X| \ge a) = b^2/a^2</math>.
</li>
</li>
<li>
[Chernoff Bound] Let <math>X_1,...,X_n</math> be independent Poisson trials such that <math>\mathbf{Pr}(X_i)=p_i</math>. Let <math>X=\sum \limits_{i=1}^n X_i</math> and <math>\mu=\mathbb{E}[X]</math>. Then for any <math>\delta>0</math>,
::<math>\Pr[X\ge (1+\delta)\mu]\le\left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}.</math>
}}
</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T02:59:33Z

Yqzhu: /* Problem 2 (Inequalities) */

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
<li>
[Chebyshev's inequality (I)] Fix <math>0 . Construct a random variable <math>X</math> with <math>\mathbb{E}[X^2] = b^2</math> for which <math>\mathbf{Pr}(|X| \ge a) = b^2/a^2</math>.
</li>
</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T02:59:12Z

Yqzhu: /* Problem 2 (Inequalities) */

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
[Chebyshev's inequality (I)] Fix <math>0 . Construct a random variable <math>X</math> with <math>\mathbb{E}[X^2] = b^2</math> for which <math>\mathbf{Pr}(|X| \ge a) = b^2/a^2</math>.
</li>
</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

概率论与数理统计 (Spring 2026)/Problem Set 3

2026-04-21T02:52:30Z

Yqzhu: Created page with "*每道题目的解答都要有完整的解题过程，中英文不限。 *我们推荐大家使用LaTeX, markdown等对作业进行排版。 *为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。 *若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照..."

*每道题目的解答都要有完整的解题过程，中英文不限。

*我们推荐大家使用LaTeX, markdown等对作业进行排版。

*为督促大家认真完成平时作业、扎实掌握课程内容，本课程期末考试将从作业题目中随机抽取部分题目进行考查。请大家务必重视每一次作业，认真理解解题思路。

*若考试中被抽取到的作业题目答错、答不完整或无法作答，将按照相关标准对作业进行扣分处理。

== Assumption throughout Problem Set 3==
Without further notice, we are working on probability space <math>(\Omega,\mathcal{F},\mathbf{Pr})</math>.

Without further notice, we assume that the expectation of random variables are well-defined.

The term <math>\log</math> used in this context refers to the natural logarithm.

== Problem 1 (Warm-up Problems) ==
<ul>
<li>[Variance (I)]
Let <math>X_1,X_2,...,X_n</math> be independent random variables, and suppose that <math>X_k</math> is Bernoulli with parameter <math>p_k</math>. Let <math>Y= X_1 + X_2 + \dots + X_n</math>. Show that, for <math>\mathbb E[Y]</math> fixed, <math>\mathrm{Var}(Y )</math> is a maximized when <math>p_1 = p_2 = \dots = p_n</math>. That is to say, the variation in the sum is greatest when individuals are most alike.
</li>
<li>[Variance (II)]
Each member of a group of <math>n</math> players rolls a (fair) 6-sided die. For any pair of players who throw the same number, the group scores <math>1</math> point. Find the mean and variance of the total score of the group.
</li>
<li>[Variance (III)]
An urn contains <math>n</math> balls numbered <math>1, 2, \ldots, n</math>. We select <math>k</math> balls uniformly at random without replacement and add up their numbers. Find the mean and variance of the sum.
</li>
<li>[Variance (IV)]
Let <math>N</math> be an integer-valued, positive random variable and let <math>\{X_i\}_{i=1}^{\infty}</math> be indepedently identically distributed random variables that are independent of <math>N</math>, too.
Precisely, for any finite subset <math>I \subseteq\mathbb{N}_+</math>, <math>\{X_i\}_{i \in I}</math> and <math>N</math> are mutually independent. Let <math>X = \sum_{i=1}^N X_i</math>, show that <math>\textbf{Var}[X] = \textbf{Var}[X_1] \mathbb{E}[N] + \mathbb{E}[X_1]^2 \textbf{Var}[N]</math>.
</li>
<li> [Moments (I)]
Show that [math]G(t) = \frac{e^t}{4} + \frac{e^{-t}}{2} + \frac{1}{4}[/math] is a moment-generating function of a random variable, and write the probability mass function of this random variable.
</li>
<li>[Moments (II)]
Let <math>X\sim \text{Geo}(p)</math> for some <math>p \in (0,1)</math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Moments (III)]
Let <math>X\sim \text{Pois}(\lambda)</math> for some <math>\lambda >0 </math>. Find <math>\mathbb{E}[X^3]</math> and <math>\mathbb{E}[X^4]</math>.
</li>
<li>[Covariance and correlation (I)]
Let <math>X</math> and <math>Y</math> be discrete random variables with correlation <math>\rho</math>. Show that <math>|\rho|= 1</math> if and only if <math>X=aY+b</math> for some real numbers <math>a,b</math>.
</li>
<li>[Covariance and correlation (II)]
Let [math]X[/math] and [math]Y[/math] be discrete random variables with mean <math>0</math>, variance <math>1</math>, and correlation [math]\rho[/math]. Show that [math]\mathbb{E}(\max\{X^2,Y^2\})\leq 1+\sqrt{1-\rho^2}[/math]. (Hint: use the identity [math]\max\{a,b\} = \frac{1}{2}(a+b+|a-b|)[/math].)
</li>
<li>[Covariance and correlation (III)]
Let [math]X[/math] and [math]Y[/math] be independent Bernoulli random variables with parameter [math]1/2[/math]. Show that [math]X+Y[/math] and [math]|X-Y|[/math] are dependent though uncorrelated.
</li>
</ul>

== Problem 2 (Inequalities) ==
<ul>
<li>
[Reverse Markov's inequality] Let <math>X</math> be a discrete random variable with bounded range <math>0 \le X \le U</math> for some <math>U > 0</math>. Show that <math>\mathbf{Pr}(X \le a) \le \frac{U-\mathbf{E}[X]}{U-a}</math> for any <math>0 < a < U</math>.
</li>
<li>
[Markov's inequality] Let <math>X</math> be a discrete random variable. Show that for all <math>\beta \geq 0</math> and all <math>x > 0</math>, <math>\mathbf{Pr}(X\geq x)\leq \mathbb{E}(e^{\beta X})e^{-\beta x}</math>.
</li>
<li>
[Cantelli's inequality] Let <math>X</math> be a discrete random variable with mean <math>0</math> and variance <math>\sigma^2</math>. Prove that for any <math>\lambda > 0</math>, <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2}{\lambda^2+\sigma^2}</math>. (Hint: You may first show that <math>\mathbf{Pr}[X \ge \lambda] \le \frac{\sigma^2 + u^2}{(\lambda + u)^2}</math> for all <math>u > 0</math>.)
</li>
<li> [Union of events]
Let <math>A_1,A_2,\ldots,A_n</math> be events with <math>\mathbf{Pr}[A_i] > 0</math> for all <math>1 \le i \le n</math>. Define <math>a = \sum_{i=1}^n \mathbf{Pr}[A_i]</math> and <math>b = \sum_{1 \le i<j\le n} \mathbf{Pr}[A_i \cap A_j]</math>. Show that <math>\mathbf{Pr}[A_1 \cup A_2 \cup \ldots \cup A_n] \ge \max\left\{2-\frac{a+2b}{a^2}, \frac{a^2}{a+2b}\right\}</math>.
</li>
</ul>

== Problem 3 (Probability meets distinct sums) ==
<ul>

Let <math>f(n)</math> denote the maximal <math>m</math> such that there exists a set of <math>m</math> distinct numbers <math>\{x_1,x_2,\ldots,x_m\}</math>
in <math>[n] = \{1,2,\ldots,n\}</math> all of whose sums are distinct. Namely, <math>\sum_{i \in S} x_i</math> are distinct for all <math>S \subseteq \{1,2,\ldots,m\}</math>.
Use the second moment method (i.e., Chebyshev's inequality) to show that <math>f(n) \le \log_2 n + \frac{1}{2} \log_2 \log_2 n + O(1)</math>. (Remark: Erdös' [https://www.erdosproblems.com/1 first open problem] asks if <math>f(n) \le \log_2 n + C</math> for some universal constant <math>C</math>.)

</ul>

概率论与数理统计 (Spring 2026)/第一次作业提交名单

2026-04-01T08:03:11Z

Yqzhu:

如有错漏邮件请及时联系助教。
<center>
{| class="wikitable"
|-
! 学号 !! 姓名
|-
| 241240046 || 黄嘉诚
|-
| 241880173 || 陆知渔
|-
| 241240064 || 黄昕宇
|-
| 241240068 || 郑飞阳
|-
| 241240036 || 唐愉兵
|-
| 241850002 || 张子腾
|-
| 241840199 || 陈诣涵
|-
| 241240049 || 罗嘉恒
|-
| 241870077 || 张辰曦
|-
| 241220095 || 王天祥
|-
| 241098114 || 于静涵
|-
| 241820122 || 商世雄
|-
| 241870240 || 杨学舟
|-
| 241240019 || 王祎泽
|-
| 241240070 || 刘梦溪
|-
| 241880503 || 郑一鸣
|-
| 241870025 || 张科杰
|-
| 241840087 || 朱枻
|-
| 241880488 || 扶嘉年
|-
| 241220003 || 沈琪皓
|-
| 241240033 || 付雨彤
|-
| 241220004 || 张宸源
|-
| 241840052 || 李彦均
|-
| 241840173 || 刘明俊
|-
| 241870097 || 丁瀚铭
|-
| 231870061 || 朱俊杰
|-
| 241240048 || 康子凯
|-
| 241240061 || 周泽钰
|-
| 241840005 || 蔡云汉
|-
| 241240004 || 陈仝
|-
| 221180155 || 许云鹏
|-
| 241870230 || 闵文楷
|-
| 241240038 || 胡彦腾
|-
| 231250084 || 谢钦煌
|-
| 241240001 || 董清扬
|-
| 241220002 || 张瑞珉
|-
| 241240050 || 李柱锃
|-
| 241240041 || 李东泽
|-
| 241240069 || 陈姝婷
|-
| 241240053 || 张家奇
|-
| 241240032 || 崔佳雪
|-
| 241840065 || 荣恒嬉
|-
| 241240029 || 谢骐泽
|-
| 241840078 || 张惠泽
|-
| 241240051 || 何明航
|-
| 221830067 || 张笑
|-
| 241240035 || 周玟序
|-
| 241240008 || 张恒畅
|-
| 241240066 || 贺子铭
|-
| 241180041 || 赵瀚清
|-
| 241240028 || 冯时
|-
| 241240007 || 杨煦天
|-
| 241840240 || 李味鸿
|-
| 241870040 || 陆子一
|-
| 251275003 || 翟悦凯
|-
| 211830049 || 王杰
|-
| 241870032 || 赵益
|-
| 241220136 || 祁书轩
|-
| 241240017 || 江子林
|-
| 241240022 || 潘诚懿
|-
| 241276007 || 胡博
|-
| 241880325 || 刘孟阳
|-
| 241840067 || 赵思景
|-
| 241276008 || 袁颀沣
|-
| 241840113 || 曾睿鸣
|-
|}
</center>
共65人。

概率论与数理统计 (Spring 2026)/第一次作业提交名单

2026-04-01T04:50:15Z

Yqzhu:

如有错漏邮件请及时联系助教。
<center>
{| class="wikitable"
|-
! 学号 !! 姓名
|-
| 241240046 || 黄嘉诚
|-
| 241880173 || 陆知渔
|-
| 241240064 || 黄昕宇
|-
| 241240068 || 郑飞阳
|-
| 241240036 || 唐愉兵
|-
| 241850002 || 张子腾
|-
| 241840199 || 陈诣涵
|-
| 241240049 || 罗嘉恒
|-
| 241870077 || 张辰曦
|-
| 241220095 || 王天祥
|-
| 241098114 || 于静涵
|-
| 241820122 || 商世雄
|-
| 241870240 || 杨学舟
|-
| 241240019 || 王祎泽
|-
| 241240070 || 刘梦溪
|-
| 241880503 || 郑一鸣
|-
| 241870025 || 张科杰
|-
| 241840087 || 朱枻
|-
| 241880488 || 扶嘉年
|-
| 241220003 || 沈琪皓
|-
| 241240033 || 付雨彤
|-
| 241220004 || 张宸源
|-
| 241840052 || 李彦均
|-
| 241840173 || 刘明俊
|-
| 241870097 || 丁瀚铭
|-
| 231870061 || 朱俊杰
|-
| 241240048 || 康子凯
|-
| 241240061 || 周泽钰
|-
| 241840005 || 蔡云汉
|-
| 241240004 || 陈仝
|-
| 221180155 || 许云鹏
|-
| 241870230 || 闵文楷
|-
| 241240038 || 胡彦腾
|-
| 231250084 || 谢钦煌
|-
| 241240001 || 董清扬
|-
| 241220002 || 张瑞珉
|-
| 241240050 || 李柱锃
|-
| 241240041 || 李东泽
|-
| 241240069 || 陈姝婷
|-
| 241240053 || 张家奇
|-
| 241240032 || 崔佳雪
|-
| 241840065 || 荣恒嬉
|-
| 241240029 || 谢骐泽
|-
| 241840078 || 张惠泽
|-
| 241240051 || 何明航
|-
| 221830067 || 张笑
|-
| 241240035 || 周玟序
|-
| 241240008 || 张恒畅
|-
| 241240066 || 贺子铭
|-
| 241180041 || 赵瀚清
|-
| 241240028 || 冯时
|-
| 241240007 || 杨煦天
|-
| 241840240 || 李味鸿
|-
| 241870040 || 陆子一
|-
| 251275003 || 翟悦凯
|-
| 211830049 || 王杰
|-
| 241870032 || 赵益
|-
| 241220136 || 祁书轩
|-
| 241240017 || 江子林
|-
| 241276007 || 胡博
|-
| 241880325 || 刘孟阳
|-
| 241840067 || 赵思景
|-
| 241276008 || 袁颀沣
|-
| 241840113 || 曾睿鸣
|-
|}
</center>
共64人。

概率论与数理统计 (Spring 2026)

2026-04-01T04:22:56Z

Yqzhu: /* Assignments */

{{Infobox
|name = Infobox
|bodystyle =
|title = '''概率论与数理统计''' 
'''Probability Theory''' & '''Mathematical Statistics'''
|titlestyle =

|image =
|imagestyle =
|caption =
|captionstyle =
|headerstyle = background:#ccf;
|labelstyle = background:#ddf;
|datastyle =

|header1 =Instructor
|label1 =
|data1 =
|header2 =
|label2 =
|data2 = '''尹一通'''
|header3 =
|label3 = Email
|data3 = yinyt@nju.edu.cn
|header4 =
|label4 = office
|data4 = 计算机系 804
|header5 =
|label5 =
|data5 = '''刘景铖'''
|header6 =
|label6 = Email
|data6 = liu@nju.edu.cn
|header7 =
|label7 = office
|data7 = 计算机系 516
|header8 = Class
|label8 =
|data8 =
|header9 =
|label9 = Class meeting
|data9 = Wednesday, 9am-12am 
仙Ⅱ-212
|header10=
|label10 = Office hour
|data10 = TBA 计算机系 804（尹一通） 计算机系 516（刘景铖）
|header11= Textbook
|label11 =
|data11 =
|header12=
|label12 =
|data12 = [[File:概率导论.jpeg|border|100px]]
|header13=
|label13 =
|data13 = '''概率导论'''（第2版·修订版） Dimitri P. Bertsekas and John N. Tsitsiklis 郑忠国童行伟译；人民邮电出版社 (2022)
|header14=
|label14 =
|data14 = [[File:Grimmett_probability.jpg|border|100px]]
|header15=
|label15 =
|data15 = '''Probability and Random Processes''' (4E) Geoffrey Grimmett and David Stirzaker Oxford University Press (2020)
|header16=
|label16 =
|data16 = [[File:Probability_and_Computing_2ed.jpg|border|100px]]
|header17=
|label17 =
|data17 = '''Probability and Computing''' (2E) Michael Mitzenmacher and Eli Upfal Cambridge University Press (2017)
|belowstyle = background:#ddf;
|below =
}}

This is the webpage for the ''Probability Theory and Mathematical Statistics'' (概率论与数理统计) class of Spring 2026. Students who take this class should check this page periodically for content updates and new announcements.

= Announcement =
* TBA

= Course info =
* '''Instructor ''':
:* [http://tcs.nju.edu.cn/yinyt/ 尹一通]：[mailto:yinyt@nju.edu.cn <yinyt@nju.edu.cn>]，计算机系 804
:* [https://liuexp.github.io 刘景铖]：[mailto:liu@nju.edu.cn <liu@nju.edu.cn>]，计算机系 516
* '''Teaching assistant''':
** 鞠哲：[mailto:juzhe@smail.nju.edu.cn <juzhe@smail.nju.edu.cn>]，计算机系 426
** 祝永祺：[mailto:652025330045@smail.nju.edu.cn <652025330045@smail.nju.edu.cn>]，计算机系 426
* '''Class meeting''':
** 周三：9am-12am，仙Ⅱ-212
* '''Office hour''':
:* TBA, 计算机系 804（尹一通）
:* TBA, 计算机系 516（刘景铖）
:* '''QQ群''': 1090092561（申请加入需提供姓名、院系、学号）

= Syllabus =
课程内容分为三大部分：
* '''经典概率论'''：概率空间、随机变量及其数字特征、多维与连续随机变量、极限定理等内容
* '''概率与计算'''：测度集中现象 (concentration of measure)、概率法 (the probabilistic method)、离散随机过程的相关专题
* '''数理统计'''：参数估计、假设检验、贝叶斯估计、线性回归等统计推断等概念

对于第一和第二部分，要求清楚掌握基本概念，深刻理解关键的现象与规律以及背后的原理，并可以灵活运用所学方法求解相关问题。对于第三部分，要求熟悉数理统计的若干基本概念，以及典型的统计模型和统计推断问题。

经过本课程的训练，力求使学生能够熟悉掌握概率的语言，并会利用概率思维来理解客观世界并对其建模，以及驾驭概率的数学工具来分析和求解专业问题。

=== 教材与参考书 Course Materials ===
* '''[BT]''' 概率导论（第2版·修订版），[美]伯特瑟卡斯（Dimitri P.Bertsekas）[美]齐齐克利斯（John N.Tsitsiklis）著，郑忠国童行伟译，人民邮电出版社（2022）。
* '''[GS]''' ''Probability and Random Processes'', by Geoffrey Grimmett and David Stirzaker; Oxford University Press; 4th edition (2020).
* '''[MU]''' ''Probability and Computing: Randomization and Probabilistic Techniques in Algorithms and Data Analysis'', by Michael Mitzenmacher, Eli Upfal; Cambridge University Press; 2nd edition (2017).

=== 成绩 Grading Policy ===
* 课程成绩：本课程将会有若干次作业和一次期末考试。最终成绩将由平时作业成绩和期末考试成绩综合得出。
* 迟交：如果有特殊的理由，无法按时完成作业，请提前联系授课老师，给出正当理由。否则迟交的作业将不被接受。

=== 学术诚信 Academic Integrity ===
学术诚信是所有从事学术活动的学生和学者最基本的职业道德底线，本课程将不遗余力的维护学术诚信规范，违反这一底线的行为将不会被容忍。

作业完成的原则：署你名字的工作必须是你个人的贡献。在完成作业的过程中，允许讨论，前提是讨论的所有参与者均处于同等完成度。但关键想法的执行、以及作业文本的写作必须独立完成，并在作业中致谢（acknowledge）所有参与讨论的人。符合规则的讨论与致谢将不会影响得分。不允许其他任何形式的合作——尤其是与已经完成作业的同学“讨论”。

本课程将对剽窃行为采取零容忍的态度。在完成作业过程中，对他人工作（出版物、互联网资料、其他人的作业等）直接的文本抄袭和对关键思想、关键元素的抄袭，按照 [http://www.acm.org/publications/policies/plagiarism_policy ACM Policy on Plagiarism]的解释，都将视为剽窃。剽窃者成绩将被取消。如果发现互相抄袭行为， 抄袭和被抄袭双方的成绩都将被取消。因此请主动防止自己的作业被他人抄袭。

学术诚信影响学生个人的品行，也关乎整个教育系统的正常运转。为了一点分数而做出学术不端的行为，不仅使自己沦为一个欺骗者，也使他人的诚实努力失去意义。让我们一起努力维护一个诚信的环境。

= Assignments =
*[[概率论与数理统计 (Spring 2026)/Problem Set 1|Problem Set 1]] 请在 2026/4/1 上课之前(9am UTC+8)提交到 [mailto:pr2026_nju@163.com pr2026_nju@163.com] (文件名为'学号_姓名_A1.pdf').
** [[概率论与数理统计 (Spring 2026)/第一次作业提交名单|第一次作业提交名单]]

= Lectures =
# [http://tcs.nju.edu.cn/slides/prob2026/Intro.pdf 课程简介]
# [http://tcs.nju.edu.cn/slides/prob2026/ProbSpace.pdf 概率空间]
#* 阅读：'''[BT] 第1章''' 或 '''[GS] Chapter 1'''
#* [[概率论与数理统计 (Spring 2026)/Entropy and volume of Hamming balls|Entropy and volume of Hamming balls]]
#* [[概率论与数理统计 (Spring 2026)/Karger's min-cut algorithm| Karger's min-cut algorithm]]
# [http://tcs.nju.edu.cn/slides/prob2026/RandVar.pdf 随机变量]
#* 阅读：'''[BT] 第2章''' 或 '''[GS] Chapter 2, Sections 3.1~3.5, 3.7'''
#* 阅读：'''[MU] Chapter 2'''

= Concepts =
* [https://plato.stanford.edu/entries/probability-interpret/ Interpretations of probability]
* [https://en.wikipedia.org/wiki/History_of_probability History of probability]
* Example problems:
** [https://dornsifecms.usc.edu/assets/sites/520/docs/VonNeumann-ams12p36-38.pdf von Neumann's Bernoulli factory] and other [https://peteroupc.github.io/bernoulli.html Bernoulli factory algorithms]
** [https://en.wikipedia.org/wiki/Boy_or_Girl_paradox Boy or Girl paradox]
** [https://en.wikipedia.org/wiki/Monty_Hall_problem Monty Hall problem]
** [https://en.wikipedia.org/wiki/Bertrand_paradox_(probability) Bertrand paradox]
** [https://en.wikipedia.org/wiki/Hard_spheres Hard spheres model] and [https://en.wikipedia.org/wiki/Ising_model Ising model]
** [https://en.wikipedia.org/wiki/PageRank ''PageRank''] and stationary [https://en.wikipedia.org/wiki/Random_walk random walk]
** [https://en.wikipedia.org/wiki/Diffusion_process Diffusion process] and [https://en.wikipedia.org/wiki/Diffusion_model diffusion model]
*[https://en.wikipedia.org/wiki/Probability_space Probability space]
** [https://en.wikipedia.org/wiki/Sample_space Sample space]
** [https://en.wikipedia.org/wiki/Event_(probability_theory) Event] and [https://en.wikipedia.org/wiki/Σ-algebra <math>\sigma</math>-algebra]
** Kolmogorov's [https://en.wikipedia.org/wiki/Probability_axioms axioms of probability]
* [https://en.wikipedia.org/wiki/Discrete_uniform_distribution Classical] and [https://en.wikipedia.org/wiki/Geometric_probability goemetric probability]
* [https://en.wikipedia.org/wiki/Boole%27s_inequality Union bound]
** [https://en.wikipedia.org/wiki/Inclusion%E2%80%93exclusion_principle Inclusion-Exclusion principle]
** [https://en.wikipedia.org/wiki/Boole%27s_inequality#Bonferroni_inequalities Bonferroni inequalities]
* [https://en.wikipedia.org/wiki/Conditional_probability Conditional probability]
** [https://en.wikipedia.org/wiki/Chain_rule_(probability) Chain rule]
** [https://en.wikipedia.org/wiki/Law_of_total_probability Law of total probability]
** [https://en.wikipedia.org/wiki/Bayes%27_theorem Bayes' law]
* [https://en.wikipedia.org/wiki/Independence_(probability_theory) Independence]
** [https://en.wikipedia.org/wiki/Pairwise_independence Pairwise independence]

概率论与数理统计 (Spring 2026)/第一次作业提交名单

2026-04-01T04:21:38Z

Yqzhu:

如有错漏邮件请及时联系助教。
<center>
{| class="wikitable"
|-
! 学号 !! 姓名
|-
| 241240046 || 黄嘉诚
|-
| 241880173 || 陆知渔
|-
| 241240064 || 黄昕宇
|-
| 241240068 || 郑飞阳
|-
| 241240036 || 唐愉兵
|-
| 241850002 || 张子腾
|-
| 241840199 || 陈诣涵
|-
| 241240049 || 罗嘉恒
|-
| 241870077 || 张辰曦
|-
| 241220095 || 王天祥
|-
| 241098114 || 于静涵
|-
| 241820122 || 商世雄
|-
| 241870240 || 杨学舟
|-
| 241240019 || 王祎泽
|-
| 241240070 || 刘梦溪
|-
| 241880503 || 郑一鸣
|-
| 241870025 || 张科杰
|-
| 241840087 || 朱枻
|-
| 241880488 || 扶嘉年
|-
| 241220003 || 沈琪皓
|-
| 241240033 || 付雨彤
|-
| 241220004 || 张宸源
|-
| 241840052 || 李彦均
|-
| 241840173 || 刘明俊
|-
| 241870097 || 丁瀚铭
|-
| 231870061 || 朱俊杰
|-
| 241240048 || 康子凯
|-
| 241240061 || 周泽钰
|-
| 241840005 || 蔡云汉
|-
| 241240004 || 陈仝
|-
| 221180155 || 许云鹏
|-
| 241870230 || 闵文楷
|-
| 241240038 || 胡彦腾
|-
| 231250084 || 谢钦煌
|-
| 241240001 || 董清扬
|-
| 241220002 || 张瑞珉
|-
| 241240050 || 李柱锃
|-
| 241240041 || 李东泽
|-
| 241240069 || 陈姝婷
|-
| 241240053 || 张家奇
|-
| 241240032 || 崔佳雪
|-
| 241840065 || 荣恒嬉
|-
| 241240029 || 谢骐泽
|-
| 241840078 || 张惠泽
|-
| 241240051 || 何明航
|-
| 221830067 || 张笑
|-
| 241240035 || 周玟序
|-
| 241240008 || 张恒畅
|-
| 241240066 || 贺子铭
|-
| 241180041 || 赵瀚清
|-
| 241240028 || 冯时
|-
| 241840240 || 李味鸿
|-
| 241870040 || 陆子一
|-
| 251275003 || 翟悦凯
|-
| 211830049 || 王杰
|-
| 241870032 || 赵益
|-
| 241220136 || 祁书轩
|-
| 241240017 || 江子林
|-
| 241276007 || 胡博
|-
| 241880325 || 刘孟阳
|-
| 241840067 || 赵思景
|-
| 241276008 || 袁颀沣
|-
| 241840113 || 曾睿鸣
|-
|}
</center>
共63人。

概率论与数理统计 (Spring 2026)/第一次作业提交名单

2026-04-01T04:21:30Z

Yqzhu:

如有错漏邮件请及时联系助教。
<center>
{| class="wikitable"
|-
! 学号 !! 姓名
|-
| 241240046 || 黄嘉诚
|-
| 241880173 || 陆知渔
|-
| 241240064 || 黄昕宇
|-
| 241240068 || 郑飞阳
|-
| 241240036 || 唐愉兵
|-
| 241850002 || 张子腾
|-
| 241840199 || 陈诣涵
|-
| 241240049 || 罗嘉恒
|-
| 241870077 || 张辰曦
|-
| 241220095 || 王天祥
|-
| 241098114 || 于静涵
|-
| 241820122 || 商世雄
|-
| 241870240 || 杨学舟
|-
| 241240019 || 王祎泽
|-
| 241240070 || 刘梦溪
|-
| 241880503 || 郑一鸣
|-
| 241870025 || 张科杰
|-
| 241840087 || 朱枻
|-
| 241880488 || 扶嘉年
|-
| 241220003 || 沈琪皓
|-
| 241240033 || 付雨彤
|-
| 241220004 || 张宸源
|-
| 241840052 || 李彦均
|-
| 241840173 || 刘明俊
|-
| 241870097 || 丁瀚铭
|-
| 231870061 || 朱俊杰
|-
| 241240048 || 康子凯
|-
| 241240061 || 周泽钰
|-
| 241840005 || 蔡云汉
|-
| 241240004 || 陈仝
|-
| 221180155 || 许云鹏
|-
| 241870230 || 闵文楷
|-
| 241240038 || 胡彦腾
|-
| 231250084 || 谢钦煌
|-
| 241240001 || 董清扬
|-
| 241220002 || 张瑞珉
|-
| 241240050 || 李柱锃
|-
| 241240041 || 李东泽
|-
| 241240069 || 陈姝婷
|-
| 241240053 || 张家奇
|-
| 241240032 || 崔佳雪
|-
| 241840065 || 荣恒嬉
|-
| 241240029 || 谢骐泽
|-
| 241840078 || 张惠泽
|-
| 241240051 || 何明航
|-
| 221830067 || 张笑
|-
| 241240035 || 周玟序
|-
| 241240008 || 张恒畅
|-
| 241240066 || 贺子铭
|-
| 241180041 || 赵瀚清
|-
| 241240028 || 冯时
|-
| 241840240 || 李味鸿
|-
| 241870040 || 陆子一
|-
| 251275003 || 翟悦凯
|-
| 211830049 || 王杰
|-
| 241870032 || 赵益
|-
| 241220136 || 祁书轩
|-
| 241240017 || 江子林
|-
| 241276007 || 胡博
|-
| 241880325 || 刘孟阳
|-
| 241840067 || 赵思景
|-
| 241276008 || 袁颀沣
|-
| 241840113 || 曾睿鸣
|-
|}
</center>
共63人

概率论与数理统计 (Spring 2026)/第一次作业提交名单

2026-04-01T04:20:41Z

Yqzhu:

如有错漏邮件请及时联系助教。
<center>
{| class="wikitable"
|-
! 学号 !! 姓名
|-
| 241240046 || 黄嘉诚
|-
| 241880173 || 陆知渔
|-
| 241240064 || 黄昕宇
|-
| 241240068 || 郑飞阳
|-
| 241240036 || 唐愉兵
|-
| 241850002 || 张子腾
|-
| 241840199 || 陈诣涵
|-
| 241240049 || 罗嘉恒
|-
| 241870077 || 张辰曦
|-
| 241220095 || 王天祥
|-
| 241098114 || 于静涵
|-
| 241820122 || 商世雄
|-
| 241870240 || 杨学舟
|-
| 241240019 || 王祎泽
|-
| 241240070 || 刘梦溪
|-
| 241880503 || 郑一鸣
|-
| 241870025 || 张科杰
|-
| 241840087 || 朱枻
|-
| 241880488 || 扶嘉年
|-
| 241220003 || 沈琪皓
|-
| 241240033 || 付雨彤
|-
| 241220004 || 张宸源
|-
| 241840052 || 李彦均
|-
| 241840173 || 刘明俊
|-
| 241870097 || 丁瀚铭
|-
| 231870061 || 朱俊杰
|-
| 241240048 || 康子凯
|-
| 241240061 || 周泽钰
|-
| 241840005 || 蔡云汉
|-
| 241240004 || 陈仝
|-
| 221180155 || 许云鹏
|-
| 241870230 || 闵文楷
|-
| 241240038 || 胡彦腾
|-
| 231250084 || 谢钦煌
|-
| 241240001 || 董清扬
|-
| 241220002 || 张瑞珉
|-
| 241240050 || 李柱锃
|-
| 241240041 || 李东泽
|-
| 241240069 || 陈姝婷
|-
| 241240053 || 张家奇
|-
| 241240032 || 崔佳雪
|-
| 241840065 || 荣恒嬉
|-
| 241240029 || 谢骐泽
|-
| 241840078 || 张惠泽
|-
| 241240051 || 何明航
|-
| 221830067 || 张笑
|-
| 241240035 || 周玟序
|-
| 241240008 || 张恒畅
|-
| 241240066 || 贺子铭
|-
| 241180041 || 赵瀚清
|-
| 241240028 || 冯时
|-
| 241840240 || 李味鸿
|-
| 241870040 || 陆子一
|-
| 251275003 || 翟悦凯
|-
| 211830049 || 王杰
|-
| 241870032 || 赵益
|-
| 241220136 || 祁书轩
|-
| 241240017 || 江子林
|-
| 241276007 || 胡博
|-
| 241880325 || 刘孟阳
|-
| 241840067 || 赵思景
|-
| 241276008 || 袁颀沣
|-
| 241840113 || 曾睿鸣
|-
|}
</center>

概率论与数理统计 (Spring 2026)/第一次作业提交名单

2026-03-31T14:22:07Z

Yqzhu:

概率论与数理统计 (Spring 2026)/第一次作业提交名单

2026-03-31T14:06:43Z

Yqzhu:

概率论与数理统计 (Spring 2026)/第一次作业提交名单

2026-03-31T12:56:57Z

Yqzhu:

概率论与数理统计 (Spring 2026)/第一次作业提交名单

2026-03-31T11:54:29Z

Yqzhu:

概率论与数理统计 (Spring 2026)/第一次作业提交名单

2026-03-31T05:42:56Z

Yqzhu: