https://tcs.nju.edu.cn/wiki/api.php?action=feedcontributions&feedformat=atom&user=EtoneTCS Wiki - User contributions [en]2026-11-02T23:21:53ZUser contributionsMediaWiki 1.46.2https://tcs.nju.edu.cn/wiki/index.php?title=Assignment_3,_Fall_2026&diff=14024Assignment 3, Fall 20262026-11-01T15:06:37Z<p>Etone: Created page with "Held-Hou(1980) 讨论了当外部强迫的经向分布呈二次勒让德多项式，即<math>\dfrac{\Theta_E(\phi,z)}{\Theta_o}=1-\dfrac{2}{3}\Delta_H P_2(\sin \phi)+\Delta_v(\dfrac{z}{H}-\dfrac{1}{2})</math>的情况下，哈德莱环流内的风场、温度场、环流的空间范围等将怎样随纬度和外力强迫的强度而变化。 如果将外力强迫的空间分布改为<math>\dfrac{\Theta_E(\phi,z)}{\Theta_o}=1-\Delta_H(\sin^3 \phi - \dfrac{1}{4})+\Delta_v..."</p>
<hr />
<div>Held-Hou(1980) 讨论了当外部强迫的经向分布呈二次勒让德多项式，即<math>\dfrac{\Theta_E(\phi,z)}{\Theta_o}=1-\dfrac{2}{3}\Delta_H P_2(\sin \phi)+\Delta_v(\dfrac{z}{H}-\dfrac{1}{2})</math>的情况下，哈德莱环流内的风场、温度场、环流的空间范围等将怎样随纬度和外力强迫的强度而变化。<br />
<br />
如果将外力强迫的空间分布改为<math>\dfrac{\Theta_E(\phi,z)}{\Theta_o}=1-\Delta_H(\sin^3 \phi - \dfrac{1}{4})+\Delta_v(\dfrac{z}{H}-\dfrac{1}{2})</math>，<br />
<br />
<br />
#请推导出哈德莱环流内的高空风场和垂直平均位温场<math>\dfrac{\tilde{\Theta}}{\Theta_o} </math>将如何随纬度分布;<br />
#同样利用小角度假设，请推导出环流的空间范围<math>\phi_H </math> 的表达式。如果设 <math>r \equiv \dfrac{gH}{\Omega^2a^2} </math>, 请分别画出当 <math>\Delta_H=1/3</math> 和 <math>\Delta_H=1/6</math> 时, 与Held-Hou的情况相比，<math>\phi_H</math> 怎样随 <math>r</math> 而变化。<br />
#<font color="red" size="2">选做题目：</font>在此情况下，近地面风场的分布有怎样变化。</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=General_Circulation(Fall_2026)&diff=14023General Circulation(Fall 2026)2026-11-01T15:05:02Z<p>Etone: /* Assignments */</p>
<hr />
<div>{{Infobox<br />
|name = Infobox<br />
|bodystyle = <br />
|title = 大气环流 <br><br />
General Circulation of the Atmosphere<br />
|titlestyle = <br />
<br />
|headerstyle = background:#ccf;<br />
|labelstyle = background:#ddf;<br />
|datastyle = <br />
<br />
|header1 =Instructor<br />
|label1 = <br />
|data1 = <br />
|header2 = <br />
|label2 = <br />
|data2 = 张洋<br />
|header3 = <br />
|label3 = Email<br />
|data3 = yangzhang@nju.edu.cn <br />
|header4 =<br />
|label4= office<br />
|data4= 仙林大气楼 B410<br />
|header5 = Class<br />
|label5 = <br />
|data5 = <br />
|header6 =<br />
|label6 = Class meetings<br />
|data6 = 周一 下午 2:00-4:00，逸B-303<br />
|header7 =<br />
|label7 = Place<br />
|data7 = <br />
|header8 =<br />
|label8 = Office hours<br />
|data8 = 周四 下午13:00-14:00 <br>仙林大气楼B410 <br />
|header9 = Reference book<br />
|label9 = <br />
|data9 = <br />
|header10 =<br />
|label10 = <br />
|data10 = {{Infobox<br />
|name = <br />
|bodystyle = <br />
|title = <br />
|titlestyle = <br />
|image = [[File:James-Circulating.jpg|border|100px]]<br />
|imagestyle = <br />
|caption = Introduction to Circulating Atmospheres, <br>''I. James'', Cambridge Press, 1995<br />
|captionstyle = }}<br />
|header11 =<br />
|label11 = <br />
|data11 = {{Infobox<br />
|name = <br />
|bodystyle = <br />
|title = <br />
|titlestyle = <br />
|image = [[File:Oort.jpg|border|100px]]<br />
|imagestyle = <br />
|caption =Physics of Climate, ''Peixoto, J. P.'' and ''A. H. Oort'', Springer-Verlag New York, 1992<br />
|captionstyle = <br />
}}<br />
|header12 =<br />
|label12 = <br />
|data12 = {{Infobox<br />
|name = <br />
|bodystyle = <br />
|title = <br />
|titlestyle = <br />
|image = [[File:geoff.jpg|border|100px]]<br />
|imagestyle = <br />
|caption =Atmospheric and Oceanic Fluid Dynamics: Fundamentals and Large-scale Circulation, ''Vallis, G. K.'', Cambridge University Press, 2006<br />
|captionstyle = <br />
}}<br />
|belowstyle = background:#ddf;<br />
|below = <br />
}}<br />
<br />
<br />
This is the page for the class ''General Circulation of the Atmosphere (大气环流)'' for the Fall 2026 semester. Students who take this class should check this page periodically for content updates and new announcements.<br />
<br />
= Announcement =<br />
* 课程交流QQ群：1075545767，请各位选课的同学入群。【2026.9.21】<br />
<br />
= Course info =<br />
* '''Instructor ''': 张洋，<br />
:*office: 仙林气象楼 B410<br />
:*email: yangzhang@nju.edu.cn<br />
* '''Class meeting''': 周一 下午 2:00-4:00；逸B-303<br />
* '''Office hour''': 周四 <font color="red" size="2>下午1:00-2:00，仙林大气楼B410</font><br />
* '''Prerequisites''': 动力气象，天气学，气候学<br />
* '''Grading''': 平时作业（50％）＋ 期末考试（50％）<br />
本课程将大致布置4次作业，每次作业一二道题目左右。题目将选择每个课题最具有代表性、需要一定思维强度和动手能力的训练用题目,意在使学生通过顺利完成作业来建立环流系统的物理模型、以对课程内容得到深刻全面地理解和掌握。期末考试题目数量将会比平时作业多,覆盖面更广,但会比作业题目简单,只涉及对基本内容的掌握和对环流理论的直接应用。<br />
<br />
= Assignments =<br />
# [[Assignment 1, Fall 2026|Reanalysis data and the earth's climatology]] [Due:2026.10.12]<font color="red" size="1.5">请在本次作业的截止日期前将本次作业的图片部分和文字部分打包作为附件发送至邮箱 circulation_nju@126.com </font><br />
# [[Assignment 2, Fall 2026|Simple energy balance climate model]][Due:2026.10.26]<br />
# [[Assignment 3, Fall 2026|Hadley Cell]][Due:2026.11.16]<br />
<br />
= Course intro =<br />
“大气环流”常指地球大气较大空间范围、较长时间尺度上的空气流动，及其对地球大气热量、动量、能量和水汽的全球输送。虽然从十七、十八世纪起人们就开始研究大尺度的大气运动(如Hadley在1735年提出的信风理论)，但大气环流真正发展成为一门较完备的学科方向却是近半个世纪的事情。随着四五十年代探空资料等高空气象要素的取得，以及六十年代卫星等覆盖全球的观测资料的加入，大气环流的空间结构和时间变化开始被系统、全面地揭示。与此同时，大气环流的数值模拟，也开始成为研究大气环流的一个主要方法，并发展至今成为了解和预估未来气候变化的主要手段。随着观测和模拟手段的进步，大气环流的理论研究也在近三十年开始快速地发展，人们对各种环流系统的维持和变化有了更全面、更深刻、也更为现代的理解。<br />
<br />
现代的大气环流是大气动力学、天气学和气候学相结合的产物。大气环流，既是各种天气现象产生的背景流场，又是各种气候状态形成的动力机制。大气环流在低频、季节、年际、年代际等时间尺度的变化，不但会引起天气现象的变化，也影响着气候状态的形成。而在大气科学领域面临着诸如全球暖化、气候变化、环流异常等重大科学问题的今天，大气环流研究的重要性被推到了前所未有的高度,大气环流也成为活跃发展又充满挑战的学科方向。<br />
<br />
本课程将讲述在过去几十年里大气环流在观测、理论和模拟上取得的进展。希望学生借此课程能熟悉大气环流的基本分布和形态，掌握各主要环流系统的维持和变化机制，建立各环流系统形成的物理模型，了解现阶段的大气环流模式，知道大气环流方向有待解决的科学问题。<br />
<br />
作为一门课程，大气环流内容的讲述常可以有两条线索。一条是全球尺度上大气热量、动量、能量和水汽的分布与输送，Lorenz（1967）和 Peixoto and Oort（1992）是按此线索介绍大气环流的优秀教材；另一条线索，是各纬度、各区域内大气环流系统的形成、维持和变化机制，James（1995）和 Vallis（2006）是按此线索介绍大气环流的经典讲义。根据现阶段大气环流方向的研究特点，本课程的讲述将主要按照后一种方式来展开，并辅以介绍各环流系统对大气各要素场的输送。在介绍各环流系统时，本课程将以观测、理论和模拟为顺序，从各大气环流系统的观测事实入手，介绍大气环流系统的分布特征和时空变化特征；着重介绍关于环流系统的各种动力学模型和现阶段对环流系统的理解；辅以对环流系统模拟研究的介绍；最后通过三者的对比，讨论各环流系统有待研究的问题。<br />
<br />
= Syllabus =<br />
本课程具体的内容安排如下：第一章为大气环流的概述，介绍大气环流发展的历史、包含的内容以及大气环流研究的常用观测资料和分析方法。第二章介绍大气环流产生的外部强迫：辐射强迫和下界面过程。第三至六章介绍大气环流中的各个环流系统及它们的动力机制。第七章详细介绍各复杂度的大气环流模式。第八章介绍大气环流领域现阶段最大的一个开放课题：全球暖化背景下的大气环流。这一章既是对前几章所介绍的大气环流理论的应用与检验，又是对未来大气环流研究方向的探讨。借此让学生熟悉并理解大气环流领域亟需解决的课题。具体课程安排和参考书目如下。<br />
== Course schedule ==<br />
*大气环流概述 (Introduction) （6课时）<br />
*大气环流的外部强迫（3课时）<br />
**辐射强迫 (Radiative forcing)<br />
**下界面过程 (Surface boundaries)<br />
*经向环流系统 (Zonally-averaged circulations)<br />
**Hadley 环流（4课时）<br />
**Ferrel 环流，急流，中纬度的波流相互作用（8课时）<br />
*纬向环流系统（Non-zonal circulations）（6课时）<br />
**Storm tracks<br />
**Monsoon<br />
**ENSO and Walker circulation<br />
*能量和水汽循环 (Angular momentum, energy and water vapor)（2课时）<br />
*不同复杂度的大气环流模式 (General circulation in a hierarchy of models)（2课时）<br />
*全球暖化背景下的大气环流 (General circulation in the global warming scenario)（2课时）<br />
<br />
[[点击此处看详细课程安排 (click for more)]]<br />
<br />
== References ==<br />
*观测部分：Peixoto, J. P. and A. H. Oort, 1992: Physics of Climate. Springer-Verlag New York, Inc., 520 pp. 中文译本：气候物理学，1995，吴国雄、刘辉等译校，气象出版社。<br />
*综合介绍：James, I., 1995: Introduction to circulating atmospheres. Cambridge University Press, 448 pp. <br />
*理论部分：Vallis, G. K., 2006: Atmospheric and Oceanic Fluid Dynamics: Fundamentals and Large-scale Circulation. Cambridge University Press. 745 pp.<br />
* [[其它参考书目（点击看详情）]]</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=Assignment_2,_Fall_2026&diff=14007Assignment 2, Fall 20262026-10-11T05:10:05Z<p>Etone: </p>
<hr />
<div><font color="red" size="2">请在题目一和题目二中任选一题作为本章的作业题目。另外的一题可作为选做题。</font><br />
<br />
==Question 1==<br />
假设在大气层顶（TOA），在多年全年平均的情况下，入射的太阳辐射随纬度的分布满足 <br />
<br />
<math>\begin{align}<br />
Q&=Q_o \cdot s(x), \\<br />
s(x)&=s_o \cdot P_o(x)+ s_2 \cdot P_2(x),<br />
\end{align}</math> <br />
<br />
其中，<math>P_o(x)=1</math>, <math>P_2(x)=\frac{1}{2}(3x^2-1)</math>, <math>s_o=1</math>, <math>s_2=-0.473</math>, <math>x=sin \phi</math>, <math>\phi</math> 为纬度。<br />
<br />
如果假设大气层顶的向外净长波辐射为<math>I</math>，其随纬度的分布满足 <br />
<br />
<math>\begin{align}<br />
I&=Q_o \cdot i(x), \\<br />
i(x)&=i_o \cdot P_o(x)+ i_2 \cdot P_2(x),<br />
\end{align}</math><br />
<br />
其中， <math>i_o=0.687</math>, <math>i_2=-0.165</math>。 假设行星反照率为<math>\alpha</math>，且忽略它随纬度和经度的变化:<br />
<br />
#请写出在能量平衡的情况下，大气和海洋的总经向能量输送应满足什么条件？<br />
#按此条件，请利用合适的边界条件，请推导出大气和海洋的总经向能量输送将如何随纬度<math>\phi</math>和太阳辐射强度<math>Q_o</math>变化。<br />
<br />
==Question 2==<br />
<br />
在第二章中，我们学习了Budyko的能量平衡气候模型（Simple Energy Balance Climate Model），并用此模型来讨论了存在冰雪线的情况下，气候系统的稳定性问题。冰雪线的南北移动会引起行星反照率及热量的动力输送的变化。我们发现这两种过程的相互作用能够引起气候系统的不稳定。如果使用同样的参数，即<math>A=211.1 W/m^{2}\,</math>, <math>B=1.55 W/(m^{2}{^{\circ}\text{C}})</math>, <math>Q_o=340 W/m^2\,</math>， <math>T_{\text{snow}}=-10^{\circ}\text{C}</math>, <math>s(x)=1-0.241(3x^2-1)\,</math>, 并同样假设:<br />
<br />
<math>\mathcal{A}(T)=<br />
\begin{cases}<br />
\alpha& T<T_{\text{snow}},\\<br />
\beta & T>T_{\text{snow}},\\<br />
\dfrac{\alpha+\beta}{2} & T=T_{\text{snow}}.<br />
\end{cases}</math><br />
<br />
如果行星反照率发生了改变，使得<math>\alpha=0.45 </math>， 请讨论：<br />
# 为了保持全球平均温度<math>\bar{T}</math>不变，对于当前的<math>Q</math>和冰雪线, <math>\beta</math>应该取多少？<br />
# 对于新的 <math>\alpha</math>和<math>\beta</math>值，<math>C</math>的取值应该为多少？<br />
# 请计算出新的<math>Q(x_s)</math>。<br />
# 请与课堂上所讲的<math>\alpha=0.4</math>, <math>\beta=0.7</math>的情况相比较（请将两种情况下<math>Q(x_s)</math>的曲线画在同一张图上），讨论两种情况的异同，尤其是气候稳定性在<math>\alpha</math>和<math>\beta</math>值改变后发生了何种变化，请讨论为什么会发生这些变化？</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=Assignment_2,_Fall_2026&diff=14006Assignment 2, Fall 20262026-10-11T04:27:45Z<p>Etone: Created page with "<font color="red" size="2">请在题目一和题目二中任选一题作为本章的作业题目。另外的一题可作为选做题。</font> ==Question 1== 在第二章中，我们学习了Budyko的能量平衡气候模型（Simple Energy Balance Climate Model），并用此模型来讨论了存在冰雪线的情况下，气候系统的稳定性问题。冰雪线的南北移动会引起行星反照率及热量的动力输送的变化。我们发现这两种过程的..."</p>
<hr />
<div><font color="red" size="2">请在题目一和题目二中任选一题作为本章的作业题目。另外的一题可作为选做题。</font><br />
<br />
==Question 1==<br />
<br />
在第二章中，我们学习了Budyko的能量平衡气候模型（Simple Energy Balance Climate Model），并用此模型来讨论了存在冰雪线的情况下，气候系统的稳定性问题。冰雪线的南北移动会引起行星反照率及热量的动力输送的变化。我们发现这两种过程的相互作用能够引起气候系统的不稳定。如果使用同样的参数，即<math>A=211.1 W/m^{2}\,</math>, <math>B=1.55 W/(m^{2}{^{\circ}\text{C}})</math>, <math>Q_o=340 W/m^2\,</math>， <math>T_{\text{snow}}=-10^{\circ}\text{C}</math>, <math>s(x)=1-0.241(3x^2-1)\,</math>, 并同样假设:<br />
<br />
<math>\mathcal{A}(T)=<br />
\begin{cases}<br />
\alpha& T<T_{\text{snow}},\\<br />
\beta & T>T_{\text{snow}},\\<br />
\dfrac{\alpha+\beta}{2} & T=T_{\text{snow}}.<br />
\end{cases}</math><br />
<br />
如果行星反照率发生了改变，使得<math>\alpha=0.45 </math>， 请讨论：<br />
# 为了保持全球平均温度<math>\bar{T}</math>不变，对于当前的<math>Q</math>和冰雪线, <math>\beta</math>应该取多少？<br />
# 对于新的 <math>\alpha</math>和<math>\beta</math>值，<math>C</math>的取值应该为多少？<br />
# 请计算出新的<math>Q(x_s)</math>。<br />
# 请与课堂上所讲的<math>\alpha=0.4</math>, <math>\beta=0.7</math>的情况相比较（请将两种情况下<math>Q(x_s)</math>的曲线画在同一张图上），讨论两种情况的异同，尤其是气候稳定性在<math>\alpha</math>和<math>\beta</math>值改变后发生了何种变化，请讨论为什么会发生这些变化？<br />
<br />
==Question 2==<br />
假设在大气层顶（TOA），在多年全年平均的情况下，入射的太阳辐射随纬度的分布满足 <br />
<br />
<math>\begin{align}<br />
Q&=Q_o \cdot s(x), \\<br />
s(x)&=s_o \cdot P_o(x)+ s_2 \cdot P_2(x),<br />
\end{align}</math> <br />
<br />
其中，<math>P_o(x)=1</math>, <math>P_2(x)=\frac{1}{2}(3x^2-1)</math>, <math>s_o=1</math>, <math>s_2=-0.473</math>, <math>x=sin \phi</math>, <math>\phi</math> 为纬度。<br />
<br />
如果假设大气层顶的向外净长波辐射为<math>I</math>，其随纬度的分布满足 <br />
<br />
<math>\begin{align}<br />
I&=Q_o \cdot i(x), \\<br />
i(x)&=i_o \cdot P_o(x)+ i_2 \cdot P_2(x),<br />
\end{align}</math><br />
<br />
其中， <math>i_o=0.687</math>, <math>i_2=-0.165</math>。 假设行星反照率为<math>\alpha</math>，且忽略它随纬度和经度的变化:<br />
<br />
#请写出在能量平衡的情况下，大气和海洋的总经向能量输送应满足什么条件？<br />
#按此条件，请利用合适的边界条件，请推导出大气和海洋的总经向能量输送将如何随纬度<math>\phi</math>和太阳辐射强度<math>Q_o</math>变化。</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=General_Circulation(Fall_2026)&diff=14005General Circulation(Fall 2026)2026-10-11T04:26:20Z<p>Etone: /* Assignments */</p>
<hr />
<div>{{Infobox<br />
|name = Infobox<br />
|bodystyle = <br />
|title = 大气环流 <br><br />
General Circulation of the Atmosphere<br />
|titlestyle = <br />
<br />
|headerstyle = background:#ccf;<br />
|labelstyle = background:#ddf;<br />
|datastyle = <br />
<br />
|header1 =Instructor<br />
|label1 = <br />
|data1 = <br />
|header2 = <br />
|label2 = <br />
|data2 = 张洋<br />
|header3 = <br />
|label3 = Email<br />
|data3 = yangzhang@nju.edu.cn <br />
|header4 =<br />
|label4= office<br />
|data4= 仙林大气楼 B410<br />
|header5 = Class<br />
|label5 = <br />
|data5 = <br />
|header6 =<br />
|label6 = Class meetings<br />
|data6 = 周一 下午 2:00-4:00，逸B-303<br />
|header7 =<br />
|label7 = Place<br />
|data7 = <br />
|header8 =<br />
|label8 = Office hours<br />
|data8 = 周四 下午13:00-14:00 <br>仙林大气楼B410 <br />
|header9 = Reference book<br />
|label9 = <br />
|data9 = <br />
|header10 =<br />
|label10 = <br />
|data10 = {{Infobox<br />
|name = <br />
|bodystyle = <br />
|title = <br />
|titlestyle = <br />
|image = [[File:James-Circulating.jpg|border|100px]]<br />
|imagestyle = <br />
|caption = Introduction to Circulating Atmospheres, <br>''I. James'', Cambridge Press, 1995<br />
|captionstyle = }}<br />
|header11 =<br />
|label11 = <br />
|data11 = {{Infobox<br />
|name = <br />
|bodystyle = <br />
|title = <br />
|titlestyle = <br />
|image = [[File:Oort.jpg|border|100px]]<br />
|imagestyle = <br />
|caption =Physics of Climate, ''Peixoto, J. P.'' and ''A. H. Oort'', Springer-Verlag New York, 1992<br />
|captionstyle = <br />
}}<br />
|header12 =<br />
|label12 = <br />
|data12 = {{Infobox<br />
|name = <br />
|bodystyle = <br />
|title = <br />
|titlestyle = <br />
|image = [[File:geoff.jpg|border|100px]]<br />
|imagestyle = <br />
|caption =Atmospheric and Oceanic Fluid Dynamics: Fundamentals and Large-scale Circulation, ''Vallis, G. K.'', Cambridge University Press, 2006<br />
|captionstyle = <br />
}}<br />
|belowstyle = background:#ddf;<br />
|below = <br />
}}<br />
<br />
<br />
This is the page for the class ''General Circulation of the Atmosphere (大气环流)'' for the Fall 2026 semester. Students who take this class should check this page periodically for content updates and new announcements.<br />
<br />
= Announcement =<br />
* 课程交流QQ群：1075545767，请各位选课的同学入群。【2026.9.21】<br />
<br />
= Course info =<br />
* '''Instructor ''': 张洋，<br />
:*office: 仙林气象楼 B410<br />
:*email: yangzhang@nju.edu.cn<br />
* '''Class meeting''': 周一 下午 2:00-4:00；逸B-303<br />
* '''Office hour''': 周四 <font color="red" size="2>下午1:00-2:00，仙林大气楼B410</font><br />
* '''Prerequisites''': 动力气象，天气学，气候学<br />
* '''Grading''': 平时作业（50％）＋ 期末考试（50％）<br />
本课程将大致布置4次作业，每次作业一二道题目左右。题目将选择每个课题最具有代表性、需要一定思维强度和动手能力的训练用题目,意在使学生通过顺利完成作业来建立环流系统的物理模型、以对课程内容得到深刻全面地理解和掌握。期末考试题目数量将会比平时作业多,覆盖面更广,但会比作业题目简单,只涉及对基本内容的掌握和对环流理论的直接应用。<br />
<br />
= Assignments =<br />
# [[Assignment 1, Fall 2026|Reanalysis data and the earth's climatology]] [Due:2026.10.12]<font color="red" size="1.5">请在本次作业的截止日期前将本次作业的图片部分和文字部分打包作为附件发送至邮箱 circulation_nju@126.com </font><br />
# [[Assignment 2, Fall 2026|Simple energy balance climate model]][Due:2026.10.26]<br />
<br />
= Course intro =<br />
“大气环流”常指地球大气较大空间范围、较长时间尺度上的空气流动，及其对地球大气热量、动量、能量和水汽的全球输送。虽然从十七、十八世纪起人们就开始研究大尺度的大气运动(如Hadley在1735年提出的信风理论)，但大气环流真正发展成为一门较完备的学科方向却是近半个世纪的事情。随着四五十年代探空资料等高空气象要素的取得，以及六十年代卫星等覆盖全球的观测资料的加入，大气环流的空间结构和时间变化开始被系统、全面地揭示。与此同时，大气环流的数值模拟，也开始成为研究大气环流的一个主要方法，并发展至今成为了解和预估未来气候变化的主要手段。随着观测和模拟手段的进步，大气环流的理论研究也在近三十年开始快速地发展，人们对各种环流系统的维持和变化有了更全面、更深刻、也更为现代的理解。<br />
<br />
现代的大气环流是大气动力学、天气学和气候学相结合的产物。大气环流，既是各种天气现象产生的背景流场，又是各种气候状态形成的动力机制。大气环流在低频、季节、年际、年代际等时间尺度的变化，不但会引起天气现象的变化，也影响着气候状态的形成。而在大气科学领域面临着诸如全球暖化、气候变化、环流异常等重大科学问题的今天，大气环流研究的重要性被推到了前所未有的高度,大气环流也成为活跃发展又充满挑战的学科方向。<br />
<br />
本课程将讲述在过去几十年里大气环流在观测、理论和模拟上取得的进展。希望学生借此课程能熟悉大气环流的基本分布和形态，掌握各主要环流系统的维持和变化机制，建立各环流系统形成的物理模型，了解现阶段的大气环流模式，知道大气环流方向有待解决的科学问题。<br />
<br />
作为一门课程，大气环流内容的讲述常可以有两条线索。一条是全球尺度上大气热量、动量、能量和水汽的分布与输送，Lorenz（1967）和 Peixoto and Oort（1992）是按此线索介绍大气环流的优秀教材；另一条线索，是各纬度、各区域内大气环流系统的形成、维持和变化机制，James（1995）和 Vallis（2006）是按此线索介绍大气环流的经典讲义。根据现阶段大气环流方向的研究特点，本课程的讲述将主要按照后一种方式来展开，并辅以介绍各环流系统对大气各要素场的输送。在介绍各环流系统时，本课程将以观测、理论和模拟为顺序，从各大气环流系统的观测事实入手，介绍大气环流系统的分布特征和时空变化特征；着重介绍关于环流系统的各种动力学模型和现阶段对环流系统的理解；辅以对环流系统模拟研究的介绍；最后通过三者的对比，讨论各环流系统有待研究的问题。<br />
<br />
= Syllabus =<br />
本课程具体的内容安排如下：第一章为大气环流的概述，介绍大气环流发展的历史、包含的内容以及大气环流研究的常用观测资料和分析方法。第二章介绍大气环流产生的外部强迫：辐射强迫和下界面过程。第三至六章介绍大气环流中的各个环流系统及它们的动力机制。第七章详细介绍各复杂度的大气环流模式。第八章介绍大气环流领域现阶段最大的一个开放课题：全球暖化背景下的大气环流。这一章既是对前几章所介绍的大气环流理论的应用与检验，又是对未来大气环流研究方向的探讨。借此让学生熟悉并理解大气环流领域亟需解决的课题。具体课程安排和参考书目如下。<br />
== Course schedule ==<br />
*大气环流概述 (Introduction) （6课时）<br />
*大气环流的外部强迫（3课时）<br />
**辐射强迫 (Radiative forcing)<br />
**下界面过程 (Surface boundaries)<br />
*经向环流系统 (Zonally-averaged circulations)<br />
**Hadley 环流（4课时）<br />
**Ferrel 环流，急流，中纬度的波流相互作用（8课时）<br />
*纬向环流系统（Non-zonal circulations）（6课时）<br />
**Storm tracks<br />
**Monsoon<br />
**ENSO and Walker circulation<br />
*能量和水汽循环 (Angular momentum, energy and water vapor)（2课时）<br />
*不同复杂度的大气环流模式 (General circulation in a hierarchy of models)（2课时）<br />
*全球暖化背景下的大气环流 (General circulation in the global warming scenario)（2课时）<br />
<br />
[[点击此处看详细课程安排 (click for more)]]<br />
<br />
== References ==<br />
*观测部分：Peixoto, J. P. and A. H. Oort, 1992: Physics of Climate. Springer-Verlag New York, Inc., 520 pp. 中文译本：气候物理学，1995，吴国雄、刘辉等译校，气象出版社。<br />
*综合介绍：James, I., 1995: Introduction to circulating atmospheres. Cambridge University Press, 448 pp. <br />
*理论部分：Vallis, G. K., 2006: Atmospheric and Oceanic Fluid Dynamics: Fundamentals and Large-scale Circulation. Cambridge University Press. 745 pp.<br />
* [[其它参考书目（点击看详情）]]</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=General_Circulation(Fall_2026)&diff=13986General Circulation(Fall 2026)2026-09-30T00:43:51Z<p>Etone: </p>
<hr />
<div>{{Infobox<br />
|name = Infobox<br />
|bodystyle = <br />
|title = 大气环流 <br><br />
General Circulation of the Atmosphere<br />
|titlestyle = <br />
<br />
|headerstyle = background:#ccf;<br />
|labelstyle = background:#ddf;<br />
|datastyle = <br />
<br />
|header1 =Instructor<br />
|label1 = <br />
|data1 = <br />
|header2 = <br />
|label2 = <br />
|data2 = 张洋<br />
|header3 = <br />
|label3 = Email<br />
|data3 = yangzhang@nju.edu.cn <br />
|header4 =<br />
|label4= office<br />
|data4= 仙林大气楼 B410<br />
|header5 = Class<br />
|label5 = <br />
|data5 = <br />
|header6 =<br />
|label6 = Class meetings<br />
|data6 = 周一 下午 2:00-4:00，逸B-303<br />
|header7 =<br />
|label7 = Place<br />
|data7 = <br />
|header8 =<br />
|label8 = Office hours<br />
|data8 = 周四 下午13:00-14:00 <br>仙林大气楼B410 <br />
|header9 = Reference book<br />
|label9 = <br />
|data9 = <br />
|header10 =<br />
|label10 = <br />
|data10 = {{Infobox<br />
|name = <br />
|bodystyle = <br />
|title = <br />
|titlestyle = <br />
|image = [[File:James-Circulating.jpg|border|100px]]<br />
|imagestyle = <br />
|caption = Introduction to Circulating Atmospheres, <br>''I. James'', Cambridge Press, 1995<br />
|captionstyle = }}<br />
|header11 =<br />
|label11 = <br />
|data11 = {{Infobox<br />
|name = <br />
|bodystyle = <br />
|title = <br />
|titlestyle = <br />
|image = [[File:Oort.jpg|border|100px]]<br />
|imagestyle = <br />
|caption =Physics of Climate, ''Peixoto, J. P.'' and ''A. H. Oort'', Springer-Verlag New York, 1992<br />
|captionstyle = <br />
}}<br />
|header12 =<br />
|label12 = <br />
|data12 = {{Infobox<br />
|name = <br />
|bodystyle = <br />
|title = <br />
|titlestyle = <br />
|image = [[File:geoff.jpg|border|100px]]<br />
|imagestyle = <br />
|caption =Atmospheric and Oceanic Fluid Dynamics: Fundamentals and Large-scale Circulation, ''Vallis, G. K.'', Cambridge University Press, 2006<br />
|captionstyle = <br />
}}<br />
|belowstyle = background:#ddf;<br />
|below = <br />
}}<br />
<br />
<br />
This is the page for the class ''General Circulation of the Atmosphere (大气环流)'' for the Fall 2026 semester. Students who take this class should check this page periodically for content updates and new announcements.<br />
<br />
= Announcement =<br />
* 课程交流QQ群：1075545767，请各位选课的同学入群。【2026.9.21】<br />
<br />
= Course info =<br />
* '''Instructor ''': 张洋，<br />
:*office: 仙林气象楼 B410<br />
:*email: yangzhang@nju.edu.cn<br />
* '''Class meeting''': 周一 下午 2:00-4:00；逸B-303<br />
* '''Office hour''': 周四 <font color="red" size="2>下午1:00-2:00，仙林大气楼B410</font><br />
* '''Prerequisites''': 动力气象，天气学，气候学<br />
* '''Grading''': 平时作业（50％）＋ 期末考试（50％）<br />
本课程将大致布置4次作业，每次作业一二道题目左右。题目将选择每个课题最具有代表性、需要一定思维强度和动手能力的训练用题目,意在使学生通过顺利完成作业来建立环流系统的物理模型、以对课程内容得到深刻全面地理解和掌握。期末考试题目数量将会比平时作业多,覆盖面更广,但会比作业题目简单,只涉及对基本内容的掌握和对环流理论的直接应用。<br />
<br />
= Assignments =<br />
# [[Assignment 1, Fall 2026|Reanalysis data and the earth's climatology]] [Due:2026.10.12]<font color="red" size="1.5">请在本次作业的截止日期前将本次作业的图片部分和文字部分打包作为附件发送至邮箱 circulation_nju@126.com </font><br />
<br />
<br />
= Course intro =<br />
“大气环流”常指地球大气较大空间范围、较长时间尺度上的空气流动，及其对地球大气热量、动量、能量和水汽的全球输送。虽然从十七、十八世纪起人们就开始研究大尺度的大气运动(如Hadley在1735年提出的信风理论)，但大气环流真正发展成为一门较完备的学科方向却是近半个世纪的事情。随着四五十年代探空资料等高空气象要素的取得，以及六十年代卫星等覆盖全球的观测资料的加入，大气环流的空间结构和时间变化开始被系统、全面地揭示。与此同时，大气环流的数值模拟，也开始成为研究大气环流的一个主要方法，并发展至今成为了解和预估未来气候变化的主要手段。随着观测和模拟手段的进步，大气环流的理论研究也在近三十年开始快速地发展，人们对各种环流系统的维持和变化有了更全面、更深刻、也更为现代的理解。<br />
<br />
现代的大气环流是大气动力学、天气学和气候学相结合的产物。大气环流，既是各种天气现象产生的背景流场，又是各种气候状态形成的动力机制。大气环流在低频、季节、年际、年代际等时间尺度的变化，不但会引起天气现象的变化，也影响着气候状态的形成。而在大气科学领域面临着诸如全球暖化、气候变化、环流异常等重大科学问题的今天，大气环流研究的重要性被推到了前所未有的高度,大气环流也成为活跃发展又充满挑战的学科方向。<br />
<br />
本课程将讲述在过去几十年里大气环流在观测、理论和模拟上取得的进展。希望学生借此课程能熟悉大气环流的基本分布和形态，掌握各主要环流系统的维持和变化机制，建立各环流系统形成的物理模型，了解现阶段的大气环流模式，知道大气环流方向有待解决的科学问题。<br />
<br />
作为一门课程，大气环流内容的讲述常可以有两条线索。一条是全球尺度上大气热量、动量、能量和水汽的分布与输送，Lorenz（1967）和 Peixoto and Oort（1992）是按此线索介绍大气环流的优秀教材；另一条线索，是各纬度、各区域内大气环流系统的形成、维持和变化机制，James（1995）和 Vallis（2006）是按此线索介绍大气环流的经典讲义。根据现阶段大气环流方向的研究特点，本课程的讲述将主要按照后一种方式来展开，并辅以介绍各环流系统对大气各要素场的输送。在介绍各环流系统时，本课程将以观测、理论和模拟为顺序，从各大气环流系统的观测事实入手，介绍大气环流系统的分布特征和时空变化特征；着重介绍关于环流系统的各种动力学模型和现阶段对环流系统的理解；辅以对环流系统模拟研究的介绍；最后通过三者的对比，讨论各环流系统有待研究的问题。<br />
<br />
= Syllabus =<br />
本课程具体的内容安排如下：第一章为大气环流的概述，介绍大气环流发展的历史、包含的内容以及大气环流研究的常用观测资料和分析方法。第二章介绍大气环流产生的外部强迫：辐射强迫和下界面过程。第三至六章介绍大气环流中的各个环流系统及它们的动力机制。第七章详细介绍各复杂度的大气环流模式。第八章介绍大气环流领域现阶段最大的一个开放课题：全球暖化背景下的大气环流。这一章既是对前几章所介绍的大气环流理论的应用与检验，又是对未来大气环流研究方向的探讨。借此让学生熟悉并理解大气环流领域亟需解决的课题。具体课程安排和参考书目如下。<br />
== Course schedule ==<br />
*大气环流概述 (Introduction) （6课时）<br />
*大气环流的外部强迫（3课时）<br />
**辐射强迫 (Radiative forcing)<br />
**下界面过程 (Surface boundaries)<br />
*经向环流系统 (Zonally-averaged circulations)<br />
**Hadley 环流（4课时）<br />
**Ferrel 环流，急流，中纬度的波流相互作用（8课时）<br />
*纬向环流系统（Non-zonal circulations）（6课时）<br />
**Storm tracks<br />
**Monsoon<br />
**ENSO and Walker circulation<br />
*能量和水汽循环 (Angular momentum, energy and water vapor)（2课时）<br />
*不同复杂度的大气环流模式 (General circulation in a hierarchy of models)（2课时）<br />
*全球暖化背景下的大气环流 (General circulation in the global warming scenario)（2课时）<br />
<br />
[[点击此处看详细课程安排 (click for more)]]<br />
<br />
== References ==<br />
*观测部分：Peixoto, J. P. and A. H. Oort, 1992: Physics of Climate. Springer-Verlag New York, Inc., 520 pp. 中文译本：气候物理学，1995，吴国雄、刘辉等译校，气象出版社。<br />
*综合介绍：James, I., 1995: Introduction to circulating atmospheres. Cambridge University Press, 448 pp. <br />
*理论部分：Vallis, G. K., 2006: Atmospheric and Oceanic Fluid Dynamics: Fundamentals and Large-scale Circulation. Cambridge University Press. 745 pp.<br />
* [[其它参考书目（点击看详情）]]</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=%E9%AB%98%E7%BA%A7%E7%AE%97%E6%B3%95_(Fall_2026)&diff=13984高级算法 (Fall 2026)2026-09-28T05:12:01Z<p>Etone: /* Lecture Notes */</p>
<hr />
<div>{{Infobox<br />
|name = Infobox<br />
|bodystyle = <br />
|title = <font size=3>高级算法 <br />
<br>Advanced Algorithms</font><br />
|titlestyle = <br />
<br />
|image = <br />
|imagestyle = <br />
|caption = <br />
|captionstyle = <br />
|headerstyle = background:#ccf;<br />
|labelstyle = background:#ddf;<br />
|datastyle = <br />
<br />
|header1 =Instructor<br />
|label1 = <br />
|data1 = <br />
|header2 = <br />
|label2 = <br />
|data2 = '''尹一通'''<br />
|header3 = <br />
|label3 = Email<br />
|data3 = yinyt@nju.edu.cn <br />
|header4 =<br />
|label4= office<br />
|data4= 计算机系 804<br />
|header5 = <br />
|label5 = <br />
|data5 = '''栗师'''<br />
|header6 = <br />
|label6 = Email<br />
|data6 = shili@nju.edu.cn <br />
|header7 =<br />
|label7= office<br />
|data7= 计算机系 605<br />
|header8 = <br />
|label8 = <br />
|data8 = '''刘景铖'''<br />
|header9 = <br />
|label9 = Email<br />
|data9 = liu@nju.edu.cn <br />
|header10 =<br />
|label10= office<br />
|data10= 计算机系 516<br />
|header11 = Class<br />
|label11 = <br />
|data11 = <br />
|header12 =<br />
|label12 = Class meetings<br />
|data12 = 周一 5-6节 (单) 仙Ⅰ-319<br />
周三 5-6节 仙Ⅰ-319<br />
|header13 =<br />
|label13 = Place<br />
|data13 = <br />
|header14 =<br />
|label14 = Office hours<br />
|data14 = 周一4-5pm（尹一通 804）<br/><br />
周四4-5pm（刘景铖 516）<br/><br />
By appointment<br />
|header15 = Textbooks<br />
|label15 = <br />
|data15 = <br />
|header16 =<br />
|label16 = <br />
|data16 = [[File:MR-randomized-algorithms.png|border|100px]]<br />
|header17 =<br />
|label17 = <br />
|data17 = Motwani and Raghavan. <br>''Randomized Algorithms''.<br> Cambridge Univ Press, 1995.<br />
|header18 =<br />
|label18 = <br />
|data18 = [[File:Approximation_Algorithms.jpg|border|100px]]<br />
|header19 =<br />
|label19 = <br />
|data19 = Vazirani. <br>''Approximation Algorithms''. <br> Springer-Verlag, 2001.<br />
|belowstyle = background:#ddf;<br />
|below = <br />
}}<br />
<br />
This is the webpage for the ''Advanced Algorithms'' class of fall 2026. Students who take this class should check this page periodically for content updates and new announcements. <br />
<br />
= Announcement =<br />
<br />
= Course info =<br />
* '''Instructor ''': <br />
:* [http://tcs.nju.edu.cn/yinyt/ 尹一通]：[mailto:yinyt@nju.edu.cn <yinyt@nju.edu.cn>]，计算机系 804 <br />
:*[https://tcs.nju.edu.cn/shili/ 栗师]：[mailto:shili@nju.edu.cn <shili@nju.edu.cn>]，计算机系 605<br />
:* [https://liuexp.github.io 刘景铖]：[mailto:liu@nju.edu.cn <liu@nju.edu.cn>]，计算机系 516 <br />
* '''Teaching Assistant''': <br />
** 于逸潇：[mailto:yixiaoyu@smail.nju.edu.cn <yixiaoyu@smail.nju.edu.cn>]<br />
** 张弈垚：[mailto:zhangyiyao@smail.nju.edu.cn <zhangyiyao@smail.nju.edu.cn>]<br />
* '''Class meeting''': <br />
** 周一 5-6节 1-17周(单) 仙Ⅰ-319<br />
** 周三 5-6节 1-18周 仙Ⅰ-319<br />
* '''Office hour''': <br />
** 周一4-5pm（尹一通 804）<br />
** 周四4-5pm（刘景铖 516）<br />
** By appointment<br />
* '''QQ群''': 1098567018<br />
<br />
= Syllabus =<br />
随着计算机算法理论的不断发展，现代计算机算法的设计与分析大量地使用非初等的数学工具以及非传统的算法思想。“高级算法”这门课程就是面向计算机算法的这一发展趋势而设立的。课程将针对传统算法课程未系统涉及、却在计算机科学各领域的科研和实践中扮演重要角色的高等算法设计思想和算法分析工具进行系统讲授。<br />
<br />
=== 先修课程 Prerequisites ===<br />
* 必须：离散数学，概率论，线性代数。<br />
* 推荐：算法设计与分析。<br />
<br />
=== Course materials ===<br />
* [[高级算法 (Fall 2026) / Course materials|<font size=3>教材和参考书</font>]]<br />
<br />
=== 成绩 Grades ===<br />
* 课程成绩：本课程将会有若干次作业和一次期末考试。最终成绩将由平时作业成绩和期末考试成绩综合得出。<br />
* 迟交：如果有特殊的理由，无法按时完成作业，请提前联系授课老师，给出正当理由。否则迟交的作业将不被接受。<br />
<br />
=== <font color=red> 学术诚信 Academic Integrity </font>===<br />
学术诚信是所有从事学术活动的学生和学者最基本的职业道德底线，本课程将不遗余力的维护学术诚信规范，违反这一底线的行为将不会被容忍。<br />
<br />
作业完成的原则：署你名字的工作必须是你个人的贡献。在完成作业的过程中，允许讨论，前提是讨论的所有参与者均处于同等完成度。但关键想法的执行、以及作业文本的写作必须独立完成，并在作业中致谢（acknowledge）所有参与讨论的人。不允许其他任何形式的合作——尤其是与已经完成作业的同学“讨论”。<br />
<br />
本课程将对剽窃行为采取零容忍的态度。在完成作业过程中，对他人工作（出版物、互联网资料、其他人的作业等）直接的文本抄袭和对关键思想、关键元素的抄袭，按照 [http://www.acm.org/publications/policies/plagiarism_policy ACM Policy on Plagiarism]的解释，都将视为剽窃。剽窃者成绩将被取消。如果发现互相抄袭行为，<font color=red> 抄袭和被抄袭双方的成绩都将被取消</font>。因此请主动防止自己的作业被他人抄袭。<br />
<br />
学术诚信影响学生个人的品行，也关乎整个教育系统的正常运转。为了一点分数而做出学术不端的行为，不仅使自己沦为一个欺骗者，也使他人的诚实努力失去意义。让我们一起努力维护一个诚信的环境。<br />
<br />
= Assignments =<br />
Late policy: In general, we will accomodate late submission requests ONLY IF you made such requests ahead of time. <br />
<br />
<br />
= Lecture Notes =<br />
# [[高级算法 (Fall 2025)/Min Cut, Max Cut, and Spectral Cut|Min Cut, Max Cut, and Spectral Cut]] ([http://tcs.nju.edu.cn/slides/aa2026/Cut.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-cuts-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-cuts-zh.pdf AI生成讲义])<br />
#* [[高级算法 (Fall 2025)/Probability Basics|Probability basics]]<br />
# [[高级算法 (Fall 2026)/Fingerprinting| Fingerprinting]] ([http://tcs.nju.edu.cn/slides/aa2026/Fingerprinting.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-fingerprinting-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-fingerprinting-zh.pdf AI生成讲义])<br />
#* [[高级算法 (Fall 2026)/Finite Field Basics|Finite field basics]]<br />
# [[高级算法 (Fall 2026)/Hashing and Sketching|Hashing and Sketching]] ([http://tcs.nju.edu.cn/slides/aa2026/Hashing.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-hashing-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-hashing-zh.pdf AI生成讲义]) <br />
#* [[高级算法 (Fall 2026)/Limited independence|Limited independence]]<br />
#* [[高级算法 (Fall 2026)/Basic deviation inequalities|Basic deviation inequalities]]<br />
# [[高级算法 (Fall 2026)/Concentration of measure|Concentration of measure]] ([http://tcs.nju.edu.cn/slides/aa2026/Concentration.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-concentration-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-concentration-zh.pdf AI生成讲义]) <br />
#* [[高级算法 (Fall 2026)/Conditional expectations|Conditional expectations]]<br />
# [[高级算法 (Fall 2026)/Dimension Reduction|Dimension Reduction]] ([http://tcs.nju.edu.cn/slides/aa2026/NNS.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-JLT-LSH-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-JLT-LSH-zh.pdf AI生成讲义]) <br />
#* [https://www.cs.princeton.edu/~hy2/teaching/fall22-cos521/notes/JL.pdf Professor Huacheng Yu's note on Johnson-Lindenstrauss Theorem]<br />
#* [http://people.csail.mit.edu/gregory/annbook/introduction.pdf An introduction of LSH]<br />
# ''Lovász'' Local Lemma ([http://tcs.nju.edu.cn/slides/aa2026/LLL.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-LLL-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-LLL-zh.pdf AI生成讲义])<br />
<br />
= Related Online Courses=<br />
* [https://www.cs.cmu.edu/~15850/ Advanced Algorithms] by Anupam Gupta at CMU.<br />
* [http://people.csail.mit.edu/moitra/854.html Advanced Algorithms] by Ankur Moitra at MIT.<br />
* [http://courses.csail.mit.edu/6.854/current/ Advanced Algorithms] by David Karger and Aleksander Mądry at MIT.<br />
* [http://web.stanford.edu/class/cs168/index.html The Modern Algorithmic Toolbox] by Tim Roughgarden and Gregory Valiant at Stanford.<br />
* [https://www.cs.princeton.edu/courses/archive/fall18/cos521/ Advanced Algorithm Design] by Pravesh Kothari and Christopher Musco at Princeton.<br />
* [http://www.cs.cmu.edu/afs/cs.cmu.edu/academic/class/15859-f11/www/ Linear and Semidefinite Programming (Advanced Algorithms)] by Anupam Gupta and Ryan O'Donnell at CMU.<br />
* [https://www.cs.cmu.edu/~odonnell/papers/cs-theory-toolkit-lecture-notes.pdf CS Theory Toolkit] by Ryan O'Donnell at CMU.<br />
* [https://cs.uwaterloo.ca/~lapchi/cs860/index.html Eigenvalues and Polynomials] by Lap Chi Lau at University of Waterloo.<br />
* The [https://www.cs.cornell.edu/jeh/book.pdf "Foundations of Data Science" book] by Avrim Blum, John Hopcroft, and Ravindran Kannan.</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=General_Circulation(Fall_2026)&diff=13983General Circulation(Fall 2026)2026-09-27T08:16:11Z<p>Etone: </p>
<hr />
<div>{{Infobox<br />
|name = Infobox<br />
|bodystyle = <br />
|title = 大气环流 <br><br />
General Circulation of the Atmosphere<br />
|titlestyle = <br />
<br />
|headerstyle = background:#ccf;<br />
|labelstyle = background:#ddf;<br />
|datastyle = <br />
<br />
|header1 =Instructor<br />
|label1 = <br />
|data1 = <br />
|header2 = <br />
|label2 = <br />
|data2 = 张洋<br />
|header3 = <br />
|label3 = Email<br />
|data3 = yangzhang@nju.edu.cn <br />
|header4 =<br />
|label4= office<br />
|data4= 仙林大气楼 B410<br />
|header5 = Class<br />
|label5 = <br />
|data5 = <br />
|header6 =<br />
|label6 = Class meetings<br />
|data6 = 周一 下午 2:00-4:00，逸B-303<br />
|header7 =<br />
|label7 = Place<br />
|data7 = <br />
|header8 =<br />
|label8 = Office hours<br />
|data8 = 周四 下午13:00-2:00 <br>仙林大气楼B410 <br />
|header9 = Reference book<br />
|label9 = <br />
|data9 = <br />
|header10 =<br />
|label10 = <br />
|data10 = {{Infobox<br />
|name = <br />
|bodystyle = <br />
|title = <br />
|titlestyle = <br />
|image = [[File:James-Circulating.jpg|border|100px]]<br />
|imagestyle = <br />
|caption = Introduction to Circulating Atmospheres, <br>''I. James'', Cambridge Press, 1995<br />
|captionstyle = }}<br />
|header11 =<br />
|label11 = <br />
|data11 = {{Infobox<br />
|name = <br />
|bodystyle = <br />
|title = <br />
|titlestyle = <br />
|image = [[File:Oort.jpg|border|100px]]<br />
|imagestyle = <br />
|caption =Physics of Climate, ''Peixoto, J. P.'' and ''A. H. Oort'', Springer-Verlag New York, 1992<br />
|captionstyle = <br />
}}<br />
|header12 =<br />
|label12 = <br />
|data12 = {{Infobox<br />
|name = <br />
|bodystyle = <br />
|title = <br />
|titlestyle = <br />
|image = [[File:geoff.jpg|border|100px]]<br />
|imagestyle = <br />
|caption =Atmospheric and Oceanic Fluid Dynamics: Fundamentals and Large-scale Circulation, ''Vallis, G. K.'', Cambridge University Press, 2006<br />
|captionstyle = <br />
}}<br />
|belowstyle = background:#ddf;<br />
|below = <br />
}}<br />
<br />
<br />
This is the page for the class ''General Circulation of the Atmosphere (大气环流)'' for the Fall 2026 semester. Students who take this class should check this page periodically for content updates and new announcements.<br />
<br />
= Announcement =<br />
* 课程交流QQ群：1075545767，请各位选课的同学入群。【2026.9.21】<br />
<br />
= Course info =<br />
* '''Instructor ''': 张洋，<br />
:*office: 仙林气象楼 B410<br />
:*email: yangzhang@nju.edu.cn<br />
* '''Class meeting''': 周一 下午 2:00-4:00；逸B-303<br />
* '''Office hour''': 周四 <font color="red" size="2>下午1:00-2:00，仙林大气楼B410</font><br />
* '''Prerequisites''': 动力气象，天气学，气候学<br />
* '''Grading''': 平时作业（50％）＋ 期末考试（50％）<br />
本课程将大致布置4次作业，每次作业一二道题目左右。题目将选择每个课题最具有代表性、需要一定思维强度和动手能力的训练用题目,意在使学生通过顺利完成作业来建立环流系统的物理模型、以对课程内容得到深刻全面地理解和掌握。期末考试题目数量将会比平时作业多,覆盖面更广,但会比作业题目简单,只涉及对基本内容的掌握和对环流理论的直接应用。<br />
<br />
= Assignments =<br />
# [[Assignment 1, Fall 2026|Reanalysis data and the earth's climatology]] [Due:2026.10.12]<font color="red" size="1.5">请在本次作业的截止日期前将本次作业的图片部分和文字部分打包作为附件发送至邮箱 circulation_nju@126.com </font><br />
<br />
<br />
= Course intro =<br />
“大气环流”常指地球大气较大空间范围、较长时间尺度上的空气流动，及其对地球大气热量、动量、能量和水汽的全球输送。虽然从十七、十八世纪起人们就开始研究大尺度的大气运动(如Hadley在1735年提出的信风理论)，但大气环流真正发展成为一门较完备的学科方向却是近半个世纪的事情。随着四五十年代探空资料等高空气象要素的取得，以及六十年代卫星等覆盖全球的观测资料的加入，大气环流的空间结构和时间变化开始被系统、全面地揭示。与此同时，大气环流的数值模拟，也开始成为研究大气环流的一个主要方法，并发展至今成为了解和预估未来气候变化的主要手段。随着观测和模拟手段的进步，大气环流的理论研究也在近三十年开始快速地发展，人们对各种环流系统的维持和变化有了更全面、更深刻、也更为现代的理解。<br />
<br />
现代的大气环流是大气动力学、天气学和气候学相结合的产物。大气环流，既是各种天气现象产生的背景流场，又是各种气候状态形成的动力机制。大气环流在低频、季节、年际、年代际等时间尺度的变化，不但会引起天气现象的变化，也影响着气候状态的形成。而在大气科学领域面临着诸如全球暖化、气候变化、环流异常等重大科学问题的今天，大气环流研究的重要性被推到了前所未有的高度,大气环流也成为活跃发展又充满挑战的学科方向。<br />
<br />
本课程将讲述在过去几十年里大气环流在观测、理论和模拟上取得的进展。希望学生借此课程能熟悉大气环流的基本分布和形态，掌握各主要环流系统的维持和变化机制，建立各环流系统形成的物理模型，了解现阶段的大气环流模式，知道大气环流方向有待解决的科学问题。<br />
<br />
作为一门课程，大气环流内容的讲述常可以有两条线索。一条是全球尺度上大气热量、动量、能量和水汽的分布与输送，Lorenz（1967）和 Peixoto and Oort（1992）是按此线索介绍大气环流的优秀教材；另一条线索，是各纬度、各区域内大气环流系统的形成、维持和变化机制，James（1995）和 Vallis（2006）是按此线索介绍大气环流的经典讲义。根据现阶段大气环流方向的研究特点，本课程的讲述将主要按照后一种方式来展开，并辅以介绍各环流系统对大气各要素场的输送。在介绍各环流系统时，本课程将以观测、理论和模拟为顺序，从各大气环流系统的观测事实入手，介绍大气环流系统的分布特征和时空变化特征；着重介绍关于环流系统的各种动力学模型和现阶段对环流系统的理解；辅以对环流系统模拟研究的介绍；最后通过三者的对比，讨论各环流系统有待研究的问题。<br />
<br />
= Syllabus =<br />
本课程具体的内容安排如下：第一章为大气环流的概述，介绍大气环流发展的历史、包含的内容以及大气环流研究的常用观测资料和分析方法。第二章介绍大气环流产生的外部强迫：辐射强迫和下界面过程。第三至六章介绍大气环流中的各个环流系统及它们的动力机制。第七章详细介绍各复杂度的大气环流模式。第八章介绍大气环流领域现阶段最大的一个开放课题：全球暖化背景下的大气环流。这一章既是对前几章所介绍的大气环流理论的应用与检验，又是对未来大气环流研究方向的探讨。借此让学生熟悉并理解大气环流领域亟需解决的课题。具体课程安排和参考书目如下。<br />
== Course schedule ==<br />
*大气环流概述 (Introduction) （6课时）<br />
*大气环流的外部强迫（3课时）<br />
**辐射强迫 (Radiative forcing)<br />
**下界面过程 (Surface boundaries)<br />
*经向环流系统 (Zonally-averaged circulations)<br />
**Hadley 环流（4课时）<br />
**Ferrel 环流，急流，中纬度的波流相互作用（8课时）<br />
*纬向环流系统（Non-zonal circulations）（6课时）<br />
**Storm tracks<br />
**Monsoon<br />
**ENSO and Walker circulation<br />
*能量和水汽循环 (Angular momentum, energy and water vapor)（2课时）<br />
*不同复杂度的大气环流模式 (General circulation in a hierarchy of models)（2课时）<br />
*全球暖化背景下的大气环流 (General circulation in the global warming scenario)（2课时）<br />
<br />
[[点击此处看详细课程安排 (click for more)]]<br />
<br />
== References ==<br />
*观测部分：Peixoto, J. P. and A. H. Oort, 1992: Physics of Climate. Springer-Verlag New York, Inc., 520 pp. 中文译本：气候物理学，1995，吴国雄、刘辉等译校，气象出版社。<br />
*综合介绍：James, I., 1995: Introduction to circulating atmospheres. Cambridge University Press, 448 pp. <br />
*理论部分：Vallis, G. K., 2006: Atmospheric and Oceanic Fluid Dynamics: Fundamentals and Large-scale Circulation. Cambridge University Press. 745 pp.<br />
* [[其它参考书目（点击看详情）]]</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=Assignment_1,_Fall_2026&diff=13982Assignment 1, Fall 20262026-09-27T08:13:18Z<p>Etone: /* Bonus Question */</p>
<hr />
<div>==Question #1==<br />
<br />
请使用多年（>20年）的NCEP/NCAR（或其他标明的）月平均再分析资料，画出各季节（至少画出冬夏两季）纬向平均温度场、纬向平均纬向风场的高度－纬度剖面分布，并简述其分布特征和季节变化特征。<br />
<br />
==Question #2==<br />
<br />
请使用多年（>20年）的NCEP/NCAR（或其他标明的）月平均再分析资料，画出各季节（至少画出冬夏两季）温度场、纬向风场在各高度（850、500、100 hPa， 对于温度场请再画出1000hPa）上的水平分布，并简述其分布特征和季节变化特征。<br />
<br />
==Bonus Question==<br />
<br />
请使用多年(>20年)的月平均再分析资料(NCEP/NCAR或其他)，画出冬夏两季由平均环流和定常波引起的经向热量通量以及经向西风动量通量纬向平均后的高度-纬度剖面分布，并简述它们的分布特征。<br />
<br />
==On the netcdf file==<br />
<br />
Netcdf (Network Common Data Form) is a set of self-describing, machine-independent data formats that support the creation, access, and sharing of array-oriented scientific data. The project homepage is hosted by the [https://www.unidata.ucar.edu/software/netcdf Unidata program] at the University Corporation for Atmospheric Research (UCAR). They are also the chief source of netCDF software, standards development, updates etc.。<br />
[https://psl.noaa.gov/data/help/read-our-files.html 读取netcdf文件的几种方法]<br />
<br />
==Read .nc using Matlab==<br />
从 <font color="red"> Matlab 2008b </font> 开始，Matlab 开始提供自带的读写.nc文件的函数。以下为读写.nc的一个样本matlab文件。对于图片，Matlab提供多种形式的存储。可在弹出图片的窗口左上方的下拉菜单中选择图片的存储方式，或者在命令行后用“print”命令存储图片（可以输入 help print 获得帮助）。关于Matlab的更多信息，可以去[http://www.mathworks.com/help Matlab网站]。<br />
<br />
==下载区==<br />
本次作业所用的数据资料可以用来自NCEP-DOE Reanalysis 2 的月平均资料，时间跨度为1979/1-2009/12。关于本资料的详细内容，请[https://psl.noaa.gov/data/gridded/data.ncep.reanalysis2.html 参见网站]。 以下为完成本次作业所需的数据文件和matlab文件。<br />
<br />
NCEP-DOE 月平均温度场资料 (1979/1-2009/12): [http://tcs.nju.edu.cn/yzhang/air.mon.mean.nc nc file]，[http://tcs.nju.edu.cn/yzhang/air_readme.txt 说明文件]<br />
<br />
NCEP-DOE 月平均纬向风(U)资料 (1979/1-2009/12): [http://tcs.nju.edu.cn/yzhang/uwnd.mon.mean.nc nc file]，[http://tcs.nju.edu.cn/yzhang/uwnd_readme.txt 说明文件]<br />
<br />
NCEP-DOE 月平均经向风(V)资料 (1979/1-2009/12): [http://tcs.nju.edu.cn/yzhang/vwnd.mon.mean.nc nc file]，[http://tcs.nju.edu.cn/yzhang/vwnd_readme.txt 说明文件]<br />
<br />
NCEP-land mask（用于在题目2中画出具体的海陆分布）: [http://tcs.nju.edu.cn/yzhang/land.nc nc文件]<br />
<br />
题目1和2计算分析用的Matlab 文件： [http://tcs.nju.edu.cn/yzhang/sample_for_Q1.m 第一题m文件], [http://tcs.nju.edu.cn/yzhang/sample_for_Q2.m 第二题m文件]</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=Assignment_1,_Fall_2026&diff=13981Assignment 1, Fall 20262026-09-27T08:11:30Z<p>Etone: </p>
<hr />
<div>==Question #1==<br />
<br />
请使用多年（>20年）的NCEP/NCAR（或其他标明的）月平均再分析资料，画出各季节（至少画出冬夏两季）纬向平均温度场、纬向平均纬向风场的高度－纬度剖面分布，并简述其分布特征和季节变化特征。<br />
<br />
==Question #2==<br />
<br />
请使用多年（>20年）的NCEP/NCAR（或其他标明的）月平均再分析资料，画出各季节（至少画出冬夏两季）温度场、纬向风场在各高度（850、500、100 hPa， 对于温度场请再画出1000hPa）上的水平分布，并简述其分布特征和季节变化特征。<br />
<br />
==Bonus Question==<br />
<br />
请使用多年(>20年)的月平均再分析资料(NCEP/NCAR或其他)，画出冬夏两季由平均环流和定常波引起的经向热量通量纬向平均后的高度-纬度剖面分布，并简述它们的分布特征。<br />
<br />
==On the netcdf file==<br />
<br />
Netcdf (Network Common Data Form) is a set of self-describing, machine-independent data formats that support the creation, access, and sharing of array-oriented scientific data. The project homepage is hosted by the [https://www.unidata.ucar.edu/software/netcdf Unidata program] at the University Corporation for Atmospheric Research (UCAR). They are also the chief source of netCDF software, standards development, updates etc.。<br />
[https://psl.noaa.gov/data/help/read-our-files.html 读取netcdf文件的几种方法]<br />
<br />
==Read .nc using Matlab==<br />
从 <font color="red"> Matlab 2008b </font> 开始，Matlab 开始提供自带的读写.nc文件的函数。以下为读写.nc的一个样本matlab文件。对于图片，Matlab提供多种形式的存储。可在弹出图片的窗口左上方的下拉菜单中选择图片的存储方式，或者在命令行后用“print”命令存储图片（可以输入 help print 获得帮助）。关于Matlab的更多信息，可以去[http://www.mathworks.com/help Matlab网站]。<br />
<br />
==下载区==<br />
本次作业所用的数据资料可以用来自NCEP-DOE Reanalysis 2 的月平均资料，时间跨度为1979/1-2009/12。关于本资料的详细内容，请[https://psl.noaa.gov/data/gridded/data.ncep.reanalysis2.html 参见网站]。 以下为完成本次作业所需的数据文件和matlab文件。<br />
<br />
NCEP-DOE 月平均温度场资料 (1979/1-2009/12): [http://tcs.nju.edu.cn/yzhang/air.mon.mean.nc nc file]，[http://tcs.nju.edu.cn/yzhang/air_readme.txt 说明文件]<br />
<br />
NCEP-DOE 月平均纬向风(U)资料 (1979/1-2009/12): [http://tcs.nju.edu.cn/yzhang/uwnd.mon.mean.nc nc file]，[http://tcs.nju.edu.cn/yzhang/uwnd_readme.txt 说明文件]<br />
<br />
NCEP-DOE 月平均经向风(V)资料 (1979/1-2009/12): [http://tcs.nju.edu.cn/yzhang/vwnd.mon.mean.nc nc file]，[http://tcs.nju.edu.cn/yzhang/vwnd_readme.txt 说明文件]<br />
<br />
NCEP-land mask（用于在题目2中画出具体的海陆分布）: [http://tcs.nju.edu.cn/yzhang/land.nc nc文件]<br />
<br />
题目1和2计算分析用的Matlab 文件： [http://tcs.nju.edu.cn/yzhang/sample_for_Q1.m 第一题m文件], [http://tcs.nju.edu.cn/yzhang/sample_for_Q2.m 第二题m文件]</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=Assignment_1,_Fall_2026&diff=13980Assignment 1, Fall 20262026-09-27T08:09:04Z<p>Etone: Created page with "==Question #1== 请使用多年（>20年）的NCEP/NCAR（或其他标明的）月平均再分析资料，画出各季节（至少画出冬夏两季）纬向平均温度场、纬向平均纬向风场的高度－纬度剖面分布，并简述其分布特征和季节变化特征。 ==Question #2== 请使用多年（>20年）的NCEP/NCAR（或其他标明的）月平均再分析资料，画出各季节（至少画出冬夏两季）温度场、纬向风场在各高度（8..."</p>
<hr />
<div>==Question #1==<br />
<br />
请使用多年（>20年）的NCEP/NCAR（或其他标明的）月平均再分析资料，画出各季节（至少画出冬夏两季）纬向平均温度场、纬向平均纬向风场的高度－纬度剖面分布，并简述其分布特征和季节变化特征。<br />
<br />
==Question #2==<br />
<br />
请使用多年（>20年）的NCEP/NCAR（或其他标明的）月平均再分析资料，画出各季节（至少画出冬夏两季）温度场、纬向风场在各高度（850、500、100 hPa， 对于温度场请再画出1000hPa）上的水平分布，并简述其分布特征和季节变化特征。<br />
<br />
==Bonus Question==<br />
<br />
请使用多年(>20年)的月平均再分析资料(NCEP/NCAR或其他)，画出冬夏两季由平均环流和定常波引起的经向热量通量纬向平均后的高度-纬度剖面分布，并简述它们的分布特征。<br />
<br />
==On the netcdf file==<br />
<br />
Netcdf (Network Common Data Form) is a set of self-describing, machine-independent data formats that support the creation, access, and sharing of array-oriented scientific data. The project homepage is hosted by the [https://www.unidata.ucar.edu/software/netcdf Unidata program] at the University Corporation for Atmospheric Research (UCAR). They are also the chief source of netCDF software, standards development, updates etc.。<br />
[https://psl.noaa.gov/data/gridded_help/read-our-files.html 读取netcdf文件的几种方法]<br />
<br />
==Read .nc using Matlab==<br />
从 <font color="red"> Matlab 2008b </font> 开始，Matlab 开始提供自带的读写.nc文件的函数。以下为读写.nc的一个样本matlab文件。对于图片，Matlab提供多种形式的存储。可在弹出图片的窗口左上方的下拉菜单中选择图片的存储方式，或者在命令行后用“print”命令存储图片（可以输入 help print 获得帮助）。关于Matlab的更多信息，可以去[http://www.mathworks.com/help Matlab网站]。<br />
<br />
==下载区==<br />
本次作业所用的数据资料可以用来自NCEP-DOE Reanalysis 2 的月平均资料，时间跨度为1979/1-2009/12。关于本资料的详细内容，请[https://psl.noaa.gov/data/gridded/data.ncep.reanalysis2.html 参见网站]。 以下为完成本次作业所需的数据文件和matlab文件。<br />
<br />
NCEP-DOE 月平均温度场资料 (1979/1-2009/12): [http://tcs.nju.edu.cn/yzhang/air.mon.mean.nc nc file]，[http://tcs.nju.edu.cn/yzhang/air_readme.txt 说明文件]<br />
<br />
NCEP-DOE 月平均纬向风(U)资料 (1979/1-2009/12): [http://tcs.nju.edu.cn/yzhang/uwnd.mon.mean.nc nc file]，[http://tcs.nju.edu.cn/yzhang/uwnd_readme.txt 说明文件]<br />
<br />
NCEP-DOE 月平均经向风(V)资料 (1979/1-2009/12): [http://tcs.nju.edu.cn/yzhang/vwnd.mon.mean.nc nc file]，[http://tcs.nju.edu.cn/yzhang/vwnd_readme.txt 说明文件]<br />
<br />
NCEP-land mask（用于在题目2中画出具体的海陆分布）: [http://tcs.nju.edu.cn/yzhang/land.nc nc文件]<br />
<br />
题目1和2计算分析用的Matlab 文件： [http://tcs.nju.edu.cn/yzhang/sample_for_Q1.m 第一题m文件], [http://tcs.nju.edu.cn/yzhang/sample_for_Q2.m 第二题m文件]</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=General_Circulation(Fall_2026)&diff=13979General Circulation(Fall 2026)2026-09-27T08:04:22Z<p>Etone: /* Announcement */</p>
<hr />
<div>{{Infobox<br />
|name = Infobox<br />
|bodystyle = <br />
|title = 大气环流 <br><br />
General Circulation of the Atmosphere<br />
|titlestyle = <br />
<br />
|headerstyle = background:#ccf;<br />
|labelstyle = background:#ddf;<br />
|datastyle = <br />
<br />
|header1 =Instructor<br />
|label1 = <br />
|data1 = <br />
|header2 = <br />
|label2 = <br />
|data2 = 张洋<br />
|header3 = <br />
|label3 = Email<br />
|data3 = yangzhang@nju.edu.cn <br />
|header4 =<br />
|label4= office<br />
|data4= 仙林大气楼 B410<br />
|header5 = Class<br />
|label5 = <br />
|data5 = <br />
|header6 =<br />
|label6 = Class meetings<br />
|data6 = 周一 下午 2:00-4:00，逸B-303<br />
|header7 =<br />
|label7 = Place<br />
|data7 = <br />
|header8 =<br />
|label8 = Office hours<br />
|data8 = 周四 下午13:00-2:00 <br>仙林大气楼B410 <br />
|header9 = Reference book<br />
|label9 = <br />
|data9 = <br />
|header10 =<br />
|label10 = <br />
|data10 = {{Infobox<br />
|name = <br />
|bodystyle = <br />
|title = <br />
|titlestyle = <br />
|image = [[File:James-Circulating.jpg|border|100px]]<br />
|imagestyle = <br />
|caption = Introduction to Circulating Atmospheres, <br>''I. James'', Cambridge Press, 1995<br />
|captionstyle = }}<br />
|header11 =<br />
|label11 = <br />
|data11 = {{Infobox<br />
|name = <br />
|bodystyle = <br />
|title = <br />
|titlestyle = <br />
|image = [[File:Oort.jpg|border|100px]]<br />
|imagestyle = <br />
|caption =Physics of Climate, ''Peixoto, J. P.'' and ''A. H. Oort'', Springer-Verlag New York, 1992<br />
|captionstyle = <br />
}}<br />
|header12 =<br />
|label12 = <br />
|data12 = {{Infobox<br />
|name = <br />
|bodystyle = <br />
|title = <br />
|titlestyle = <br />
|image = [[File:geoff.jpg|border|100px]]<br />
|imagestyle = <br />
|caption =Atmospheric and Oceanic Fluid Dynamics: Fundamentals and Large-scale Circulation, ''Vallis, G. K.'', Cambridge University Press, 2006<br />
|captionstyle = <br />
}}<br />
|belowstyle = background:#ddf;<br />
|below = <br />
}}<br />
<br />
<br />
This is the page for the class ''General Circulation of the Atmosphere (大气环流)'' for the Fall 2025 semester. Students who take this class should check this page periodically for content updates and new announcements.<br />
<br />
= Announcement =<br />
* 课程交流QQ群：1075545767，请各位选课的同学入群。【2026.9.21】<br />
<br />
= Course info =<br />
* '''Instructor ''': 张洋，<br />
:*office: 仙林气象楼 B410<br />
:*email: yangzhang@nju.edu.cn<br />
* '''Class meeting''': 周一 下午 2:00-4:00；逸B-303<br />
* '''Office hour''': 周四 <font color="red" size="2>下午1:00-2:00，仙林大气楼B410</font><br />
* '''Prerequisites''': 动力气象，天气学，气候学<br />
* '''Grading''': 平时作业（50％）＋ 期末考试（50％）<br />
本课程将大致布置4次作业，每次作业一二道题目左右。题目将选择每个课题最具有代表性、需要一定思维强度和动手能力的训练用题目,意在使学生通过顺利完成作业来建立环流系统的物理模型、以对课程内容得到深刻全面地理解和掌握。期末考试题目数量将会比平时作业多,覆盖面更广,但会比作业题目简单,只涉及对基本内容的掌握和对环流理论的直接应用。<br />
<br />
= Assignments =<br />
# [[Assignment 1, Fall 2026|Reanalysis data and the earth's climatology]] [Due:2026.10.12]<font color="red" size="1.5">请在本次作业的截止日期前将本次作业的图片部分和文字部分打包作为附件发送至邮箱 circulation_nju@126.com </font><br />
<br />
<br />
= Course intro =<br />
“大气环流”常指地球大气较大空间范围、较长时间尺度上的空气流动，及其对地球大气热量、动量、能量和水汽的全球输送。虽然从十七、十八世纪起人们就开始研究大尺度的大气运动(如Hadley在1735年提出的信风理论)，但大气环流真正发展成为一门较完备的学科方向却是近半个世纪的事情。随着四五十年代探空资料等高空气象要素的取得，以及六十年代卫星等覆盖全球的观测资料的加入，大气环流的空间结构和时间变化开始被系统、全面地揭示。与此同时，大气环流的数值模拟，也开始成为研究大气环流的一个主要方法，并发展至今成为了解和预估未来气候变化的主要手段。随着观测和模拟手段的进步，大气环流的理论研究也在近三十年开始快速地发展，人们对各种环流系统的维持和变化有了更全面、更深刻、也更为现代的理解。<br />
<br />
现代的大气环流是大气动力学、天气学和气候学相结合的产物。大气环流，既是各种天气现象产生的背景流场，又是各种气候状态形成的动力机制。大气环流在低频、季节、年际、年代际等时间尺度的变化，不但会引起天气现象的变化，也影响着气候状态的形成。而在大气科学领域面临着诸如全球暖化、气候变化、环流异常等重大科学问题的今天，大气环流研究的重要性被推到了前所未有的高度,大气环流也成为活跃发展又充满挑战的学科方向。<br />
<br />
本课程将讲述在过去几十年里大气环流在观测、理论和模拟上取得的进展。希望学生借此课程能熟悉大气环流的基本分布和形态，掌握各主要环流系统的维持和变化机制，建立各环流系统形成的物理模型，了解现阶段的大气环流模式，知道大气环流方向有待解决的科学问题。<br />
<br />
作为一门课程，大气环流内容的讲述常可以有两条线索。一条是全球尺度上大气热量、动量、能量和水汽的分布与输送，Lorenz（1967）和 Peixoto and Oort（1992）是按此线索介绍大气环流的优秀教材；另一条线索，是各纬度、各区域内大气环流系统的形成、维持和变化机制，James（1995）和 Vallis（2006）是按此线索介绍大气环流的经典讲义。根据现阶段大气环流方向的研究特点，本课程的讲述将主要按照后一种方式来展开，并辅以介绍各环流系统对大气各要素场的输送。在介绍各环流系统时，本课程将以观测、理论和模拟为顺序，从各大气环流系统的观测事实入手，介绍大气环流系统的分布特征和时空变化特征；着重介绍关于环流系统的各种动力学模型和现阶段对环流系统的理解；辅以对环流系统模拟研究的介绍；最后通过三者的对比，讨论各环流系统有待研究的问题。<br />
<br />
= Syllabus =<br />
本课程具体的内容安排如下：第一章为大气环流的概述，介绍大气环流发展的历史、包含的内容以及大气环流研究的常用观测资料和分析方法。第二章介绍大气环流产生的外部强迫：辐射强迫和下界面过程。第三至六章介绍大气环流中的各个环流系统及它们的动力机制。第七章详细介绍各复杂度的大气环流模式。第八章介绍大气环流领域现阶段最大的一个开放课题：全球暖化背景下的大气环流。这一章既是对前几章所介绍的大气环流理论的应用与检验，又是对未来大气环流研究方向的探讨。借此让学生熟悉并理解大气环流领域亟需解决的课题。具体课程安排和参考书目如下。<br />
== Course schedule ==<br />
*大气环流概述 (Introduction) （6课时）<br />
*大气环流的外部强迫（3课时）<br />
**辐射强迫 (Radiative forcing)<br />
**下界面过程 (Surface boundaries)<br />
*经向环流系统 (Zonally-averaged circulations)<br />
**Hadley 环流（4课时）<br />
**Ferrel 环流，急流，中纬度的波流相互作用（8课时）<br />
*纬向环流系统（Non-zonal circulations）（6课时）<br />
**Storm tracks<br />
**Monsoon<br />
**ENSO and Walker circulation<br />
*能量和水汽循环 (Angular momentum, energy and water vapor)（2课时）<br />
*不同复杂度的大气环流模式 (General circulation in a hierarchy of models)（2课时）<br />
*全球暖化背景下的大气环流 (General circulation in the global warming scenario)（2课时）<br />
<br />
[[点击此处看详细课程安排 (click for more)]]<br />
<br />
== References ==<br />
*观测部分：Peixoto, J. P. and A. H. Oort, 1992: Physics of Climate. Springer-Verlag New York, Inc., 520 pp. 中文译本：气候物理学，1995，吴国雄、刘辉等译校，气象出版社。<br />
*综合介绍：James, I., 1995: Introduction to circulating atmospheres. Cambridge University Press, 448 pp. <br />
*理论部分：Vallis, G. K., 2006: Atmospheric and Oceanic Fluid Dynamics: Fundamentals and Large-scale Circulation. Cambridge University Press. 745 pp.<br />
* [[其它参考书目（点击看详情）]]</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=General_Circulation(Fall_2026)&diff=13978General Circulation(Fall 2026)2026-09-27T08:02:27Z<p>Etone: /* Course info */</p>
<hr />
<div>{{Infobox<br />
|name = Infobox<br />
|bodystyle = <br />
|title = 大气环流 <br><br />
General Circulation of the Atmosphere<br />
|titlestyle = <br />
<br />
|headerstyle = background:#ccf;<br />
|labelstyle = background:#ddf;<br />
|datastyle = <br />
<br />
|header1 =Instructor<br />
|label1 = <br />
|data1 = <br />
|header2 = <br />
|label2 = <br />
|data2 = 张洋<br />
|header3 = <br />
|label3 = Email<br />
|data3 = yangzhang@nju.edu.cn <br />
|header4 =<br />
|label4= office<br />
|data4= 仙林大气楼 B410<br />
|header5 = Class<br />
|label5 = <br />
|data5 = <br />
|header6 =<br />
|label6 = Class meetings<br />
|data6 = 周一 下午 2:00-4:00，逸B-303<br />
|header7 =<br />
|label7 = Place<br />
|data7 = <br />
|header8 =<br />
|label8 = Office hours<br />
|data8 = 周四 下午13:00-2:00 <br>仙林大气楼B410 <br />
|header9 = Reference book<br />
|label9 = <br />
|data9 = <br />
|header10 =<br />
|label10 = <br />
|data10 = {{Infobox<br />
|name = <br />
|bodystyle = <br />
|title = <br />
|titlestyle = <br />
|image = [[File:James-Circulating.jpg|border|100px]]<br />
|imagestyle = <br />
|caption = Introduction to Circulating Atmospheres, <br>''I. James'', Cambridge Press, 1995<br />
|captionstyle = }}<br />
|header11 =<br />
|label11 = <br />
|data11 = {{Infobox<br />
|name = <br />
|bodystyle = <br />
|title = <br />
|titlestyle = <br />
|image = [[File:Oort.jpg|border|100px]]<br />
|imagestyle = <br />
|caption =Physics of Climate, ''Peixoto, J. P.'' and ''A. H. Oort'', Springer-Verlag New York, 1992<br />
|captionstyle = <br />
}}<br />
|header12 =<br />
|label12 = <br />
|data12 = {{Infobox<br />
|name = <br />
|bodystyle = <br />
|title = <br />
|titlestyle = <br />
|image = [[File:geoff.jpg|border|100px]]<br />
|imagestyle = <br />
|caption =Atmospheric and Oceanic Fluid Dynamics: Fundamentals and Large-scale Circulation, ''Vallis, G. K.'', Cambridge University Press, 2006<br />
|captionstyle = <br />
}}<br />
|belowstyle = background:#ddf;<br />
|below = <br />
}}<br />
<br />
<br />
This is the page for the class ''General Circulation of the Atmosphere (大气环流)'' for the Fall 2025 semester. Students who take this class should check this page periodically for content updates and new announcements.<br />
<br />
= Announcement =<br />
* 课程交流QQ群：631262169，请各位选课的同学入群。【2025.9.8】<br />
<br />
= Course info =<br />
* '''Instructor ''': 张洋，<br />
:*office: 仙林气象楼 B410<br />
:*email: yangzhang@nju.edu.cn<br />
* '''Class meeting''': 周一 下午 2:00-4:00；逸B-303<br />
* '''Office hour''': 周四 <font color="red" size="2>下午1:00-2:00，仙林大气楼B410</font><br />
* '''Prerequisites''': 动力气象，天气学，气候学<br />
* '''Grading''': 平时作业（50％）＋ 期末考试（50％）<br />
本课程将大致布置4次作业，每次作业一二道题目左右。题目将选择每个课题最具有代表性、需要一定思维强度和动手能力的训练用题目,意在使学生通过顺利完成作业来建立环流系统的物理模型、以对课程内容得到深刻全面地理解和掌握。期末考试题目数量将会比平时作业多,覆盖面更广,但会比作业题目简单,只涉及对基本内容的掌握和对环流理论的直接应用。<br />
<br />
= Assignments =<br />
# [[Assignment 1, Fall 2026|Reanalysis data and the earth's climatology]] [Due:2026.10.12]<font color="red" size="1.5">请在本次作业的截止日期前将本次作业的图片部分和文字部分打包作为附件发送至邮箱 circulation_nju@126.com </font><br />
<br />
<br />
= Course intro =<br />
“大气环流”常指地球大气较大空间范围、较长时间尺度上的空气流动，及其对地球大气热量、动量、能量和水汽的全球输送。虽然从十七、十八世纪起人们就开始研究大尺度的大气运动(如Hadley在1735年提出的信风理论)，但大气环流真正发展成为一门较完备的学科方向却是近半个世纪的事情。随着四五十年代探空资料等高空气象要素的取得，以及六十年代卫星等覆盖全球的观测资料的加入，大气环流的空间结构和时间变化开始被系统、全面地揭示。与此同时，大气环流的数值模拟，也开始成为研究大气环流的一个主要方法，并发展至今成为了解和预估未来气候变化的主要手段。随着观测和模拟手段的进步，大气环流的理论研究也在近三十年开始快速地发展，人们对各种环流系统的维持和变化有了更全面、更深刻、也更为现代的理解。<br />
<br />
现代的大气环流是大气动力学、天气学和气候学相结合的产物。大气环流，既是各种天气现象产生的背景流场，又是各种气候状态形成的动力机制。大气环流在低频、季节、年际、年代际等时间尺度的变化，不但会引起天气现象的变化，也影响着气候状态的形成。而在大气科学领域面临着诸如全球暖化、气候变化、环流异常等重大科学问题的今天，大气环流研究的重要性被推到了前所未有的高度,大气环流也成为活跃发展又充满挑战的学科方向。<br />
<br />
本课程将讲述在过去几十年里大气环流在观测、理论和模拟上取得的进展。希望学生借此课程能熟悉大气环流的基本分布和形态，掌握各主要环流系统的维持和变化机制，建立各环流系统形成的物理模型，了解现阶段的大气环流模式，知道大气环流方向有待解决的科学问题。<br />
<br />
作为一门课程，大气环流内容的讲述常可以有两条线索。一条是全球尺度上大气热量、动量、能量和水汽的分布与输送，Lorenz（1967）和 Peixoto and Oort（1992）是按此线索介绍大气环流的优秀教材；另一条线索，是各纬度、各区域内大气环流系统的形成、维持和变化机制，James（1995）和 Vallis（2006）是按此线索介绍大气环流的经典讲义。根据现阶段大气环流方向的研究特点，本课程的讲述将主要按照后一种方式来展开，并辅以介绍各环流系统对大气各要素场的输送。在介绍各环流系统时，本课程将以观测、理论和模拟为顺序，从各大气环流系统的观测事实入手，介绍大气环流系统的分布特征和时空变化特征；着重介绍关于环流系统的各种动力学模型和现阶段对环流系统的理解；辅以对环流系统模拟研究的介绍；最后通过三者的对比，讨论各环流系统有待研究的问题。<br />
<br />
= Syllabus =<br />
本课程具体的内容安排如下：第一章为大气环流的概述，介绍大气环流发展的历史、包含的内容以及大气环流研究的常用观测资料和分析方法。第二章介绍大气环流产生的外部强迫：辐射强迫和下界面过程。第三至六章介绍大气环流中的各个环流系统及它们的动力机制。第七章详细介绍各复杂度的大气环流模式。第八章介绍大气环流领域现阶段最大的一个开放课题：全球暖化背景下的大气环流。这一章既是对前几章所介绍的大气环流理论的应用与检验，又是对未来大气环流研究方向的探讨。借此让学生熟悉并理解大气环流领域亟需解决的课题。具体课程安排和参考书目如下。<br />
== Course schedule ==<br />
*大气环流概述 (Introduction) （6课时）<br />
*大气环流的外部强迫（3课时）<br />
**辐射强迫 (Radiative forcing)<br />
**下界面过程 (Surface boundaries)<br />
*经向环流系统 (Zonally-averaged circulations)<br />
**Hadley 环流（4课时）<br />
**Ferrel 环流，急流，中纬度的波流相互作用（8课时）<br />
*纬向环流系统（Non-zonal circulations）（6课时）<br />
**Storm tracks<br />
**Monsoon<br />
**ENSO and Walker circulation<br />
*能量和水汽循环 (Angular momentum, energy and water vapor)（2课时）<br />
*不同复杂度的大气环流模式 (General circulation in a hierarchy of models)（2课时）<br />
*全球暖化背景下的大气环流 (General circulation in the global warming scenario)（2课时）<br />
<br />
[[点击此处看详细课程安排 (click for more)]]<br />
<br />
== References ==<br />
*观测部分：Peixoto, J. P. and A. H. Oort, 1992: Physics of Climate. Springer-Verlag New York, Inc., 520 pp. 中文译本：气候物理学，1995，吴国雄、刘辉等译校，气象出版社。<br />
*综合介绍：James, I., 1995: Introduction to circulating atmospheres. Cambridge University Press, 448 pp. <br />
*理论部分：Vallis, G. K., 2006: Atmospheric and Oceanic Fluid Dynamics: Fundamentals and Large-scale Circulation. Cambridge University Press. 745 pp.<br />
* [[其它参考书目（点击看详情）]]</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=General_Circulation(Fall_2026)&diff=13977General Circulation(Fall 2026)2026-09-27T08:02:06Z<p>Etone: Created page with "{{Infobox |name = Infobox |bodystyle = |title = 大气环流 <br> General Circulation of the Atmosphere |titlestyle = |headerstyle = background:#ccf; |labelstyle = background:#ddf; |datastyle = |header1 =Instructor |label1 = |data1 = |header2 = |label2 = |data2 = 张洋 |header3 = |label3 = Email |data3 = yangzhang@nju.edu.cn |header4 = |label4= office |data4= 仙林大气楼 B410 |header5 = Class |label5 = |data5 = |hea..."</p>
<hr />
<div>{{Infobox<br />
|name = Infobox<br />
|bodystyle = <br />
|title = 大气环流 <br><br />
General Circulation of the Atmosphere<br />
|titlestyle = <br />
<br />
|headerstyle = background:#ccf;<br />
|labelstyle = background:#ddf;<br />
|datastyle = <br />
<br />
|header1 =Instructor<br />
|label1 = <br />
|data1 = <br />
|header2 = <br />
|label2 = <br />
|data2 = 张洋<br />
|header3 = <br />
|label3 = Email<br />
|data3 = yangzhang@nju.edu.cn <br />
|header4 =<br />
|label4= office<br />
|data4= 仙林大气楼 B410<br />
|header5 = Class<br />
|label5 = <br />
|data5 = <br />
|header6 =<br />
|label6 = Class meetings<br />
|data6 = 周一 下午 2:00-4:00，逸B-303<br />
|header7 =<br />
|label7 = Place<br />
|data7 = <br />
|header8 =<br />
|label8 = Office hours<br />
|data8 = 周四 下午13:00-2:00 <br>仙林大气楼B410 <br />
|header9 = Reference book<br />
|label9 = <br />
|data9 = <br />
|header10 =<br />
|label10 = <br />
|data10 = {{Infobox<br />
|name = <br />
|bodystyle = <br />
|title = <br />
|titlestyle = <br />
|image = [[File:James-Circulating.jpg|border|100px]]<br />
|imagestyle = <br />
|caption = Introduction to Circulating Atmospheres, <br>''I. James'', Cambridge Press, 1995<br />
|captionstyle = }}<br />
|header11 =<br />
|label11 = <br />
|data11 = {{Infobox<br />
|name = <br />
|bodystyle = <br />
|title = <br />
|titlestyle = <br />
|image = [[File:Oort.jpg|border|100px]]<br />
|imagestyle = <br />
|caption =Physics of Climate, ''Peixoto, J. P.'' and ''A. H. Oort'', Springer-Verlag New York, 1992<br />
|captionstyle = <br />
}}<br />
|header12 =<br />
|label12 = <br />
|data12 = {{Infobox<br />
|name = <br />
|bodystyle = <br />
|title = <br />
|titlestyle = <br />
|image = [[File:geoff.jpg|border|100px]]<br />
|imagestyle = <br />
|caption =Atmospheric and Oceanic Fluid Dynamics: Fundamentals and Large-scale Circulation, ''Vallis, G. K.'', Cambridge University Press, 2006<br />
|captionstyle = <br />
}}<br />
|belowstyle = background:#ddf;<br />
|below = <br />
}}<br />
<br />
<br />
This is the page for the class ''General Circulation of the Atmosphere (大气环流)'' for the Fall 2025 semester. Students who take this class should check this page periodically for content updates and new announcements.<br />
<br />
= Announcement =<br />
* 课程交流QQ群：631262169，请各位选课的同学入群。【2025.9.8】<br />
<br />
= Course info =<br />
* '''Instructor ''': 张洋，<br />
:*office: 仙林气象楼 B410<br />
:*email: yangzhang@nju.edu.cn<br />
* '''Class meeting''': 周一 下午 2:00-4:00；逸B-303<br />
* '''Office hour''': 周五 <font color="red" size="2>下午1:00-2:00，仙林大气楼B410</font><br />
* '''Prerequisites''': 动力气象，天气学，气候学<br />
* '''Grading''': 平时作业（50％）＋ 期末考试（50％）<br />
本课程将大致布置4次作业，每次作业一二道题目左右。题目将选择每个课题最具有代表性、需要一定思维强度和动手能力的训练用题目,意在使学生通过顺利完成作业来建立环流系统的物理模型、以对课程内容得到深刻全面地理解和掌握。期末考试题目数量将会比平时作业多,覆盖面更广,但会比作业题目简单,只涉及对基本内容的掌握和对环流理论的直接应用。<br />
<br />
= Assignments =<br />
# [[Assignment 1, Fall 2026|Reanalysis data and the earth's climatology]] [Due:2026.10.12]<font color="red" size="1.5">请在本次作业的截止日期前将本次作业的图片部分和文字部分打包作为附件发送至邮箱 circulation_nju@126.com </font><br />
<br />
<br />
= Course intro =<br />
“大气环流”常指地球大气较大空间范围、较长时间尺度上的空气流动，及其对地球大气热量、动量、能量和水汽的全球输送。虽然从十七、十八世纪起人们就开始研究大尺度的大气运动(如Hadley在1735年提出的信风理论)，但大气环流真正发展成为一门较完备的学科方向却是近半个世纪的事情。随着四五十年代探空资料等高空气象要素的取得，以及六十年代卫星等覆盖全球的观测资料的加入，大气环流的空间结构和时间变化开始被系统、全面地揭示。与此同时，大气环流的数值模拟，也开始成为研究大气环流的一个主要方法，并发展至今成为了解和预估未来气候变化的主要手段。随着观测和模拟手段的进步，大气环流的理论研究也在近三十年开始快速地发展，人们对各种环流系统的维持和变化有了更全面、更深刻、也更为现代的理解。<br />
<br />
现代的大气环流是大气动力学、天气学和气候学相结合的产物。大气环流，既是各种天气现象产生的背景流场，又是各种气候状态形成的动力机制。大气环流在低频、季节、年际、年代际等时间尺度的变化，不但会引起天气现象的变化，也影响着气候状态的形成。而在大气科学领域面临着诸如全球暖化、气候变化、环流异常等重大科学问题的今天，大气环流研究的重要性被推到了前所未有的高度,大气环流也成为活跃发展又充满挑战的学科方向。<br />
<br />
本课程将讲述在过去几十年里大气环流在观测、理论和模拟上取得的进展。希望学生借此课程能熟悉大气环流的基本分布和形态，掌握各主要环流系统的维持和变化机制，建立各环流系统形成的物理模型，了解现阶段的大气环流模式，知道大气环流方向有待解决的科学问题。<br />
<br />
作为一门课程，大气环流内容的讲述常可以有两条线索。一条是全球尺度上大气热量、动量、能量和水汽的分布与输送，Lorenz（1967）和 Peixoto and Oort（1992）是按此线索介绍大气环流的优秀教材；另一条线索，是各纬度、各区域内大气环流系统的形成、维持和变化机制，James（1995）和 Vallis（2006）是按此线索介绍大气环流的经典讲义。根据现阶段大气环流方向的研究特点，本课程的讲述将主要按照后一种方式来展开，并辅以介绍各环流系统对大气各要素场的输送。在介绍各环流系统时，本课程将以观测、理论和模拟为顺序，从各大气环流系统的观测事实入手，介绍大气环流系统的分布特征和时空变化特征；着重介绍关于环流系统的各种动力学模型和现阶段对环流系统的理解；辅以对环流系统模拟研究的介绍；最后通过三者的对比，讨论各环流系统有待研究的问题。<br />
<br />
= Syllabus =<br />
本课程具体的内容安排如下：第一章为大气环流的概述，介绍大气环流发展的历史、包含的内容以及大气环流研究的常用观测资料和分析方法。第二章介绍大气环流产生的外部强迫：辐射强迫和下界面过程。第三至六章介绍大气环流中的各个环流系统及它们的动力机制。第七章详细介绍各复杂度的大气环流模式。第八章介绍大气环流领域现阶段最大的一个开放课题：全球暖化背景下的大气环流。这一章既是对前几章所介绍的大气环流理论的应用与检验，又是对未来大气环流研究方向的探讨。借此让学生熟悉并理解大气环流领域亟需解决的课题。具体课程安排和参考书目如下。<br />
== Course schedule ==<br />
*大气环流概述 (Introduction) （6课时）<br />
*大气环流的外部强迫（3课时）<br />
**辐射强迫 (Radiative forcing)<br />
**下界面过程 (Surface boundaries)<br />
*经向环流系统 (Zonally-averaged circulations)<br />
**Hadley 环流（4课时）<br />
**Ferrel 环流，急流，中纬度的波流相互作用（8课时）<br />
*纬向环流系统（Non-zonal circulations）（6课时）<br />
**Storm tracks<br />
**Monsoon<br />
**ENSO and Walker circulation<br />
*能量和水汽循环 (Angular momentum, energy and water vapor)（2课时）<br />
*不同复杂度的大气环流模式 (General circulation in a hierarchy of models)（2课时）<br />
*全球暖化背景下的大气环流 (General circulation in the global warming scenario)（2课时）<br />
<br />
[[点击此处看详细课程安排 (click for more)]]<br />
<br />
== References ==<br />
*观测部分：Peixoto, J. P. and A. H. Oort, 1992: Physics of Climate. Springer-Verlag New York, Inc., 520 pp. 中文译本：气候物理学，1995，吴国雄、刘辉等译校，气象出版社。<br />
*综合介绍：James, I., 1995: Introduction to circulating atmospheres. Cambridge University Press, 448 pp. <br />
*理论部分：Vallis, G. K., 2006: Atmospheric and Oceanic Fluid Dynamics: Fundamentals and Large-scale Circulation. Cambridge University Press. 745 pp.<br />
* [[其它参考书目（点击看详情）]]</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=Main_Page&diff=13976Main Page2026-09-27T07:54:29Z<p>Etone: /* Home Pages for Courses and Seminars */</p>
<hr />
<div>This is a course/seminar wiki run by the [http://tcs.nju.edu.cn theory group] in the Department of Computer Science and Technology at Nanjing University.<br />
<br />
== Home Pages for Courses and Seminars==<br />
;Current semester<br />
<br />
* [[高级算法 (Fall 2026)|高级算法 Advanced Algorithms (Fall 2026)]]<br />
<br />
* [[计算方法 Numerical method (Fall 2026)|计算方法 Numerical method (Fall 2026)]]<br />
<br />
* [[数据科学基础 (Fall 2026)|数据科学基础 (Fall 2026)]]<br />
<br />
* [[Theory@Suzhou_2026 | Suzhou Theory day]] (Nov 22, 2026)<br />
<br />
;Past courses<br />
<br />
* Advanced Algorithms: [[高级算法 (Spring 2026)| Spring 2026 (Suzhou)]], [[高级算法 (Fall 2025)|Fall 2025]], [[高级算法 (Spring 2025)|Spring 2025 (Suzhou)]], [[高级算法 (Fall 2024)|Fall 2024]], [[高级算法 (Fall 2023)|Fall 2023]], [[高级算法 (Fall 2022)|Fall 2022]], [[高级算法 (Fall 2021)|Fall 2021]], [[高级算法 (Fall 2020)|Fall 2020]], [[高级算法 (Fall 2019)|Fall 2019]], [[高级算法 (Fall 2018)|Fall 2018]], [[高级算法 (Fall 2017)|Fall 2017]], [[随机算法 \ 高级算法 (Fall 2016)|Fall 2016]].<br />
<br />
*Algorithm Design and Analysis: [https://tcs.nju.edu.cn/shili/courses/2024spring-algo/ Spring 2024]<br />
<br />
* Combinatorics: [[组合数学 (Spring 2026)|Spring 2026]], [[组合数学 (Spring 2025)|Spring 2025]], [[组合数学 (Spring 2024)|Spring 2024]], [[组合数学 (Spring 2023)|Spring 2023]], [[组合数学 (Fall 2019)|Fall 2019]], [[组合数学 (Fall 2017)|Fall 2017]], [[组合数学 (Fall 2016)|Fall 2016]], [[组合数学 (Fall 2015)|Fall 2015]], [[组合数学 (Spring 2014)|Spring 2014]], [[组合数学 (Spring 2013)|Spring 2013]], [[组合数学 (Fall 2011)|Fall 2011]], [[Combinatorics (Fall 2010)|Fall 2010]].<br />
<br />
* Computational Complexity: [[计算复杂性 (Spring 2026)| Spring 2026]], [[计算复杂性 (Spring 2025)|Spring 2025]], [[计算复杂性 (Spring 2024)|Spring 2024]], [[计算复杂性 (Spring 2023)|Spring 2023]], [[计算复杂性 (Fall 2019)|Fall 2019]], [[计算复杂性 (Fall 2018)|Fall 2018]].<br />
<br />
* Foundations of Data Science: [[数据科学基础 (Fall 2025)|Fall 2025]], [[数据科学基础 (Fall 2024)|Fall 2024]]<br />
<br />
* Numerical Method: [[计算方法 Numerical method (Spring 2026)|Spring 2026]], [[计算方法 Numerical method (Spring 2025)|Spring 2025]], [[计算方法 Numerical method (Spring 2024)|Spring 2024]], [[计算方法 Numerical method (Spring 2023)|Spring 2023]], [https://liuexp.github.io/numerical.html Spring 2022].<br />
<br />
* Probability Theory: [[概率论与数理统计 (Spring 2026)| Spring 2026]], [[概率论与数理统计 (Spring 2025)|Spring 2025]], [[概率论与数理统计 (Spring 2024)|Spring 2024]], [[概率论与数理统计 (Spring 2023)|Spring 2023]].<br />
<br />
* Quantum Computation: [[量子计算 (Spring 2022)|Spring 2022]], [[量子计算 (Spring 2021)|Spring 2021]], [[量子计算 (Fall 2019)|Fall 2019]].<br />
<br />
* Randomized Algorithms: [[随机算法 (Fall 2015)|Fall 2015]], [[随机算法 (Spring 2014)|Spring 2014]], [[随机算法 (Spring 2013)|Spring 2013]], [[随机算法 (Fall 2011)|Fall 2011]], [[Randomized Algorithms (Spring 2010)|Spring 2010]].<br />
<br />
;Past seminars, workshops and summer schools<br />
*计算理论之美暑期学校: [[计算理论之美 (Summer 2026) | 2026]], [[计算理论之美 (Summer 2025)|2025]], [[计算理论之美 (Summer 2024)|2024]], [[计算理论之美 (Summer 2023)|2023]], [[计算理论之美 (Summer 2021)|2021]]<br />
*[[Theory Seminar|理论计算机科学讨论班]]<br />
*[[Study Group|理论计算机科学学习小组]]<br />
*[[TCSPhD2020| 理论计算机科学优秀博士生论坛2020]]<br />
*[[Quantum|量子算法与物理实现研讨会]]<br />
*Theory Day: [[Theory@Suzhou 2025 | 2025 (Suzhou)]], [[Theory@Nanjing 2019|2019]], [[Theory@Nanjing 2018|2018]], [[Theory@Nanjing 2017|2017]]<br />
*[[\Delta Seminar on Logic, Philosophy, and Computer Science|Δ Seminar on Logic, Philosophy, and Computer Science]]<br />
*[[近似算法讨论班 (Fall 2011)|近似算法 Approximation Algorithms, Fall 2011.]]<br />
<br />
; 其它链接<br />
* [[General Circulation(Fall 2026)|大气环流 General Circulation of the Atmosphere, Fall 2026]]<br />
* [[General Circulation(Fall 2025)|大气环流 General Circulation of the Atmosphere, Fall 2025]]<br />
<br />
<br />
* [[概率论 (Summer 2014)| 概率与计算 (上海交大 Summer 2014)]]</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=%E9%AB%98%E7%BA%A7%E7%AE%97%E6%B3%95_(Fall_2026)&diff=13967高级算法 (Fall 2026)2026-09-23T05:58:44Z<p>Etone: /* Lecture Notes */</p>
<hr />
<div>{{Infobox<br />
|name = Infobox<br />
|bodystyle = <br />
|title = <font size=3>高级算法 <br />
<br>Advanced Algorithms</font><br />
|titlestyle = <br />
<br />
|image = <br />
|imagestyle = <br />
|caption = <br />
|captionstyle = <br />
|headerstyle = background:#ccf;<br />
|labelstyle = background:#ddf;<br />
|datastyle = <br />
<br />
|header1 =Instructor<br />
|label1 = <br />
|data1 = <br />
|header2 = <br />
|label2 = <br />
|data2 = '''尹一通'''<br />
|header3 = <br />
|label3 = Email<br />
|data3 = yinyt@nju.edu.cn <br />
|header4 =<br />
|label4= office<br />
|data4= 计算机系 804<br />
|header5 = <br />
|label5 = <br />
|data5 = '''栗师'''<br />
|header6 = <br />
|label6 = Email<br />
|data6 = shili@nju.edu.cn <br />
|header7 =<br />
|label7= office<br />
|data7= 计算机系 605<br />
|header8 = <br />
|label8 = <br />
|data8 = '''刘景铖'''<br />
|header9 = <br />
|label9 = Email<br />
|data9 = liu@nju.edu.cn <br />
|header10 =<br />
|label10= office<br />
|data10= 计算机系 516<br />
|header11 = Class<br />
|label11 = <br />
|data11 = <br />
|header12 =<br />
|label12 = Class meetings<br />
|data12 = 周一 5-6节 (单) 仙Ⅰ-319<br />
周三 5-6节 仙Ⅰ-319<br />
|header13 =<br />
|label13 = Place<br />
|data13 = <br />
|header14 =<br />
|label14 = Office hours<br />
|data14 = 周一4-5pm（尹一通 804）<br/><br />
周四4-5pm（刘景铖 516）<br/><br />
By appointment<br />
|header15 = Textbooks<br />
|label15 = <br />
|data15 = <br />
|header16 =<br />
|label16 = <br />
|data16 = [[File:MR-randomized-algorithms.png|border|100px]]<br />
|header17 =<br />
|label17 = <br />
|data17 = Motwani and Raghavan. <br>''Randomized Algorithms''.<br> Cambridge Univ Press, 1995.<br />
|header18 =<br />
|label18 = <br />
|data18 = [[File:Approximation_Algorithms.jpg|border|100px]]<br />
|header19 =<br />
|label19 = <br />
|data19 = Vazirani. <br>''Approximation Algorithms''. <br> Springer-Verlag, 2001.<br />
|belowstyle = background:#ddf;<br />
|below = <br />
}}<br />
<br />
This is the webpage for the ''Advanced Algorithms'' class of fall 2026. Students who take this class should check this page periodically for content updates and new announcements. <br />
<br />
= Announcement =<br />
<br />
= Course info =<br />
* '''Instructor ''': <br />
:* [http://tcs.nju.edu.cn/yinyt/ 尹一通]：[mailto:yinyt@nju.edu.cn <yinyt@nju.edu.cn>]，计算机系 804 <br />
:*[https://tcs.nju.edu.cn/shili/ 栗师]：[mailto:shili@nju.edu.cn <shili@nju.edu.cn>]，计算机系 605<br />
:* [https://liuexp.github.io 刘景铖]：[mailto:liu@nju.edu.cn <liu@nju.edu.cn>]，计算机系 516 <br />
* '''Teaching Assistant''': <br />
** 于逸潇：[mailto:yixiaoyu@smail.nju.edu.cn <yixiaoyu@smail.nju.edu.cn>]<br />
** 张弈垚：[mailto:zhangyiyao@smail.nju.edu.cn <zhangyiyao@smail.nju.edu.cn>]<br />
* '''Class meeting''': <br />
** 周一 5-6节 1-17周(单) 仙Ⅰ-319<br />
** 周三 5-6节 1-18周 仙Ⅰ-319<br />
* '''Office hour''': <br />
** 周一4-5pm（尹一通 804）<br />
** 周四4-5pm（刘景铖 516）<br />
** By appointment<br />
* '''QQ群''': 1098567018<br />
<br />
= Syllabus =<br />
随着计算机算法理论的不断发展，现代计算机算法的设计与分析大量地使用非初等的数学工具以及非传统的算法思想。“高级算法”这门课程就是面向计算机算法的这一发展趋势而设立的。课程将针对传统算法课程未系统涉及、却在计算机科学各领域的科研和实践中扮演重要角色的高等算法设计思想和算法分析工具进行系统讲授。<br />
<br />
=== 先修课程 Prerequisites ===<br />
* 必须：离散数学，概率论，线性代数。<br />
* 推荐：算法设计与分析。<br />
<br />
=== Course materials ===<br />
* [[高级算法 (Fall 2026) / Course materials|<font size=3>教材和参考书</font>]]<br />
<br />
=== 成绩 Grades ===<br />
* 课程成绩：本课程将会有若干次作业和一次期末考试。最终成绩将由平时作业成绩和期末考试成绩综合得出。<br />
* 迟交：如果有特殊的理由，无法按时完成作业，请提前联系授课老师，给出正当理由。否则迟交的作业将不被接受。<br />
<br />
=== <font color=red> 学术诚信 Academic Integrity </font>===<br />
学术诚信是所有从事学术活动的学生和学者最基本的职业道德底线，本课程将不遗余力的维护学术诚信规范，违反这一底线的行为将不会被容忍。<br />
<br />
作业完成的原则：署你名字的工作必须是你个人的贡献。在完成作业的过程中，允许讨论，前提是讨论的所有参与者均处于同等完成度。但关键想法的执行、以及作业文本的写作必须独立完成，并在作业中致谢（acknowledge）所有参与讨论的人。不允许其他任何形式的合作——尤其是与已经完成作业的同学“讨论”。<br />
<br />
本课程将对剽窃行为采取零容忍的态度。在完成作业过程中，对他人工作（出版物、互联网资料、其他人的作业等）直接的文本抄袭和对关键思想、关键元素的抄袭，按照 [http://www.acm.org/publications/policies/plagiarism_policy ACM Policy on Plagiarism]的解释，都将视为剽窃。剽窃者成绩将被取消。如果发现互相抄袭行为，<font color=red> 抄袭和被抄袭双方的成绩都将被取消</font>。因此请主动防止自己的作业被他人抄袭。<br />
<br />
学术诚信影响学生个人的品行，也关乎整个教育系统的正常运转。为了一点分数而做出学术不端的行为，不仅使自己沦为一个欺骗者，也使他人的诚实努力失去意义。让我们一起努力维护一个诚信的环境。<br />
<br />
= Assignments =<br />
Late policy: In general, we will accomodate late submission requests ONLY IF you made such requests ahead of time. <br />
<br />
<br />
= Lecture Notes =<br />
# [[高级算法 (Fall 2025)/Min Cut, Max Cut, and Spectral Cut|Min Cut, Max Cut, and Spectral Cut]] ([http://tcs.nju.edu.cn/slides/aa2026/Cut.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-cuts-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-cuts-zh.pdf AI生成讲义])<br />
#* [[高级算法 (Fall 2025)/Probability Basics|Probability basics]]<br />
# [[高级算法 (Fall 2026)/Fingerprinting| Fingerprinting]] ([http://tcs.nju.edu.cn/slides/aa2026/Fingerprinting.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-fingerprinting-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-fingerprinting-zh.pdf AI生成讲义])<br />
#* [[高级算法 (Fall 2026)/Finite Field Basics|Finite field basics]]<br />
# [[高级算法 (Fall 2026)/Hashing and Sketching|Hashing and Sketching]] ([http://tcs.nju.edu.cn/slides/aa2026/Hashing.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-hashing-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-hashing-zh.pdf AI生成讲义]) <br />
#* [[高级算法 (Fall 2026)/Limited independence|Limited independence]]<br />
#* [[高级算法 (Fall 2026)/Basic deviation inequalities|Basic deviation inequalities]]<br />
# [[高级算法 (Fall 2026)/Concentration of measure|Concentration of measure]] ([http://tcs.nju.edu.cn/slides/aa2026/Concentration.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-concentration-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-concentration-zh.pdf AI生成讲义]) <br />
#* [[高级算法 (Fall 2026)/Conditional expectations|Conditional expectations]]<br />
# [[高级算法 (Fall 2026)/Dimension Reduction|Dimension Reduction]] ([http://tcs.nju.edu.cn/slides/aa2026/NNS.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-JLT-LSH-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-JLT-LSH-zh.pdf AI生成讲义]) <br />
#* [https://www.cs.princeton.edu/~hy2/teaching/fall22-cos521/notes/JL.pdf Professor Huacheng Yu's note on Johnson-Lindenstrauss Theorem]<br />
#* [http://people.csail.mit.edu/gregory/annbook/introduction.pdf An introduction of LSH]<br />
<br />
= Related Online Courses=<br />
* [https://www.cs.cmu.edu/~15850/ Advanced Algorithms] by Anupam Gupta at CMU.<br />
* [http://people.csail.mit.edu/moitra/854.html Advanced Algorithms] by Ankur Moitra at MIT.<br />
* [http://courses.csail.mit.edu/6.854/current/ Advanced Algorithms] by David Karger and Aleksander Mądry at MIT.<br />
* [http://web.stanford.edu/class/cs168/index.html The Modern Algorithmic Toolbox] by Tim Roughgarden and Gregory Valiant at Stanford.<br />
* [https://www.cs.princeton.edu/courses/archive/fall18/cos521/ Advanced Algorithm Design] by Pravesh Kothari and Christopher Musco at Princeton.<br />
* [http://www.cs.cmu.edu/afs/cs.cmu.edu/academic/class/15859-f11/www/ Linear and Semidefinite Programming (Advanced Algorithms)] by Anupam Gupta and Ryan O'Donnell at CMU.<br />
* [https://www.cs.cmu.edu/~odonnell/papers/cs-theory-toolkit-lecture-notes.pdf CS Theory Toolkit] by Ryan O'Donnell at CMU.<br />
* [https://cs.uwaterloo.ca/~lapchi/cs860/index.html Eigenvalues and Polynomials] by Lap Chi Lau at University of Waterloo.<br />
* The [https://www.cs.cornell.edu/jeh/book.pdf "Foundations of Data Science" book] by Avrim Blum, John Hopcroft, and Ravindran Kannan.</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=%E9%AB%98%E7%BA%A7%E7%AE%97%E6%B3%95_(Fall_2026)/Dimension_Reduction&diff=13966高级算法 (Fall 2026)/Dimension Reduction2026-09-20T05:41:21Z<p>Etone: Created page with "= Metric Embedding= A '''metric space''' is a pair <math>(X,d)</math>, where <math>X</math> is a set and <math>d</math> is a '''metric''' (or '''distance''') on <math>X</math>, i.e., a function :<math>d:X^2\to\mathbb{R}_{\ge 0}</math> such that for any <math>x,y,z\in X</math>, the following axioms hold: # (identity of indiscernibles) <math>d(x,y)=0\Leftrightarrow x=y</math> # (symmetry) <math>d(x,y)=d(y,x)</math> # (triangle inequality) <math>d(x,z)\le d(x,y)+d(y,z)</ma..."</p>
<hr />
<div>= Metric Embedding=<br />
A '''metric space''' is a pair <math>(X,d)</math>, where <math>X</math> is a set and <math>d</math> is a '''metric''' (or '''distance''') on <math>X</math>, i.e., a function<br />
:<math>d:X^2\to\mathbb{R}_{\ge 0}</math> <br />
such that for any <math>x,y,z\in X</math>, the following axioms hold:<br />
# (identity of indiscernibles) <math>d(x,y)=0\Leftrightarrow x=y</math><br />
# (symmetry) <math>d(x,y)=d(y,x)</math><br />
# (triangle inequality) <math>d(x,z)\le d(x,y)+d(y,z)</math><br />
<br />
Let <math>(X,d_X)</math> and <math>(Y,d_Y)</math> be two metric spaces. A mapping<br />
:<math>\phi:X\to Y</math><br />
is called an '''embedding''' of metric space <math>X</math> into <math>Y</math>. The embedding is said to be with '''distortion''' <math>\alpha\ge1</math> if for any <math>x,y\in X</math> it holds that<br />
:<math>\frac{1}{\alpha}\cdot d(x,y)\le d(\phi(x),\phi(y))\le \alpha\cdot d(x,y)</math>.<br />
<br />
In Computer Science, a typical scenario for the metric embedding is as follows. We want to solve some difficult computation problem on a metric space <math>(X,d)</math>. Instead of solving this problem directly on the original metric space, we embed the metric into a new metric space <math>(Y,d_Y)</math> (with low distortion) where the computation problem is much easier to solve.<br />
<br />
One particular important case for the metric embedding is to embed a high-dimensional metric space to a new metric space whose dimension is much lower. This is called dimension reduction. This can be very helpful because various very common computation tasks can be very hard to solve on high-dimensional space due to the [https://en.wikipedia.org/wiki/Curse_of_dimensionality curse of dimensionality].<br />
<br />
= The Johnson-Lindenstrauss Theorem =<br />
The '''Johnson-Lindenstrauss Theorem''' or '''Johnson-Lindenstrauss Transformation''' (both shorten as '''JLT''') is a fundamental result for dimension reduction in Euclidian space.<br />
<br />
Recall that in Euclidian space <math>\mathbf{R}^d</math>, for any two points <math>x,y\in\mathbf{R}^d</math>, the Euclidian distance between them is given by <math>\|x-y\|=\sqrt{(x_1-y_1)^2+(x_2-y_2)^2+\cdots +(x_d-y_d)^2}</math>, where <math>\|\cdot\|=\|\cdot\|_2</math> denotes the Euclidian norm (a.k.a. the <math>\ell_2</math>-norm).<br />
<br />
The JLT says that in Euclidian space, it is always possible to embed a set of <math>n</math> points in ''arbitrary'' dimension to <math>O(\log n)</math> dimension with constant distortion. The theorem itself is stated formally as follows.<br />
<br />
{{Theorem<br />
|Johnson-Lindenstrauss Theorem, 1984|<br />
:For any <math>0<\epsilon<1/2</math> and any positive integer <math>n</math>, there is a positive integer <math>k=O(\epsilon^{-2}\log n)</math> such that the following holds:<br />
:For any set <math>S\subset\mathbf{R}^d</math> with <math>|S|=n</math>, where <math>d</math> is arbitrary, there is an embedding <math>\phi:\mathbf{R}^d\rightarrow\mathbf{R}^k</math> such that<br />
::<math>\forall x,y\in S,\quad (1-\epsilon)\|x-y\|^2\le\|\phi(x)-\phi(y)\|^2\le(1+\epsilon)\|x-y\|^2</math>.<br />
}}<br />
<br />
The Johnson-Lindenstrauss Theorem is usually stated for the <math>\ell_2^2</math>-norm <math>\|\cdot\|^2</math> instead of the Euclidian norm <math>\|\cdot\|</math> itself. Note that this does not change anything other than the constant faction in <math>k=O(\epsilon^{-2}\log n)</math> because <math>(1\pm\epsilon)^{\frac{1}{2}}=1\pm\Theta(\epsilon)</math>. The reason for stating the theorem in <math>\ell_2^2</math>-norm is because <math>\|\cdot\|^2</math> is a sum (rather than a square root) which is easier to analyze.<br />
<br />
In fact, the embedding <math>\phi:\mathbf{R}^d\rightarrow\mathbf{R}^k</math> can be as simple as a linear transformation <math>A\in\mathbf{R}^{k\times d}</math> so that <math>\phi(x)=Ax</math> for any <math>x\in\mathbf{R}^{d}</math>.<br />
Therefore, the above theorem can be stated more precisely as follows.<br />
<br />
{{Theorem<br />
|Johnson-Lindenstrauss Theorem (linear embedding)|<br />
:For any <math>0<\epsilon<1/2</math> and any positive integer <math>n</math>, there is a positive integer <math>k=O(\epsilon^{-2}\log n)</math> such that the following holds:<br />
:For any set <math>S\subset\mathbf{R}^d</math> with <math>|S|=n</math>, where <math>d</math> is arbitrary, there is a linear transformation <math>A\in\mathbf{R}^{k\times d}</math> such that<br />
::<math>\forall x,y\in S,\quad (1-\epsilon)\|x-y\|^2\le\|Ax-Ay\|^2\le(1+\epsilon)\|x-y\|^2</math>.<br />
}}<br />
<br />
The theorem is proved by the probabilistic method. Specifically, we construct a random matrix <math>A\in\mathbf{R}^{k\times d}</math> and show that with high probability (<math>1-O(1/n)</math>) it is a good embedding satisfying:<br />
:<math>\forall x,y\in S,\quad (1-\epsilon)\|x-y\|\le\|Ax-Ay\|\le(1+\epsilon)\|x-y\|</math>.<br />
Therefore, if such random matrix <math>A</math> is efficient to construct, it immediately gives us an efficient randomized algorithm for dimension reduction in the Euclidian space.<br />
<br />
There are several such constructions of the random matrix <math>A\in\mathbf{R}^{k\times d}</math>, including:<br />
* projection onto uniform random <math>k</math>-dimensional subspace of <math>\mathbf{R}^{d}</math>; (the original construction of Johnson and Lindenstrauss in 1984; a simplified analysis due to [https://cseweb.ucsd.edu/~dasgupta/papers/jl.pdf Dasgupta and Gupta in 1999])<br />
* random matrix with i.i.d. Gaussian entries; (due to [https://www.cs.princeton.edu/courses/archive/spr04/cos598B/bib/IndykM-curse.pdf Indyk and Motwani in 1998])<br />
* random matrix with i.i.d. -1/+1 entries; (due to [https://www.sciencedirect.com/science/article/pii/S0022000003000254?via%3Dihub#BIB8 Achlioptas in 2003])<br />
<br />
It was proved by [https://arxiv.org/abs/1609.02094 Kasper Green Larsen and Jelani Nelson in 2016] that the JLT is optimal: there is a set of <math>n</math> points from <math>\mathbf{R}^d</math> such that any embedding <math>\phi:\mathbf{R}^d\rightarrow\mathbf{R}^k</math> having the distortion asserted in the JLT must have dimension <math>k</math> of the target space satisfy <math>k=\Omega(\epsilon^{-2}\log n)</math>.<br />
<br />
==JLT via Gaussian matrix==<br />
Here we prove the Johnson-Lindenstrauss theorem with the second construction, by the random matrix <math>A\in\mathbf{R}^{k\times d}</math> with i.i.d. Gaussian entries. The construction of <math>A</math> is very simple:<br />
:{|border="2" width="100%" cellspacing="4" cellpadding="3" rules="all" style="margin:1em 1em 1em 0; border:solid 1px #AAAAAA; border-collapse:collapse;empty-cells:show;"<br />
|<br />
*Each entry of <math>A\in\mathbf{R}^{k\times d}</math> is drawn independently from the Gaussian distribution <math>\mathcal{N}(0,1/k)</math>.<br />
|}<br />
Recall that a [https://en.wikipedia.org/wiki/Normal_distribution Gaussian distribution] <math>\mathcal{N}(\mu,\sigma^2)</math> is specified by its mean <math>\mu</math> and standard deviation <math>\sigma</math> such that for a random variable <math>X</math> distributed as <math>\mathcal{N}(\mu,\sigma^2)</math>, we have<br />
:<math>\mathbf{E}[X]=\mu</math> and <math>\mathbf{Var}[X]=\sigma^2</math>,<br />
and the probability density function is given by <math>p(x)=\frac{1}{\sqrt{2\pi\sigma^2}}\mathrm{e}^{-\frac{(x-\mu)^2}{2\sigma^2}}</math>, therefore<br />
:<math>\Pr[X\le t]=\int_{-\infty}^t\frac{1}{\sqrt{2\pi\sigma^2}}\mathrm{e}^{-\frac{(x-\mu)^2}{2\sigma^2}}\,\mathrm{d}x</math>.<br />
<br />
Fix any two points <math>x,y</math> out of the <math>n</math> points in <math>S\subset \mathbf{R}^d</math>, if we can show that<br />
:<math>\Pr\left[(1-\epsilon)\|x-y\|^2\le\|Ax-Ay\|^2\le(1+\epsilon)\|x-y\|^2\right]\ge 1-\frac{1}{n^3}</math>,<br />
then by the union bound, the following event<br />
:<math>\forall x,y\in S,\quad (1-\epsilon)\|x-y\|^2\le\|Ax-Ay\|^2\le(1+\epsilon)\|x-y\|^2</math><br />
holds with probability <math>1-O(1/n)</math>. The Johnson-Lindenstrauss theorem follows.<br />
<br />
Furthermore, dividing both sides of the inequalities <math>(1-\epsilon)\|x-y\|^2\le\|Ax-Ay\|^2\le(1+\epsilon)\|x-y\|^2</math> by the factor <math>\|x-y\|^2</math> gives us<br />
:<math>(1-\epsilon)\le\frac{\|Ax-Ay\|^2}{\|x-y\|^2}=\left\|A\frac{(x-y)}{\|x-y\|^2}\right\|^2\le(1+\epsilon)</math>,<br />
where <math>\frac{(x-y)}{\|x-y\|^2}</math> is a unit vector.<br />
<br />
Therefore, the Johnson-Lindenstrauss theorem is proved once the following theorem on the unit vector is proved.<br />
{{Theorem<br />
|Theorem (JLT on unit vector)|<br />
:For any positive <math>\epsilon,\delta<1/2</math> there is a positive integer <math>k=O\left(\epsilon^{-2}\log \frac{1}{\delta}\right)</math> such that for the random matrix <math>A\in\mathbf{R}^{k\times d}</math> with each entry drawn independently from the Gaussian distribution <math>\mathcal{N}(0,1/k)</math>, for any unit vector <math>u\in\mathbf{R}^d</math> with <math>\|u\|=1</math>, <br />
::<math>\Pr\left[\left|\|Au\|^2-1\right|>\epsilon\right]<\delta</math>.<br />
}}<br />
<br />
For any <math>u\in\mathbf{R}^d</math>, we have <math>\|Au\|^2=\sum_{i=1}^k(Au)_i^2</math>, where the <math>(Au)_i</math>'s are independent of each other, because each <br />
:<math>(Au)_i=\langle A_{i\cdot},u\rangle=\sum_{j=1}^dA_{ij}u_j</math><br />
is determined by a distinct row vector <math>A_{i\cdot}</math> of the matrix <math>A</math> with independent entries. <br />
<br />
Moreover, since each entry <math>A_{ij}\sim\mathcal{N}(0,1/k)</math> independently and recall that for any two independent Gaussian random variables <math>X_1\sim \mathcal{N}(\mu_1,\sigma_1^2)</math> and <math>X_2\sim \mathcal{N}(\mu_2,\sigma_2^2)</math>, their weighted sum <math>aX_1+bX_2</math> follows the Gaussian distribution <math>\mathcal{N}(a\mu_1+b\mu_2,a^2\sigma_1^2+b^2\sigma_2^2)</math>, therefore <math>(Au)_i=\langle A_{i\cdot},u\rangle=\sum_{j=1}^dA_{ij}u_j</math> follows the Gaussian distribution<br />
:<math>(Au)_i\sim\mathcal{N}\left(0,\sum_{j=1}^d\frac{u_j^2}{k}\right)</math>,<br />
which is precisely <math>\mathcal{N}\left(0,\frac{1}{k}\right)</math> if <math>u\in\mathbf{R}^d</math> is a unit vector, i.e. <math>\|u\|^2=\sum_{j=1}^du_j^2=1</math>.<br />
<br />
In summary, the random variable <math>\|Au\|^2</math> that we are interested in, can be represented as a sum-of-square form<br />
:<math>\|Au\|^2=\sum_{i=1}^kY_i^2</math>,<br />
where each <math>Y_i=(Au)_i</math> independently follows the Gaussian distribution <math>\mathcal{N}\left(0,\frac{1}{k}\right)</math>.<br />
<br />
==Concentration of <math>\chi^2</math>-distribution ==<br />
By the above argument, the Johnson-Lindenstrauss theorem is proved by the following concentration inequality for the sum of the squares of <math>k</math> Gaussian distributions.<br />
{{Theorem<br />
|Chernoff bound for sum-of-squares of Gaussian distributions|<br />
:For any positive <math>\epsilon,\delta<1/2</math> there is a positive integer <math>k=O\left(\epsilon^{-2}\log \frac{1}{\delta}\right)</math> such that for i.i.d. Gaussian random variables <math>Y_1,Y_2,\ldots,Y_k\sim\mathcal{N}(0,1/k)</math>, <br />
::<math>\Pr\left[\left|\sum_{i=1}^kY_i^2-1\right|>\epsilon\right]<\delta</math>.<br />
}}<br />
<br />
First, we see that this is indeed a concentration inequality that bounds the deviation of a random variable from its mean.<br />
<br />
For each <math>Y_i\sim\mathcal{N}\left(0,\frac{1}{k}\right)</math>, we know that <math>\mathbf{E}[Y_i]=0</math> and <math>\mathbf{Var}[Y_i]=\mathbf{E}[Y_i^2]-\mathbf{E}[Y_i]^2=\frac{1}{k}</math>, thus<br />
:<math>\mathbf{E}[Y_i^2]=\mathbf{Var}[Y_i]+\mathbf{E}[Y_i]^2=\frac{1}{k}</math>.<br />
By linearity of expectation, it holds that<br />
:<math>\mathbf{E}\left[\sum_{i=1}^kY_i^2\right]=\sum_{i=1}^k\mathbf{E}\left[Y_i^2\right]=1</math>.<br />
<br />
We prove an equivalent concentration bound stated for the [https://en.wikipedia.org/wiki/Chi-squared_distribution '''<math>\chi^2</math>-distribution'''] (sum of the squares of <math>k</math> '''standard''' Gaussian distributions).<br />
<br />
{{Theorem<br />
|Chernoff bound for the <math>\chi^2</math>-distribution|<br />
:For i.i.d. standard Gaussian random variables <math>X_1,X_2,\ldots,X_k\sim\mathcal{N}(0,1)</math>, for <math>0<\epsilon<1</math>,<br />
:*<math>\Pr\left[\sum_{i=1}^kX_i^2>(1+\epsilon)k\right]<\mathrm{e}^{-\epsilon^2k/8}</math>,<br />
:*<math>\Pr\left[\sum_{i=1}^kX_i^2<(1-\epsilon)k\right]<\mathrm{e}^{-\epsilon^2k/8}</math>.<br />
}}<br />
<br />
Note that this indeed implies the above concentration bound by setting <math>X_i=\sqrt{k}\cdot Y_i</math> and <math>\delta\ge\mathrm{e}^{-\epsilon^2k/8}</math>.<br />
<br />
{{Proof|<br />
We first prove the upper tail. Let <math>\lambda>0</math> be a parameter to be determined.<br />
:<math><br />
\begin{align}<br />
\Pr\left[\sum_{i=1}^kX_i^2>(1+\epsilon)k\right]<br />
&=\Pr\left[\mathrm{e}^{\lambda \sum_{i=1}^kX_i^2}>\mathrm{e}^{(1+\epsilon)\lambda k}\right]\\<br />
&< \mathrm{e}^{-(1+\epsilon)\lambda k}\cdot \mathbf{E}\left[\mathrm{e}^{\lambda \sum_{i=1}^kX_i^2}\right] && \text{(Markov's inequality)}\\<br />
&= \mathrm{e}^{-(1+\epsilon)\lambda k}\cdot \prod_{i=1}^k \mathbf{E}\left[\mathrm{e}^{\lambda X_i^2}\right]. && \text{(independence)}<br />
\end{align}<br />
</math><br />
{{Theorem|Proposition|<br />
:For standard Gaussian <math>X\sim\mathcal{N}(0,1)</math>, <math>\mathbf{E}\left[\mathrm{e}^{\lambda X^2}\right]=\frac{1}{\sqrt{1-2\lambda}}</math>.<br />
}}<br />
{{Proof|<br />
:<math><br />
\begin{align}<br />
\mathbf{E}\left[\mathrm{e}^{\lambda X^2}\right]<br />
&=<br />
\frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty}\mathrm{e}^{\lambda x^2}\mathrm{e}^{-x^2/2}\,\mathrm{d}x\\<br />
&=<br />
\frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty}\mathrm{e}^{-(1-2\lambda)x^2/2}\,\mathrm{d}x\\<br />
&=<br />
\frac{1}{\sqrt{1-2\lambda}}\frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty}\mathrm{e}^{-y^2/2}\,\mathrm{d}y &&(\text{set }y=\sqrt{1-2\lambda}x)\\<br />
&=\frac{1}{\sqrt{1-2\lambda}}.<br />
\end{align}<br />
</math><br />
The last equation is because <math>\frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty}\mathrm{e}^{-y^2/2}\,\mathrm{d}y</math> is the total probability mass of a standard Gaussian distribution (which is obviously 1).<br />
}}<br />
Then continue the above calculation:<br />
:<math><br />
\begin{align}<br />
\Pr\left[\sum_{i=1}^kX_i^2>(1+\epsilon)k\right]<br />
&< <br />
\mathrm{e}^{-(1+\epsilon)\lambda k}\cdot \left(\frac{1}{\sqrt{1-2\lambda}}\right)^k\\<br />
&= <br />
\mathrm{e}^{-\epsilon\lambda k}\cdot \left(\frac{\mathrm{e}^{-\lambda}}{\sqrt{1-2\lambda}}\right)^k\\<br />
&\le <br />
\mathrm{e}^{-\epsilon\lambda k + 2\lambda^2 k} && (\text{for }\lambda<1/4)\\<br />
&=<br />
\mathrm{e}^{-\epsilon k/8}. && (\text{by setting the stationary point }\lambda=\epsilon/4)<br />
\end{align}<br />
</math><br />
This proved the upper tail. The lower tail can be symmetrically proved by optimizing over a parameter <math>\lambda<0</math>.<br />
}}<br />
<br />
As we argued above, this finishes the proof of the Johnson-Lindenstrauss Theorem.<br />
<br />
= Nearest Neighbor Search (NNS)=<br />
{{Theorem|Nearest Neighbor Search (NNS)|<br />
*'''Data''': a set <math>S</math> of <math>n</math> points <math>y_1,y_2,\ldots,y_n\in X</math> from a metric space <math>(X,\mathrm{dist})</math>; <br />
*'''Query''': a point <math>x\in X</math>;<br />
*'''Answer''': a nearest neighbor of the query point <math>x</math> among all data points, i.e. a data point <math>y_{i^*}\in S</math> such that <math>\mathrm{dist}(x,y_{i^*})=\min_{y_i\in S}\mathrm{dist}(x,y_i)</math>.<br />
}}<br />
<br />
<br />
{{Theorem|<math>c</math>-ANN (Approximate Nearest Neighbor)|<br />
:<math>c>1</math> is an approximate ratio.<br />
*'''Data''': a set <math>S</math> of <math>n</math> points <math>y_1,y_2,\ldots,y_n\in X</math> from a metric space <math>(X,\mathrm{dist})</math>; <br />
*'''Query''': a point <math>x\in X</math>;<br />
*'''Answer''': a <math>c</math>-approximate nearest neighbor of the query point <math>x</math> among all data points, i.e. a data point <math>y_{i^*}\in S</math> such that <math>\mathrm{dist}(x,y_{i^*})\le c\, \min_{y_i\in S}\mathrm{dist}(x,y_i)</math>.<br />
}}<br />
<br />
<br />
{{Theorem|<math>(c,r)</math>-ANN (Approximate ''Near'' Neighbor)|<br />
:<math>c>1</math> is an approximate ratio; <math>r>0</math> is a radius;<br />
*'''Data''': a set <math>S</math> of <math>n</math> points <math>y_1,y_2,\ldots,y_n\in X</math> from a metric space <math>(X,\mathrm{dist})</math>; <br />
*'''Query''': a point <math>x\in X</math>;<br />
*'''Answer''': <math><br />
\begin{cases}<br />
\text{a data point }y_{i^*}\in S\text{ such that }\mathrm{dist}(x,y_{i^*})\le c\cdot r & \text{if }\exists y_i\in S, \mathrm{dist}(x,y_{i})\le r,\\<br />
\bot & \text{if }\forall y_i\in S, \mathrm{dist}(x,y_{i})>c\cdot r,\\<br />
\text{arbitrary} & \text{otherwise}.<br />
\end{cases}<br />
</math><br />
}}<br />
<br />
{{Theorem|Theorem|<br />
:For any instance <math>S=\{y_1,y_2,\ldots,y_n\}</math> of the nearest neighbor problem, let <math>D_{\min}\triangleq\min_{y_i,y_j\in S}\mathrm{dist}(y_i,y_j)</math>, <math>D_{\max}\triangleq\max_{y_i,y_j\in S}\mathrm{dist}(y_i,y_j)</math> and <math>R\triangleq \frac{D_{\max}}{D_{\min}}</math>. <br />
:Suppose that for any <math>r>0</math> there always exists a data structure for the <math>(c,r)</math>-ANN problem, using space at most <math>s</math>, answering each query in time <math>t</math>, correctly with probability <math>1-\delta</math>. <br />
:Then there exists a data structure for the <math>c</math>-ANN problem, using space <math>O(s\log_c R)</math>, answering each query in time <math>O(t\log\log_c R)</math>, correctly with probability <math>1-O(\delta\log\log_c R)</math>.<br />
}}<br />
<br />
==Solving ANN by dimension reduction==<br />
<br />
= Locality-Sensitive Hashing (LSH)=</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=%E9%AB%98%E7%BA%A7%E7%AE%97%E6%B3%95_(Fall_2026)&diff=13965高级算法 (Fall 2026)2026-09-20T05:40:55Z<p>Etone: /* Lecture Notes */</p>
<hr />
<div>{{Infobox<br />
|name = Infobox<br />
|bodystyle = <br />
|title = <font size=3>高级算法 <br />
<br>Advanced Algorithms</font><br />
|titlestyle = <br />
<br />
|image = <br />
|imagestyle = <br />
|caption = <br />
|captionstyle = <br />
|headerstyle = background:#ccf;<br />
|labelstyle = background:#ddf;<br />
|datastyle = <br />
<br />
|header1 =Instructor<br />
|label1 = <br />
|data1 = <br />
|header2 = <br />
|label2 = <br />
|data2 = '''尹一通'''<br />
|header3 = <br />
|label3 = Email<br />
|data3 = yinyt@nju.edu.cn <br />
|header4 =<br />
|label4= office<br />
|data4= 计算机系 804<br />
|header5 = <br />
|label5 = <br />
|data5 = '''栗师'''<br />
|header6 = <br />
|label6 = Email<br />
|data6 = shili@nju.edu.cn <br />
|header7 =<br />
|label7= office<br />
|data7= 计算机系 605<br />
|header8 = <br />
|label8 = <br />
|data8 = '''刘景铖'''<br />
|header9 = <br />
|label9 = Email<br />
|data9 = liu@nju.edu.cn <br />
|header10 =<br />
|label10= office<br />
|data10= 计算机系 516<br />
|header11 = Class<br />
|label11 = <br />
|data11 = <br />
|header12 =<br />
|label12 = Class meetings<br />
|data12 = 周一 5-6节 (单) 仙Ⅰ-319<br />
周三 5-6节 仙Ⅰ-319<br />
|header13 =<br />
|label13 = Place<br />
|data13 = <br />
|header14 =<br />
|label14 = Office hours<br />
|data14 = 周一4-5pm（尹一通 804）<br/><br />
周四4-5pm（刘景铖 516）<br/><br />
By appointment<br />
|header15 = Textbooks<br />
|label15 = <br />
|data15 = <br />
|header16 =<br />
|label16 = <br />
|data16 = [[File:MR-randomized-algorithms.png|border|100px]]<br />
|header17 =<br />
|label17 = <br />
|data17 = Motwani and Raghavan. <br>''Randomized Algorithms''.<br> Cambridge Univ Press, 1995.<br />
|header18 =<br />
|label18 = <br />
|data18 = [[File:Approximation_Algorithms.jpg|border|100px]]<br />
|header19 =<br />
|label19 = <br />
|data19 = Vazirani. <br>''Approximation Algorithms''. <br> Springer-Verlag, 2001.<br />
|belowstyle = background:#ddf;<br />
|below = <br />
}}<br />
<br />
This is the webpage for the ''Advanced Algorithms'' class of fall 2026. Students who take this class should check this page periodically for content updates and new announcements. <br />
<br />
= Announcement =<br />
<br />
= Course info =<br />
* '''Instructor ''': <br />
:* [http://tcs.nju.edu.cn/yinyt/ 尹一通]：[mailto:yinyt@nju.edu.cn <yinyt@nju.edu.cn>]，计算机系 804 <br />
:*[https://tcs.nju.edu.cn/shili/ 栗师]：[mailto:shili@nju.edu.cn <shili@nju.edu.cn>]，计算机系 605<br />
:* [https://liuexp.github.io 刘景铖]：[mailto:liu@nju.edu.cn <liu@nju.edu.cn>]，计算机系 516 <br />
* '''Teaching Assistant''': <br />
** 于逸潇：[mailto:yixiaoyu@smail.nju.edu.cn <yixiaoyu@smail.nju.edu.cn>]<br />
** 张弈垚：[mailto:zhangyiyao@smail.nju.edu.cn <zhangyiyao@smail.nju.edu.cn>]<br />
* '''Class meeting''': <br />
** 周一 5-6节 1-17周(单) 仙Ⅰ-319<br />
** 周三 5-6节 1-18周 仙Ⅰ-319<br />
* '''Office hour''': <br />
** 周一4-5pm（尹一通 804）<br />
** 周四4-5pm（刘景铖 516）<br />
** By appointment<br />
* '''QQ群''': 1098567018<br />
<br />
= Syllabus =<br />
随着计算机算法理论的不断发展，现代计算机算法的设计与分析大量地使用非初等的数学工具以及非传统的算法思想。“高级算法”这门课程就是面向计算机算法的这一发展趋势而设立的。课程将针对传统算法课程未系统涉及、却在计算机科学各领域的科研和实践中扮演重要角色的高等算法设计思想和算法分析工具进行系统讲授。<br />
<br />
=== 先修课程 Prerequisites ===<br />
* 必须：离散数学，概率论，线性代数。<br />
* 推荐：算法设计与分析。<br />
<br />
=== Course materials ===<br />
* [[高级算法 (Fall 2026) / Course materials|<font size=3>教材和参考书</font>]]<br />
<br />
=== 成绩 Grades ===<br />
* 课程成绩：本课程将会有若干次作业和一次期末考试。最终成绩将由平时作业成绩和期末考试成绩综合得出。<br />
* 迟交：如果有特殊的理由，无法按时完成作业，请提前联系授课老师，给出正当理由。否则迟交的作业将不被接受。<br />
<br />
=== <font color=red> 学术诚信 Academic Integrity </font>===<br />
学术诚信是所有从事学术活动的学生和学者最基本的职业道德底线，本课程将不遗余力的维护学术诚信规范，违反这一底线的行为将不会被容忍。<br />
<br />
作业完成的原则：署你名字的工作必须是你个人的贡献。在完成作业的过程中，允许讨论，前提是讨论的所有参与者均处于同等完成度。但关键想法的执行、以及作业文本的写作必须独立完成，并在作业中致谢（acknowledge）所有参与讨论的人。不允许其他任何形式的合作——尤其是与已经完成作业的同学“讨论”。<br />
<br />
本课程将对剽窃行为采取零容忍的态度。在完成作业过程中，对他人工作（出版物、互联网资料、其他人的作业等）直接的文本抄袭和对关键思想、关键元素的抄袭，按照 [http://www.acm.org/publications/policies/plagiarism_policy ACM Policy on Plagiarism]的解释，都将视为剽窃。剽窃者成绩将被取消。如果发现互相抄袭行为，<font color=red> 抄袭和被抄袭双方的成绩都将被取消</font>。因此请主动防止自己的作业被他人抄袭。<br />
<br />
学术诚信影响学生个人的品行，也关乎整个教育系统的正常运转。为了一点分数而做出学术不端的行为，不仅使自己沦为一个欺骗者，也使他人的诚实努力失去意义。让我们一起努力维护一个诚信的环境。<br />
<br />
= Assignments =<br />
Late policy: In general, we will accomodate late submission requests ONLY IF you made such requests ahead of time. <br />
<br />
<br />
= Lecture Notes =<br />
# [[高级算法 (Fall 2025)/Min Cut, Max Cut, and Spectral Cut|Min Cut, Max Cut, and Spectral Cut]] ([http://tcs.nju.edu.cn/slides/aa2026/Cut.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-cuts-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-cuts-zh.pdf AI生成讲义])<br />
#* [[高级算法 (Fall 2025)/Probability Basics|Probability basics]]<br />
# [[高级算法 (Fall 2026)/Fingerprinting| Fingerprinting]] ([http://tcs.nju.edu.cn/slides/aa2026/Fingerprinting.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-fingerprinting-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-fingerprinting-zh.pdf AI生成讲义])<br />
#* [[高级算法 (Fall 2026)/Finite Field Basics|Finite field basics]]<br />
# [[高级算法 (Fall 2026)/Hashing and Sketching|Hashing and Sketching]] ([http://tcs.nju.edu.cn/slides/aa2026/Hashing.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-hashing-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-hashing-zh.pdf AI生成讲义]) <br />
#* [[高级算法 (Fall 2026)/Limited independence|Limited independence]]<br />
#* [[高级算法 (Fall 2026)/Basic deviation inequalities|Basic deviation inequalities]]<br />
# [[高级算法 (Fall 2026)/Concentration of measure|Concentration of measure]] ([http://tcs.nju.edu.cn/slides/aa2026/Concentration.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-concentration-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-concentration-zh.pdf AI生成讲义]) <br />
#* [[高级算法 (Fall 2026)/Conditional expectations|Conditional expectations]]<br />
# [[高级算法 (Fall 2026)/Dimension Reduction|Dimension Reduction]] <br />
#* [https://www.cs.princeton.edu/~hy2/teaching/fall22-cos521/notes/JL.pdf Professor Huacheng Yu's note on Johnson-Lindenstrauss Theorem]<br />
#* [http://people.csail.mit.edu/gregory/annbook/introduction.pdf An introduction of LSH]<br />
<br />
= Related Online Courses=<br />
* [https://www.cs.cmu.edu/~15850/ Advanced Algorithms] by Anupam Gupta at CMU.<br />
* [http://people.csail.mit.edu/moitra/854.html Advanced Algorithms] by Ankur Moitra at MIT.<br />
* [http://courses.csail.mit.edu/6.854/current/ Advanced Algorithms] by David Karger and Aleksander Mądry at MIT.<br />
* [http://web.stanford.edu/class/cs168/index.html The Modern Algorithmic Toolbox] by Tim Roughgarden and Gregory Valiant at Stanford.<br />
* [https://www.cs.princeton.edu/courses/archive/fall18/cos521/ Advanced Algorithm Design] by Pravesh Kothari and Christopher Musco at Princeton.<br />
* [http://www.cs.cmu.edu/afs/cs.cmu.edu/academic/class/15859-f11/www/ Linear and Semidefinite Programming (Advanced Algorithms)] by Anupam Gupta and Ryan O'Donnell at CMU.<br />
* [https://www.cs.cmu.edu/~odonnell/papers/cs-theory-toolkit-lecture-notes.pdf CS Theory Toolkit] by Ryan O'Donnell at CMU.<br />
* [https://cs.uwaterloo.ca/~lapchi/cs860/index.html Eigenvalues and Polynomials] by Lap Chi Lau at University of Waterloo.<br />
* The [https://www.cs.cornell.edu/jeh/book.pdf "Foundations of Data Science" book] by Avrim Blum, John Hopcroft, and Ravindran Kannan.</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=%E9%AB%98%E7%BA%A7%E7%AE%97%E6%B3%95_(Fall_2026)&diff=13959高级算法 (Fall 2026)2026-09-16T15:33:28Z<p>Etone: /* Lecture Notes */</p>
<hr />
<div>{{Infobox<br />
|name = Infobox<br />
|bodystyle = <br />
|title = <font size=3>高级算法 <br />
<br>Advanced Algorithms</font><br />
|titlestyle = <br />
<br />
|image = <br />
|imagestyle = <br />
|caption = <br />
|captionstyle = <br />
|headerstyle = background:#ccf;<br />
|labelstyle = background:#ddf;<br />
|datastyle = <br />
<br />
|header1 =Instructor<br />
|label1 = <br />
|data1 = <br />
|header2 = <br />
|label2 = <br />
|data2 = '''尹一通'''<br />
|header3 = <br />
|label3 = Email<br />
|data3 = yinyt@nju.edu.cn <br />
|header4 =<br />
|label4= office<br />
|data4= 计算机系 804<br />
|header5 = <br />
|label5 = <br />
|data5 = '''栗师'''<br />
|header6 = <br />
|label6 = Email<br />
|data6 = shili@nju.edu.cn <br />
|header7 =<br />
|label7= office<br />
|data7= 计算机系 605<br />
|header8 = <br />
|label8 = <br />
|data8 = '''刘景铖'''<br />
|header9 = <br />
|label9 = Email<br />
|data9 = liu@nju.edu.cn <br />
|header10 =<br />
|label10= office<br />
|data10= 计算机系 516<br />
|header11 = Class<br />
|label11 = <br />
|data11 = <br />
|header12 =<br />
|label12 = Class meetings<br />
|data12 = 周一 5-6节 (单) 仙Ⅰ-319<br />
周三 5-6节 仙Ⅰ-319<br />
|header13 =<br />
|label13 = Place<br />
|data13 = <br />
|header14 =<br />
|label14 = Office hours<br />
|data14 = 周一4-5pm（尹一通 804）<br/><br />
周四4-5pm（刘景铖 516）<br/><br />
By appointment<br />
|header15 = Textbooks<br />
|label15 = <br />
|data15 = <br />
|header16 =<br />
|label16 = <br />
|data16 = [[File:MR-randomized-algorithms.png|border|100px]]<br />
|header17 =<br />
|label17 = <br />
|data17 = Motwani and Raghavan. <br>''Randomized Algorithms''.<br> Cambridge Univ Press, 1995.<br />
|header18 =<br />
|label18 = <br />
|data18 = [[File:Approximation_Algorithms.jpg|border|100px]]<br />
|header19 =<br />
|label19 = <br />
|data19 = Vazirani. <br>''Approximation Algorithms''. <br> Springer-Verlag, 2001.<br />
|belowstyle = background:#ddf;<br />
|below = <br />
}}<br />
<br />
This is the webpage for the ''Advanced Algorithms'' class of fall 2026. Students who take this class should check this page periodically for content updates and new announcements. <br />
<br />
= Announcement =<br />
<br />
= Course info =<br />
* '''Instructor ''': <br />
:* [http://tcs.nju.edu.cn/yinyt/ 尹一通]：[mailto:yinyt@nju.edu.cn <yinyt@nju.edu.cn>]，计算机系 804 <br />
:*[https://tcs.nju.edu.cn/shili/ 栗师]：[mailto:shili@nju.edu.cn <shili@nju.edu.cn>]，计算机系 605<br />
:* [https://liuexp.github.io 刘景铖]：[mailto:liu@nju.edu.cn <liu@nju.edu.cn>]，计算机系 516 <br />
* '''Teaching Assistant''': <br />
** 于逸潇：[mailto:yixiaoyu@smail.nju.edu.cn <yixiaoyu@smail.nju.edu.cn>]<br />
** 张弈垚：[mailto:zhangyiyao@smail.nju.edu.cn <zhangyiyao@smail.nju.edu.cn>]<br />
* '''Class meeting''': <br />
** 周一 5-6节 1-17周(单) 仙Ⅰ-319<br />
** 周三 5-6节 1-18周 仙Ⅰ-319<br />
* '''Office hour''': <br />
** 周一4-5pm（尹一通 804）<br />
** 周四4-5pm（刘景铖 516）<br />
** By appointment<br />
* '''QQ群''': 1098567018<br />
<br />
= Syllabus =<br />
随着计算机算法理论的不断发展，现代计算机算法的设计与分析大量地使用非初等的数学工具以及非传统的算法思想。“高级算法”这门课程就是面向计算机算法的这一发展趋势而设立的。课程将针对传统算法课程未系统涉及、却在计算机科学各领域的科研和实践中扮演重要角色的高等算法设计思想和算法分析工具进行系统讲授。<br />
<br />
=== 先修课程 Prerequisites ===<br />
* 必须：离散数学，概率论，线性代数。<br />
* 推荐：算法设计与分析。<br />
<br />
=== Course materials ===<br />
* [[高级算法 (Fall 2026) / Course materials|<font size=3>教材和参考书</font>]]<br />
<br />
=== 成绩 Grades ===<br />
* 课程成绩：本课程将会有若干次作业和一次期末考试。最终成绩将由平时作业成绩和期末考试成绩综合得出。<br />
* 迟交：如果有特殊的理由，无法按时完成作业，请提前联系授课老师，给出正当理由。否则迟交的作业将不被接受。<br />
<br />
=== <font color=red> 学术诚信 Academic Integrity </font>===<br />
学术诚信是所有从事学术活动的学生和学者最基本的职业道德底线，本课程将不遗余力的维护学术诚信规范，违反这一底线的行为将不会被容忍。<br />
<br />
作业完成的原则：署你名字的工作必须是你个人的贡献。在完成作业的过程中，允许讨论，前提是讨论的所有参与者均处于同等完成度。但关键想法的执行、以及作业文本的写作必须独立完成，并在作业中致谢（acknowledge）所有参与讨论的人。不允许其他任何形式的合作——尤其是与已经完成作业的同学“讨论”。<br />
<br />
本课程将对剽窃行为采取零容忍的态度。在完成作业过程中，对他人工作（出版物、互联网资料、其他人的作业等）直接的文本抄袭和对关键思想、关键元素的抄袭，按照 [http://www.acm.org/publications/policies/plagiarism_policy ACM Policy on Plagiarism]的解释，都将视为剽窃。剽窃者成绩将被取消。如果发现互相抄袭行为，<font color=red> 抄袭和被抄袭双方的成绩都将被取消</font>。因此请主动防止自己的作业被他人抄袭。<br />
<br />
学术诚信影响学生个人的品行，也关乎整个教育系统的正常运转。为了一点分数而做出学术不端的行为，不仅使自己沦为一个欺骗者，也使他人的诚实努力失去意义。让我们一起努力维护一个诚信的环境。<br />
<br />
= Assignments =<br />
Late policy: In general, we will accomodate late submission requests ONLY IF you made such requests ahead of time. <br />
<br />
<br />
= Lecture Notes =<br />
# [[高级算法 (Fall 2025)/Min Cut, Max Cut, and Spectral Cut|Min Cut, Max Cut, and Spectral Cut]] ([http://tcs.nju.edu.cn/slides/aa2026/Cut.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-cuts-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-cuts-zh.pdf AI生成讲义])<br />
#* [[高级算法 (Fall 2025)/Probability Basics|Probability basics]]<br />
# [[高级算法 (Fall 2026)/Fingerprinting| Fingerprinting]] ([http://tcs.nju.edu.cn/slides/aa2026/Fingerprinting.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-fingerprinting-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-fingerprinting-zh.pdf AI生成讲义])<br />
#* [[高级算法 (Fall 2026)/Finite Field Basics|Finite field basics]]<br />
# [[高级算法 (Fall 2026)/Hashing and Sketching|Hashing and Sketching]] ([http://tcs.nju.edu.cn/slides/aa2026/Hashing.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-hashing-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-hashing-zh.pdf AI生成讲义]) <br />
#* [[高级算法 (Fall 2026)/Limited independence|Limited independence]]<br />
#* [[高级算法 (Fall 2026)/Basic deviation inequalities|Basic deviation inequalities]]<br />
# [[高级算法 (Fall 2026)/Concentration of measure|Concentration of measure]] ([http://tcs.nju.edu.cn/slides/aa2026/Concentration.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-concentration-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-concentration-zh.pdf AI生成讲义]) <br />
#* [[高级算法 (Fall 2026)/Conditional expectations|Conditional expectations]]<br />
<br />
= Related Online Courses=<br />
* [https://www.cs.cmu.edu/~15850/ Advanced Algorithms] by Anupam Gupta at CMU.<br />
* [http://people.csail.mit.edu/moitra/854.html Advanced Algorithms] by Ankur Moitra at MIT.<br />
* [http://courses.csail.mit.edu/6.854/current/ Advanced Algorithms] by David Karger and Aleksander Mądry at MIT.<br />
* [http://web.stanford.edu/class/cs168/index.html The Modern Algorithmic Toolbox] by Tim Roughgarden and Gregory Valiant at Stanford.<br />
* [https://www.cs.princeton.edu/courses/archive/fall18/cos521/ Advanced Algorithm Design] by Pravesh Kothari and Christopher Musco at Princeton.<br />
* [http://www.cs.cmu.edu/afs/cs.cmu.edu/academic/class/15859-f11/www/ Linear and Semidefinite Programming (Advanced Algorithms)] by Anupam Gupta and Ryan O'Donnell at CMU.<br />
* [https://www.cs.cmu.edu/~odonnell/papers/cs-theory-toolkit-lecture-notes.pdf CS Theory Toolkit] by Ryan O'Donnell at CMU.<br />
* [https://cs.uwaterloo.ca/~lapchi/cs860/index.html Eigenvalues and Polynomials] by Lap Chi Lau at University of Waterloo.<br />
* The [https://www.cs.cornell.edu/jeh/book.pdf "Foundations of Data Science" book] by Avrim Blum, John Hopcroft, and Ravindran Kannan.</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=%E9%AB%98%E7%BA%A7%E7%AE%97%E6%B3%95_(Fall_2026)&diff=13958高级算法 (Fall 2026)2026-09-16T15:32:30Z<p>Etone: /* Lecture Notes */</p>
<hr />
<div>{{Infobox<br />
|name = Infobox<br />
|bodystyle = <br />
|title = <font size=3>高级算法 <br />
<br>Advanced Algorithms</font><br />
|titlestyle = <br />
<br />
|image = <br />
|imagestyle = <br />
|caption = <br />
|captionstyle = <br />
|headerstyle = background:#ccf;<br />
|labelstyle = background:#ddf;<br />
|datastyle = <br />
<br />
|header1 =Instructor<br />
|label1 = <br />
|data1 = <br />
|header2 = <br />
|label2 = <br />
|data2 = '''尹一通'''<br />
|header3 = <br />
|label3 = Email<br />
|data3 = yinyt@nju.edu.cn <br />
|header4 =<br />
|label4= office<br />
|data4= 计算机系 804<br />
|header5 = <br />
|label5 = <br />
|data5 = '''栗师'''<br />
|header6 = <br />
|label6 = Email<br />
|data6 = shili@nju.edu.cn <br />
|header7 =<br />
|label7= office<br />
|data7= 计算机系 605<br />
|header8 = <br />
|label8 = <br />
|data8 = '''刘景铖'''<br />
|header9 = <br />
|label9 = Email<br />
|data9 = liu@nju.edu.cn <br />
|header10 =<br />
|label10= office<br />
|data10= 计算机系 516<br />
|header11 = Class<br />
|label11 = <br />
|data11 = <br />
|header12 =<br />
|label12 = Class meetings<br />
|data12 = 周一 5-6节 (单) 仙Ⅰ-319<br />
周三 5-6节 仙Ⅰ-319<br />
|header13 =<br />
|label13 = Place<br />
|data13 = <br />
|header14 =<br />
|label14 = Office hours<br />
|data14 = 周一4-5pm（尹一通 804）<br/><br />
周四4-5pm（刘景铖 516）<br/><br />
By appointment<br />
|header15 = Textbooks<br />
|label15 = <br />
|data15 = <br />
|header16 =<br />
|label16 = <br />
|data16 = [[File:MR-randomized-algorithms.png|border|100px]]<br />
|header17 =<br />
|label17 = <br />
|data17 = Motwani and Raghavan. <br>''Randomized Algorithms''.<br> Cambridge Univ Press, 1995.<br />
|header18 =<br />
|label18 = <br />
|data18 = [[File:Approximation_Algorithms.jpg|border|100px]]<br />
|header19 =<br />
|label19 = <br />
|data19 = Vazirani. <br>''Approximation Algorithms''. <br> Springer-Verlag, 2001.<br />
|belowstyle = background:#ddf;<br />
|below = <br />
}}<br />
<br />
This is the webpage for the ''Advanced Algorithms'' class of fall 2026. Students who take this class should check this page periodically for content updates and new announcements. <br />
<br />
= Announcement =<br />
<br />
= Course info =<br />
* '''Instructor ''': <br />
:* [http://tcs.nju.edu.cn/yinyt/ 尹一通]：[mailto:yinyt@nju.edu.cn <yinyt@nju.edu.cn>]，计算机系 804 <br />
:*[https://tcs.nju.edu.cn/shili/ 栗师]：[mailto:shili@nju.edu.cn <shili@nju.edu.cn>]，计算机系 605<br />
:* [https://liuexp.github.io 刘景铖]：[mailto:liu@nju.edu.cn <liu@nju.edu.cn>]，计算机系 516 <br />
* '''Teaching Assistant''': <br />
** 于逸潇：[mailto:yixiaoyu@smail.nju.edu.cn <yixiaoyu@smail.nju.edu.cn>]<br />
** 张弈垚：[mailto:zhangyiyao@smail.nju.edu.cn <zhangyiyao@smail.nju.edu.cn>]<br />
* '''Class meeting''': <br />
** 周一 5-6节 1-17周(单) 仙Ⅰ-319<br />
** 周三 5-6节 1-18周 仙Ⅰ-319<br />
* '''Office hour''': <br />
** 周一4-5pm（尹一通 804）<br />
** 周四4-5pm（刘景铖 516）<br />
** By appointment<br />
* '''QQ群''': 1098567018<br />
<br />
= Syllabus =<br />
随着计算机算法理论的不断发展，现代计算机算法的设计与分析大量地使用非初等的数学工具以及非传统的算法思想。“高级算法”这门课程就是面向计算机算法的这一发展趋势而设立的。课程将针对传统算法课程未系统涉及、却在计算机科学各领域的科研和实践中扮演重要角色的高等算法设计思想和算法分析工具进行系统讲授。<br />
<br />
=== 先修课程 Prerequisites ===<br />
* 必须：离散数学，概率论，线性代数。<br />
* 推荐：算法设计与分析。<br />
<br />
=== Course materials ===<br />
* [[高级算法 (Fall 2026) / Course materials|<font size=3>教材和参考书</font>]]<br />
<br />
=== 成绩 Grades ===<br />
* 课程成绩：本课程将会有若干次作业和一次期末考试。最终成绩将由平时作业成绩和期末考试成绩综合得出。<br />
* 迟交：如果有特殊的理由，无法按时完成作业，请提前联系授课老师，给出正当理由。否则迟交的作业将不被接受。<br />
<br />
=== <font color=red> 学术诚信 Academic Integrity </font>===<br />
学术诚信是所有从事学术活动的学生和学者最基本的职业道德底线，本课程将不遗余力的维护学术诚信规范，违反这一底线的行为将不会被容忍。<br />
<br />
作业完成的原则：署你名字的工作必须是你个人的贡献。在完成作业的过程中，允许讨论，前提是讨论的所有参与者均处于同等完成度。但关键想法的执行、以及作业文本的写作必须独立完成，并在作业中致谢（acknowledge）所有参与讨论的人。不允许其他任何形式的合作——尤其是与已经完成作业的同学“讨论”。<br />
<br />
本课程将对剽窃行为采取零容忍的态度。在完成作业过程中，对他人工作（出版物、互联网资料、其他人的作业等）直接的文本抄袭和对关键思想、关键元素的抄袭，按照 [http://www.acm.org/publications/policies/plagiarism_policy ACM Policy on Plagiarism]的解释，都将视为剽窃。剽窃者成绩将被取消。如果发现互相抄袭行为，<font color=red> 抄袭和被抄袭双方的成绩都将被取消</font>。因此请主动防止自己的作业被他人抄袭。<br />
<br />
学术诚信影响学生个人的品行，也关乎整个教育系统的正常运转。为了一点分数而做出学术不端的行为，不仅使自己沦为一个欺骗者，也使他人的诚实努力失去意义。让我们一起努力维护一个诚信的环境。<br />
<br />
= Assignments =<br />
Late policy: In general, we will accomodate late submission requests ONLY IF you made such requests ahead of time. <br />
<br />
<br />
= Lecture Notes =<br />
# [[高级算法 (Fall 2025)/Min Cut, Max Cut, and Spectral Cut|Min Cut, Max Cut, and Spectral Cut]] ([http://tcs.nju.edu.cn/slides/aa2026/Cut.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-cuts-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-cuts-zh.pdf AI生成讲义])<br />
#* [[高级算法 (Fall 2025)/Probability Basics|Probability basics]]<br />
# [[高级算法 (Fall 2026)/Fingerprinting| Fingerprinting]] ([http://tcs.nju.edu.cn/slides/aa2026/Fingerprinting.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-fingerprinting-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-fingerprinting-zh.pdf AI生成讲义])<br />
#* [[高级算法 (Fall 2026)/Finite Field Basics|Finite field basics]]<br />
# [[高级算法 (Fall 2026)/Hashing and Sketching|Hashing and Sketching]] ([http://tcs.nju.edu.cn/slides/aa2026/Hashing.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-hashing-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-hashing-zh.pdf AI生成讲义]) <br />
#* [[高级算法 (Fall 2026)/Limited independence|Limited independence]]<br />
#* [[高级算法 (Fall 2026)/Basic deviation inequalities|Basic deviation inequalities]]<br />
# [[高级算法 (Fall 2026)/Concentration of measure|Concentration of measure]] ([http://tcs.nju.edu.cn/slides/aa2026/Concentration.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-concentration-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-concentration.pdf-zh.pdf AI生成讲义]) <br />
#* [[高级算法 (Fall 2026)/Conditional expectations|Conditional expectations]]<br />
<br />
= Related Online Courses=<br />
* [https://www.cs.cmu.edu/~15850/ Advanced Algorithms] by Anupam Gupta at CMU.<br />
* [http://people.csail.mit.edu/moitra/854.html Advanced Algorithms] by Ankur Moitra at MIT.<br />
* [http://courses.csail.mit.edu/6.854/current/ Advanced Algorithms] by David Karger and Aleksander Mądry at MIT.<br />
* [http://web.stanford.edu/class/cs168/index.html The Modern Algorithmic Toolbox] by Tim Roughgarden and Gregory Valiant at Stanford.<br />
* [https://www.cs.princeton.edu/courses/archive/fall18/cos521/ Advanced Algorithm Design] by Pravesh Kothari and Christopher Musco at Princeton.<br />
* [http://www.cs.cmu.edu/afs/cs.cmu.edu/academic/class/15859-f11/www/ Linear and Semidefinite Programming (Advanced Algorithms)] by Anupam Gupta and Ryan O'Donnell at CMU.<br />
* [https://www.cs.cmu.edu/~odonnell/papers/cs-theory-toolkit-lecture-notes.pdf CS Theory Toolkit] by Ryan O'Donnell at CMU.<br />
* [https://cs.uwaterloo.ca/~lapchi/cs860/index.html Eigenvalues and Polynomials] by Lap Chi Lau at University of Waterloo.<br />
* The [https://www.cs.cornell.edu/jeh/book.pdf "Foundations of Data Science" book] by Avrim Blum, John Hopcroft, and Ravindran Kannan.</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=%E9%AB%98%E7%BA%A7%E7%AE%97%E6%B3%95_(Fall_2026)/Conditional_expectations&diff=13946高级算法 (Fall 2026)/Conditional expectations2026-09-15T12:00:16Z<p>Etone: Created page with "= Conditional Expectations = The '''conditional expectation''' of a random variable <math>Y</math> with respect to an event <math>\mathcal{E}</math> is defined by :<math> \mathbf{E}[Y\mid \mathcal{E}]=\sum_{y}y\Pr[Y=y\mid\mathcal{E}]. </math> In particular, if the event <math>\mathcal{E}</math> is <math>X=a</math>, the conditional expectation :<math> \mathbf{E}[Y\mid X=a] </math> defines a function :<math> f(a)=\mathbf{E}[Y\mid X=a]. </math> Thus, <math>\mathbf{E}[Y\mid..."</p>
<hr />
<div>= Conditional Expectations =<br />
The '''conditional expectation''' of a random variable <math>Y</math> with respect to an event <math>\mathcal{E}</math> is defined by<br />
:<math><br />
\mathbf{E}[Y\mid \mathcal{E}]=\sum_{y}y\Pr[Y=y\mid\mathcal{E}].<br />
</math><br />
In particular, if the event <math>\mathcal{E}</math> is <math>X=a</math>, the conditional expectation<br />
:<math><br />
\mathbf{E}[Y\mid X=a]<br />
</math><br />
defines a function<br />
:<math><br />
f(a)=\mathbf{E}[Y\mid X=a].<br />
</math><br />
Thus, <math>\mathbf{E}[Y\mid X]</math> can be regarded as a random variable <math>f(X)</math>.<br />
<br />
;Example<br />
:Suppose that we uniformly sample a human from all human beings. Let <math>Y</math> be his/her height, and let <math>X</math> be the country where he/she is from. For any country <math>a</math>, <math>\mathbf{E}[Y\mid X=a]</math> gives the average height of that country. And <math>\mathbf{E}[Y\mid X]</math> is the random variable which can be defined in either ways:<br />
:* We choose a human uniformly at random from all human beings, and <math>\mathbf{E}[Y\mid X]</math> is the average height of the country where he/she comes from.<br />
:* We choose a country at random with a probability ''proportional to its population'', and <math>\mathbf{E}[Y\mid X]</math> is the average height of the chosen country.<br />
<br />
The following proposition states some fundamental facts about conditional expectation.<br />
<br />
{{Theorem<br />
|Proposition (fundamental facts about conditional expectation)|<br />
:Let <math>X,Y</math> and <math>Z</math> be arbitrary random variables. Let <math>f</math> and <math>g</math> be arbitrary functions. Then<br />
:# <math>\mathbf{E}[X]=\mathbf{E}[\mathbf{E}[X\mid Y]]</math>.<br />
:# <math>\mathbf{E}[X\mid Z]=\mathbf{E}[\mathbf{E}[X\mid Y,Z]\mid Z]</math>.<br />
:# <math>\mathbf{E}[\mathbf{E}[f(X)g(X,Y)\mid X]]=\mathbf{E}[f(X)\cdot \mathbf{E}[g(X,Y)\mid X]]</math>.<br />
}}<br />
The proposition can be formally verified by computing these expectations. Although these equations look formal, the intuitive interpretations to them are very clear.<br />
<br />
The first equation:<br />
:<math>\mathbf{E}[X]=\mathbf{E}[\mathbf{E}[X\mid Y]]</math><br />
says that there are two ways to compute an average. Suppose again that <math>X</math> is the height of a uniform random human and <math>Y</math> is the country where he/she is from. There are two ways to compute the average human height: one is to directly average over the heights of all humans; the other is that first compute the average height for each country, and then average over these heights weighted by the populations of the countries.<br />
<br />
The second equation:<br />
:<math>\mathbf{E}[X\mid Z]=\mathbf{E}[\mathbf{E}[X\mid Y,Z]\mid Z]</math><br />
is the same as the first one, restricted to a particular subspace. As the previous example, inaddition to the height <math>X</math> and the country <math>Y</math>, let <math>Z</math> be the gender of the individual. Thus, <math>\mathbf{E}[X\mid Z]</math> is the average height of a human being of a given sex. Again, this can be computed either directly or on a country-by-country basis.<br />
<br />
The third equation:<br />
:<math>\mathbf{E}[\mathbf{E}[f(X)g(X,Y)\mid X]]=\mathbf{E}[f(X)\cdot \mathbf{E}[g(X,Y)\mid X]]</math>.<br />
looks obscure at the first glance, especially when considering that <math>X</math> and <math>Y</math> are not necessarily independent. Nevertheless, the equation follows the simple fact that conditioning on any <math>X=a</math>, the function value <math>f(X)=f(a)</math> becomes a constant, thus can be safely taken outside the expectation due to the linearity of expectation. For any value <math>X=a</math>,<br />
:<math><br />
\mathbf{E}[f(X)g(X,Y)\mid X=a]=\mathbf{E}[f(a)g(X,Y)\mid X=a]=f(a)\cdot \mathbf{E}[g(X,Y)\mid X=a].<br />
</math><br />
<br />
The proposition holds in more general cases when <math>X, Y</math> and <math>Z</math> are a sequence of random variables.<br />
<br />
= Martingales =<br />
A '''martingale''' is a random sequence <math>X_0,X_1,\ldots</math> satisfying the following so-called ''martingale property''.<br />
<br />
{{Theorem<br />
|Definition (martingale)|<br />
:A sequence of random variables <math>X_0,X_1,\ldots</math> is a '''martingale''' if for all <math>i> 0</math>,<br />
:: <math>\begin{align}<br />
\mathbf{E}[X_{i}\mid X_0,\ldots,X_{i-1}]=X_{i-1}.<br />
\end{align}</math><br />
}}<br />
<br />
==Examples ==<br />
;coin flips<br />
:A fair coin is flipped for a number of times. Let <math>Z_j\in\{-1,1\}</math> denote the outcome of the <math>j</math>th flip. Let <br />
::<math>X_0=0\quad \mbox{ and } \quad X_i=\sum_{j\le i}Z_j</math>. <br />
:The random variables <math>X_0,X_1,\ldots</math> defines a martingale.<br />
{{Proof| We first observe that <math>\mathbf{E}[X_i\mid X_0,\ldots,X_{i-1}] = \mathbf{E}[X_i\mid X_{i-1}]</math>, which intuitively says that the next number of HEADs depends only on the current number of HEADs. This property is also called the '''Markov property''' in statistic processes.<br />
::<math><br />
\begin{align}<br />
\mathbf{E}[X_i\mid X_0,\ldots,X_{i-1}] <br />
&= \mathbf{E}[X_i\mid X_{i-1}]\\<br />
&= \mathbf{E}[X_{i-1}+Z_{i}\mid X_{i-1}]\\<br />
&= \mathbf{E}[X_{i-1}\mid X_{i-1}]+\mathbf{E}[Z_{i}\mid X_{i-1}]\\<br />
&= X_{i-1}+\mathbf{E}[Z_{i}\mid X_{i-1}]\\<br />
&= X_{i-1}+\mathbf{E}[Z_{i}] &\quad (\mbox{independence of coin flips})\\<br />
&= X_{i-1}<br />
\end{align}<br />
</math><br />
}}<br />
<br />
;edge exposure in a random graph<br />
:Consider a '''random graph''' <math>G</math> generated as follows. Let <math>[n]</math> be the set of vertices, and let <math>[m]={[n]\choose 2}</math> be the set of all possible edges. For convenience, we enumerate these potential edges by <math>e_1,\ldots, e_m</math>. For each potential edge <math>e_j</math>, we independently flip a fair coin to decide whether the edge <math>e_j</math> appears in <math>G</math>. Let <math>I_j</math> be the random variable that indicates whether <math>e_j\in G</math>. We are interested in some graph-theoretical parameter, say [http://mathworld.wolfram.com/ChromaticNumber.html chromatic number], of the random graph <math>G</math>. Let <math>\chi(G)</math> be the chromatic number of <math>G</math>. Let <math>X_0=\mathbf{E}[\chi(G)]</math>, and for each <math>i\ge 1</math>, let <math>X_i=\mathbf{E}[\chi(G)\mid I_1,\ldots,I_{i}]</math>, namely, the expected chromatic number of the random graph after fixing the first <math>i</math> edges. This process is called edges exposure of a random graph, as we "exposing" the edges one by one in a random graph.<br />
<br />
It is nontrivial to formally verify that the edge exposure sequence for a random graph is a martingale. However, we will later see that this construction can be put into a more general context.<br />
<br />
==Generalization ==<br />
<br />
The martingale can be generalized to be with respect to another sequence of random variables.<br />
{{Theorem<br />
|Definition (martingale, general version)| <br />
:A sequence of random variables <math>Y_0,Y_1,\ldots</math> is a martingale with respect to the sequence <math>X_0,X_1,\ldots</math> if, for all <math>i\ge 0</math>, the following conditions hold:<br />
:* <math>Y_i</math> is a function of <math>X_0,X_1,\ldots,X_i</math>;<br />
:* <math>\begin{align}<br />
\mathbf{E}[Y_{i+1}\mid X_0,\ldots,X_{i}]=Y_{i}.<br />
\end{align}</math><br />
}}<br />
Therefore, a sequence <math>X_0,X_1,\ldots</math> is a martingale if it is a martingale with respect to itself.<br />
<br />
The purpose of this generalization is that we are usually more interested in a function of a sequence of random variables, rather than the sequence itself.<br />
<br />
The following definition describes a very general approach for constructing an important type of martingales.<br />
<br />
{{Theorem<br />
|Definition (The Doob sequence)|<br />
: The Doob sequence of a function <math>f</math> with respect to a sequence of random variables <math>X_1,\ldots,X_n</math> is defined by<br />
::<math><br />
Y_i=\mathbf{E}[f(X_1,\ldots,X_n)\mid X_1,\ldots,X_{i}], \quad 0\le i\le n.<br />
</math><br />
:In particular, <math>Y_0=\mathbf{E}[f(X_1,\ldots,X_n)]</math> and <math>Y_n=f(X_1,\ldots,X_n)</math>.<br />
}}<br />
<br />
The Doob sequence of a function defines a martingale. That is<br />
::<math><br />
\mathbf{E}[Y_i\mid X_1,\ldots,X_{i-1}]=Y_{i-1}, <br />
</math><br />
for any <math>0\le i\le n</math>.<br />
<br />
To prove this claim, we recall the definition that <math>Y_i=\mathbf{E}[f(X_1,\ldots,X_n)\mid X_1,\ldots,X_{i}]</math>, thus,<br />
:<math><br />
\begin{align}<br />
\mathbf{E}[Y_i\mid X_1,\ldots,X_{i-1}]<br />
&=\mathbf{E}[\mathbf{E}[f(X_1,\ldots,X_n)\mid X_1,\ldots,X_{i}]\mid X_1,\ldots,X_{i-1}]\\<br />
&=\mathbf{E}[f(X_1,\ldots,X_n)\mid X_1,\ldots,X_{i-1}]\\<br />
&=Y_{i-1},<br />
\end{align}<br />
</math><br />
where the second equation is due to the fundamental fact about conditional expectation introduced in the first section.<br />
<br />
The Doob martingale describes a very natural procedure to determine a function value of a sequence of random variables. Suppose that we want to predict the value of a function <math>f(X_1,\ldots,X_n)</math> of random variables <math>X_1,\ldots,X_n</math>. The Doob sequence <math>Y_0,Y_1,\ldots,Y_n</math> represents a sequence of refined estimates of the value of <math>f(X_1,\ldots,X_n)</math>, gradually using more information on the values of the random variables <math>X_1,\ldots,X_n</math>. The first element <math>Y_0</math> is just the expectation of <math>f(X_1,\ldots,X_n)</math>. Element <math>Y_i</math> is the expected value of <math>f(X_1,\ldots,X_n)</math> when the values of <math>X_1,\ldots,X_{i}</math> are known, and <math>Y_n=f(X_1,\ldots,X_n)</math> when <math>f(X_1,\ldots,X_n)</math> is fully determined by <math>X_1,\ldots,X_n</math>.<br />
<br />
The following two Doob martingales arise in evaluating the parameters of random graphs. <br />
<br />
===edge exposure martingale===<br />
:Let <math>G</math> be a random graph on <math>n</math> vertices. Let <math>f</math> be a real-valued function of graphs, such as, chromatic number, number of triangles, the size of the largest clique or independent set, etc. Denote that <math>m={n\choose 2}</math>. Fix an arbitrary numbering of potential edges between the <math>n</math> vertices, and denote the edges as <math>e_1,\ldots,e_m</math>. Let <br />
::<math><br />
X_i=\begin{cases}<br />
1& \mbox{if }e_i\in G,\\<br />
0& \mbox{otherwise}.<br />
\end{cases}<br />
</math><br />
:Let <math>Y_0=\mathbf{E}[f(G)]</math> and for <math>i=1,\ldots,m</math>, let <math>Y_i=\mathbf{E}[f(G)\mid X_1,\ldots,X_i]</math>.<br />
:The sequence <math>Y_0,Y_1,\ldots,Y_n</math> gives a Doob martingale that is commonly called the '''edge exposure martingale'''.<br />
<br />
===vertex exposure martingale===<br />
: Instead of revealing edges one at a time, we could reveal the set of edges connected to a given vertex, one vertex at a time. Suppose that the vertex set is <math>[n]</math>. Let <math>X_i</math> be the subgraph of <math>G</math> induced by the vertex set <math>[i]</math>, i.e. the first <math>i</math> vertices.<br />
:Let <math>Y_0=\mathbf{E}[f(G)]</math> and for <math>i=1,\ldots,n</math>, let <math>Y_i=\mathbf{E}[f(G)\mid X_1,\ldots,X_i]</math>.<br />
:The sequence <math>Y_0,Y_1,\ldots,Y_n</math> gives a Doob martingale that is commonly called the '''vertex exposure martingale'''.</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=%E9%AB%98%E7%BA%A7%E7%AE%97%E6%B3%95_(Fall_2026)/Concentration_of_measure&diff=13945高级算法 (Fall 2026)/Concentration of measure2026-09-15T11:59:57Z<p>Etone: Created page with "=Chernoff Bound= Suppose that we have a fair coin. If we toss it once, then the outcome is completely unpredictable. But if we toss it, say for 1000 times, then the number of HEADs is very likely to be around 500. This phenomenon, as illustrated in the following figure, is called the '''concentration''' of measure. The Chernoff bound is an inequality that characterizes the concentration phenomenon for the sum of independent trials. File:Coinflip.png|border|450px|cent..."</p>
<hr />
<div>=Chernoff Bound=<br />
<br />
Suppose that we have a fair coin. If we toss it once, then the outcome is completely unpredictable. But if we toss it, say for 1000 times, then the number of HEADs is very likely to be around 500. This phenomenon, as illustrated in the following figure, is called the '''concentration''' of measure. The Chernoff bound is an inequality that characterizes the concentration phenomenon for the sum of independent trials.<br />
<br />
[[File:Coinflip.png|border|450px|center]]<br />
<br />
Before formally stating the Chernoff bound, let's introduce the '''moment generating function'''.<br />
<br />
== Moment generating functions ==<br />
The more we know about the moments of a random variable <math>X</math>, the more information we would have about <math>X</math>. There is a so-called '''moment generating function''', which "packs" all the information about the moments of <math>X</math> into one function.<br />
<br />
{{Theorem<br />
|Definition|<br />
:The moment generating function of a random variable <math>X</math> is defined as <math>\mathbf{E}\left[\mathrm{e}^{\lambda X}\right]</math> where <math>\lambda</math> is the parameter of the function.<br />
}}<br />
<br />
By Taylor's expansion and the linearity of expectations,<br />
:<math>\begin{align}<br />
\mathbf{E}\left[\mathrm{e}^{\lambda X}\right]<br />
&=<br />
\mathbf{E}\left[\sum_{k=0}^\infty\frac{\lambda^k}{k!}X^k\right]\\<br />
&=\sum_{k=0}^\infty\frac{\lambda^k}{k!}\mathbf{E}\left[X^k\right]<br />
\end{align}</math> <br />
<br />
The moment generating function <math>\mathbf{E}\left[\mathrm{e}^{\lambda X}\right]</math> is a function of <math>\lambda</math>.<br />
<br />
== The Chernoff bound ==<br />
The Chernoff bounds are exponentially sharp tail inequalities for the sum of independent trials.<br />
The bounds are obtained by applying Markov's inequality to the moment generating function of the sum of independent trials, with some appropriate choice of the parameter <math>\lambda</math>.<br />
{{Theorem<br />
|Chernoff bound (the upper tail)|<br />
:Let <math>X=\sum_{i=1}^n X_i</math>, where <math>X_1, X_2, \ldots, X_n</math> are independent Poisson trials. Let <math>\mu=\mathbf{E}[X]</math>. <br />
:Then for any <math>\delta>0</math>,<br />
::<math>\Pr[X\ge (1+\delta)\mu]\le\left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}.</math><br />
}}<br />
{{Proof| For any <math>\lambda>0</math>, <math>X\ge (1+\delta)\mu</math> is equivalent to that <math>e^{\lambda X}\ge e^{\lambda (1+\delta)\mu}</math>, thus<br />
:<math>\begin{align}<br />
\Pr[X\ge (1+\delta)\mu] <br />
&=<br />
\Pr\left[e^{\lambda X}\ge e^{\lambda (1+\delta)\mu}\right]\\<br />
&\le<br />
\frac{\mathbf{E}\left[e^{\lambda X}\right]}{e^{\lambda (1+\delta)\mu}},<br />
\end{align}</math> <br />
where the last step follows by Markov's inequality.<br />
<br />
Computing the moment generating function <math>\mathbf{E}[e^{\lambda X}]</math>:<br />
:<math>\begin{align}<br />
\mathbf{E}\left[e^{\lambda X}\right]<br />
&=<br />
\mathbf{E}\left[e^{\lambda \sum_{i=1}^n X_i}\right]\\<br />
&=<br />
\mathbf{E}\left[\prod_{i=1}^n e^{\lambda X_i}\right]\\<br />
&=<br />
\prod_{i=1}^n \mathbf{E}\left[e^{\lambda X_i}\right].<br />
& (\mbox{for independent random variables})<br />
\end{align}</math><br />
<br />
Let <math>p_i=\Pr[X_i=1]</math> for <math>i=1,2,\ldots,n</math>. Then,<br />
:<math>\mu=\mathbf{E}[X]=\mathbf{E}\left[\sum_{i=1}^n X_i\right]=\sum_{i=1}^n\mathbf{E}[X_i]=\sum_{i=1}^n p_i</math>.<br />
<br />
We bound the moment generating function for each individual <math>X_i</math> as follows.<br />
:<math>\begin{align}<br />
\mathbf{E}\left[e^{\lambda X_i}\right]<br />
&=<br />
p_i\cdot e^{\lambda\cdot 1}+(1-p_i)\cdot e^{\lambda\cdot 0}\\<br />
&=<br />
1+p_i(e^\lambda -1)\\<br />
&\le<br />
e^{p_i(e^\lambda-1)},<br />
\end{align}</math><br />
where in the last step we apply the Taylor's expansion so that <math>e^y\ge 1+y</math> where <math>y=p_i(e^\lambda-1)\ge 0</math>. (By doing this, we can transform the product to the sum of <math>p_i</math>, which is <math>\mu</math>.) <br />
<br />
Therefore,<br />
:<math>\begin{align}<br />
\mathbf{E}\left[e^{\lambda X}\right]<br />
&=<br />
\prod_{i=1}^n \mathbf{E}\left[e^{\lambda X_i}\right]\\<br />
&\le<br />
\prod_{i=1}^n e^{p_i(e^\lambda-1)}\\<br />
&=<br />
\exp\left(\sum_{i=1}^n p_i(e^{\lambda}-1)\right)\\<br />
&=<br />
e^{(e^\lambda-1)\mu}.<br />
\end{align}</math><br />
Thus, we have shown that for any <math>\lambda>0</math>,<br />
:<math>\begin{align}<br />
\Pr[X\ge (1+\delta)\mu] <br />
&\le<br />
\frac{\mathbf{E}\left[e^{\lambda X}\right]}{e^{\lambda (1+\delta)\mu}}\\<br />
&\le<br />
\frac{e^{(e^\lambda-1)\mu}}{e^{\lambda (1+\delta)\mu}}\\<br />
&=<br />
\left(\frac{e^{(e^\lambda-1)}}{e^{\lambda (1+\delta)}}\right)^\mu<br />
\end{align}</math>.<br />
For any <math>\delta>0</math>, we can let <math>\lambda=\ln(1+\delta)>0</math> to get<br />
:<math>\Pr[X\ge (1+\delta)\mu]\le\left(\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{\mu}.</math><br />
}}<br />
<br />
The idea of the proof is actually quite clear: we apply Markov's inequality to <math>e^{\lambda X}</math> and for the rest, we just estimate the moment generating function <math>\mathbf{E}[e^{\lambda X}]</math>. To make the bound as tight as possible, we minimized the <math>\frac{e^{(e^\lambda-1)}}{e^{\lambda (1+\delta)}}</math> by setting <math>\lambda=\ln(1+\delta)</math>, which can be justified by taking derivatives of <math>\frac{e^{(e^\lambda-1)}}{e^{\lambda (1+\delta)}}</math>.<br />
<br />
----<br />
<br />
We then proceed to the lower tail, the probability that the random variable deviates below the mean value:<br />
<br />
{{Theorem<br />
|Chernoff bound (the lower tail)|<br />
:Let <math>X=\sum_{i=1}^n X_i</math>, where <math>X_1, X_2, \ldots, X_n</math> are independent Poisson trials. Let <math>\mu=\mathbf{E}[X]</math>. <br />
:Then for any <math>0<\delta<1</math>,<br />
::<math>\Pr[X\le (1-\delta)\mu]\le\left(\frac{e^{-\delta}}{(1-\delta)^{(1-\delta)}}\right)^{\mu}.</math><br />
}}<br />
{{Proof| For any <math>\lambda<0</math>, by the same analysis as in the upper tail version,<br />
:<math>\begin{align}<br />
\Pr[X\le (1-\delta)\mu] <br />
&=<br />
\Pr\left[e^{\lambda X}\ge e^{\lambda (1-\delta)\mu}\right]\\<br />
&\le<br />
\frac{\mathbf{E}\left[e^{\lambda X}\right]}{e^{\lambda (1-\delta)\mu}}\\<br />
&\le<br />
\left(\frac{e^{(e^\lambda-1)}}{e^{\lambda (1-\delta)}}\right)^\mu.<br />
\end{align}</math> <br />
For any <math>0<\delta<1</math>, we can let <math>\lambda=\ln(1-\delta)<0</math> to get<br />
:<math>\Pr[X\ge (1-\delta)\mu]\le\left(\frac{e^{-\delta}}{(1-\delta)^{(1-\delta)}}\right)^{\mu}.</math><br />
}}<br />
<br />
== Useful forms of the Chernoff bounds==<br />
Some useful special forms of the bounds can be derived directly from the above general forms of the bounds. We now know better why we say that the bounds are exponentially sharp.<br />
<br />
{{Theorem<br />
|Useful forms of the Chernoff bound|<br />
:Let <math>X=\sum_{i=1}^n X_i</math>, where <math>X_1, X_2, \ldots, X_n</math> are independent Poisson trials. Let <math>\mu=\mathbf{E}[X]</math>. Then<br />
:1. for <math>0<\delta\le 1</math>,<br />
::<math>\Pr[X\ge (1+\delta)\mu]<\exp\left(-\frac{\mu\delta^2}{3}\right);</math><br />
::<math>\Pr[X\le (1-\delta)\mu]<\exp\left(-\frac{\mu\delta^2}{2}\right);</math><br />
:2. for <math>t\ge 2e\mu</math>,<br />
::<math>\Pr[X\ge t]\le 2^{-t}.</math><br />
}}<br />
{{Proof| To obtain the bounds in (1), we need to show that for <math>0<\delta< 1</math>, <math>\frac{e^{\delta}}{(1+\delta)^{(1+\delta)}}\le e^{-\delta^2/3}</math> and <math>\frac{e^{-\delta}}{(1-\delta)^{(1-\delta)}}\le e^{-\delta^2/2}</math>. We can verify both inequalities by standard analysis techniques.<br />
<br />
To obtain the bound in (2), let <math>t=(1+\delta)\mu</math>. Then <math>\delta=t/\mu-1\ge 2e-1</math>. Hence,<br />
:<math>\begin{align}<br />
\Pr[X\ge(1+\delta)\mu]<br />
&\le<br />
\left(\frac{e^\delta}{(1+\delta)^{(1+\delta)}}\right)^\mu\\<br />
&\le<br />
\left(\frac{e}{1+\delta}\right)^{(1+\delta)\mu}\\<br />
&\le<br />
\left(\frac{e}{2e}\right)^t\\<br />
&\le<br />
2^{-t}<br />
\end{align}</math><br />
}}<br />
<br />
== Applications to balls-into-bins ==<br />
Throwing <math>m</math> balls uniformly and independently to <math>n</math> bins, what is the maximum load of all bins with high probability? In the last class, we gave an analysis of this problem by using a counting argument.<br />
<br />
Now we give a more "advanced" analysis by using Chernoff bounds.<br />
<br />
<br />
For any <math>i\in[n]</math> and <math>j\in[m]</math>, let <math>X_{ij}</math> be the indicator variable for the event that ball <math>j</math> is thrown to bin <math>i</math>. Obviously<br />
:<math>\mathbf{E}[X_{ij}]=\Pr[\mbox{ball }j\mbox{ is thrown to bin }i]=\frac{1}{n}</math><br />
Let <math>Y_i=\sum_{j\in[m]}X_{ij}</math> be the load of bin <math>i</math>. <br />
<br />
<br />
Then the expected load of bin <math>i</math> is<br />
<br />
<math>(*)\qquad \mu=\mathbf{E}[Y_i]=\mathbf{E}\left[\sum_{j\in[m]}X_{ij}\right]=\sum_{j\in[m]}\mathbf{E}[X_{ij}]=m/n. </math><br />
<br />
For the case <math>m=n</math>, it holds that <math>\mu=1</math><br />
<br />
Note that <math>Y_i</math> is a sum of <math>m</math> mutually independent indicator variable. Applying Chernoff bound, for any particular bin <math>i\in[n]</math>,<br />
:<math><br />
\Pr[Y_i>(1+\delta)\mu] \le \left(\frac{e^{\delta}}{(1+\delta)^{1+\delta}}\right)^\mu.<br />
</math><br />
<br />
=== The <math>m=n</math> case ===<br />
<br />
When <math>m=n</math>, <math>\mu=1</math>. Write <math>c=1+\delta</math>. The above bound can be written as<br />
:<math><br />
\Pr[Y_i>c] \le \frac{e^{c-1}}{c^c}.<br />
</math><br />
<br />
Let <math>c=\frac{e\ln n}{\ln\ln n}</math>, we evaluate <math>\frac{e^{c-1}}{c^c}</math> by taking logarithm to its reciprocal.<br />
:<math><br />
\begin{align}<br />
\ln\left(\frac{c^c}{e^{c-1}}\right)<br />
&=<br />
c\ln c-c+1\\<br />
&=<br />
c(\ln c-1)+1\\<br />
&=<br />
\frac{e\ln n}{\ln\ln n}\left(\ln\ln n-\ln\ln\ln n\right)+1\\<br />
&\ge<br />
\frac{e\ln n}{\ln\ln n}\cdot\frac{2}{e}\ln\ln n+1\\<br />
&\ge<br />
2\ln n.<br />
\end{align}<br />
</math><br />
Thus,<br />
:<math><br />
\Pr\left[Y_i>\frac{e\ln n}{\ln\ln n}\right] \le \frac{1}{n^2}.<br />
</math><br />
<br />
Applying the union bound, the probability that there exists a bin with load <math>>12\ln n</math> is<br />
:<math>n\cdot \Pr\left[Y_1>\frac{e\ln n}{\ln\ln n}\right] \le \frac{1}{n}</math>.<br />
Therefore, for <math>m=n</math>, with high probability, the maximum load is <math>O\left(\frac{e\ln n}{\ln\ln n}\right)</math>.<br />
<br />
=== The <math>m> \ln n</math> case===<br />
When <math>m\ge n\ln n</math>, then according to <math>(*)</math>, <math>\mu=\frac{m}{n}\ge \ln n</math><br />
<br />
We can apply an easier form of the Chernoff bounds,<br />
:<math><br />
\Pr[Y_i\ge 2e\mu]\le 2^{-2e\mu}\le 2^{-2e\ln n}<\frac{1}{n^2}.<br />
</math><br />
By the union bound, the probability that there exists a bin with load <math>\ge 2e\frac{m}{n}</math> is,<br />
:<math>n\cdot \Pr\left[Y_1>2e\frac{m}{n}\right] = n\cdot \Pr\left[Y_1>2e\mu\right]\le \frac{1}{n}</math>.<br />
Therefore, for <math>m\ge n\ln n</math>, with high probability, the maximum load is <math>O\left(\frac{m}{n}\right)</math>.<br />
<br />
<br />
= Martingales =<br />
"Martingale" originally refers to a betting strategy in which the gambler doubles his bet after every loss. Assuming unlimited wealth, this strategy is guaranteed to eventually have a positive net profit. For example, starting from an initial stake 1, after <math>n</math> losses, if the <math>(n+1)</math>th bet wins, then it gives a net profit of<br />
:<math><br />
2^n-\sum_{i=1}^{n}2^{i-1}=1,<br />
</math><br />
which is a positive number.<br />
<br />
However, the assumption of unlimited wealth is unrealistic. For limited wealth, with geometrically increasing bet, it is very likely to end up bankrupt. You should never try this strategy in real life.<br />
<br />
Suppose that the gambler is allowed to use any strategy. His stake on the next beting is decided based on the results of all the bettings so far. This gives us a highly dependent sequence of random variables <math>X_0,X_1,\ldots,</math>, where <math>X_0</math> is his initial capital, and <math>X_i</math> represents his capital after the <math>i</math>th betting. Up to different betting strategies, <math>X_i</math> can be arbitrarily dependent on <math>X_0,\ldots,X_{i-1}</math>. However, as long as the game is fair, namely, winning and losing with equal chances, conditioning on the past variables <math>X_0,\ldots,X_{i-1}</math>, we will expect no change in the value of the present variable <math>X_{i}</math> on average. Random variables satisfying this property is called a '''martingale''' sequence.<br />
<br />
{{Theorem<br />
|Definition (martingale)|<br />
:A sequence of random variables <math>X_0,X_1,\ldots</math> is a '''martingale''' if for all <math>i> 0</math>,<br />
:: <math>\begin{align}<br />
\mathbf{E}[X_{i}\mid X_0,\ldots,X_{i-1}]=X_{i-1}.<br />
\end{align}</math><br />
}}<br />
<br />
The martingale can be generalized to be with respect to another sequence of random variables.<br />
{{Theorem<br />
|Definition (martingale, general version)| <br />
:A sequence of random variables <math>Y_0,Y_1,\ldots</math> is a martingale with respect to the sequence <math>X_0,X_1,\ldots</math> if, for all <math>i\ge 0</math>, the following conditions hold:<br />
:* <math>Y_i</math> is a function of <math>X_0,X_1,\ldots,X_i</math>;<br />
:* <math>\begin{align}<br />
\mathbf{E}[Y_{i+1}\mid X_0,\ldots,X_{i}]=Y_{i}.<br />
\end{align}</math><br />
}}<br />
Therefore, a sequence <math>X_0,X_1,\ldots</math> is a martingale if it is a martingale with respect to itself.<br />
<br />
The purpose of this generalization is that we are usually more interested in a function of a sequence of random variables, rather than the sequence itself.<br />
<br />
==Azuma's Inequality==<br />
<br />
The Azuma's inequality is a martingale tail inequality.<br />
<br />
{{Theorem<br />
|Azuma's Inequality|<br />
:Let <math>X_0,X_1,\ldots</math> be a martingale such that, for all <math>k\ge 1</math>,<br />
::<math><br />
|X_{k}-X_{k-1}|\le c_k,<br />
</math><br />
:Then<br />
::<math>\begin{align}<br />
\Pr\left[|X_n-X_0|\ge t\right]\le 2\exp\left(-\frac{t^2}{2\sum_{k=1}^nc_k^2}\right).<br />
\end{align}</math><br />
}}<br />
Unlike the Chernoff bounds, there is no assumption of independence, which makes the martingale inequalities more useful.<br />
<br />
The following '''bounded difference condition''' <br />
:<math><br />
|X_{k}-X_{k-1}|\le c_k<br />
</math><br />
says that the martingale <math>X_0,X_1,\ldots</math> as a process evolving over time, never makes big change in a single step. <br />
<br />
The Azuma's inequality says that for any martingale satisfying the bounded difference condition, it is unlikely that process wanders far from its starting point.<br />
<br />
A special case is when the differences are bounded by a constant. The following corollary is directly implied by the Azuma's inequality. <br />
<br />
{{Theorem<br />
|Corollary|<br />
:Let <math>X_0,X_1,\ldots</math> be a martingale such that, for all <math>k\ge 1</math>,<br />
::<math><br />
|X_{k}-X_{k-1}|\le c,<br />
</math><br />
:Then<br />
::<math>\begin{align}<br />
\Pr\left[|X_n-X_0|\ge ct\sqrt{n}\right]\le 2 e^{-t^2/2}.<br />
\end{align}</math><br />
}}<br />
<br />
This corollary states that for any martingale sequence whose diferences are bounded by a constant, the probability that it deviates <math>\omega(\sqrt{n})</math> far away from the starting point after <math>n</math> steps is bounded by <math>o(1)</math>.<br />
<br />
=== Generalization ===<br />
<br />
Azuma's inequality can be generalized to a martingale with respect another sequence.<br />
{{Theorem<br />
|Azuma's Inequality (general version)|<br />
:Let <math>Y_0,Y_1,\ldots</math> be a martingale with respect to the sequence <math>X_0,X_1,\ldots</math> such that, for all <math>k\ge 1</math>,<br />
::<math><br />
|Y_{k}-Y_{k-1}|\le c_k,<br />
</math><br />
:Then<br />
::<math>\begin{align}<br />
\Pr\left[|Y_n-Y_0|\ge t\right]\le 2\exp\left(-\frac{t^2}{2\sum_{k=1}^nc_k^2}\right).<br />
\end{align}</math><br />
}}<br />
<br />
=== The Proof of Azuma's Inueqality===<br />
We will only give the formal proof of the non-generalized version. The proof of the general version is almost identical, with the only difference that we work on random sequence <math>Y_i</math> conditioning on sequence <math>X_i</math>. <br />
<br />
The proof of Azuma's Inequality uses several ideas which are used in the proof of the Chernoff bounds. We first observe that the total deviation of the martingale sequence can be represented as the sum of deferences in every steps. Thus, as the Chernoff bounds, we are looking for a bound of the deviation of the sum of random variables. The strategy of the proof is almost the same as the proof of Chernoff bounds: we first apply Markov's inequality to the moment generating function, then we bound the moment generating function, and at last we optimize the parameter of the moment generating function. However, unlike the Chernoff bounds, the martingale differences are not independent any more. So we replace the use of the independence in the Chernoff bound by the martingale property. The proof is detailed as follows.<br />
<br />
In order to bound the probability of <math>|X_n-X_0|\ge t</math>, we first bound the upper tail <math>\Pr[X_n-X_0\ge t]</math>. The bound of the lower tail can be symmetrically proved with the <math>X_i</math> replaced by <math>-X_i</math>.<br />
<br />
==== Represent the deviation as the sum of differences ====<br />
We define the '''martingale difference sequence''': for <math>i\ge 1</math>, let<br />
:<math><br />
Y_i=X_i-X_{i-1}.<br />
</math><br />
It holds that<br />
:<math><br />
\begin{align}<br />
\mathbf{E}[Y_i\mid X_0,\ldots,X_{i-1}]<br />
&=\mathbf{E}[X_i-X_{i-1}\mid X_0,\ldots,X_{i-1}]\\<br />
&=\mathbf{E}[X_i\mid X_0,\ldots,X_{i-1}]-\mathbf{E}[X_{i-1}\mid X_0,\ldots,X_{i-1}]\\<br />
&=X_{i-1}-X_{i-1}\\<br />
&=0.<br />
\end{align}<br />
</math><br />
The second to the last equation is due to the fact that <math>X_0,X_1,\ldots</math> is a martingale and the definition of conditional expectation.<br />
<br />
Let <math>Z_n</math> be the accumulated differences<br />
:<math><br />
Z_n=\sum_{i=1}^n Y_i.<br />
</math><br />
<br />
The deviation <math>(X_n-X_0)</math> can be computed by the accumulated differences:<br />
:<math><br />
\begin{align}<br />
X_n-X_0<br />
&=(X_1-X_{0})+(X_2-X_1)+\cdots+(X_n-X_{n-1})\\<br />
&=\sum_{i=1}^n Y_i\\<br />
&=Z_n.<br />
\end{align}<br />
</math><br />
<br />
We then only need to upper bound the probability of the event <math>Z_n\ge t</math>.<br />
<br />
==== Apply Markov's inequality to the moment generating function ====<br />
The event <math>Z_n\ge t</math> is equivalent to that <math>e^{\lambda Z_n}\ge e^{\lambda t}</math> for <math>\lambda>0</math>. Apply Markov's inequality, we have<br />
:<math><br />
\begin{align}<br />
\Pr\left[Z_n\ge t\right]<br />
&=\Pr\left[e^{\lambda Z_n}\ge e^{\lambda t}\right]\\<br />
&\le \frac{\mathbf{E}\left[e^{\lambda Z_n}\right]}{e^{\lambda t}}.<br />
\end{align}<br />
</math><br />
This is exactly the same as what we did to prove the Chernoff bound. Next, we need to bound the moment generating function <math>\mathbf{E}\left[e^{\lambda Z_n}\right]</math>.<br />
<br />
==== Bound the moment generating functions ====<br />
The moment generating function<br />
:<math><br />
\begin{align}<br />
\mathbf{E}\left[e^{\lambda Z_n}\right]<br />
&=\mathbf{E}\left[\mathbf{E}\left[e^{\lambda Z_n}\mid X_0,\ldots,X_{n-1}\right]\right]\\<br />
&=\mathbf{E}\left[\mathbf{E}\left[e^{\lambda (Z_{n-1}+Y_n)}\mid X_0,\ldots,X_{n-1}\right]\right]\\<br />
&=\mathbf{E}\left[\mathbf{E}\left[e^{\lambda Z_{n-1}}\cdot e^{\lambda Y_n}\mid X_0,\ldots,X_{n-1}\right]\right]\\<br />
&=\mathbf{E}\left[e^{\lambda Z_{n-1}}\cdot\mathbf{E}\left[e^{\lambda Y_n}\mid X_0,\ldots,X_{n-1}\right]\right]<br />
\end{align}<br />
</math><br />
The first and the last equations are due to the fundamental facts about conditional expectation which are proved by us in the first section.<br />
<br />
We then upper bound the <math>\mathbf{E}\left[e^{\lambda Y_n}\mid X_0,\ldots,X_{n-1}\right]</math> by a constant. To do so, we need the following technical lemma which is proved by the convexity of <math>e^{\lambda Y_n}</math>.<br />
<br />
{{Theorem<br />
|Lemma|<br />
:Let <math>X</math> be a random variable such that <math>\mathbf{E}[X]=0</math> and <math>|X|\le c</math>. Then for <math>\lambda>0</math>,<br />
::<math><br />
\mathbf{E}[e^{\lambda X}]\le e^{\lambda^2c^2/2}.<br />
</math><br />
}}<br />
{{Proof| Observe that for <math>\lambda>0</math>, the function <math>e^{\lambda X}</math> of the variable <math>X</math> is convex in the interval <math>[-c,c]</math>. We draw a line between the two endpoints points <math>(-c, e^{-\lambda c})</math> and <math>(c, e^{\lambda c})</math>. The curve of <math>e^{\lambda X}</math> lies entirely below this line. Thus,<br />
:<math><br />
\begin{align}<br />
e^{\lambda X}<br />
&\le \frac{c-X}{2c}e^{-\lambda c}+\frac{c+X}{2c}e^{\lambda c}\\<br />
&=\frac{e^{\lambda c}+e^{-\lambda c}}{2}+\frac{X}{2c}(e^{\lambda c}-e^{-\lambda c}).<br />
\end{align}<br />
</math><br />
<br />
Since <math>\mathbf{E}[X]=0</math>, we have<br />
:<math><br />
\begin{align}<br />
\mathbf{E}[e^{\lambda X}]<br />
&\le \mathbf{E}[\frac{e^{\lambda c}+e^{-\lambda c}}{2}+\frac{X}{2c}(e^{\lambda c}-e^{-\lambda c})]\\<br />
&=\frac{e^{\lambda c}+e^{-\lambda c}}{2}+\frac{e^{\lambda c}-e^{-\lambda c}}{2c}\mathbf{E}[X]\\<br />
&=\frac{e^{\lambda c}+e^{-\lambda c}}{2}.<br />
\end{align}<br />
</math><br />
<br />
By expanding both sides as Taylor's series, it can be verified that <math>\frac{e^{\lambda c}+e^{-\lambda c}}{2}\le e^{\lambda^2c^2/2}</math>.<br />
}}<br />
<br />
Apply the above lemma to the random variable<br />
:<math><br />
(Y_n \mid X_0,\ldots,X_{n-1})<br />
</math><br />
<br />
We have already shown that its expectation <br />
<math><br />
\mathbf{E}[(Y_n \mid X_0,\ldots,X_{n-1})]=0,<br />
</math><br />
and by the bounded difference condition of Azuma's inequality, we have<br />
<math><br />
|Y_n|=|(X_n-X_{n-1})|\le c_n.<br />
</math><br />
Thus, due to the above lemma, it holds that<br />
:<math><br />
\mathbf{E}[e^{\lambda Y_n}\mid X_0,\ldots,X_{n-1}]\le e^{\lambda^2c_n^2/2}.<br />
</math><br />
<br />
Back to our analysis of the expectation <math>\mathbf{E}\left[e^{\lambda Z_n}\right]</math>, we have <br />
:<math><br />
\begin{align}<br />
\mathbf{E}\left[e^{\lambda Z_n}\right]<br />
&=\mathbf{E}\left[e^{\lambda Z_{n-1}}\cdot\mathbf{E}\left[e^{\lambda Y_n}\mid X_0,\ldots,X_{n-1}\right]\right]\\<br />
&\le \mathbf{E}\left[e^{\lambda Z_{n-1}}\cdot e^{\lambda^2c_n^2/2}\right]\\<br />
&= e^{\lambda^2c_n^2/2}\cdot\mathbf{E}\left[e^{\lambda Z_{n-1}}\right] .<br />
\end{align}<br />
</math><br />
<br />
Apply the same analysis to <math>\mathbf{E}\left[e^{\lambda Z_{n-1}}\right]</math>, we can solve the above recursion by<br />
:<math><br />
\begin{align}<br />
\mathbf{E}\left[e^{\lambda Z_n}\right]<br />
&\le \prod_{k=1}^n e^{\lambda^2c_k^2/2}\\<br />
&= \exp\left(\lambda^2\sum_{k=1}^n c_k^2/2\right).<br />
\end{align}<br />
</math><br />
<br />
Go back to the Markov's inequality,<br />
:<math><br />
\begin{align}<br />
\Pr\left[Z_n\ge t\right]<br />
&\le \frac{\mathbf{E}\left[e^{\lambda Z_n}\right]}{e^{\lambda t}}\\<br />
&\le \exp\left(\lambda^2\sum_{k=1}^n c_k^2/2-\lambda t\right).<br />
\end{align}<br />
</math><br />
<br />
We then only need to choose a proper <math>\lambda>0</math>.<br />
<br />
==== Optimization ====<br />
By choosing <math>\lambda=\frac{t}{\sum_{k=1}^n c_k^2}</math>, we have that <br />
:<math><br />
\exp\left(\lambda^2\sum_{k=1}^n c_k^2/2-\lambda t\right)=\exp\left(-\frac{t^2}{2\sum_{k=1}^n c_k^2}\right).<br />
</math><br />
Thus, the probability<br />
:<math><br />
\begin{align}<br />
\Pr\left[X_n-X_0\ge t\right]<br />
&=\Pr\left[Z_n\ge t\right]\\<br />
&\le \exp\left(\lambda^2\sum_{k=1}^n c_k^2/2-\lambda t\right)\\<br />
&= \exp\left(-\frac{t^2}{2\sum_{k=1}^n c_k^2}\right).<br />
\end{align}<br />
</math><br />
The upper tail of Azuma's inequality is proved. By replacing <math>X_i</math> by <math>-X_i</math>, the lower tail can be treated just as the upper tail. Applying the union bound, Azuma's inequality is proved.<br />
<br />
==The Doob martingales ==<br />
The following definition describes a very general approach for constructing an important type of martingales.<br />
<br />
{{Theorem<br />
|Definition (The Doob sequence)|<br />
: The Doob sequence of a function <math>f</math> with respect to a sequence of random variables <math>X_1,\ldots,X_n</math> is defined by<br />
::<math><br />
Y_i=\mathbf{E}[f(X_1,\ldots,X_n)\mid X_1,\ldots,X_{i}], \quad 0\le i\le n.<br />
</math><br />
:In particular, <math>Y_0=\mathbf{E}[f(X_1,\ldots,X_n)]</math> and <math>Y_n=f(X_1,\ldots,X_n)</math>.<br />
}}<br />
<br />
The Doob sequence of a function defines a martingale. That is<br />
::<math><br />
\mathbf{E}[Y_i\mid X_1,\ldots,X_{i-1}]=Y_{i-1}, <br />
</math><br />
for any <math>0\le i\le n</math>.<br />
<br />
To prove this claim, we recall the definition that <math>Y_i=\mathbf{E}[f(X_1,\ldots,X_n)\mid X_1,\ldots,X_{i}]</math>, thus,<br />
:<math><br />
\begin{align}<br />
\mathbf{E}[Y_i\mid X_1,\ldots,X_{i-1}]<br />
&=\mathbf{E}[\mathbf{E}[f(X_1,\ldots,X_n)\mid X_1,\ldots,X_{i}]\mid X_1,\ldots,X_{i-1}]\\<br />
&=\mathbf{E}[f(X_1,\ldots,X_n)\mid X_1,\ldots,X_{i-1}]\\<br />
&=Y_{i-1},<br />
\end{align}<br />
</math><br />
where the second equation is due to the fundamental fact about conditional expectation introduced in the first section.<br />
<br />
The Doob martingale describes a very natural procedure to determine a function value of a sequence of random variables. Suppose that we want to predict the value of a function <math>f(X_1,\ldots,X_n)</math> of random variables <math>X_1,\ldots,X_n</math>. The Doob sequence <math>Y_0,Y_1,\ldots,Y_n</math> represents a sequence of refined estimates of the value of <math>f(X_1,\ldots,X_n)</math>, gradually using more information on the values of the random variables <math>X_1,\ldots,X_n</math>. The first element <math>Y_0</math> is just the expectation of <math>f(X_1,\ldots,X_n)</math>. Element <math>Y_i</math> is the expected value of <math>f(X_1,\ldots,X_n)</math> when the values of <math>X_1,\ldots,X_{i}</math> are known, and <math>Y_n=f(X_1,\ldots,X_n)</math> when <math>f(X_1,\ldots,X_n)</math> is fully determined by <math>X_1,\ldots,X_n</math>.<br />
<br />
The following two Doob martingales arise in evaluating the parameters of random graphs. <br />
<br />
;edge exposure martingale<br />
:Let <math>G</math> be a random graph on <math>n</math> vertices. Let <math>f</math> be a real-valued function of graphs, such as, chromatic number, number of triangles, the size of the largest clique or independent set, etc. Denote that <math>m={n\choose 2}</math>. Fix an arbitrary numbering of potential edges between the <math>n</math> vertices, and denote the edges as <math>e_1,\ldots,e_m</math>. Let <br />
::<math><br />
X_i=\begin{cases}<br />
1& \mbox{if }e_i\in G,\\<br />
0& \mbox{otherwise}.<br />
\end{cases}<br />
</math><br />
:Let <math>Y_0=\mathbf{E}[f(G)]</math> and for <math>i=1,\ldots,m</math>, let <math>Y_i=\mathbf{E}[f(G)\mid X_1,\ldots,X_i]</math>.<br />
:The sequence <math>Y_0,Y_1,\ldots,Y_n</math> gives a Doob martingale that is commonly called the '''edge exposure martingale'''.<br />
<br />
;vertex exposure martingale<br />
: Instead of revealing edges one at a time, we could reveal the set of edges connected to a given vertex, one vertex at a time. Suppose that the vertex set is <math>[n]</math>. Let <math>X_i</math> be the subgraph of <math>G</math> induced by the vertex set <math>[i]</math>, i.e. the first <math>i</math> vertices.<br />
:Let <math>Y_0=\mathbf{E}[f(G)]</math> and for <math>i=1,\ldots,n</math>, let <math>Y_i=\mathbf{E}[f(G)\mid X_1,\ldots,X_i]</math>.<br />
:The sequence <math>Y_0,Y_1,\ldots,Y_n</math> gives a Doob martingale that is commonly called the '''vertex exposure martingale'''.<br />
<br />
===Chromatic number===<br />
The random graph <math>G(n,p)</math> is the graph on <math>n</math> vertices <math>[n]</math>, obtained by selecting each pair of vertices to be an edge, randomly and independently, with probability <math>p</math>. We denote <math>G\sim G(n,p)</math> if <math>G</math> is generated in this way.<br />
<br />
{{Theorem<br />
|Theorem [Shamir and Spencer (1987)]|<br />
:Let <math>G\sim G(n,p)</math>. Let <math>\chi(G)</math> be the chromatic number of <math>G</math>. Then<br />
::<math>\begin{align}<br />
\Pr\left[|\chi(G)-\mathbf{E}[\chi(G)]|\ge t\sqrt{n}\right]\le 2e^{-t^2/2}.<br />
\end{align}</math><br />
}}<br />
{{Proof| Consider the vertex exposure martingale <br />
:<math><br />
Y_i=\mathbf{E}[\chi(G)\mid X_1,\ldots,X_i]<br />
</math><br />
where each <math>X_k</math> exposes the induced subgraph of <math>G</math> on vertex set <math>[k]</math>. A single vertex can always be given a new color so that the graph is properly colored, thus the bounded difference condition <br />
:<math><br />
|Y_i-Y_{i-1}|\le 1<br />
</math><br />
is satisfied. Now apply the Azuma's inequality for the martingale <math>Y_1,\ldots,Y_n</math> with respect to <math>X_1,\ldots,X_n</math>.<br />
}}<br />
<br />
For <math>t=\omega(1)</math>, the theorem states that the chromatic number of a random graph is tightly concentrated around its mean. The proof gives no clue as to where the mean is. This actually shows how powerful the martingale inequalities are: we can prove that a distribution is concentrated to its expectation without actually knowing the expectation. <br />
<br />
=== Hoeffding's Inequality===<br />
The following theorem states the so-called Hoeffding's inequality. It is a generalized version of the Chernoff bounds. Recall that the Chernoff bounds hold for the sum of independent ''trials''. When the random variables are not trials, the Hoeffding's inequality is useful, since it holds for the sum of any independent random variables whose ranges are bounded.<br />
{{Theorem<br />
|Hoeffding's inequality|<br />
: Let <math>X=\sum_{i=1}^nX_i</math>, where <math>X_1,\ldots,X_n</math> are independent random variables with <math>a_i\le X_i\le b_i</math> for each <math>1\le i\le n</math>. Let <math>\mu=\mathbf{E}[X]</math>. Then<br />
::<math><br />
\Pr[|X-\mu|\ge t]\le 2\exp\left(-\frac{t^2}{2\sum_{i=1}^n(b_i-a_i)^2}\right).<br />
</math><br />
}}<br />
{{Proof| Define the Doob martingale sequence <math>Y_i=\mathbf{E}\left[\sum_{j=1}^n X_j\,\Big|\, X_1,\ldots,X_{i}\right]</math>. Obviously <math>Y_0=\mu</math> and <math>Y_n=X</math>.<br />
<br />
:<math><br />
\begin{align}<br />
|Y_i-Y_{i-1}|<br />
&=<br />
\left|\mathbf{E}\left[\sum_{j=1}^n X_j\,\Big|\, X_0,\ldots,X_{i}\right]-\mathbf{E}\left[\sum_{j=1}^n X_j\,\Big|\, X_0,\ldots,X_{i-1}\right]\right|\\<br />
&=\left|\sum_{j=1}^i X_i+\sum_{j=i+1}^n\mathbf{E}[X_j]-\sum_{j=1}^{i-1} X_i-\sum_{j=i}^n\mathbf{E}[X_j]\right|\\<br />
&=\left|X_i-\mathbf{E}[X_{i}]\right|\\<br />
&\le b_i-a_i<br />
\end{align}<br />
</math><br />
Apply Azuma's inequality for the martingale <math>Y_0,\ldots,Y_n</math> with respect to <math>X_1,\ldots, X_n</math>, the Hoeffding's inequality is proved.<br />
}}<br />
<br />
==The Bounded Difference Method==<br />
Combining Azuma's inequality with the construction of Doob martingales, we have the powerful ''Bounded Difference Method'' for concentration of measures.<br />
<br />
=== For arbitrary random variables ===<br />
Given a sequence of random variables <math>X_1,\ldots,X_n</math> and a function <math>f</math>. The Doob sequence constructs a martingale from them. Combining this construction with Azuma's inequality, we can get a very powerful theorem called "the method of averaged bounded differences" which bounds the concentration for arbitrary function on arbitrary random variables (not necessarily a martingale).<br />
<br />
{{Theorem<br />
|Theorem (Method of averaged bounded differences)|<br />
:Let <math>\boldsymbol{X}=(X_1,\ldots, X_n)</math> be arbitrary random variables and let <math>f</math> be a function of <math>X_0,X_1,\ldots, X_n</math> satisfying that, for all <math>1\le i\le n</math>,<br />
::<math><br />
|\mathbf{E}[f(\boldsymbol{X})\mid X_1,\ldots,X_i]-\mathbf{E}[f(\boldsymbol{X})\mid X_1,\ldots,X_{i-1}]|\le c_i,<br />
</math><br />
:Then<br />
::<math>\begin{align}<br />
\Pr\left[|f(\boldsymbol{X})-\mathbf{E}[f(\boldsymbol{X})]|\ge t\right]\le 2\exp\left(-\frac{t^2}{2\sum_{i=1}^nc_i^2}\right).<br />
\end{align}</math><br />
}}<br />
{{Proof| Define the Doob Martingale sequence <math>Y_0,Y_1,\ldots,Y_n</math> by setting <math>Y_0=\mathbf{E}[f(X_1,\ldots,X_n)]</math> and, for <math>1\le i\le n</math>, <math>Y_i=\mathbf{E}[f(X_1,\ldots,X_n)\mid X_1,\ldots,X_i]</math>. Then the above theorem is a restatement of the Azuma's inequality holding for <math>Y_0,Y_1,\ldots,Y_n</math>.<br />
}}<br />
<br />
=== For independent random variables ===<br />
The condition of bounded averaged differences is usually hard to check. This severely limits the usefulness of the method. To overcome this, we introduce a property which is much easier to check, called the Lipschitz condition.<br />
<br />
{{Theorem<br />
|Definition (Lipschitz condition)|<br />
:A function <math>f(x_1,\ldots,x_n)</math> satisfies the Lipschitz condition, if for any <math>x_1,\ldots,x_n</math> and any <math>y_i</math>,<br />
::<math>\begin{align}<br />
|f(x_1,\ldots,x_{i-1},x_i,x_{i+1},\ldots,x_n)-f(x_1,\ldots,x_{i-1},y_i,x_{i+1},\ldots,x_n)|\le 1.<br />
\end{align}</math><br />
}}<br />
In other words, the function satisfies the Lipschitz condition if an arbitrary change in the value of any one argument does not change the value of the function by more than 1. <br />
<br />
The diference of 1 can be replaced by arbitrary constants, which gives a generalized version of Lipschitz condition.<br />
{{Theorem<br />
|Definition (Lipschitz condition, general version)|<br />
:A function <math>f(x_1,\ldots,x_n)</math> satisfies the Lipschitz condition with constants <math>c_i</math>, <math>1\le i\le n</math>, if for any <math>x_1,\ldots,x_n</math> and any <math>y_i</math>,<br />
::<math>\begin{align}<br />
|f(x_1,\ldots,x_{i-1},x_i,x_{i+1},\ldots,x_n)-f(x_1,\ldots,x_{i-1},y_i,x_{i+1},\ldots,x_n)|\le c_i.<br />
\end{align}</math><br />
}}<br />
<br />
The following "method of bounded differences" can be developed for functions satisfying the Lipschitz condition. Unfortunately, in order to imply the condition of averaged bounded differences from the Lipschitz condition, we have to restrict the method to independent random variables.<br />
{{Theorem<br />
|Corollary (Method of bounded differences)|<br />
:Let <math>\boldsymbol{X}=(X_1,\ldots, X_n)</math> be <math>n</math> '''independent''' random variables and let <math>f</math> be a function satisfying the Lipschitz condition with constants <math>c_i</math>, <math>1\le i\le n</math>. Then<br />
::<math>\begin{align}<br />
\Pr\left[|f(\boldsymbol{X})-\mathbf{E}[f(\boldsymbol{X})]|\ge t\right]\le 2\exp\left(-\frac{t^2}{2\sum_{i=1}^nc_i^2}\right).<br />
\end{align}</math><br />
}}<br />
<br />
{{Proof| For convenience, we denote that <math>\boldsymbol{X}_{[i,j]}=(X_i,X_{i+1},\ldots, X_j)</math> for any <math>1\le i\le j\le n</math>.<br />
<br />
We first show that the Lipschitz condition with constants <math>c_i</math>, <math>1\le i\le n</math>, implies another condition called the averaged Lipschitz condition (ALC): for any <math>a_i,b_i</math>, <math>1\le i\le n</math>,<br />
:<math><br />
\left|\mathbf{E}\left[f(\boldsymbol{X})\mid \boldsymbol{X}_{[1,i-1]},X_i=a_i\right]-\mathbf{E}\left[f(\boldsymbol{X})\mid \boldsymbol{X}_{[1,i-1]},X_i=b_i\right]\right|\le c_i.<br />
</math><br />
And this condition implies the averaged bounded difference condition: for all <math>1\le i\le n</math>,<br />
::<math><br />
\left|\mathbf{E}\left[f(\boldsymbol{X})\mid \boldsymbol{X}_{[1,i]}\right]-\mathbf{E}\left[f(\boldsymbol{X})\mid \boldsymbol{X}_{[1,i-1]}\right]\right|\le c_i.<br />
</math><br />
Then by applying the method of averaged bounded differences, the corollary can be proved.<br />
<br />
For any <math>a</math>, by the law of total expectation,<br />
:<math><br />
\begin{align}<br />
&\quad\, \mathbf{E}\left[f(\boldsymbol{X})\mid \boldsymbol{X}_{[1,i-1]},X_i=a\right]\\<br />
&=\sum_{a_{i+1},\ldots,a_n}\mathbf{E}\left[f(\boldsymbol{X})\mid \boldsymbol{X}_{[1,i-1]},X_i=a, \boldsymbol{X}_{[i+1,n]}=\boldsymbol{a}_{[i+1,n]}\right]\cdot\Pr\left[\boldsymbol{X}_{[i+1,n]}=\boldsymbol{a}_{[i+1,n]}\mid \boldsymbol{X}_{[1,i-1]},X_i=a\right]\\<br />
&=\sum_{a_{i+1},\ldots,a_n}\mathbf{E}\left[f(\boldsymbol{X})\mid \boldsymbol{X}_{[1,i-1]},X_i=a, \boldsymbol{X}_{[i+1,n]}=\boldsymbol{a}_{[i+1,n]}\right]\cdot\Pr\left[\boldsymbol{X}_{[i+1,n]}=\boldsymbol{a}_{[i+1,n]}\right] \qquad (\mbox{independence})\\<br />
&= \sum_{a_{i+1},\ldots,a_n} f(\boldsymbol{X}_{[1,i-1]},a,\boldsymbol{a}_{[i+1,n]})\cdot\Pr\left[\boldsymbol{X}_{[i+1,n]}=\boldsymbol{a}_{[i+1,n]}\right].<br />
\end{align}<br />
</math><br />
<br />
Let <math>a=a_i</math> and <math>b_i</math>, and take the diference. Then<br />
:<math><br />
\begin{align}<br />
&\quad\, \left|\mathbf{E}\left[f(\boldsymbol{X})\mid \boldsymbol{X}_{[1,i-1]},X_i=a_i\right]-\mathbf{E}\left[f(\boldsymbol{X})\mid \boldsymbol{X}_{[1,i-1]},X_i=b_i\right]\right|\\<br />
&=\left|\sum_{a_{i+1},\ldots,a_n}\left(f(\boldsymbol{X}_{[1,i-1]},a_i,\boldsymbol{a}_{[i+1,n]})-f(\boldsymbol{X}_{[1,i-1]},b_i,\boldsymbol{a}_{[i+1,n]})\right)\Pr\left[\boldsymbol{X}_{[i+1,n]}=\boldsymbol{a}_{[i+1,n]}\right]\right|\\<br />
&\le \sum_{a_{i+1},\ldots,a_n}\left|f(\boldsymbol{X}_{[1,i-1]},a_i,\boldsymbol{a}_{[i+1,n]})-f(\boldsymbol{X}_{[1,i-1]},b_i,\boldsymbol{a}_{[i+1,n]})\right|\Pr\left[\boldsymbol{X}_{[i+1,n]}=\boldsymbol{a}_{[i+1,n]}\right]\\<br />
&\le \sum_{a_{i+1},\ldots,a_n}c_i\Pr\left[\boldsymbol{X}_{[i+1,n]}=\boldsymbol{a}_{[i+1,n]}\right] \qquad (\mbox{Lipschitz condition})\\<br />
&=c_i.<br />
\end{align}<br />
</math><br />
<br />
Thus, the Lipschitz condition is transformed to the ALC. We then deduce the averaged bounded difference condition from ALC.<br />
<br />
By the law of total expectation,<br />
:<math><br />
\mathbf{E}\left[f(\boldsymbol{X})\mid \boldsymbol{X}_{[1,i-1]}\right]=\sum_{a}\mathbf{E}\left[f(\boldsymbol{X})\mid \boldsymbol{X}_{[1,i-1]},X_i=a\right]\cdot\Pr[X_i=a\mid \boldsymbol{X}_{[1,i-1]}].<br />
</math><br />
<br />
We can trivially write <math>\mathbf{E}\left[f(\boldsymbol{X})\mid \boldsymbol{X}_{[1,i]}\right]</math> as<br />
:<math><br />
\mathbf{E}\left[f(\boldsymbol{X})\mid \boldsymbol{X}_{[1,i]}\right]=\sum_{a}\mathbf{E}\left[f(\boldsymbol{X})\mid \boldsymbol{X}_{[1,i]}\right]\cdot\Pr\left[X_i=a\mid \boldsymbol{X}_{[1,i-1]}\right].<br />
</math><br />
<br />
Hence, the difference is<br />
:<math><br />
\begin{align}<br />
&\quad \left|\mathbf{E}\left[f(\boldsymbol{X})\mid \boldsymbol{X}_{[1,i]}\right]-\mathbf{E}\left[f(\boldsymbol{X})\mid \boldsymbol{X}_{[1,i-1]}\right]\right|\\<br />
&=\left|\sum_{a}\left(\mathbf{E}\left[f(\boldsymbol{X})\mid \boldsymbol{X}_{[1,i]}\right]-\mathbf{E}\left[f(\boldsymbol{X})\mid \boldsymbol{X}_{[1,i-1]},X_i=a\right]\right)\cdot\Pr\left[X_i=a\mid \boldsymbol{X}_{[1,i-1]}\right]\right| \\<br />
&\le \sum_{a}\left|\mathbf{E}\left[f(\boldsymbol{X})\mid \boldsymbol{X}_{[1,i]}\right]-\mathbf{E}\left[f(\boldsymbol{X})\mid \boldsymbol{X}_{[1,i-1]},X_i=a\right]\right|\cdot\Pr\left[X_i=a\mid \boldsymbol{X}_{[1,i-1]}\right] \\<br />
&\le \sum_a c_i\Pr\left[X_i=a\mid \boldsymbol{X}_{[1,i-1]}\right] \qquad (\mbox{due to ALC})\\<br />
&=c_i.<br />
\end{align}<br />
</math><br />
<br />
The averaged bounded diference condition is implied. Applying the method of averaged bounded diferences, the corollary follows.<br />
}}<br />
<br />
=== Applications ===<br />
<br />
==== Occupancy problem ====<br />
Throwing <math>m</math> balls uniformly and independently at random to <math>n</math> bins, we ask for the occupancies of bins by the balls. In particular, we are interested in the number of empty bins.<br />
<br />
This problem can be described equivalently as follows. Let <math>f:[m]\rightarrow[n]</math> be a uniform random function from <math>[m]\rightarrow[n]</math>. We ask for the number of <math>i\in[n]</math> that <math>f^{-1}(i)</math> is empty.<br />
<br />
For any <math>i\in[n]</math>, let <math>X_i</math> indicate the emptiness of bin <math>i</math>. Let <math>X=\sum_{i=1}^nX_i</math> be the number of empty bins.<br />
:<math><br />
\mathbf{E}[X_i]=\Pr[\mbox{bin }i\mbox{ is empty}]=\left(1-\frac{1}{n}\right)^m.<br />
</math><br />
By the linearity of expectation,<br />
:<math><br />
\mathbf{E}[X]=\sum_{i=1}^n\mathbf{E}[X_i]=n\left(1-\frac{1}{n}\right)^m.<br />
</math><br />
<br />
We want to know how <math>X</math> deviates from this expectation. The complication here is that <math>X_i</math> are not independent. So we alternatively look at a sequence of independent random variables <math>Y_1,\ldots, Y_m</math>, where <math>Y_j\in[n]</math> represents the bin into which the <math>j</math>th ball falls. Clearly <math>X</math> is function of <math>Y_1,\ldots, Y_m</math>.<br />
<br />
We than observe that changing the value of any <math>Y_i</math> can change the value of <math>X</math> by at most 1, because one ball can affect the emptiness of at most one bin. <br />
Thus as a function of independent random variables <math>Y_1,\ldots, Y_m</math>, <math>X</math> satisfies the Lipschitz condition. Apply the method of bounded differences, it holds that <br />
:<math><br />
\Pr\left[\left|X-n\left(1-\frac{1}{n}\right)^m\right|\ge t\sqrt{m}\right]=\Pr[|X-\mathbf{E}[X]|\ge t\sqrt{m}]\le 2e^{-t^2/2}<br />
</math><br />
<br />
Thus, for sufficiently large <math>n</math> and <math>m</math>, the number of empty bins is tightly concentrated around <math>n\left(1-\frac{1}{n}\right)^m\approx \frac{n}{e^{m/n}}</math><br />
<br />
==== Pattern Matching ====<br />
Let <math>\boldsymbol{X}=(X_1,\ldots,X_n)</math> be a sequence of characters chosen independently and uniformly at random from an alphabet <math>\Sigma</math>, where <math>m=|\Sigma|</math>. Let <math>\pi\in\Sigma^k</math> be an arbitrarily fixed string of <math>k</math> characters from <math>\Sigma</math>, called a ''pattern''. Let <math>Y</math> be the number of occurrences of the pattern <math>\pi</math> as a substring of the random string <math>X</math>.<br />
<br />
By the linearity of expectation, it is obvious that<br />
:<math><br />
\mathbf{E}[Y]=(n-k+1)\left(\frac{1}{m}\right)^k.<br />
</math><br />
<br />
We now look at the concentration of <math>Y</math>. The complication again lies in the dependencies between the matches. Yet we will see that <math>Y</math> is well tightly concentrated around its expectation if <math>k</math> is relatively small compared to <math>n</math>.<br />
<br />
For a fixed pattern <math>\pi</math>, the random variable <math>Y</math> is a function of the independent random variables <math>(X_1,\ldots,X_n)</math>. Any character <math>X_i</math> participates in no more than <math>k</math> matches, thus changing the value of any <math>X_i</math> can affect the value of <math>Y</math> by at most <math>k</math>. <math>Y</math> satisfies the Lipschitz condition with constant <math>k</math>. Apply the method of bounded differences,<br />
:<math><br />
\Pr\left[\left|Y-\frac{n-k+1}{m^k}\right|\ge tk\sqrt{n}\right]=\Pr\left[\left|Y-\mathbf{E}[Y]\right|\ge tk\sqrt{n}\right]\le 2e^{-t^2/2}<br />
</math><br />
<br />
==== Combining unit vectors ====<br />
Let <math>u_1,\ldots,u_n</math> be <math>n</math> unit vectors from some normed space. That is, <math>\|u_i\|=1</math> for any <math>1\le i\le n</math>, where <math>\|\cdot\|</math> denote the vector norm (e.g. <math>\ell_1,\ell_2,\ell_\infty</math>) of the space.<br />
<br />
Let <math>\epsilon_1,\ldots,\epsilon_n\in\{-1,+1\}</math> be independently chosen and <math>\Pr[\epsilon_i=-1]=\Pr[\epsilon_i=1]=1/2</math>. <br />
<br />
Let <br />
:<math>v=\epsilon_1u_1+\cdots+\epsilon_nu_n,<br />
</math><br />
and <br />
:<math><br />
X=\|v\|.<br />
</math><br />
This kind of construction is very useful in combinatorial proofs of metric problems. We will show that by this construction, the random variable <math>X</math> is well concentrated around its mean.<br />
<br />
<math>X</math> is a function of independent random variables <math>\epsilon_1,\ldots,\epsilon_n</math>.<br />
By the triangle inequality for norms, it is easy to verify that changing the sign of a unit vector <math>u_i</math> can only change the value of <math>X</math> for at most 2, thus <math>X</math> satisfies the Lipschitz condition with constant 2. The concentration result follows by applying the method of bounded differences:<br />
:<math><br />
\Pr[|X-\mathbf{E}[X]|\ge 2t\sqrt{n}]\le 2e^{-t^2/2}.<br />
</math></div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=%E9%AB%98%E7%BA%A7%E7%AE%97%E6%B3%95_(Fall_2026)&diff=13944高级算法 (Fall 2026)2026-09-15T11:59:31Z<p>Etone: /* Lecture Notes */</p>
<hr />
<div>{{Infobox<br />
|name = Infobox<br />
|bodystyle = <br />
|title = <font size=3>高级算法 <br />
<br>Advanced Algorithms</font><br />
|titlestyle = <br />
<br />
|image = <br />
|imagestyle = <br />
|caption = <br />
|captionstyle = <br />
|headerstyle = background:#ccf;<br />
|labelstyle = background:#ddf;<br />
|datastyle = <br />
<br />
|header1 =Instructor<br />
|label1 = <br />
|data1 = <br />
|header2 = <br />
|label2 = <br />
|data2 = '''尹一通'''<br />
|header3 = <br />
|label3 = Email<br />
|data3 = yinyt@nju.edu.cn <br />
|header4 =<br />
|label4= office<br />
|data4= 计算机系 804<br />
|header5 = <br />
|label5 = <br />
|data5 = '''栗师'''<br />
|header6 = <br />
|label6 = Email<br />
|data6 = shili@nju.edu.cn <br />
|header7 =<br />
|label7= office<br />
|data7= 计算机系 605<br />
|header8 = <br />
|label8 = <br />
|data8 = '''刘景铖'''<br />
|header9 = <br />
|label9 = Email<br />
|data9 = liu@nju.edu.cn <br />
|header10 =<br />
|label10= office<br />
|data10= 计算机系 516<br />
|header11 = Class<br />
|label11 = <br />
|data11 = <br />
|header12 =<br />
|label12 = Class meetings<br />
|data12 = 周一 5-6节 (单) 仙Ⅰ-319<br />
周三 5-6节 仙Ⅰ-319<br />
|header13 =<br />
|label13 = Place<br />
|data13 = <br />
|header14 =<br />
|label14 = Office hours<br />
|data14 = 周一4-5pm（尹一通 804）<br/><br />
周四4-5pm（刘景铖 516）<br/><br />
By appointment<br />
|header15 = Textbooks<br />
|label15 = <br />
|data15 = <br />
|header16 =<br />
|label16 = <br />
|data16 = [[File:MR-randomized-algorithms.png|border|100px]]<br />
|header17 =<br />
|label17 = <br />
|data17 = Motwani and Raghavan. <br>''Randomized Algorithms''.<br> Cambridge Univ Press, 1995.<br />
|header18 =<br />
|label18 = <br />
|data18 = [[File:Approximation_Algorithms.jpg|border|100px]]<br />
|header19 =<br />
|label19 = <br />
|data19 = Vazirani. <br>''Approximation Algorithms''. <br> Springer-Verlag, 2001.<br />
|belowstyle = background:#ddf;<br />
|below = <br />
}}<br />
<br />
This is the webpage for the ''Advanced Algorithms'' class of fall 2026. Students who take this class should check this page periodically for content updates and new announcements. <br />
<br />
= Announcement =<br />
<br />
= Course info =<br />
* '''Instructor ''': <br />
:* [http://tcs.nju.edu.cn/yinyt/ 尹一通]：[mailto:yinyt@nju.edu.cn <yinyt@nju.edu.cn>]，计算机系 804 <br />
:*[https://tcs.nju.edu.cn/shili/ 栗师]：[mailto:shili@nju.edu.cn <shili@nju.edu.cn>]，计算机系 605<br />
:* [https://liuexp.github.io 刘景铖]：[mailto:liu@nju.edu.cn <liu@nju.edu.cn>]，计算机系 516 <br />
* '''Teaching Assistant''': <br />
** 于逸潇：[mailto:yixiaoyu@smail.nju.edu.cn <yixiaoyu@smail.nju.edu.cn>]<br />
** 张弈垚：[mailto:zhangyiyao@smail.nju.edu.cn <zhangyiyao@smail.nju.edu.cn>]<br />
* '''Class meeting''': <br />
** 周一 5-6节 1-17周(单) 仙Ⅰ-319<br />
** 周三 5-6节 1-18周 仙Ⅰ-319<br />
* '''Office hour''': <br />
** 周一4-5pm（尹一通 804）<br />
** 周四4-5pm（刘景铖 516）<br />
** By appointment<br />
* '''QQ群''': 1098567018<br />
<br />
= Syllabus =<br />
随着计算机算法理论的不断发展，现代计算机算法的设计与分析大量地使用非初等的数学工具以及非传统的算法思想。“高级算法”这门课程就是面向计算机算法的这一发展趋势而设立的。课程将针对传统算法课程未系统涉及、却在计算机科学各领域的科研和实践中扮演重要角色的高等算法设计思想和算法分析工具进行系统讲授。<br />
<br />
=== 先修课程 Prerequisites ===<br />
* 必须：离散数学，概率论，线性代数。<br />
* 推荐：算法设计与分析。<br />
<br />
=== Course materials ===<br />
* [[高级算法 (Fall 2026) / Course materials|<font size=3>教材和参考书</font>]]<br />
<br />
=== 成绩 Grades ===<br />
* 课程成绩：本课程将会有若干次作业和一次期末考试。最终成绩将由平时作业成绩和期末考试成绩综合得出。<br />
* 迟交：如果有特殊的理由，无法按时完成作业，请提前联系授课老师，给出正当理由。否则迟交的作业将不被接受。<br />
<br />
=== <font color=red> 学术诚信 Academic Integrity </font>===<br />
学术诚信是所有从事学术活动的学生和学者最基本的职业道德底线，本课程将不遗余力的维护学术诚信规范，违反这一底线的行为将不会被容忍。<br />
<br />
作业完成的原则：署你名字的工作必须是你个人的贡献。在完成作业的过程中，允许讨论，前提是讨论的所有参与者均处于同等完成度。但关键想法的执行、以及作业文本的写作必须独立完成，并在作业中致谢（acknowledge）所有参与讨论的人。不允许其他任何形式的合作——尤其是与已经完成作业的同学“讨论”。<br />
<br />
本课程将对剽窃行为采取零容忍的态度。在完成作业过程中，对他人工作（出版物、互联网资料、其他人的作业等）直接的文本抄袭和对关键思想、关键元素的抄袭，按照 [http://www.acm.org/publications/policies/plagiarism_policy ACM Policy on Plagiarism]的解释，都将视为剽窃。剽窃者成绩将被取消。如果发现互相抄袭行为，<font color=red> 抄袭和被抄袭双方的成绩都将被取消</font>。因此请主动防止自己的作业被他人抄袭。<br />
<br />
学术诚信影响学生个人的品行，也关乎整个教育系统的正常运转。为了一点分数而做出学术不端的行为，不仅使自己沦为一个欺骗者，也使他人的诚实努力失去意义。让我们一起努力维护一个诚信的环境。<br />
<br />
= Assignments =<br />
Late policy: In general, we will accomodate late submission requests ONLY IF you made such requests ahead of time. <br />
<br />
<br />
= Lecture Notes =<br />
# [[高级算法 (Fall 2025)/Min Cut, Max Cut, and Spectral Cut|Min Cut, Max Cut, and Spectral Cut]] ([http://tcs.nju.edu.cn/slides/aa2026/Cut.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-cuts-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-cuts-zh.pdf AI生成讲义])<br />
#* [[高级算法 (Fall 2025)/Probability Basics|Probability basics]]<br />
# [[高级算法 (Fall 2026)/Fingerprinting| Fingerprinting]] ([http://tcs.nju.edu.cn/slides/aa2026/Fingerprinting.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-fingerprinting-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-fingerprinting-zh.pdf AI生成讲义])<br />
#* [[高级算法 (Fall 2026)/Finite Field Basics|Finite field basics]]<br />
# [[高级算法 (Fall 2026)/Hashing and Sketching|Hashing and Sketching]] ([http://tcs.nju.edu.cn/slides/aa2026/Hashing.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-hashing-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-hashing-zh.pdf AI生成讲义]) <br />
#* [[高级算法 (Fall 2026)/Limited independence|Limited independence]]<br />
#* [[高级算法 (Fall 2026)/Basic deviation inequalities|Basic deviation inequalities]]<br />
# [[高级算法 (Fall 2026)/Concentration of measure|Concentration of measure]] <br />
#* [[高级算法 (Fall 2026)/Conditional expectations|Conditional expectations]]<br />
<br />
= Related Online Courses=<br />
* [https://www.cs.cmu.edu/~15850/ Advanced Algorithms] by Anupam Gupta at CMU.<br />
* [http://people.csail.mit.edu/moitra/854.html Advanced Algorithms] by Ankur Moitra at MIT.<br />
* [http://courses.csail.mit.edu/6.854/current/ Advanced Algorithms] by David Karger and Aleksander Mądry at MIT.<br />
* [http://web.stanford.edu/class/cs168/index.html The Modern Algorithmic Toolbox] by Tim Roughgarden and Gregory Valiant at Stanford.<br />
* [https://www.cs.princeton.edu/courses/archive/fall18/cos521/ Advanced Algorithm Design] by Pravesh Kothari and Christopher Musco at Princeton.<br />
* [http://www.cs.cmu.edu/afs/cs.cmu.edu/academic/class/15859-f11/www/ Linear and Semidefinite Programming (Advanced Algorithms)] by Anupam Gupta and Ryan O'Donnell at CMU.<br />
* [https://www.cs.cmu.edu/~odonnell/papers/cs-theory-toolkit-lecture-notes.pdf CS Theory Toolkit] by Ryan O'Donnell at CMU.<br />
* [https://cs.uwaterloo.ca/~lapchi/cs860/index.html Eigenvalues and Polynomials] by Lap Chi Lau at University of Waterloo.<br />
* The [https://www.cs.cornell.edu/jeh/book.pdf "Foundations of Data Science" book] by Avrim Blum, John Hopcroft, and Ravindran Kannan.</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=%E9%AB%98%E7%BA%A7%E7%AE%97%E6%B3%95_(Fall_2026)&diff=13931高级算法 (Fall 2026)2026-09-09T08:30:10Z<p>Etone: /* Lecture Notes */</p>
<hr />
<div>{{Infobox<br />
|name = Infobox<br />
|bodystyle = <br />
|title = <font size=3>高级算法 <br />
<br>Advanced Algorithms</font><br />
|titlestyle = <br />
<br />
|image = <br />
|imagestyle = <br />
|caption = <br />
|captionstyle = <br />
|headerstyle = background:#ccf;<br />
|labelstyle = background:#ddf;<br />
|datastyle = <br />
<br />
|header1 =Instructor<br />
|label1 = <br />
|data1 = <br />
|header2 = <br />
|label2 = <br />
|data2 = '''尹一通'''<br />
|header3 = <br />
|label3 = Email<br />
|data3 = yinyt@nju.edu.cn <br />
|header4 =<br />
|label4= office<br />
|data4= 计算机系 804<br />
|header5 = <br />
|label5 = <br />
|data5 = '''栗师'''<br />
|header6 = <br />
|label6 = Email<br />
|data6 = shili@nju.edu.cn <br />
|header7 =<br />
|label7= office<br />
|data7= 计算机系 605<br />
|header8 = <br />
|label8 = <br />
|data8 = '''刘景铖'''<br />
|header9 = <br />
|label9 = Email<br />
|data9 = liu@nju.edu.cn <br />
|header10 =<br />
|label10= office<br />
|data10= 计算机系 516<br />
|header11 = Class<br />
|label11 = <br />
|data11 = <br />
|header12 =<br />
|label12 = Class meetings<br />
|data12 = 周一 5-6节 (单) 仙Ⅰ-319<br />
周三 5-6节 仙Ⅰ-319<br />
|header13 =<br />
|label13 = Place<br />
|data13 = <br />
|header14 =<br />
|label14 = Office hours<br />
|data14 = 周一4-5pm（尹一通 804）<br/><br />
周四4-5pm（刘景铖 516）<br/><br />
By appointment<br />
|header15 = Textbooks<br />
|label15 = <br />
|data15 = <br />
|header16 =<br />
|label16 = <br />
|data16 = [[File:MR-randomized-algorithms.png|border|100px]]<br />
|header17 =<br />
|label17 = <br />
|data17 = Motwani and Raghavan. <br>''Randomized Algorithms''.<br> Cambridge Univ Press, 1995.<br />
|header18 =<br />
|label18 = <br />
|data18 = [[File:Approximation_Algorithms.jpg|border|100px]]<br />
|header19 =<br />
|label19 = <br />
|data19 = Vazirani. <br>''Approximation Algorithms''. <br> Springer-Verlag, 2001.<br />
|belowstyle = background:#ddf;<br />
|below = <br />
}}<br />
<br />
This is the webpage for the ''Advanced Algorithms'' class of fall 2026. Students who take this class should check this page periodically for content updates and new announcements. <br />
<br />
= Announcement =<br />
<br />
= Course info =<br />
* '''Instructor ''': <br />
:* [http://tcs.nju.edu.cn/yinyt/ 尹一通]：[mailto:yinyt@nju.edu.cn <yinyt@nju.edu.cn>]，计算机系 804 <br />
:*[https://tcs.nju.edu.cn/shili/ 栗师]：[mailto:shili@nju.edu.cn <shili@nju.edu.cn>]，计算机系 605<br />
:* [https://liuexp.github.io 刘景铖]：[mailto:liu@nju.edu.cn <liu@nju.edu.cn>]，计算机系 516 <br />
* '''Teaching Assistant''': <br />
** 于逸潇：[mailto:yixiaoyu@smail.nju.edu.cn <yixiaoyu@smail.nju.edu.cn>]<br />
** 张弈垚：[mailto:zhangyiyao@smail.nju.edu.cn <zhangyiyao@smail.nju.edu.cn>]<br />
* '''Class meeting''': <br />
** 周一 5-6节 1-17周(单) 仙Ⅰ-319<br />
** 周三 5-6节 1-18周 仙Ⅰ-319<br />
* '''Office hour''': <br />
** 周一4-5pm（尹一通 804）<br />
** 周四4-5pm（刘景铖 516）<br />
** By appointment<br />
* '''QQ群''': 1098567018<br />
<br />
= Syllabus =<br />
随着计算机算法理论的不断发展，现代计算机算法的设计与分析大量地使用非初等的数学工具以及非传统的算法思想。“高级算法”这门课程就是面向计算机算法的这一发展趋势而设立的。课程将针对传统算法课程未系统涉及、却在计算机科学各领域的科研和实践中扮演重要角色的高等算法设计思想和算法分析工具进行系统讲授。<br />
<br />
=== 先修课程 Prerequisites ===<br />
* 必须：离散数学，概率论，线性代数。<br />
* 推荐：算法设计与分析。<br />
<br />
=== Course materials ===<br />
* [[高级算法 (Fall 2026) / Course materials|<font size=3>教材和参考书</font>]]<br />
<br />
=== 成绩 Grades ===<br />
* 课程成绩：本课程将会有若干次作业和一次期末考试。最终成绩将由平时作业成绩和期末考试成绩综合得出。<br />
* 迟交：如果有特殊的理由，无法按时完成作业，请提前联系授课老师，给出正当理由。否则迟交的作业将不被接受。<br />
<br />
=== <font color=red> 学术诚信 Academic Integrity </font>===<br />
学术诚信是所有从事学术活动的学生和学者最基本的职业道德底线，本课程将不遗余力的维护学术诚信规范，违反这一底线的行为将不会被容忍。<br />
<br />
作业完成的原则：署你名字的工作必须是你个人的贡献。在完成作业的过程中，允许讨论，前提是讨论的所有参与者均处于同等完成度。但关键想法的执行、以及作业文本的写作必须独立完成，并在作业中致谢（acknowledge）所有参与讨论的人。不允许其他任何形式的合作——尤其是与已经完成作业的同学“讨论”。<br />
<br />
本课程将对剽窃行为采取零容忍的态度。在完成作业过程中，对他人工作（出版物、互联网资料、其他人的作业等）直接的文本抄袭和对关键思想、关键元素的抄袭，按照 [http://www.acm.org/publications/policies/plagiarism_policy ACM Policy on Plagiarism]的解释，都将视为剽窃。剽窃者成绩将被取消。如果发现互相抄袭行为，<font color=red> 抄袭和被抄袭双方的成绩都将被取消</font>。因此请主动防止自己的作业被他人抄袭。<br />
<br />
学术诚信影响学生个人的品行，也关乎整个教育系统的正常运转。为了一点分数而做出学术不端的行为，不仅使自己沦为一个欺骗者，也使他人的诚实努力失去意义。让我们一起努力维护一个诚信的环境。<br />
<br />
= Assignments =<br />
Late policy: In general, we will accomodate late submission requests ONLY IF you made such requests ahead of time. <br />
<br />
<br />
= Lecture Notes =<br />
# [[高级算法 (Fall 2025)/Min Cut, Max Cut, and Spectral Cut|Min Cut, Max Cut, and Spectral Cut]] ([http://tcs.nju.edu.cn/slides/aa2026/Cut.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-cuts-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-cuts-zh.pdf AI生成讲义])<br />
#* [[高级算法 (Fall 2025)/Probability Basics|Probability basics]]<br />
# [[高级算法 (Fall 2026)/Fingerprinting| Fingerprinting]] ([http://tcs.nju.edu.cn/slides/aa2026/Fingerprinting.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-fingerprinting-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-fingerprinting-zh.pdf AI生成讲义])<br />
#* [[高级算法 (Fall 2026)/Finite Field Basics|Finite field basics]]<br />
# [[高级算法 (Fall 2026)/Hashing and Sketching|Hashing and Sketching]] ([http://tcs.nju.edu.cn/slides/aa2026/Hashing.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-hashing-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-hashing-zh.pdf AI生成讲义]) <br />
#* [[高级算法 (Fall 2026)/Limited independence|Limited independence]]<br />
#* [[高级算法 (Fall 2026)/Basic deviation inequalities|Basic deviation inequalities]]<br />
<br />
= Related Online Courses=<br />
* [https://www.cs.cmu.edu/~15850/ Advanced Algorithms] by Anupam Gupta at CMU.<br />
* [http://people.csail.mit.edu/moitra/854.html Advanced Algorithms] by Ankur Moitra at MIT.<br />
* [http://courses.csail.mit.edu/6.854/current/ Advanced Algorithms] by David Karger and Aleksander Mądry at MIT.<br />
* [http://web.stanford.edu/class/cs168/index.html The Modern Algorithmic Toolbox] by Tim Roughgarden and Gregory Valiant at Stanford.<br />
* [https://www.cs.princeton.edu/courses/archive/fall18/cos521/ Advanced Algorithm Design] by Pravesh Kothari and Christopher Musco at Princeton.<br />
* [http://www.cs.cmu.edu/afs/cs.cmu.edu/academic/class/15859-f11/www/ Linear and Semidefinite Programming (Advanced Algorithms)] by Anupam Gupta and Ryan O'Donnell at CMU.<br />
* [https://www.cs.cmu.edu/~odonnell/papers/cs-theory-toolkit-lecture-notes.pdf CS Theory Toolkit] by Ryan O'Donnell at CMU.<br />
* [https://cs.uwaterloo.ca/~lapchi/cs860/index.html Eigenvalues and Polynomials] by Lap Chi Lau at University of Waterloo.<br />
* The [https://www.cs.cornell.edu/jeh/book.pdf "Foundations of Data Science" book] by Avrim Blum, John Hopcroft, and Ravindran Kannan.</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=%E9%AB%98%E7%BA%A7%E7%AE%97%E6%B3%95_(Fall_2026)&diff=13930高级算法 (Fall 2026)2026-09-09T08:29:22Z<p>Etone: /* Lecture Notes */</p>
<hr />
<div>{{Infobox<br />
|name = Infobox<br />
|bodystyle = <br />
|title = <font size=3>高级算法 <br />
<br>Advanced Algorithms</font><br />
|titlestyle = <br />
<br />
|image = <br />
|imagestyle = <br />
|caption = <br />
|captionstyle = <br />
|headerstyle = background:#ccf;<br />
|labelstyle = background:#ddf;<br />
|datastyle = <br />
<br />
|header1 =Instructor<br />
|label1 = <br />
|data1 = <br />
|header2 = <br />
|label2 = <br />
|data2 = '''尹一通'''<br />
|header3 = <br />
|label3 = Email<br />
|data3 = yinyt@nju.edu.cn <br />
|header4 =<br />
|label4= office<br />
|data4= 计算机系 804<br />
|header5 = <br />
|label5 = <br />
|data5 = '''栗师'''<br />
|header6 = <br />
|label6 = Email<br />
|data6 = shili@nju.edu.cn <br />
|header7 =<br />
|label7= office<br />
|data7= 计算机系 605<br />
|header8 = <br />
|label8 = <br />
|data8 = '''刘景铖'''<br />
|header9 = <br />
|label9 = Email<br />
|data9 = liu@nju.edu.cn <br />
|header10 =<br />
|label10= office<br />
|data10= 计算机系 516<br />
|header11 = Class<br />
|label11 = <br />
|data11 = <br />
|header12 =<br />
|label12 = Class meetings<br />
|data12 = 周一 5-6节 (单) 仙Ⅰ-319<br />
周三 5-6节 仙Ⅰ-319<br />
|header13 =<br />
|label13 = Place<br />
|data13 = <br />
|header14 =<br />
|label14 = Office hours<br />
|data14 = 周一4-5pm（尹一通 804）<br/><br />
周四4-5pm（刘景铖 516）<br/><br />
By appointment<br />
|header15 = Textbooks<br />
|label15 = <br />
|data15 = <br />
|header16 =<br />
|label16 = <br />
|data16 = [[File:MR-randomized-algorithms.png|border|100px]]<br />
|header17 =<br />
|label17 = <br />
|data17 = Motwani and Raghavan. <br>''Randomized Algorithms''.<br> Cambridge Univ Press, 1995.<br />
|header18 =<br />
|label18 = <br />
|data18 = [[File:Approximation_Algorithms.jpg|border|100px]]<br />
|header19 =<br />
|label19 = <br />
|data19 = Vazirani. <br>''Approximation Algorithms''. <br> Springer-Verlag, 2001.<br />
|belowstyle = background:#ddf;<br />
|below = <br />
}}<br />
<br />
This is the webpage for the ''Advanced Algorithms'' class of fall 2026. Students who take this class should check this page periodically for content updates and new announcements. <br />
<br />
= Announcement =<br />
<br />
= Course info =<br />
* '''Instructor ''': <br />
:* [http://tcs.nju.edu.cn/yinyt/ 尹一通]：[mailto:yinyt@nju.edu.cn <yinyt@nju.edu.cn>]，计算机系 804 <br />
:*[https://tcs.nju.edu.cn/shili/ 栗师]：[mailto:shili@nju.edu.cn <shili@nju.edu.cn>]，计算机系 605<br />
:* [https://liuexp.github.io 刘景铖]：[mailto:liu@nju.edu.cn <liu@nju.edu.cn>]，计算机系 516 <br />
* '''Teaching Assistant''': <br />
** 于逸潇：[mailto:yixiaoyu@smail.nju.edu.cn <yixiaoyu@smail.nju.edu.cn>]<br />
** 张弈垚：[mailto:zhangyiyao@smail.nju.edu.cn <zhangyiyao@smail.nju.edu.cn>]<br />
* '''Class meeting''': <br />
** 周一 5-6节 1-17周(单) 仙Ⅰ-319<br />
** 周三 5-6节 1-18周 仙Ⅰ-319<br />
* '''Office hour''': <br />
** 周一4-5pm（尹一通 804）<br />
** 周四4-5pm（刘景铖 516）<br />
** By appointment<br />
* '''QQ群''': 1098567018<br />
<br />
= Syllabus =<br />
随着计算机算法理论的不断发展，现代计算机算法的设计与分析大量地使用非初等的数学工具以及非传统的算法思想。“高级算法”这门课程就是面向计算机算法的这一发展趋势而设立的。课程将针对传统算法课程未系统涉及、却在计算机科学各领域的科研和实践中扮演重要角色的高等算法设计思想和算法分析工具进行系统讲授。<br />
<br />
=== 先修课程 Prerequisites ===<br />
* 必须：离散数学，概率论，线性代数。<br />
* 推荐：算法设计与分析。<br />
<br />
=== Course materials ===<br />
* [[高级算法 (Fall 2026) / Course materials|<font size=3>教材和参考书</font>]]<br />
<br />
=== 成绩 Grades ===<br />
* 课程成绩：本课程将会有若干次作业和一次期末考试。最终成绩将由平时作业成绩和期末考试成绩综合得出。<br />
* 迟交：如果有特殊的理由，无法按时完成作业，请提前联系授课老师，给出正当理由。否则迟交的作业将不被接受。<br />
<br />
=== <font color=red> 学术诚信 Academic Integrity </font>===<br />
学术诚信是所有从事学术活动的学生和学者最基本的职业道德底线，本课程将不遗余力的维护学术诚信规范，违反这一底线的行为将不会被容忍。<br />
<br />
作业完成的原则：署你名字的工作必须是你个人的贡献。在完成作业的过程中，允许讨论，前提是讨论的所有参与者均处于同等完成度。但关键想法的执行、以及作业文本的写作必须独立完成，并在作业中致谢（acknowledge）所有参与讨论的人。不允许其他任何形式的合作——尤其是与已经完成作业的同学“讨论”。<br />
<br />
本课程将对剽窃行为采取零容忍的态度。在完成作业过程中，对他人工作（出版物、互联网资料、其他人的作业等）直接的文本抄袭和对关键思想、关键元素的抄袭，按照 [http://www.acm.org/publications/policies/plagiarism_policy ACM Policy on Plagiarism]的解释，都将视为剽窃。剽窃者成绩将被取消。如果发现互相抄袭行为，<font color=red> 抄袭和被抄袭双方的成绩都将被取消</font>。因此请主动防止自己的作业被他人抄袭。<br />
<br />
学术诚信影响学生个人的品行，也关乎整个教育系统的正常运转。为了一点分数而做出学术不端的行为，不仅使自己沦为一个欺骗者，也使他人的诚实努力失去意义。让我们一起努力维护一个诚信的环境。<br />
<br />
= Assignments =<br />
Late policy: In general, we will accomodate late submission requests ONLY IF you made such requests ahead of time. <br />
<br />
<br />
= Lecture Notes =<br />
# [[高级算法 (Fall 2025)/Min Cut, Max Cut, and Spectral Cut|Min Cut, Max Cut, and Spectral Cut]] ([http://tcs.nju.edu.cn/slides/aa2026/Cut.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-cuts-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-cuts-zh.pdf AI生成讲义])<br />
#* [[高级算法 (Fall 2025)/Probability Basics|Probability basics]]<br />
# [[高级算法 (Fall 2026)/Fingerprinting| Fingerprinting]] ([http://tcs.nju.edu.cn/slides/aa2026/Fingerprinting.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-fingerprinting-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-fingerprinting-zh.pdf AI生成讲义])<br />
#* [[高级算法 (Fall 2026)/Finite Field Basics|Finite field basics]]<br />
# [[高级算法 (Fall 2026)/Hashing and Sketching|Hashing and Sketching]] ([[Media:Hashing-26.pdf|slides]]) ([[Media:AA26-note-hashing-en.pdf|AI-generated lecture note]], [[Media:AA26-note-hashing-zh.pdf|AI生成讲义]]) <br />
#* [[高级算法 (Fall 2026)/Limited independence|Limited independence]]<br />
#* [[高级算法 (Fall 2026)/Basic deviation inequalities|Basic deviation inequalities]]<br />
<br />
= Related Online Courses=<br />
* [https://www.cs.cmu.edu/~15850/ Advanced Algorithms] by Anupam Gupta at CMU.<br />
* [http://people.csail.mit.edu/moitra/854.html Advanced Algorithms] by Ankur Moitra at MIT.<br />
* [http://courses.csail.mit.edu/6.854/current/ Advanced Algorithms] by David Karger and Aleksander Mądry at MIT.<br />
* [http://web.stanford.edu/class/cs168/index.html The Modern Algorithmic Toolbox] by Tim Roughgarden and Gregory Valiant at Stanford.<br />
* [https://www.cs.princeton.edu/courses/archive/fall18/cos521/ Advanced Algorithm Design] by Pravesh Kothari and Christopher Musco at Princeton.<br />
* [http://www.cs.cmu.edu/afs/cs.cmu.edu/academic/class/15859-f11/www/ Linear and Semidefinite Programming (Advanced Algorithms)] by Anupam Gupta and Ryan O'Donnell at CMU.<br />
* [https://www.cs.cmu.edu/~odonnell/papers/cs-theory-toolkit-lecture-notes.pdf CS Theory Toolkit] by Ryan O'Donnell at CMU.<br />
* [https://cs.uwaterloo.ca/~lapchi/cs860/index.html Eigenvalues and Polynomials] by Lap Chi Lau at University of Waterloo.<br />
* The [https://www.cs.cornell.edu/jeh/book.pdf "Foundations of Data Science" book] by Avrim Blum, John Hopcroft, and Ravindran Kannan.</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=%E9%AB%98%E7%BA%A7%E7%AE%97%E6%B3%95_(Fall_2026)&diff=13917高级算法 (Fall 2026)2026-09-09T04:44:07Z<p>Etone: /* Lecture Notes */</p>
<hr />
<div>{{Infobox<br />
|name = Infobox<br />
|bodystyle = <br />
|title = <font size=3>高级算法 <br />
<br>Advanced Algorithms</font><br />
|titlestyle = <br />
<br />
|image = <br />
|imagestyle = <br />
|caption = <br />
|captionstyle = <br />
|headerstyle = background:#ccf;<br />
|labelstyle = background:#ddf;<br />
|datastyle = <br />
<br />
|header1 =Instructor<br />
|label1 = <br />
|data1 = <br />
|header2 = <br />
|label2 = <br />
|data2 = '''尹一通'''<br />
|header3 = <br />
|label3 = Email<br />
|data3 = yinyt@nju.edu.cn <br />
|header4 =<br />
|label4= office<br />
|data4= 计算机系 804<br />
|header5 = <br />
|label5 = <br />
|data5 = '''栗师'''<br />
|header6 = <br />
|label6 = Email<br />
|data6 = shili@nju.edu.cn <br />
|header7 =<br />
|label7= office<br />
|data7= 计算机系 605<br />
|header8 = <br />
|label8 = <br />
|data8 = '''刘景铖'''<br />
|header9 = <br />
|label9 = Email<br />
|data9 = liu@nju.edu.cn <br />
|header10 =<br />
|label10= office<br />
|data10= 计算机系 516<br />
|header11 = Class<br />
|label11 = <br />
|data11 = <br />
|header12 =<br />
|label12 = Class meetings<br />
|data12 = 周一 5-6节 (单) 仙Ⅰ-319<br />
周三 5-6节 仙Ⅰ-319<br />
|header13 =<br />
|label13 = Place<br />
|data13 = <br />
|header14 =<br />
|label14 = Office hours<br />
|data14 = 周一4-5pm（尹一通 804）<br/><br />
周四4-5pm（刘景铖 516）<br/><br />
By appointment<br />
|header15 = Textbooks<br />
|label15 = <br />
|data15 = <br />
|header16 =<br />
|label16 = <br />
|data16 = [[File:MR-randomized-algorithms.png|border|100px]]<br />
|header17 =<br />
|label17 = <br />
|data17 = Motwani and Raghavan. <br>''Randomized Algorithms''.<br> Cambridge Univ Press, 1995.<br />
|header18 =<br />
|label18 = <br />
|data18 = [[File:Approximation_Algorithms.jpg|border|100px]]<br />
|header19 =<br />
|label19 = <br />
|data19 = Vazirani. <br>''Approximation Algorithms''. <br> Springer-Verlag, 2001.<br />
|belowstyle = background:#ddf;<br />
|below = <br />
}}<br />
<br />
This is the webpage for the ''Advanced Algorithms'' class of fall 2026. Students who take this class should check this page periodically for content updates and new announcements. <br />
<br />
= Announcement =<br />
<br />
= Course info =<br />
* '''Instructor ''': <br />
:* [http://tcs.nju.edu.cn/yinyt/ 尹一通]：[mailto:yinyt@nju.edu.cn <yinyt@nju.edu.cn>]，计算机系 804 <br />
:*[https://tcs.nju.edu.cn/shili/ 栗师]：[mailto:shili@nju.edu.cn <shili@nju.edu.cn>]，计算机系 605<br />
:* [https://liuexp.github.io 刘景铖]：[mailto:liu@nju.edu.cn <liu@nju.edu.cn>]，计算机系 516 <br />
* '''Teaching Assistant''': <br />
** 于逸潇：[mailto:yixiaoyu@smail.nju.edu.cn <yixiaoyu@smail.nju.edu.cn>]<br />
** 张弈垚：[mailto:zhangyiyao@smail.nju.edu.cn <zhangyiyao@smail.nju.edu.cn>]<br />
* '''Class meeting''': <br />
** 周一 5-6节 1-17周(单) 仙Ⅰ-319<br />
** 周三 5-6节 1-18周 仙Ⅰ-319<br />
* '''Office hour''': <br />
** 周一4-5pm（尹一通 804）<br />
** 周四4-5pm（刘景铖 516）<br />
** By appointment<br />
* '''QQ群''': 1098567018<br />
<br />
= Syllabus =<br />
随着计算机算法理论的不断发展，现代计算机算法的设计与分析大量地使用非初等的数学工具以及非传统的算法思想。“高级算法”这门课程就是面向计算机算法的这一发展趋势而设立的。课程将针对传统算法课程未系统涉及、却在计算机科学各领域的科研和实践中扮演重要角色的高等算法设计思想和算法分析工具进行系统讲授。<br />
<br />
=== 先修课程 Prerequisites ===<br />
* 必须：离散数学，概率论，线性代数。<br />
* 推荐：算法设计与分析。<br />
<br />
=== Course materials ===<br />
* [[高级算法 (Fall 2026) / Course materials|<font size=3>教材和参考书</font>]]<br />
<br />
=== 成绩 Grades ===<br />
* 课程成绩：本课程将会有若干次作业和一次期末考试。最终成绩将由平时作业成绩和期末考试成绩综合得出。<br />
* 迟交：如果有特殊的理由，无法按时完成作业，请提前联系授课老师，给出正当理由。否则迟交的作业将不被接受。<br />
<br />
=== <font color=red> 学术诚信 Academic Integrity </font>===<br />
学术诚信是所有从事学术活动的学生和学者最基本的职业道德底线，本课程将不遗余力的维护学术诚信规范，违反这一底线的行为将不会被容忍。<br />
<br />
作业完成的原则：署你名字的工作必须是你个人的贡献。在完成作业的过程中，允许讨论，前提是讨论的所有参与者均处于同等完成度。但关键想法的执行、以及作业文本的写作必须独立完成，并在作业中致谢（acknowledge）所有参与讨论的人。不允许其他任何形式的合作——尤其是与已经完成作业的同学“讨论”。<br />
<br />
本课程将对剽窃行为采取零容忍的态度。在完成作业过程中，对他人工作（出版物、互联网资料、其他人的作业等）直接的文本抄袭和对关键思想、关键元素的抄袭，按照 [http://www.acm.org/publications/policies/plagiarism_policy ACM Policy on Plagiarism]的解释，都将视为剽窃。剽窃者成绩将被取消。如果发现互相抄袭行为，<font color=red> 抄袭和被抄袭双方的成绩都将被取消</font>。因此请主动防止自己的作业被他人抄袭。<br />
<br />
学术诚信影响学生个人的品行，也关乎整个教育系统的正常运转。为了一点分数而做出学术不端的行为，不仅使自己沦为一个欺骗者，也使他人的诚实努力失去意义。让我们一起努力维护一个诚信的环境。<br />
<br />
= Assignments =<br />
Late policy: In general, we will accomodate late submission requests ONLY IF you made such requests ahead of time. <br />
<br />
<br />
= Lecture Notes =<br />
# [[高级算法 (Fall 2025)/Min Cut, Max Cut, and Spectral Cut|Min Cut, Max Cut, and Spectral Cut]] ([http://tcs.nju.edu.cn/slides/aa2026/Cut.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-cuts-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-cuts-zh.pdf AI生成讲义])<br />
#* [[高级算法 (Fall 2025)/Probability Basics|Probability basics]]<br />
# [[高级算法 (Fall 2026)/Fingerprinting| Fingerprinting]] ([http://tcs.nju.edu.cn/slides/aa2026/Fingerprinting.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-fingerprinting-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-fingerprinting-zh.pdf AI生成讲义])<br />
#* [[高级算法 (Fall 2026)/Finite Field Basics|Finite field basics]]<br />
# [[高级算法 (Fall 2026)/Hashing and Sketching|Hashing and Sketching]] ([[Media:Hashing-26.pdf|slides]]) ([[Media:AA26-note-hashing-en.pdf|AI-generated lecture note]], [[Media:AA26-note-hashing-zh.pdf|AI生成讲义]])<br />
#* [[高级算法 (Fall 2026)/Limited independence|Limited independence]]<br />
#* [[高级算法 (Fall 2026)/Basic deviation inequalities|Basic deviation inequalities]]<br />
<br />
= Related Online Courses=<br />
* [https://www.cs.cmu.edu/~15850/ Advanced Algorithms] by Anupam Gupta at CMU.<br />
* [http://people.csail.mit.edu/moitra/854.html Advanced Algorithms] by Ankur Moitra at MIT.<br />
* [http://courses.csail.mit.edu/6.854/current/ Advanced Algorithms] by David Karger and Aleksander Mądry at MIT.<br />
* [http://web.stanford.edu/class/cs168/index.html The Modern Algorithmic Toolbox] by Tim Roughgarden and Gregory Valiant at Stanford.<br />
* [https://www.cs.princeton.edu/courses/archive/fall18/cos521/ Advanced Algorithm Design] by Pravesh Kothari and Christopher Musco at Princeton.<br />
* [http://www.cs.cmu.edu/afs/cs.cmu.edu/academic/class/15859-f11/www/ Linear and Semidefinite Programming (Advanced Algorithms)] by Anupam Gupta and Ryan O'Donnell at CMU.<br />
* [https://www.cs.cmu.edu/~odonnell/papers/cs-theory-toolkit-lecture-notes.pdf CS Theory Toolkit] by Ryan O'Donnell at CMU.<br />
* [https://cs.uwaterloo.ca/~lapchi/cs860/index.html Eigenvalues and Polynomials] by Lap Chi Lau at University of Waterloo.<br />
* The [https://www.cs.cornell.edu/jeh/book.pdf "Foundations of Data Science" book] by Avrim Blum, John Hopcroft, and Ravindran Kannan.</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=%E9%AB%98%E7%BA%A7%E7%AE%97%E6%B3%95_(Fall_2026)&diff=13916高级算法 (Fall 2026)2026-09-09T04:43:29Z<p>Etone: /* Lecture Notes */</p>
<hr />
<div>{{Infobox<br />
|name = Infobox<br />
|bodystyle = <br />
|title = <font size=3>高级算法 <br />
<br>Advanced Algorithms</font><br />
|titlestyle = <br />
<br />
|image = <br />
|imagestyle = <br />
|caption = <br />
|captionstyle = <br />
|headerstyle = background:#ccf;<br />
|labelstyle = background:#ddf;<br />
|datastyle = <br />
<br />
|header1 =Instructor<br />
|label1 = <br />
|data1 = <br />
|header2 = <br />
|label2 = <br />
|data2 = '''尹一通'''<br />
|header3 = <br />
|label3 = Email<br />
|data3 = yinyt@nju.edu.cn <br />
|header4 =<br />
|label4= office<br />
|data4= 计算机系 804<br />
|header5 = <br />
|label5 = <br />
|data5 = '''栗师'''<br />
|header6 = <br />
|label6 = Email<br />
|data6 = shili@nju.edu.cn <br />
|header7 =<br />
|label7= office<br />
|data7= 计算机系 605<br />
|header8 = <br />
|label8 = <br />
|data8 = '''刘景铖'''<br />
|header9 = <br />
|label9 = Email<br />
|data9 = liu@nju.edu.cn <br />
|header10 =<br />
|label10= office<br />
|data10= 计算机系 516<br />
|header11 = Class<br />
|label11 = <br />
|data11 = <br />
|header12 =<br />
|label12 = Class meetings<br />
|data12 = 周一 5-6节 (单) 仙Ⅰ-319<br />
周三 5-6节 仙Ⅰ-319<br />
|header13 =<br />
|label13 = Place<br />
|data13 = <br />
|header14 =<br />
|label14 = Office hours<br />
|data14 = 周一4-5pm（尹一通 804）<br/><br />
周四4-5pm（刘景铖 516）<br/><br />
By appointment<br />
|header15 = Textbooks<br />
|label15 = <br />
|data15 = <br />
|header16 =<br />
|label16 = <br />
|data16 = [[File:MR-randomized-algorithms.png|border|100px]]<br />
|header17 =<br />
|label17 = <br />
|data17 = Motwani and Raghavan. <br>''Randomized Algorithms''.<br> Cambridge Univ Press, 1995.<br />
|header18 =<br />
|label18 = <br />
|data18 = [[File:Approximation_Algorithms.jpg|border|100px]]<br />
|header19 =<br />
|label19 = <br />
|data19 = Vazirani. <br>''Approximation Algorithms''. <br> Springer-Verlag, 2001.<br />
|belowstyle = background:#ddf;<br />
|below = <br />
}}<br />
<br />
This is the webpage for the ''Advanced Algorithms'' class of fall 2026. Students who take this class should check this page periodically for content updates and new announcements. <br />
<br />
= Announcement =<br />
<br />
= Course info =<br />
* '''Instructor ''': <br />
:* [http://tcs.nju.edu.cn/yinyt/ 尹一通]：[mailto:yinyt@nju.edu.cn <yinyt@nju.edu.cn>]，计算机系 804 <br />
:*[https://tcs.nju.edu.cn/shili/ 栗师]：[mailto:shili@nju.edu.cn <shili@nju.edu.cn>]，计算机系 605<br />
:* [https://liuexp.github.io 刘景铖]：[mailto:liu@nju.edu.cn <liu@nju.edu.cn>]，计算机系 516 <br />
* '''Teaching Assistant''': <br />
** 于逸潇：[mailto:yixiaoyu@smail.nju.edu.cn <yixiaoyu@smail.nju.edu.cn>]<br />
** 张弈垚：[mailto:zhangyiyao@smail.nju.edu.cn <zhangyiyao@smail.nju.edu.cn>]<br />
* '''Class meeting''': <br />
** 周一 5-6节 1-17周(单) 仙Ⅰ-319<br />
** 周三 5-6节 1-18周 仙Ⅰ-319<br />
* '''Office hour''': <br />
** 周一4-5pm（尹一通 804）<br />
** 周四4-5pm（刘景铖 516）<br />
** By appointment<br />
* '''QQ群''': 1098567018<br />
<br />
= Syllabus =<br />
随着计算机算法理论的不断发展，现代计算机算法的设计与分析大量地使用非初等的数学工具以及非传统的算法思想。“高级算法”这门课程就是面向计算机算法的这一发展趋势而设立的。课程将针对传统算法课程未系统涉及、却在计算机科学各领域的科研和实践中扮演重要角色的高等算法设计思想和算法分析工具进行系统讲授。<br />
<br />
=== 先修课程 Prerequisites ===<br />
* 必须：离散数学，概率论，线性代数。<br />
* 推荐：算法设计与分析。<br />
<br />
=== Course materials ===<br />
* [[高级算法 (Fall 2026) / Course materials|<font size=3>教材和参考书</font>]]<br />
<br />
=== 成绩 Grades ===<br />
* 课程成绩：本课程将会有若干次作业和一次期末考试。最终成绩将由平时作业成绩和期末考试成绩综合得出。<br />
* 迟交：如果有特殊的理由，无法按时完成作业，请提前联系授课老师，给出正当理由。否则迟交的作业将不被接受。<br />
<br />
=== <font color=red> 学术诚信 Academic Integrity </font>===<br />
学术诚信是所有从事学术活动的学生和学者最基本的职业道德底线，本课程将不遗余力的维护学术诚信规范，违反这一底线的行为将不会被容忍。<br />
<br />
作业完成的原则：署你名字的工作必须是你个人的贡献。在完成作业的过程中，允许讨论，前提是讨论的所有参与者均处于同等完成度。但关键想法的执行、以及作业文本的写作必须独立完成，并在作业中致谢（acknowledge）所有参与讨论的人。不允许其他任何形式的合作——尤其是与已经完成作业的同学“讨论”。<br />
<br />
本课程将对剽窃行为采取零容忍的态度。在完成作业过程中，对他人工作（出版物、互联网资料、其他人的作业等）直接的文本抄袭和对关键思想、关键元素的抄袭，按照 [http://www.acm.org/publications/policies/plagiarism_policy ACM Policy on Plagiarism]的解释，都将视为剽窃。剽窃者成绩将被取消。如果发现互相抄袭行为，<font color=red> 抄袭和被抄袭双方的成绩都将被取消</font>。因此请主动防止自己的作业被他人抄袭。<br />
<br />
学术诚信影响学生个人的品行，也关乎整个教育系统的正常运转。为了一点分数而做出学术不端的行为，不仅使自己沦为一个欺骗者，也使他人的诚实努力失去意义。让我们一起努力维护一个诚信的环境。<br />
<br />
= Assignments =<br />
Late policy: In general, we will accomodate late submission requests ONLY IF you made such requests ahead of time. <br />
<br />
<br />
= Lecture Notes =<br />
# [[高级算法 (Fall 2025)/Min Cut, Max Cut, and Spectral Cut|Min Cut, Max Cut, and Spectral Cut]] ([http://tcs.nju.edu.cn/slides/aa2026/Cut.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-cuts-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-cuts-zh.pdf AI生成讲义])<br />
#* [[高级算法 (Fall 2025)/Probability Basics|Probability basics]]<br />
# [[高级算法 (Fall 2026)/Fingerprinting| Fingerprinting]] ([http://tcs.nju.edu.cn/slides/aa2026/Fingerprinting.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-fingerprinting-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-fingerprinting-zh.pdf AI生成讲义])<br />
#* [[高级算法 (Fall 2026)/Finite Field Basics|Finite field basics]]<br />
# [[高级算法 (Fall 2026)/Hashing and Sketching|Hashing and Sketching]] ([[Meida:Hashing-26.pdf|slides]]) ([[Media:AA26-note-hashing-en.pdf|AI-generated lecture note]], [[Media:AA26-note-hashing-zh.pdf|AI生成讲义]])<br />
#* [[高级算法 (Fall 2026)/Limited independence|Limited independence]]<br />
#* [[高级算法 (Fall 2026)/Basic deviation inequalities|Basic deviation inequalities]]<br />
<br />
= Related Online Courses=<br />
* [https://www.cs.cmu.edu/~15850/ Advanced Algorithms] by Anupam Gupta at CMU.<br />
* [http://people.csail.mit.edu/moitra/854.html Advanced Algorithms] by Ankur Moitra at MIT.<br />
* [http://courses.csail.mit.edu/6.854/current/ Advanced Algorithms] by David Karger and Aleksander Mądry at MIT.<br />
* [http://web.stanford.edu/class/cs168/index.html The Modern Algorithmic Toolbox] by Tim Roughgarden and Gregory Valiant at Stanford.<br />
* [https://www.cs.princeton.edu/courses/archive/fall18/cos521/ Advanced Algorithm Design] by Pravesh Kothari and Christopher Musco at Princeton.<br />
* [http://www.cs.cmu.edu/afs/cs.cmu.edu/academic/class/15859-f11/www/ Linear and Semidefinite Programming (Advanced Algorithms)] by Anupam Gupta and Ryan O'Donnell at CMU.<br />
* [https://www.cs.cmu.edu/~odonnell/papers/cs-theory-toolkit-lecture-notes.pdf CS Theory Toolkit] by Ryan O'Donnell at CMU.<br />
* [https://cs.uwaterloo.ca/~lapchi/cs860/index.html Eigenvalues and Polynomials] by Lap Chi Lau at University of Waterloo.<br />
* The [https://www.cs.cornell.edu/jeh/book.pdf "Foundations of Data Science" book] by Avrim Blum, John Hopcroft, and Ravindran Kannan.</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=File:Hashing-26.pdf&diff=13915File:Hashing-26.pdf2026-09-09T04:43:16Z<p>Etone: Etone uploaded a new version of File:Hashing-26.pdf</p>
<hr />
<div></div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=%E9%AB%98%E7%BA%A7%E7%AE%97%E6%B3%95_(Fall_2026)&diff=13914高级算法 (Fall 2026)2026-09-09T04:42:41Z<p>Etone: /* Lecture Notes */</p>
<hr />
<div>{{Infobox<br />
|name = Infobox<br />
|bodystyle = <br />
|title = <font size=3>高级算法 <br />
<br>Advanced Algorithms</font><br />
|titlestyle = <br />
<br />
|image = <br />
|imagestyle = <br />
|caption = <br />
|captionstyle = <br />
|headerstyle = background:#ccf;<br />
|labelstyle = background:#ddf;<br />
|datastyle = <br />
<br />
|header1 =Instructor<br />
|label1 = <br />
|data1 = <br />
|header2 = <br />
|label2 = <br />
|data2 = '''尹一通'''<br />
|header3 = <br />
|label3 = Email<br />
|data3 = yinyt@nju.edu.cn <br />
|header4 =<br />
|label4= office<br />
|data4= 计算机系 804<br />
|header5 = <br />
|label5 = <br />
|data5 = '''栗师'''<br />
|header6 = <br />
|label6 = Email<br />
|data6 = shili@nju.edu.cn <br />
|header7 =<br />
|label7= office<br />
|data7= 计算机系 605<br />
|header8 = <br />
|label8 = <br />
|data8 = '''刘景铖'''<br />
|header9 = <br />
|label9 = Email<br />
|data9 = liu@nju.edu.cn <br />
|header10 =<br />
|label10= office<br />
|data10= 计算机系 516<br />
|header11 = Class<br />
|label11 = <br />
|data11 = <br />
|header12 =<br />
|label12 = Class meetings<br />
|data12 = 周一 5-6节 (单) 仙Ⅰ-319<br />
周三 5-6节 仙Ⅰ-319<br />
|header13 =<br />
|label13 = Place<br />
|data13 = <br />
|header14 =<br />
|label14 = Office hours<br />
|data14 = 周一4-5pm（尹一通 804）<br/><br />
周四4-5pm（刘景铖 516）<br/><br />
By appointment<br />
|header15 = Textbooks<br />
|label15 = <br />
|data15 = <br />
|header16 =<br />
|label16 = <br />
|data16 = [[File:MR-randomized-algorithms.png|border|100px]]<br />
|header17 =<br />
|label17 = <br />
|data17 = Motwani and Raghavan. <br>''Randomized Algorithms''.<br> Cambridge Univ Press, 1995.<br />
|header18 =<br />
|label18 = <br />
|data18 = [[File:Approximation_Algorithms.jpg|border|100px]]<br />
|header19 =<br />
|label19 = <br />
|data19 = Vazirani. <br>''Approximation Algorithms''. <br> Springer-Verlag, 2001.<br />
|belowstyle = background:#ddf;<br />
|below = <br />
}}<br />
<br />
This is the webpage for the ''Advanced Algorithms'' class of fall 2026. Students who take this class should check this page periodically for content updates and new announcements. <br />
<br />
= Announcement =<br />
<br />
= Course info =<br />
* '''Instructor ''': <br />
:* [http://tcs.nju.edu.cn/yinyt/ 尹一通]：[mailto:yinyt@nju.edu.cn <yinyt@nju.edu.cn>]，计算机系 804 <br />
:*[https://tcs.nju.edu.cn/shili/ 栗师]：[mailto:shili@nju.edu.cn <shili@nju.edu.cn>]，计算机系 605<br />
:* [https://liuexp.github.io 刘景铖]：[mailto:liu@nju.edu.cn <liu@nju.edu.cn>]，计算机系 516 <br />
* '''Teaching Assistant''': <br />
** 于逸潇：[mailto:yixiaoyu@smail.nju.edu.cn <yixiaoyu@smail.nju.edu.cn>]<br />
** 张弈垚：[mailto:zhangyiyao@smail.nju.edu.cn <zhangyiyao@smail.nju.edu.cn>]<br />
* '''Class meeting''': <br />
** 周一 5-6节 1-17周(单) 仙Ⅰ-319<br />
** 周三 5-6节 1-18周 仙Ⅰ-319<br />
* '''Office hour''': <br />
** 周一4-5pm（尹一通 804）<br />
** 周四4-5pm（刘景铖 516）<br />
** By appointment<br />
* '''QQ群''': 1098567018<br />
<br />
= Syllabus =<br />
随着计算机算法理论的不断发展，现代计算机算法的设计与分析大量地使用非初等的数学工具以及非传统的算法思想。“高级算法”这门课程就是面向计算机算法的这一发展趋势而设立的。课程将针对传统算法课程未系统涉及、却在计算机科学各领域的科研和实践中扮演重要角色的高等算法设计思想和算法分析工具进行系统讲授。<br />
<br />
=== 先修课程 Prerequisites ===<br />
* 必须：离散数学，概率论，线性代数。<br />
* 推荐：算法设计与分析。<br />
<br />
=== Course materials ===<br />
* [[高级算法 (Fall 2026) / Course materials|<font size=3>教材和参考书</font>]]<br />
<br />
=== 成绩 Grades ===<br />
* 课程成绩：本课程将会有若干次作业和一次期末考试。最终成绩将由平时作业成绩和期末考试成绩综合得出。<br />
* 迟交：如果有特殊的理由，无法按时完成作业，请提前联系授课老师，给出正当理由。否则迟交的作业将不被接受。<br />
<br />
=== <font color=red> 学术诚信 Academic Integrity </font>===<br />
学术诚信是所有从事学术活动的学生和学者最基本的职业道德底线，本课程将不遗余力的维护学术诚信规范，违反这一底线的行为将不会被容忍。<br />
<br />
作业完成的原则：署你名字的工作必须是你个人的贡献。在完成作业的过程中，允许讨论，前提是讨论的所有参与者均处于同等完成度。但关键想法的执行、以及作业文本的写作必须独立完成，并在作业中致谢（acknowledge）所有参与讨论的人。不允许其他任何形式的合作——尤其是与已经完成作业的同学“讨论”。<br />
<br />
本课程将对剽窃行为采取零容忍的态度。在完成作业过程中，对他人工作（出版物、互联网资料、其他人的作业等）直接的文本抄袭和对关键思想、关键元素的抄袭，按照 [http://www.acm.org/publications/policies/plagiarism_policy ACM Policy on Plagiarism]的解释，都将视为剽窃。剽窃者成绩将被取消。如果发现互相抄袭行为，<font color=red> 抄袭和被抄袭双方的成绩都将被取消</font>。因此请主动防止自己的作业被他人抄袭。<br />
<br />
学术诚信影响学生个人的品行，也关乎整个教育系统的正常运转。为了一点分数而做出学术不端的行为，不仅使自己沦为一个欺骗者，也使他人的诚实努力失去意义。让我们一起努力维护一个诚信的环境。<br />
<br />
= Assignments =<br />
Late policy: In general, we will accomodate late submission requests ONLY IF you made such requests ahead of time. <br />
<br />
<br />
= Lecture Notes =<br />
# [[高级算法 (Fall 2025)/Min Cut, Max Cut, and Spectral Cut|Min Cut, Max Cut, and Spectral Cut]] ([http://tcs.nju.edu.cn/slides/aa2026/Cut.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-cuts-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-cuts-zh.pdf AI生成讲义])<br />
#* [[高级算法 (Fall 2025)/Probability Basics|Probability basics]]<br />
# [[高级算法 (Fall 2026)/Fingerprinting| Fingerprinting]] ([http://tcs.nju.edu.cn/slides/aa2026/Fingerprinting.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-fingerprinting-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-fingerprinting-zh.pdf AI生成讲义])<br />
#* [[高级算法 (Fall 2026)/Finite Field Basics|Finite field basics]]<br />
# [[高级算法 (Fall 2026)/Hashing and Sketching|Hashing and Sketching]] ([[file:Hashing-26.pdf|slides]]) ([[Media:AA26-note-hashing-en.pdf|AI-generated lecture note]], [[Media:AA26-note-hashing-zh.pdf|AI生成讲义]])<br />
#* [[高级算法 (Fall 2026)/Limited independence|Limited independence]]<br />
#* [[高级算法 (Fall 2026)/Basic deviation inequalities|Basic deviation inequalities]]<br />
<br />
= Related Online Courses=<br />
* [https://www.cs.cmu.edu/~15850/ Advanced Algorithms] by Anupam Gupta at CMU.<br />
* [http://people.csail.mit.edu/moitra/854.html Advanced Algorithms] by Ankur Moitra at MIT.<br />
* [http://courses.csail.mit.edu/6.854/current/ Advanced Algorithms] by David Karger and Aleksander Mądry at MIT.<br />
* [http://web.stanford.edu/class/cs168/index.html The Modern Algorithmic Toolbox] by Tim Roughgarden and Gregory Valiant at Stanford.<br />
* [https://www.cs.princeton.edu/courses/archive/fall18/cos521/ Advanced Algorithm Design] by Pravesh Kothari and Christopher Musco at Princeton.<br />
* [http://www.cs.cmu.edu/afs/cs.cmu.edu/academic/class/15859-f11/www/ Linear and Semidefinite Programming (Advanced Algorithms)] by Anupam Gupta and Ryan O'Donnell at CMU.<br />
* [https://www.cs.cmu.edu/~odonnell/papers/cs-theory-toolkit-lecture-notes.pdf CS Theory Toolkit] by Ryan O'Donnell at CMU.<br />
* [https://cs.uwaterloo.ca/~lapchi/cs860/index.html Eigenvalues and Polynomials] by Lap Chi Lau at University of Waterloo.<br />
* The [https://www.cs.cornell.edu/jeh/book.pdf "Foundations of Data Science" book] by Avrim Blum, John Hopcroft, and Ravindran Kannan.</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=%E9%AB%98%E7%BA%A7%E7%AE%97%E6%B3%95_(Fall_2026)&diff=13913高级算法 (Fall 2026)2026-09-08T13:54:23Z<p>Etone: /* Lecture Notes */</p>
<hr />
<div>{{Infobox<br />
|name = Infobox<br />
|bodystyle = <br />
|title = <font size=3>高级算法 <br />
<br>Advanced Algorithms</font><br />
|titlestyle = <br />
<br />
|image = <br />
|imagestyle = <br />
|caption = <br />
|captionstyle = <br />
|headerstyle = background:#ccf;<br />
|labelstyle = background:#ddf;<br />
|datastyle = <br />
<br />
|header1 =Instructor<br />
|label1 = <br />
|data1 = <br />
|header2 = <br />
|label2 = <br />
|data2 = '''尹一通'''<br />
|header3 = <br />
|label3 = Email<br />
|data3 = yinyt@nju.edu.cn <br />
|header4 =<br />
|label4= office<br />
|data4= 计算机系 804<br />
|header5 = <br />
|label5 = <br />
|data5 = '''栗师'''<br />
|header6 = <br />
|label6 = Email<br />
|data6 = shili@nju.edu.cn <br />
|header7 =<br />
|label7= office<br />
|data7= 计算机系 605<br />
|header8 = <br />
|label8 = <br />
|data8 = '''刘景铖'''<br />
|header9 = <br />
|label9 = Email<br />
|data9 = liu@nju.edu.cn <br />
|header10 =<br />
|label10= office<br />
|data10= 计算机系 516<br />
|header11 = Class<br />
|label11 = <br />
|data11 = <br />
|header12 =<br />
|label12 = Class meetings<br />
|data12 = 周一 5-6节 (单) 仙Ⅰ-319<br />
周三 5-6节 仙Ⅰ-319<br />
|header13 =<br />
|label13 = Place<br />
|data13 = <br />
|header14 =<br />
|label14 = Office hours<br />
|data14 = 周一4-5pm（尹一通 804）<br/><br />
周四4-5pm（刘景铖 516）<br/><br />
By appointment<br />
|header15 = Textbooks<br />
|label15 = <br />
|data15 = <br />
|header16 =<br />
|label16 = <br />
|data16 = [[File:MR-randomized-algorithms.png|border|100px]]<br />
|header17 =<br />
|label17 = <br />
|data17 = Motwani and Raghavan. <br>''Randomized Algorithms''.<br> Cambridge Univ Press, 1995.<br />
|header18 =<br />
|label18 = <br />
|data18 = [[File:Approximation_Algorithms.jpg|border|100px]]<br />
|header19 =<br />
|label19 = <br />
|data19 = Vazirani. <br>''Approximation Algorithms''. <br> Springer-Verlag, 2001.<br />
|belowstyle = background:#ddf;<br />
|below = <br />
}}<br />
<br />
This is the webpage for the ''Advanced Algorithms'' class of fall 2026. Students who take this class should check this page periodically for content updates and new announcements. <br />
<br />
= Announcement =<br />
<br />
= Course info =<br />
* '''Instructor ''': <br />
:* [http://tcs.nju.edu.cn/yinyt/ 尹一通]：[mailto:yinyt@nju.edu.cn <yinyt@nju.edu.cn>]，计算机系 804 <br />
:*[https://tcs.nju.edu.cn/shili/ 栗师]：[mailto:shili@nju.edu.cn <shili@nju.edu.cn>]，计算机系 605<br />
:* [https://liuexp.github.io 刘景铖]：[mailto:liu@nju.edu.cn <liu@nju.edu.cn>]，计算机系 516 <br />
* '''Teaching Assistant''': <br />
** 于逸潇：[mailto:yixiaoyu@smail.nju.edu.cn <yixiaoyu@smail.nju.edu.cn>]<br />
** 张弈垚：[mailto:zhangyiyao@smail.nju.edu.cn <zhangyiyao@smail.nju.edu.cn>]<br />
* '''Class meeting''': <br />
** 周一 5-6节 1-17周(单) 仙Ⅰ-319<br />
** 周三 5-6节 1-18周 仙Ⅰ-319<br />
* '''Office hour''': <br />
** 周一4-5pm（尹一通 804）<br />
** 周四4-5pm（刘景铖 516）<br />
** By appointment<br />
* '''QQ群''': 1098567018<br />
<br />
= Syllabus =<br />
随着计算机算法理论的不断发展，现代计算机算法的设计与分析大量地使用非初等的数学工具以及非传统的算法思想。“高级算法”这门课程就是面向计算机算法的这一发展趋势而设立的。课程将针对传统算法课程未系统涉及、却在计算机科学各领域的科研和实践中扮演重要角色的高等算法设计思想和算法分析工具进行系统讲授。<br />
<br />
=== 先修课程 Prerequisites ===<br />
* 必须：离散数学，概率论，线性代数。<br />
* 推荐：算法设计与分析。<br />
<br />
=== Course materials ===<br />
* [[高级算法 (Fall 2026) / Course materials|<font size=3>教材和参考书</font>]]<br />
<br />
=== 成绩 Grades ===<br />
* 课程成绩：本课程将会有若干次作业和一次期末考试。最终成绩将由平时作业成绩和期末考试成绩综合得出。<br />
* 迟交：如果有特殊的理由，无法按时完成作业，请提前联系授课老师，给出正当理由。否则迟交的作业将不被接受。<br />
<br />
=== <font color=red> 学术诚信 Academic Integrity </font>===<br />
学术诚信是所有从事学术活动的学生和学者最基本的职业道德底线，本课程将不遗余力的维护学术诚信规范，违反这一底线的行为将不会被容忍。<br />
<br />
作业完成的原则：署你名字的工作必须是你个人的贡献。在完成作业的过程中，允许讨论，前提是讨论的所有参与者均处于同等完成度。但关键想法的执行、以及作业文本的写作必须独立完成，并在作业中致谢（acknowledge）所有参与讨论的人。不允许其他任何形式的合作——尤其是与已经完成作业的同学“讨论”。<br />
<br />
本课程将对剽窃行为采取零容忍的态度。在完成作业过程中，对他人工作（出版物、互联网资料、其他人的作业等）直接的文本抄袭和对关键思想、关键元素的抄袭，按照 [http://www.acm.org/publications/policies/plagiarism_policy ACM Policy on Plagiarism]的解释，都将视为剽窃。剽窃者成绩将被取消。如果发现互相抄袭行为，<font color=red> 抄袭和被抄袭双方的成绩都将被取消</font>。因此请主动防止自己的作业被他人抄袭。<br />
<br />
学术诚信影响学生个人的品行，也关乎整个教育系统的正常运转。为了一点分数而做出学术不端的行为，不仅使自己沦为一个欺骗者，也使他人的诚实努力失去意义。让我们一起努力维护一个诚信的环境。<br />
<br />
= Assignments =<br />
Late policy: In general, we will accomodate late submission requests ONLY IF you made such requests ahead of time. <br />
<br />
<br />
= Lecture Notes =<br />
# [[高级算法 (Fall 2025)/Min Cut, Max Cut, and Spectral Cut|Min Cut, Max Cut, and Spectral Cut]] ([http://tcs.nju.edu.cn/slides/aa2026/Cut.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-cuts-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-cuts-zh.pdf AI生成讲义])<br />
#* [[高级算法 (Fall 2025)/Probability Basics|Probability basics]]<br />
# [[高级算法 (Fall 2026)/Fingerprinting| Fingerprinting]] ([http://tcs.nju.edu.cn/slides/aa2026/Fingerprinting.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-fingerprinting-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-fingerprinting-zh.pdf AI生成讲义])<br />
#* [[高级算法 (Fall 2026)/Finite Field Basics|Finite field basics]]<br />
# [[高级算法 (Fall 2026)/Hashing and Sketching|Hashing and Sketching]] ([[Media:Hashing-26.pdf|slides]]) ([[Media:AA26-note-hashing-en.pdf|AI-generated lecture note]], [[Media:AA26-note-hashing-zh.pdf|AI生成讲义]])<br />
#* [[高级算法 (Fall 2026)/Limited independence|Limited independence]]<br />
#* [[高级算法 (Fall 2026)/Basic deviation inequalities|Basic deviation inequalities]]<br />
<br />
= Related Online Courses=<br />
* [https://www.cs.cmu.edu/~15850/ Advanced Algorithms] by Anupam Gupta at CMU.<br />
* [http://people.csail.mit.edu/moitra/854.html Advanced Algorithms] by Ankur Moitra at MIT.<br />
* [http://courses.csail.mit.edu/6.854/current/ Advanced Algorithms] by David Karger and Aleksander Mądry at MIT.<br />
* [http://web.stanford.edu/class/cs168/index.html The Modern Algorithmic Toolbox] by Tim Roughgarden and Gregory Valiant at Stanford.<br />
* [https://www.cs.princeton.edu/courses/archive/fall18/cos521/ Advanced Algorithm Design] by Pravesh Kothari and Christopher Musco at Princeton.<br />
* [http://www.cs.cmu.edu/afs/cs.cmu.edu/academic/class/15859-f11/www/ Linear and Semidefinite Programming (Advanced Algorithms)] by Anupam Gupta and Ryan O'Donnell at CMU.<br />
* [https://www.cs.cmu.edu/~odonnell/papers/cs-theory-toolkit-lecture-notes.pdf CS Theory Toolkit] by Ryan O'Donnell at CMU.<br />
* [https://cs.uwaterloo.ca/~lapchi/cs860/index.html Eigenvalues and Polynomials] by Lap Chi Lau at University of Waterloo.<br />
* The [https://www.cs.cornell.edu/jeh/book.pdf "Foundations of Data Science" book] by Avrim Blum, John Hopcroft, and Ravindran Kannan.</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=File:AA26-note-hashing-zh.pdf&diff=13912File:AA26-note-hashing-zh.pdf2026-09-08T13:53:21Z<p>Etone: </p>
<hr />
<div></div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=File:AA26-note-hashing-en.pdf&diff=13911File:AA26-note-hashing-en.pdf2026-09-08T13:53:06Z<p>Etone: </p>
<hr />
<div></div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=%E9%AB%98%E7%BA%A7%E7%AE%97%E6%B3%95_(Fall_2026)&diff=13910高级算法 (Fall 2026)2026-09-08T13:52:09Z<p>Etone: /* Lecture Notes */</p>
<hr />
<div>{{Infobox<br />
|name = Infobox<br />
|bodystyle = <br />
|title = <font size=3>高级算法 <br />
<br>Advanced Algorithms</font><br />
|titlestyle = <br />
<br />
|image = <br />
|imagestyle = <br />
|caption = <br />
|captionstyle = <br />
|headerstyle = background:#ccf;<br />
|labelstyle = background:#ddf;<br />
|datastyle = <br />
<br />
|header1 =Instructor<br />
|label1 = <br />
|data1 = <br />
|header2 = <br />
|label2 = <br />
|data2 = '''尹一通'''<br />
|header3 = <br />
|label3 = Email<br />
|data3 = yinyt@nju.edu.cn <br />
|header4 =<br />
|label4= office<br />
|data4= 计算机系 804<br />
|header5 = <br />
|label5 = <br />
|data5 = '''栗师'''<br />
|header6 = <br />
|label6 = Email<br />
|data6 = shili@nju.edu.cn <br />
|header7 =<br />
|label7= office<br />
|data7= 计算机系 605<br />
|header8 = <br />
|label8 = <br />
|data8 = '''刘景铖'''<br />
|header9 = <br />
|label9 = Email<br />
|data9 = liu@nju.edu.cn <br />
|header10 =<br />
|label10= office<br />
|data10= 计算机系 516<br />
|header11 = Class<br />
|label11 = <br />
|data11 = <br />
|header12 =<br />
|label12 = Class meetings<br />
|data12 = 周一 5-6节 (单) 仙Ⅰ-319<br />
周三 5-6节 仙Ⅰ-319<br />
|header13 =<br />
|label13 = Place<br />
|data13 = <br />
|header14 =<br />
|label14 = Office hours<br />
|data14 = 周一4-5pm（尹一通 804）<br/><br />
周四4-5pm（刘景铖 516）<br/><br />
By appointment<br />
|header15 = Textbooks<br />
|label15 = <br />
|data15 = <br />
|header16 =<br />
|label16 = <br />
|data16 = [[File:MR-randomized-algorithms.png|border|100px]]<br />
|header17 =<br />
|label17 = <br />
|data17 = Motwani and Raghavan. <br>''Randomized Algorithms''.<br> Cambridge Univ Press, 1995.<br />
|header18 =<br />
|label18 = <br />
|data18 = [[File:Approximation_Algorithms.jpg|border|100px]]<br />
|header19 =<br />
|label19 = <br />
|data19 = Vazirani. <br>''Approximation Algorithms''. <br> Springer-Verlag, 2001.<br />
|belowstyle = background:#ddf;<br />
|below = <br />
}}<br />
<br />
This is the webpage for the ''Advanced Algorithms'' class of fall 2026. Students who take this class should check this page periodically for content updates and new announcements. <br />
<br />
= Announcement =<br />
<br />
= Course info =<br />
* '''Instructor ''': <br />
:* [http://tcs.nju.edu.cn/yinyt/ 尹一通]：[mailto:yinyt@nju.edu.cn <yinyt@nju.edu.cn>]，计算机系 804 <br />
:*[https://tcs.nju.edu.cn/shili/ 栗师]：[mailto:shili@nju.edu.cn <shili@nju.edu.cn>]，计算机系 605<br />
:* [https://liuexp.github.io 刘景铖]：[mailto:liu@nju.edu.cn <liu@nju.edu.cn>]，计算机系 516 <br />
* '''Teaching Assistant''': <br />
** 于逸潇：[mailto:yixiaoyu@smail.nju.edu.cn <yixiaoyu@smail.nju.edu.cn>]<br />
** 张弈垚：[mailto:zhangyiyao@smail.nju.edu.cn <zhangyiyao@smail.nju.edu.cn>]<br />
* '''Class meeting''': <br />
** 周一 5-6节 1-17周(单) 仙Ⅰ-319<br />
** 周三 5-6节 1-18周 仙Ⅰ-319<br />
* '''Office hour''': <br />
** 周一4-5pm（尹一通 804）<br />
** 周四4-5pm（刘景铖 516）<br />
** By appointment<br />
* '''QQ群''': 1098567018<br />
<br />
= Syllabus =<br />
随着计算机算法理论的不断发展，现代计算机算法的设计与分析大量地使用非初等的数学工具以及非传统的算法思想。“高级算法”这门课程就是面向计算机算法的这一发展趋势而设立的。课程将针对传统算法课程未系统涉及、却在计算机科学各领域的科研和实践中扮演重要角色的高等算法设计思想和算法分析工具进行系统讲授。<br />
<br />
=== 先修课程 Prerequisites ===<br />
* 必须：离散数学，概率论，线性代数。<br />
* 推荐：算法设计与分析。<br />
<br />
=== Course materials ===<br />
* [[高级算法 (Fall 2026) / Course materials|<font size=3>教材和参考书</font>]]<br />
<br />
=== 成绩 Grades ===<br />
* 课程成绩：本课程将会有若干次作业和一次期末考试。最终成绩将由平时作业成绩和期末考试成绩综合得出。<br />
* 迟交：如果有特殊的理由，无法按时完成作业，请提前联系授课老师，给出正当理由。否则迟交的作业将不被接受。<br />
<br />
=== <font color=red> 学术诚信 Academic Integrity </font>===<br />
学术诚信是所有从事学术活动的学生和学者最基本的职业道德底线，本课程将不遗余力的维护学术诚信规范，违反这一底线的行为将不会被容忍。<br />
<br />
作业完成的原则：署你名字的工作必须是你个人的贡献。在完成作业的过程中，允许讨论，前提是讨论的所有参与者均处于同等完成度。但关键想法的执行、以及作业文本的写作必须独立完成，并在作业中致谢（acknowledge）所有参与讨论的人。不允许其他任何形式的合作——尤其是与已经完成作业的同学“讨论”。<br />
<br />
本课程将对剽窃行为采取零容忍的态度。在完成作业过程中，对他人工作（出版物、互联网资料、其他人的作业等）直接的文本抄袭和对关键思想、关键元素的抄袭，按照 [http://www.acm.org/publications/policies/plagiarism_policy ACM Policy on Plagiarism]的解释，都将视为剽窃。剽窃者成绩将被取消。如果发现互相抄袭行为，<font color=red> 抄袭和被抄袭双方的成绩都将被取消</font>。因此请主动防止自己的作业被他人抄袭。<br />
<br />
学术诚信影响学生个人的品行，也关乎整个教育系统的正常运转。为了一点分数而做出学术不端的行为，不仅使自己沦为一个欺骗者，也使他人的诚实努力失去意义。让我们一起努力维护一个诚信的环境。<br />
<br />
= Assignments =<br />
Late policy: In general, we will accomodate late submission requests ONLY IF you made such requests ahead of time. <br />
<br />
<br />
= Lecture Notes =<br />
# [[高级算法 (Fall 2025)/Min Cut, Max Cut, and Spectral Cut|Min Cut, Max Cut, and Spectral Cut]] ([http://tcs.nju.edu.cn/slides/aa2026/Cut.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-cuts-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-cuts-zh.pdf AI生成讲义])<br />
#* [[高级算法 (Fall 2025)/Probability Basics|Probability basics]]<br />
# [[高级算法 (Fall 2026)/Fingerprinting| Fingerprinting]] ([http://tcs.nju.edu.cn/slides/aa2026/Fingerprinting.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-fingerprinting-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-fingerprinting-zh.pdf AI生成讲义])<br />
#* [[高级算法 (Fall 2026)/Finite Field Basics|Finite field basics]]<br />
# [[高级算法 (Fall 2026)/Hashing and Sketching|Hashing and Sketching]] ([[Media:Hashing-26.pdf|slides]])<br />
#* [[高级算法 (Fall 2026)/Limited independence|Limited independence]]<br />
#* [[高级算法 (Fall 2026)/Basic deviation inequalities|Basic deviation inequalities]]<br />
<br />
= Related Online Courses=<br />
* [https://www.cs.cmu.edu/~15850/ Advanced Algorithms] by Anupam Gupta at CMU.<br />
* [http://people.csail.mit.edu/moitra/854.html Advanced Algorithms] by Ankur Moitra at MIT.<br />
* [http://courses.csail.mit.edu/6.854/current/ Advanced Algorithms] by David Karger and Aleksander Mądry at MIT.<br />
* [http://web.stanford.edu/class/cs168/index.html The Modern Algorithmic Toolbox] by Tim Roughgarden and Gregory Valiant at Stanford.<br />
* [https://www.cs.princeton.edu/courses/archive/fall18/cos521/ Advanced Algorithm Design] by Pravesh Kothari and Christopher Musco at Princeton.<br />
* [http://www.cs.cmu.edu/afs/cs.cmu.edu/academic/class/15859-f11/www/ Linear and Semidefinite Programming (Advanced Algorithms)] by Anupam Gupta and Ryan O'Donnell at CMU.<br />
* [https://www.cs.cmu.edu/~odonnell/papers/cs-theory-toolkit-lecture-notes.pdf CS Theory Toolkit] by Ryan O'Donnell at CMU.<br />
* [https://cs.uwaterloo.ca/~lapchi/cs860/index.html Eigenvalues and Polynomials] by Lap Chi Lau at University of Waterloo.<br />
* The [https://www.cs.cornell.edu/jeh/book.pdf "Foundations of Data Science" book] by Avrim Blum, John Hopcroft, and Ravindran Kannan.</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=File:Hashing-26.pdf&diff=13909File:Hashing-26.pdf2026-09-08T13:47:20Z<p>Etone: </p>
<hr />
<div></div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=File:Hashing.pdf&diff=13908File:Hashing.pdf2026-09-08T13:46:04Z<p>Etone: </p>
<hr />
<div></div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=%E9%AB%98%E7%BA%A7%E7%AE%97%E6%B3%95_(Fall_2026)/Basic_deviation_inequalities&diff=13906高级算法 (Fall 2026)/Basic deviation inequalities2026-09-07T10:13:47Z<p>Etone: Created page with "=Markov's Inequality= One of the most natural information about a random variable is its expectation, which is the first moment of the random variable. Markov's inequality draws a tail bound for a random variable from its expectation. {{Theorem |Theorem (Markov's Inequality)| :Let <math>X</math> be a random variable assuming only nonnegative values. Then, for all <math>t>0</math>, ::<math>\begin{align} \Pr[X\ge t]\le \frac{\mathbf{E}[X]}{t}. \end{align}</math> }} {{Proo..."</p>
<hr />
<div>=Markov's Inequality=<br />
<br />
One of the most natural information about a random variable is its expectation, which is the first moment of the random variable. Markov's inequality draws a tail bound for a random variable from its expectation.<br />
{{Theorem<br />
|Theorem (Markov's Inequality)|<br />
:Let <math>X</math> be a random variable assuming only nonnegative values. Then, for all <math>t>0</math>,<br />
::<math>\begin{align}<br />
\Pr[X\ge t]\le \frac{\mathbf{E}[X]}{t}.<br />
\end{align}</math><br />
}}<br />
{{Proof| Let <math>Y</math> be the indicator such that <br />
:<math>\begin{align}<br />
Y &=<br />
\begin{cases}<br />
1 & \mbox{if }X\ge t,\\<br />
0 & \mbox{otherwise.}<br />
\end{cases}<br />
\end{align}</math><br />
<br />
It holds that <math>Y\le\frac{X}{t}</math>. Since <math>Y</math> is 0-1 valued, <math>\mathbf{E}[Y]=\Pr[Y=1]=\Pr[X\ge t]</math>. Therefore,<br />
:<math><br />
\Pr[X\ge t]<br />
=<br />
\mathbf{E}[Y]<br />
\le<br />
\mathbf{E}\left[\frac{X}{t}\right]<br />
=\frac{\mathbf{E}[X]}{t}.<br />
</math><br />
}}<br />
<br />
== Generalization ==<br />
For any random variable <math>X</math>, for an arbitrary non-negative real function <math>h</math>, the <math>h(X)</math> is a non-negative random variable. Applying Markov's inequality, we directly have that<br />
:<math><br />
\Pr[h(X)\ge t]\le\frac{\mathbf{E}[h(X)]}{t}.<br />
</math><br />
<br />
This trivial application of Markov's inequality gives us a powerful tool for proving tail inequalities. With the function <math>h</math> which extracts more information about the random variable, we can prove sharper tail inequalities.<br />
<br />
=Chebyshev's inequality=<br />
<br />
== Variance ==<br />
{{Theorem<br />
|Definition (variance)|<br />
:The '''variance''' of a random variable <math>X</math> is defined as<br />
::<math>\begin{align}<br />
\mathbf{Var}[X]=\mathbf{E}\left[(X-\mathbf{E}[X])^2\right]=\mathbf{E}\left[X^2\right]-(\mathbf{E}[X])^2.<br />
\end{align}</math><br />
:The '''standard deviation''' of random variable <math>X</math> is <br />
::<math><br />
\delta[X]=\sqrt{\mathbf{Var}[X]}.<br />
</math><br />
}}<br />
<br />
The variance is the diagonal case for '''covariance'''.<br />
{{Theorem<br />
|Definition (covariance)|<br />
:The '''covariance''' of two random variables <math>X</math> and <math>Y</math> is<br />
::<math>\begin{align}<br />
\mathbf{Cov}(X,Y)=\mathbf{E}\left[(X-\mathbf{E}[X])(Y-\mathbf{E}[Y])\right].<br />
\end{align}</math><br />
}}<br />
<br />
We have the following theorem for the variance of sum.<br />
<br />
{{Theorem<br />
|Theorem|<br />
:For any two random variables <math>X</math> and <math>Y</math>,<br />
::<math>\begin{align}<br />
\mathbf{Var}[X+Y]=\mathbf{Var}[X]+\mathbf{Var}[Y]+2\mathbf{Cov}(X,Y).<br />
\end{align}</math><br />
:Generally, for any random variables <math>X_1,X_2,\ldots,X_n</math>,<br />
::<math>\begin{align}<br />
\mathbf{Var}\left[\sum_{i=1}^n X_i\right]=\sum_{i=1}^n\mathbf{Var}[X_i]+\sum_{i\neq j}\mathbf{Cov}(X_i,X_j).<br />
\end{align}</math><br />
}}<br />
{{Proof| The equation for two variables is directly due to the definition of variance and covariance. The equation for <math>n</math> variables can be deduced from the equation for two variables.<br />
}}<br />
<br />
For independent random variables, the expectation of a product equals the product of expectations. <br />
{{Theorem<br />
|Theorem|<br />
:For any two independent random variables <math>X</math> and <math>Y</math>,<br />
::<math>\begin{align}<br />
\mathbf{E}[X\cdot Y]=\mathbf{E}[X]\cdot\mathbf{E}[Y].<br />
\end{align}</math><br />
}}<br />
{{Proof|<br />
:<math><br />
\begin{align}<br />
\mathbf{E}[X\cdot Y]<br />
&=<br />
\sum_{x,y}xy\Pr[X=x\wedge Y=y]\\<br />
&=<br />
\sum_{x,y}xy\Pr[X=x]\Pr[Y=y]\\<br />
&=<br />
\sum_{x}x\Pr[X=x]\sum_{y}y\Pr[Y=y]\\<br />
&=<br />
\mathbf{E}[X]\cdot\mathbf{E}[Y].<br />
\end{align}<br />
</math><br />
}}<br />
<br />
Consequently, covariance of independent random variables is always zero.<br />
{{Theorem<br />
|Theorem|<br />
:For any two independent random variables <math>X</math> and <math>Y</math>,<br />
::<math>\begin{align}<br />
\mathbf{Cov}(X,Y)=0.<br />
\end{align}</math><br />
}}<br />
{{Proof| <br />
:<math>\begin{align}<br />
\mathbf{Cov}(X,Y) <br />
&=\mathbf{E}\left[(X-\mathbf{E}[X])(Y-\mathbf{E}[Y])\right]\\<br />
&= \mathbf{E}\left[X-\mathbf{E}[X]\right]\mathbf{E}\left[Y-\mathbf{E}[Y]\right] &\qquad(\mbox{Independence})\\<br />
&=0.<br />
\end{align}</math><br />
}}<br />
<br />
The variance of the sum of pairwise independent random variables is equal to the sum of variances. <br />
{{Theorem<br />
|Theorem|<br />
:For '''pairwise''' independent random variables <math>X_1,X_2,\ldots,X_n</math>,<br />
::<math>\begin{align}<br />
\mathbf{Var}\left[\sum_{i=1}^n X_i\right]=\sum_{i=1}^n\mathbf{Var}[X_i].<br />
\end{align}</math><br />
}}<br />
<br />
;Remark<br />
:The theorem holds for '''pairwise''' independent random variables, a much weaker independence requirement than the '''mutual''' independence. This makes the second-moment methods very useful for pairwise independent random variables.<br />
<br />
=== Variance of binomial distribution ===<br />
For a Bernoulli trial with parameter <math>p</math>.<br />
:<math><br />
X=\begin{cases}<br />
1& \mbox{with probability }p\\<br />
0& \mbox{with probability }1-p<br />
\end{cases}<br />
</math><br />
The variance is<br />
:<math><br />
\mathbf{Var}[X]=\mathbf{E}[X^2]-(\mathbf{E}[X])^2=\mathbf{E}[X]-(\mathbf{E}[X])^2=p-p^2=p(1-p).<br />
</math><br />
<br />
Let <math>Y</math> be a binomial random variable with parameter <math>n</math> and <math>p</math>, i.e. <math>Y=\sum_{i=1}^nY_i</math>, where <math>Y_i</math>'s are i.i.d. Bernoulli trials with parameter <math>p</math>. The variance is<br />
:<math><br />
\begin{align}<br />
\mathbf{Var}[Y] <br />
&=<br />
\mathbf{Var}\left[\sum_{i=1}^nY_i\right]\\<br />
&=<br />
\sum_{i=1}^n\mathbf{Var}\left[Y_i\right] &\qquad (\mbox{Independence})\\<br />
&=<br />
\sum_{i=1}^np(1-p) &\qquad (\mbox{Bernoulli})\\<br />
&=<br />
p(1-p)n.<br />
\end{align}<br />
</math><br />
<br />
== Chebyshev's inequality ==<br />
{{Theorem<br />
|Theorem (Chebyshev's Inequality)|<br />
:For any <math>t>0</math>,<br />
::<math>\begin{align}<br />
\Pr\left[|X-\mathbf{E}[X]| \ge t\right] \le \frac{\mathbf{Var}[X]}{t^2}.<br />
\end{align}</math><br />
}}<br />
{{Proof| Observe that <br />
:<math>\Pr[|X-\mathbf{E}[X]| \ge t] = \Pr[(X-\mathbf{E}[X])^2 \ge t^2].</math><br />
Since <math>(X-\mathbf{E}[X])^2</math> is a nonnegative random variable, we can apply Markov's inequality, such that<br />
:<math><br />
\Pr[(X-\mathbf{E}[X])^2 \ge t^2] \le<br />
\frac{\mathbf{E}[(X-\mathbf{E}[X])^2]}{t^2}<br />
=\frac{\mathbf{Var}[X]}{t^2}.<br />
</math><br />
}}</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=%E9%AB%98%E7%BA%A7%E7%AE%97%E6%B3%95_(Fall_2026)/Limited_independence&diff=13905高级算法 (Fall 2026)/Limited independence2026-09-07T10:13:26Z<p>Etone: Created page with "= <math>k</math>-wise independence = Recall the definition of independence between events: {{Theorem |Definition (Independent events)| :Events <math>\mathcal{E}_1, \mathcal{E}_2, \ldots, \mathcal{E}_n</math> are '''mutually independent''' if, for any subset <math>I\subseteq\{1,2,\ldots,n\}</math>, ::<math>\begin{align} \Pr\left[\bigwedge_{i\in I}\mathcal{E}_i\right] &= \prod_{i\in I}\Pr[\mathcal{E}_i]. \end{align}</math> }} Similarly, we can define independence between..."</p>
<hr />
<div>= <math>k</math>-wise independence =<br />
Recall the definition of independence between events:<br />
{{Theorem<br />
|Definition (Independent events)|<br />
:Events <math>\mathcal{E}_1, \mathcal{E}_2, \ldots, \mathcal{E}_n</math> are '''mutually independent''' if, for any subset <math>I\subseteq\{1,2,\ldots,n\}</math>,<br />
::<math>\begin{align}<br />
\Pr\left[\bigwedge_{i\in I}\mathcal{E}_i\right]<br />
&=<br />
\prod_{i\in I}\Pr[\mathcal{E}_i].<br />
\end{align}</math><br />
}}<br />
Similarly, we can define independence between random variables:<br />
{{Theorem<br />
|Definition (Independent variables)|<br />
:Random variables <math>X_1, X_2, \ldots, X_n</math> are '''mutually independent''' if, for any subset <math>I\subseteq\{1,2,\ldots,n\}</math> and any values <math>x_i</math>, where <math>i\in I</math>,<br />
::<math>\begin{align}<br />
\Pr\left[\bigwedge_{i\in I}(X_i=x_i)\right]<br />
&=<br />
\prod_{i\in I}\Pr[X_i=x_i].<br />
\end{align}</math><br />
}}<br />
<br />
Mutual independence is an ideal condition of independence. The limited notion of independence is usually defined by the '''k-wise independence'''.<br />
{{Theorem<br />
|Definition (k-wise Independenc)|<br />
:1. Events <math>\mathcal{E}_1, \mathcal{E}_2, \ldots, \mathcal{E}_n</math> are '''k-wise independent''' if, for any subset <math>I\subseteq\{1,2,\ldots,n\}</math> with <math>|I|\le k</math><br />
:::<math>\begin{align}<br />
\Pr\left[\bigwedge_{i\in I}\mathcal{E}_i\right]<br />
&=<br />
\prod_{i\in I}\Pr[\mathcal{E}_i].<br />
\end{align}</math><br />
:2. Random variables <math>X_1, X_2, \ldots, X_n</math> are '''k-wise independent''' if, for any subset <math>I\subseteq\{1,2,\ldots,n\}</math> with <math>|I|\le k</math> and any values <math>x_i</math>, where <math>i\in I</math>,<br />
:::<math>\begin{align}<br />
\Pr\left[\bigwedge_{i\in I}(X_i=x_i)\right]<br />
&=<br />
\prod_{i\in I}\Pr[X_i=x_i].<br />
\end{align}</math><br />
}}<br />
<br />
A very common case is pairwise independence, i.e. the 2-wise independence.<br />
{{Theorem<br />
|Definition (pairwise Independent random variables)|<br />
:Random variables <math>X_1, X_2, \ldots, X_n</math> are '''pairwise independent''' if, for any <math>X_i,X_j</math> where <math>i\neq j</math> and any values <math>a,b</math><br />
:::<math>\begin{align}<br />
\Pr\left[X_i=a\wedge X_j=b\right]<br />
&=<br />
\Pr[X_i=a]\cdot\Pr[X_j=b].<br />
\end{align}</math><br />
}}<br />
<br />
Note that the definition of k-wise independence is hereditary:<br />
* If <math>X_1, X_2, \ldots, X_n</math> are k-wise independent, then they are also <math>\ell</math>-wise independent for any <math>\ell<k</math>.<br />
* If <math>X_1, X_2, \ldots, X_n</math> are NOT k-wise independent, then they cannot be <math>\ell</math>-wise independent for any <math>\ell>k</math>.<br />
<br />
== Pairwise Independent Bits ==<br />
Suppose we have <math>m</math> mutually independent and uniform random bits <math>X_1,\ldots, X_m</math>. We are going to extract <math>n=2^m-1</math> pairwise independent bits from these <math>m</math> mutually independent bits.<br />
<br />
Enumerate all the nonempty subsets of <math>\{1,2,\ldots,m\}</math> in some order. Let <math>S_j</math> be the <math>j</math>th subset. Let<br />
:<math><br />
Y_j=\bigoplus_{i\in S_j} X_i,<br />
</math><br />
where <math>\oplus</math> is the exclusive-or, whose truth table is as follows.<br />
:{|cellpadding="4" border="1"<br />
|-<br />
|<math>a</math><br />
|<math>b</math><br />
|<math>a</math><math>\oplus</math><math>b</math><br />
|-<br />
| 0 || 0 ||align="center"| 0<br />
|-<br />
| 0 || 1 ||align="center"| 1<br />
|-<br />
| 1 || 0 ||align="center"| 1<br />
|-<br />
| 1 || 1 ||align="center"| 0<br />
|}<br />
<br />
There are <math>n=2^m-1</math> such <math>Y_j</math>, because there are <math>2^m-1</math> nonempty subsets of <math>\{1,2,\ldots,m\}</math>. An equivalent definition of <math>Y_j</math> is<br />
:<math>Y_j=\left(\sum_{i\in S_j}X_i\right)\bmod 2</math>.<br />
Sometimes, <math>Y_j</math> is called the '''parity''' of the bits in <math>S_j</math>.<br />
<br />
We claim that <math>Y_j</math> are pairwise independent and uniform.<br />
<br />
{{Theorem<br />
|Theorem|<br />
:For any <math>Y_j</math> and any <math>b\in\{0,1\}</math>,<br />
::<math>\begin{align}<br />
\Pr\left[Y_j=b\right]<br />
&=<br />
\frac{1}{2}.<br />
\end{align}</math><br />
:For any <math>Y_j,Y_\ell</math> that <math>j\neq\ell</math> and any <math>a,b\in\{0,1\}</math>,<br />
::<math>\begin{align}<br />
\Pr\left[Y_j=a\wedge Y_\ell=b\right]<br />
&=<br />
\frac{1}{4}.<br />
\end{align}</math><br />
}}<br />
<br />
The proof is left for your exercise.<br />
<br />
Therefore, we extract exponentially many pairwise independent uniform random bits from a sequence of mutually independent uniform random bits.<br />
<br />
Note that <math>Y_j</math> are not 3-wise independent. For example, consider the subsets <math>S_1=\{1\},S_2=\{2\},S_3=\{1,2\}</math> and the corresponding random bits <math>Y_1,Y_2,Y_3</math>. Any two of <math>Y_1,Y_2,Y_3</math> would decide the value of the third one.<br />
<br />
== Pairwise Independent Variables ==<br />
We now consider constructing pairwise independent random variables ranging over <math>[p]=\{0,1,2,\ldots,p-1\}</math> for some prime <math>p</math>. Unlike the above construction, now we only need two independent random sources <math>X_0,X_1</math>, which are uniformly and independently distributed over <math>[p]</math>.<br />
<br />
Let <math>Y_0,Y_1,\ldots, Y_{p-1}</math> be defined as:<br />
:<math><br />
\begin{align}<br />
Y_i=(X_0+i\cdot X_1)\bmod p &\quad \mbox{for }i\in[p].<br />
\end{align}<br />
</math><br />
<br />
{{Theorem<br />
|Theorem|<br />
: The random variables <math>Y_0,Y_1,\ldots, Y_{p-1}</math> are pairwise independent uniform random variables over <math>[p]</math>.<br />
}}<br />
{{Proof| We first show that <math>Y_i</math> are uniform. That is, we will show that for any <math>i,a\in[p]</math>,<br />
:<math>\begin{align}<br />
\Pr\left[(X_0+i\cdot X_1)\bmod p=a\right]<br />
&=<br />
\frac{1}{p}.<br />
\end{align}</math><br />
Due to the law of total probability,<br />
:<math>\begin{align}<br />
\Pr\left[(X_0+i\cdot X_1)\bmod p=a\right]<br />
&=<br />
\sum_{j\in[p]}\Pr[X_1=j]\cdot\Pr\left[(X_0+ij)\bmod p=a\right]\\<br />
&=\frac{1}{p}\sum_{j\in[p]}\Pr\left[X_0\equiv(a-ij)\pmod{p}\right].<br />
\end{align}</math><br />
For prime <math>p</math>, for any <math>i,j,a\in[p]</math>, there is exact one value in <math>[p]</math> of <math>X_0</math> satisfying <math>X_0\equiv(a-ij)\pmod{p}</math>. Thus, <math>\Pr\left[X_0\equiv(a-ij)\pmod{p}\right]=1/p</math> and the above probability is <math>\frac{1}{p}</math>.<br />
<br />
We then show that <math>Y_i</math> are pairwise independent, i.e. we will show that for any <math>Y_i,Y_j</math> that <math>i\neq j</math> and any <math>a,b\in[p]</math>,<br />
:<math>\begin{align}<br />
\Pr\left[Y_i=a\wedge Y_j=b\right]<br />
&=<br />
\frac{1}{p^2}.<br />
\end{align}</math><br />
<br />
The event <math>Y_i=a\wedge Y_j=b</math> is equivalent to that<br />
:<math><br />
\begin{cases}<br />
(X_0+iX_1)\equiv a\pmod{p}\\<br />
(X_0+jX_1)\equiv b\pmod{p}<br />
\end{cases}<br />
</math><br />
Due to the [http://en.wikipedia.org/wiki/Chinese_remainder_theorem Chinese remainder theorem], there exists a unique solution of <math>X_0</math> and <math>X_1</math> in <math>[p]</math> to the above linear congruential system. Thus the probability of the event is <math>\frac{1}{p^2}</math>.<br />
}}<br />
<br />
=Universal Hashing =<br />
Hashing is one of the oldest tools in Computer Science. Knuth's memorandum in 1963 on analysis of hash tables is now considered to be the birth of the area of analysis of algorithms.<br />
* Knuth. Notes on "open" addressing, July 22 1963. Unpublished memorandum.<br />
<br />
The idea of hashing is simple: an unknown set <math>S</math> of <math>n</math> data '''items''' (or keys) are drawn from a large '''universe''' <math>U=[N]</math> where <math>N\gg n</math>; in order to store <math>S</math> in a table of <math>M</math> entries (slots), we assume a consistent mapping (called a '''hash function''') from the universe <math>U</math> to a small range <math>[M]</math>.<br />
<br />
This idea seems clever: we use a consistent mapping to deal with an arbitrary unknown data set. However, there is a fundamental flaw for hashing.<br />
* For sufficiently large universe (<math>N> M(n-1)</math>), for any function, there exists a bad data set <math>S</math>, such that all items in <math>S</math> are mapped to the same entry in the table.<br />
<br />
A simple use of pigeonhole principle can prove the above statement. <br />
<br />
To overcome this situation, randomization is introduced into hashing. We assume that the hash function is a random mapping from <math>[N]</math> to <math>[M]</math>. In order to ease the analysis, the following ideal assumption is used:<br />
<br />
'''Simple Uniform Hash Assumption''' ('''SUHA''' or '''UHA''', a.k.a. the random oracle model): <br />
:A ''uniform'' random function <math>h:[N]\rightarrow[M]</math> is available and the computation of <math>h</math> is efficient.<br />
<br />
== Families of universal hash functions ==<br />
The assumption of completely random function simplifies the analysis. However, in practice, truly uniform random hash function is extremely expensive to compute and store. Thus, this simple assumption can hardly represent the reality.<br />
<br />
There are two approaches for implementing practical hash functions. One is to use ''ad hoc'' implementations and wish they may work. The other approach is to construct class of hash functions which are efficient to compute and store but with weaker randomness guarantees, and then analyze the applications of hash functions based on this weaker assumption of randomness.<br />
<br />
This route was took by Carter and Wegman in 1977 while they introduced universal families of hash functions.<br />
<br />
{{Theorem<br />
|Definition (universal hash families)|<br />
:Let <math>[N]</math> be a universe with <math>N\ge M</math>. A family of hash functions <math>\mathcal{H}</math> from <math>[N]</math> to <math>[M]</math> is said to be '''<math>k</math>-universal''' if, for any items <math>x_1,x_2,\ldots,x_k\in [N]</math> and for a hash function <math>h</math> chosen uniformly at random from <math>\mathcal{H}</math>, we have <br />
::<math><br />
\Pr[h(x_1)=h(x_2)=\cdots=h(x_k)]\le\frac{1}{M^{k-1}}.<br />
</math><br />
<br />
:A family of hash functions <math>\mathcal{H}</math> from <math>[N]</math> to <math>[M]</math> is said to be '''strongly <math>k</math>-universal''' if, for any items <math>x_1,x_2,\ldots,x_k\in [N]</math>, any values <math>y_1,y_2,\ldots,y_k\in[M]</math>, and for a hash function <math>h</math> chosen uniformly at random from <math>\mathcal{H}</math>, we have <br />
::<math><br />
\Pr[h(x_1)=y_1\wedge h(x_2)=y_2 \wedge \cdots \wedge h(x_k)=y_k]=\frac{1}{M^{k}}.<br />
</math><br />
}}<br />
In particular, for a 2-universal family <math>\mathcal{H}</math>, for any elements <math>x_1,x_2\in[N]</math>, a uniform random <math>h\in\mathcal{H}</math> has<br />
:<math><br />
\Pr[h(x_1)=h(x_2)]\le\frac{1}{M}.<br />
</math><br />
For a strongly 2-universal family <math>\mathcal{H}</math>, for any elements <math>x_1,x_2\in[N]</math> and any values <math>y_1,y_2\in[M]</math>, a uniform random <math>h\in\mathcal{H}</math> has<br />
:<math><br />
\Pr[h(x_1)=y_1\wedge h(x_2)=y_2]=\frac{1}{M^2}.<br />
</math><br />
This behavior is exactly the same as uniform random hash functions on any pair of inputs. For this reason, a strongly 2-universal hash family are also called pairwise independent hash functions.<br />
<br />
== 2-universal hash families ==<br />
<br />
The construction of pairwise independent random variables via modulo a prime introduced in Section 1 already provides a way of constructing a strongly 2-universal hash family.<br />
<br />
Let <math>p</math> be a prime. The function <math>h_{a,b}:[p]\rightarrow [p]</math> is defined by<br />
:<math><br />
h_{a,b}(x)=(ax+b)\bmod p,<br />
</math><br />
and the family is<br />
:<math><br />
\mathcal{H}=\{h_{a,b}\mid a,b\in[p]\}.<br />
</math><br />
<br />
{{Theorem<br />
|Lemma|<br />
:<math>\mathcal{H}</math> is strongly 2-universal.<br />
}}<br />
{{Proof| In Section 1, we have proved the pairwise independence of the sequence of <math>(a i+b)\bmod p</math>, for <math>i=0,1,\ldots, p-1</math>, which directly implies that <math>\mathcal{H}</math> is strongly 2-universal.<br />
}}<br />
<br />
;The original construction of Carter-Wegman<br />
What if we want to have hash functions from <math>[N]</math> to <math>[M]</math> for non-prime <math>N</math> and <math>M</math>? Carter and Wegman developed the following method.<br />
<br />
Suppose that the universe is <math>[N]</math>, and the functions map <math>[N]</math> to <math>[M]</math>, where <math>N\ge M</math>. For some prime <math>p\ge N</math>, let<br />
:<math><br />
h_{a,b}(x)=((ax+b)\bmod p)\bmod M,<br />
</math><br />
and the family<br />
:<math><br />
\mathcal{H}=\{h_{a,b}\mid 1\le a\le p-1, b\in[p]\}.<br />
</math><br />
Note that unlike the first construction, now <math>a\neq 0</math>.<br />
{{Theorem<br />
|Lemma (Carter-Wegman)|<br />
:<math>\mathcal{H}</math> is 2-universal.<br />
}}<br />
{{Proof| Due to the definition of <math>\mathcal{H}</math>, there are <math>p(p-1)</math> many different hash functions in <math>\mathcal{H}</math>, because each hash function in <math>\mathcal{H}</math> corresponds to a pair of <math>1\le a\le p-1</math> and <math>b\in[p]</math>. We only need to count for any particular pair of <math>x_1,x_2\in[N]</math> that <math>x_1\neq x_2</math>, the number of hash functions that <math>h(x_1)=h(x_2)</math>.<br />
<br />
We first note that for any <math>x_1\neq x_2</math>, <math>a x_1+b\not\equiv a x_2+b \pmod p</math>. This is because <math>a x_1+b\equiv a x_2+b \pmod p</math> would imply that <math>a(x_1-x_2)\equiv 0\pmod p</math>, which can never happen since <math>1\le a\le p-1</math> and <math>x_1\neq x_2</math> (note that <math>x_1,x_2\in[N]</math> for an <math>N\le p</math>). Therefore, we can assume that <math>(a x_1+b)\bmod p=u</math> and <math>(a x_2+b)\bmod p=v</math> for <math>u\neq v</math>.<br />
<br />
By linear algebra (over finite field), for any <math>x_1,x_2\in[N]</math> that <math>x_1\neq x_2</math>, for any <math>u,v\in[p]</math> that <math>u\neq v</math>, there is exact one solution to <math>(a,b)</math> satisfying:<br />
:<math><br />
\begin{cases}<br />
a x_1+b \equiv u \pmod p\\<br />
a x_2+b \equiv v \pmod p.<br />
\end{cases}<br />
</math><br />
After modulo <math>M</math>, every <math>u\in[p]</math> has at most <math>\lceil p/M\rceil -1</math> many <math>v\in[p]</math> that <math>v\neq u</math> but <math>v\equiv u\pmod M</math>. Therefore, for every pair of <math>x_1,x_2\in[N]</math> that <math>x_1\neq x_2</math>, there exist at most <math>p(\lceil p/M\rceil -1)\le p(p-1)/M</math> pairs of <math>1\le a\le p-1</math> and <math>b\in[p]</math> such that <math>((ax_1+b)\bmod p)\bmod M=((ax_2+b)\bmod p)\bmod M</math>, which means there are at most <math> p(p-1)/M</math> many hash functions <math>h\in\mathcal{H}</math> having <math>h(x_1)=h(x_2)</math> for <math>x_1\neq x_2</math>. For <math>h</math> uniformly chosen from <math>\mathcal{H}</math>, for any <math>x_1\neq x_2</math>,<br />
:<math><br />
\Pr[h(x_1)=h(x_2)]\le \frac{p(p-1)/M}{p(p-1)}=\frac{1}{M}.<br />
</math><br />
We prove that <math>\mathcal{H}</math> is 2-universal.<br />
}}<br />
<br />
;A construction used in practice<br />
The main issue of Carter-Wegman construction is the efficiency. The mod operation is very slow, and has been so for more than 30 years.<br />
<br />
The following construction is due to Dietzfelbinger ''et al''. It was published in 1997 and has been practically used in various applications of universal hashing.<br />
<br />
The family of hash functions is from <math>[2^u]</math> to <math>[2^v]</math>. With a binary representation, the functions map binary strings of length <math>u</math> to binary strings of length <math>v</math>.<br />
Let<br />
:<math><br />
h_{a}(x)=\left\lfloor\frac{a\cdot x\bmod 2^u}{2^{u-v}}\right\rfloor,<br />
</math><br />
and the family<br />
:<math><br />
\mathcal{H}=\{h_{a}\mid a\in[2^v]\mbox{ and }a\mbox{ is odd}\}.<br />
</math><br />
<br />
This family of hash functions does not exactly meet the requirement of 2-universal family. However, Dietzfelbinger ''et al'' proved that <math>\mathcal{H}</math> is close to a 2-universal family. Specifically, for any input values <math>x_1,x_2\in[2^u]</math>, for a uniformly random <math>h\in\mathcal{H}</math>,<br />
:<math><br />
\Pr[h(x_1)=h(x_2)]\le\frac{1}{2^{v-1}}.<br />
</math><br />
So <math>\mathcal{H}</math> is within an approximation ratio of 2 to being 2-universal. The proof uses the fact that odd numbers are relative prime to a power of 2.<br />
<br />
The function is extremely simple to compute in c language.<br />
We exploit that C-multiplication (*) of unsigned u-bit numbers is done <math>\bmod 2^u</math>, and have a one-line C-code for computing the hash function:<br />
h_a(x) = (a*x)>>(u-v)<br />
The bit-wise shifting is a lot faster than modular. It explains the popularity of this scheme in practice than the original Carter-Wegman construction.</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=%E9%AB%98%E7%BA%A7%E7%AE%97%E6%B3%95_(Fall_2026)/Hashing_and_Sketching&diff=13904高级算法 (Fall 2026)/Hashing and Sketching2026-09-07T10:12:58Z<p>Etone: Created page with "=Balls into Bins= The following is the so-called balls into bins model. Consider throwing <math>m</math> balls into <math>n</math> bins uniformly and independently at random. This is equivalent to a random mapping <math>f:[m]\to[n]</math>. Needless to say, random mapping is an important random model and may have many applications in Computer Science, e.g. hashing. We are concerned with the following three questions regarding the balls into bins model: * birthday problem..."</p>
<hr />
<div>=Balls into Bins=<br />
The following is the so-called balls into bins model.<br />
Consider throwing <math>m</math> balls into <math>n</math> bins uniformly and independently at random. This is equivalent to a random mapping <math>f:[m]\to[n]</math>. Needless to say, random mapping is an important random model and may have many applications in Computer Science, e.g. hashing.<br />
<br />
We are concerned with the following three questions regarding the balls into bins model:<br />
* birthday problem: the probability that every bin contains at most one ball (the mapping is 1-1);<br />
* coupon collector problem: the probability that every bin contains at least one ball (the mapping is on-to);<br />
* occupancy problem: the maximum load of bins.<br />
<br />
== Birthday Problem==<br />
We now consider the '''birthday problem'''.<br />
There are <math>m</math> students in the class. Assume that for each student, his/her birthday is uniformly and independently distributed over the 365 days in a years. We wonder what the probability that no two students share a birthday.<br />
<br />
Due to the [http://en.wikipedia.org/wiki/Pigeonhole_principle pigeonhole principle], it is obvious that for <math>m>365</math>, there must be two students with the same birthday. Surprisingly, for any <math>m>57</math> this event occurs with more than 99% probability. This is called the [http://en.wikipedia.org/wiki/Birthday_problem '''birthday paradox''']. Despite the name, the birthday paradox is not a real paradox.<br />
<br />
We can model this problem as a balls-into-bins problem. <math>m</math> different balls (students) are uniformly and independently thrown into 365 bins (days). More generally, let <math>n</math> be the number of bins. We ask for the probability of the following event <math>\mathcal{E}</math><br />
<br />
* <math>\mathcal{E}</math>: there is no bin with more than one balls (i.e. no two students share birthday).<br />
<br />
We first analyze this by counting. There are totally <math>n^m</math> ways of assigning <math>m</math> balls to <math>n</math> bins. The number of assignments that no two balls share a bin is <math>{n\choose m}m!</math>.<br />
<br />
Thus the probability is given by:<br />
:<math>\begin{align}<br />
\Pr[\mathcal{E}]<br />
=<br />
\frac{{n\choose m}m!}{n^m}.<br />
\end{align}<br />
</math><br />
<br />
Recall that <math>{n\choose m}=\frac{n!}{(n-m)!m!}</math>. Then <br />
:<math>\begin{align}<br />
\Pr[\mathcal{E}]<br />
=<br />
\frac{{n\choose m}m!}{n^m}<br />
=<br />
\frac{n!}{n^m(n-m)!}<br />
=<br />
\frac{n}{n}\cdot\frac{n-1}{n}\cdot\frac{n-2}{n}\cdots\frac{n-(m-1)}{n}<br />
=<br />
\prod_{k=1}^{m-1}\left(1-\frac{k}{n}\right).<br />
\end{align}<br />
</math><br />
<br />
There is also a more "probabilistic" argument for the above equation. Consider again that <math>m</math> students are mapped to <math>n</math> possible birthdays uniformly at random.<br />
<br />
The first student has a birthday for sure. The probability that the second student has a different birthday from the first student is <math>\left(1-\frac{1}{n}\right)</math>. Given that the first two students have different birthdays, the probability that the third student has a different birthday from the first two students is <math>\left(1-\frac{2}{n}\right)</math>. Continuing this on, assuming that the first <math>k-1</math> students all have different birthdays, the probability that the <math>k</math>th student has a different birthday than the first <math>k-1</math>, is given by <math>\left(1-\frac{k-1}{n}\right)</math>. By the chain rule, the probability that all <math>m</math> students have different birthdays is:<br />
:<math>\begin{align}<br />
\Pr[\mathcal{E}]=\left(1-\frac{1}{n}\right)\cdot \left(1-\frac{2}{n}\right)\cdots \left(1-\frac{m-1}{n}\right)<br />
&=<br />
\prod_{k=1}^{m-1}\left(1-\frac{k}{n}\right),<br />
\end{align}<br />
</math><br />
which is the same as what we got by the counting argument.<br />
<br />
[[File:Birthday.png|border|450px|right]]<br />
<br />
There are several ways of analyzing this formular. Here is a convenient one: Due to [http://en.wikipedia.org/wiki/Taylor_series Taylor's expansion], <math>e^{-k/n}\approx 1-k/n</math>. Then<br />
:<math>\begin{align}<br />
\prod_{k=1}^{m-1}\left(1-\frac{k}{n}\right)<br />
&\approx<br />
\prod_{k=1}^{m-1}e^{-\frac{k}{n}}\\<br />
&=<br />
\exp\left(-\sum_{k=1}^{m-1}\frac{k}{n}\right)\\<br />
&=<br />
e^{-m(m-1)/2n}\\<br />
&\approx<br />
e^{-m^2/2n}.<br />
\end{align}</math><br />
The quality of this approximation is shown in the Figure.<br />
<br />
Therefore, for <math>m=\sqrt{2n\ln \frac{1}{\epsilon}}</math>, the probability that <math>\Pr[\mathcal{E}]\approx\epsilon</math>.<br />
<br />
==Universal Hashing ==<br />
Hashing is one of the oldest tools in Computer Science. Knuth's memorandum in 1963 on analysis of hash tables is now considered to be the birth of the area of analysis of algorithms.<br />
* Knuth. Notes on "open" addressing, July 22 1963. Unpublished memorandum.<br />
<br />
The idea of hashing is simple: an unknown set <math>S</math> of <math>n</math> data '''items''' (or keys) are drawn from a large '''universe''' <math>U=[N]</math> where <math>N\gg n</math>; in order to store <math>S</math> in a table of <math>M</math> entries (slots), we assume a consistent mapping (called a '''hash function''') from the universe <math>U</math> to a small range <math>[M]</math>.<br />
<br />
This idea seems clever: we use a consistent mapping to deal with an arbitrary unknown data set. However, there is a fundamental flaw for hashing.<br />
* For sufficiently large universe (<math>N> M(n-1)</math>), for any function, there exists a bad data set <math>S</math>, such that all items in <math>S</math> are mapped to the same entry in the table.<br />
<br />
A simple use of pigeonhole principle can prove the above statement. <br />
<br />
To overcome this situation, randomization is introduced into hashing. We assume that the hash function is a random mapping from <math>[N]</math> to <math>[M]</math>. In order to ease the analysis, the following ideal assumption is used:<br />
<br />
'''Simple Uniform Hash Assumption''' ('''SUHA''' or '''UHA''', a.k.a. the random oracle model): <br />
:A ''uniform'' random function <math>h:[N]\rightarrow[M]</math> is available and the computation of <math>h</math> is efficient.<br />
<br />
=== Families of universal hash functions ===<br />
The assumption of completely random function simplifies the analysis. However, in practice, truly uniform random hash function is extremely expensive to compute and store. Thus, this simple assumption can hardly represent the reality.<br />
<br />
There are two approaches for implementing practical hash functions. One is to use ''ad hoc'' implementations and wish they may work. The other approach is to construct class of hash functions which are efficient to compute and store but with weaker randomness guarantees, and then analyze the applications of hash functions based on this weaker assumption of randomness.<br />
<br />
This route was took by Carter and Wegman in 1977 while they introduced universal families of hash functions.<br />
<br />
{{Theorem<br />
|Definition (universal hash families)|<br />
:Let <math>[N]</math> be a universe with <math>N\ge M</math>. A family of hash functions <math>\mathcal{H}</math> from <math>[N]</math> to <math>[M]</math> is said to be '''<math>k</math>-universal''' if, for any distinct items <math>x_1,x_2,\ldots,x_k\in [N]</math> and for a hash function <math>h</math> chosen uniformly at random from <math>\mathcal{H}</math>, we have <br />
::<math><br />
\Pr[h(x_1)=h(x_2)=\cdots=h(x_k)]\le\frac{1}{M^{k-1}}.<br />
</math><br />
<br />
:A family of hash functions <math>\mathcal{H}</math> from <math>[N]</math> to <math>[M]</math> is said to be '''strongly <math>k</math>-universal''' if, for any distinct items <math>x_1,x_2,\ldots,x_k\in [N]</math>, any values <math>y_1,y_2,\ldots,y_k\in[M]</math>, and for a hash function <math>h</math> chosen uniformly at random from <math>\mathcal{H}</math>, we have <br />
::<math><br />
\Pr[h(x_1)=y_1\wedge h(x_2)=y_2 \wedge \cdots \wedge h(x_k)=y_k]=\frac{1}{M^{k}}.<br />
</math><br />
}}<br />
In particular, for a 2-universal family <math>\mathcal{H}</math>, for any distinct elements <math>x_1,x_2\in[N]</math>, a uniform random <math>h\in\mathcal{H}</math> has<br />
:<math><br />
\Pr[h(x_1)=h(x_2)]\le\frac{1}{M}.<br />
</math><br />
For a strongly 2-universal family <math>\mathcal{H}</math>, for any distinct elements <math>x_1,x_2\in[N]</math> and any values <math>y_1,y_2\in[M]</math>, a uniform random <math>h\in\mathcal{H}</math> has<br />
:<math><br />
\Pr[h(x_1)=y_1\wedge h(x_2)=y_2]=\frac{1}{M^2}.<br />
</math><br />
This behavior is exactly the same as uniform random hash functions on any distinct pair of inputs. For this reason, a strongly 2-universal hash family are also called pairwise independent hash functions.<br />
<br />
=== 2-universal hash families ===<br />
<br />
The construction of pairwise independent random variables via modulo a prime introduced in Section 1 already provides a way of constructing a strongly 2-universal hash family.<br />
<br />
Let <math>p</math> be a prime. The function <math>h_{a,b}:[p]\rightarrow [p]</math> is defined by<br />
:<math><br />
h_{a,b}(x)=(ax+b)\bmod p,<br />
</math><br />
and the family is<br />
:<math><br />
\mathcal{H}=\{h_{a,b}\mid a,b\in[p]\}.<br />
</math><br />
<br />
{{Theorem<br />
|Lemma|<br />
:<math>\mathcal{H}</math> is strongly 2-universal.<br />
}}<br />
{{Proof| In Section 1, we have proved the pairwise independence of the sequence of <math>(a i+b)\bmod p</math>, for <math>i=0,1,\ldots, p-1</math>, which directly implies that <math>\mathcal{H}</math> is strongly 2-universal.<br />
}}<br />
<br />
;The original construction of Carter-Wegman<br />
What if we want to have hash functions from <math>[N]</math> to <math>[M]</math> for non-prime <math>N</math> and <math>M</math>? Carter and Wegman developed the following method.<br />
<br />
Suppose that the universe is <math>[N]</math>, and the functions map <math>[N]</math> to <math>[M]</math>, where <math>N\ge M</math>. For some prime <math>p\ge N</math>, let<br />
:<math><br />
h_{a,b}(x)=((ax+b)\bmod p)\bmod M,<br />
</math><br />
and the family<br />
:<math><br />
\mathcal{H}=\{h_{a,b}\mid 1\le a\le p-1, b\in[p]\}.<br />
</math><br />
Note that unlike the first construction, now <math>a\neq 0</math>.<br />
{{Theorem<br />
|Lemma (Carter-Wegman)|<br />
:<math>\mathcal{H}</math> is 2-universal.<br />
}}<br />
{{Proof| Due to the definition of <math>\mathcal{H}</math>, there are <math>p(p-1)</math> many different hash functions in <math>\mathcal{H}</math>, because each hash function in <math>\mathcal{H}</math> corresponds to a pair of <math>1\le a\le p-1</math> and <math>b\in[p]</math>. We only need to count for any particular pair of <math>x_1,x_2\in[N]</math> that <math>x_1\neq x_2</math>, the number of hash functions that <math>h(x_1)=h(x_2)</math>.<br />
<br />
We first note that for any <math>x_1\neq x_2</math>, <math>a x_1+b\not\equiv a x_2+b \pmod p</math>. This is because <math>a x_1+b\equiv a x_2+b \pmod p</math> would imply that <math>a(x_1-x_2)\equiv 0\pmod p</math>, which can never happen since <math>1\le a\le p-1</math> and <math>x_1\neq x_2</math> (note that <math>x_1,x_2\in[N]</math> for an <math>N\le p</math>). Therefore, we can assume that <math>(a x_1+b)\bmod p=u</math> and <math>(a x_2+b)\bmod p=v</math> for <math>u\neq v</math>.<br />
<br />
By linear algebra (over finite field), for any <math>x_1,x_2\in[N]</math> that <math>x_1\neq x_2</math>, for any <math>u,v\in[p]</math> that <math>u\neq v</math>, there is exact one solution to <math>(a,b)</math> satisfying:<br />
:<math><br />
\begin{cases}<br />
a x_1+b \equiv u \pmod p\\<br />
a x_2+b \equiv v \pmod p.<br />
\end{cases}<br />
</math><br />
After modulo <math>M</math>, every <math>u\in[p]</math> has at most <math>\lceil p/M\rceil -1</math> many <math>v\in[p]</math> that <math>v\neq u</math> but <math>v\equiv u\pmod M</math>. Therefore, for every pair of <math>x_1,x_2\in[N]</math> that <math>x_1\neq x_2</math>, there exist at most <math>p(\lceil p/M\rceil -1)\le p(p-1)/M</math> pairs of <math>1\le a\le p-1</math> and <math>b\in[p]</math> such that <math>((ax_1+b)\bmod p)\bmod M=((ax_2+b)\bmod p)\bmod M</math>, which means there are at most <math> p(p-1)/M</math> many hash functions <math>h\in\mathcal{H}</math> having <math>h(x_1)=h(x_2)</math> for <math>x_1\neq x_2</math>. For <math>h</math> uniformly chosen from <math>\mathcal{H}</math>, for any <math>x_1\neq x_2</math>,<br />
:<math><br />
\Pr[h(x_1)=h(x_2)]\le \frac{p(p-1)/M}{p(p-1)}=\frac{1}{M}.<br />
</math><br />
We prove that <math>\mathcal{H}</math> is 2-universal.<br />
}}<br />
<br />
;A construction used in practice<br />
The main issue of Carter-Wegman construction is the efficiency. The mod operation is very slow, and has been so for more than 30 years.<br />
<br />
The following construction is due to Dietzfelbinger ''et al''. It was published in 1997 and has been practically used in various applications of universal hashing.<br />
<br />
The family of hash functions is from <math>[2^u]</math> to <math>[2^v]</math>. With a binary representation, the functions map binary strings of length <math>u</math> to binary strings of length <math>v</math>.<br />
Let<br />
:<math><br />
h_{a}(x)=\left\lfloor\frac{a\cdot x\bmod 2^u}{2^{u-v}}\right\rfloor,<br />
</math><br />
and the family<br />
:<math><br />
\mathcal{H}=\{h_{a}\mid a\in[2^v]\mbox{ and }a\mbox{ is odd}\}.<br />
</math><br />
<br />
This family of hash functions does not exactly meet the requirement of 2-universal family. However, Dietzfelbinger ''et al'' proved that <math>\mathcal{H}</math> is close to a 2-universal family. Specifically, for any distinct input values <math>x_1,x_2\in[2^u]</math>, for a uniformly random <math>h\in\mathcal{H}</math>,<br />
:<math><br />
\Pr[h(x_1)=h(x_2)]\le\frac{1}{2^{v-1}}.<br />
</math><br />
So <math>\mathcal{H}</math> is within an approximation ratio of 2 to being 2-universal. The proof uses the fact that odd numbers are relative prime to a power of 2.<br />
<br />
The function is extremely simple to compute in c language.<br />
We exploit that C-multiplication (*) of unsigned u-bit numbers is done <math>\bmod 2^u</math>, and have a one-line C-code for computing the hash function:<br />
h_a(x) = (a*x)>>(u-v)<br />
The bit-wise shifting is a lot faster than modular. It explains the popularity of this scheme in practice than the original Carter-Wegman construction.<br />
<br />
== Collision number ==<br />
Consider a 2-universal family <math>\mathcal{H}</math> of hash functions from <math>[N]</math> to <math>[M]</math>. Let <math>h</math> be a hash function chosen uniformly from <math>\mathcal{H}</math>. For a fixed set <math>S</math> of <math>n</math> distinct elements from <math>[N]</math>, say <math>S=\{x_1,x_2,\ldots,x_n\}</math>, the elements are mapped to the hash values <math>h(x_1), h(x_2), \ldots, h(x_n)</math>. This can be seen as throwing <math>n</math> balls to <math>M</math> bins, with pairwise independent choices of bins.<br />
<br />
As in the balls-into-bins with full independence, we are curious about the questions such as the birthday problem or the maximum load. These questions are interesting not only because they are natural to ask in a balls-into-bins setting, but in the context of hashing, they are closely related to the performance of hash functions.<br />
<br />
The old techniques for analyzing balls-into-bins rely too much on the independence of the choice of the bin for each ball, therefore can hardly be extended to the setting of 2-universal hash families. However, it turns out several balls-into-bins questions can somehow be answered by analyzing a very natural quantity: the number of '''collision pairs'''.<br />
<br />
A collision pair for hashing is a pair of elements <math>x_1,x_2\in S</math> which are mapped to the same hash value, i.e. <math>h(x_1)=h(x_2)</math>. Formally, for a fixed set of elements <math>S=\{x_1,x_2,\ldots,x_n\}</math>, for any <math>1\le i,j\le n</math>, let the random variable<br />
:<math><br />
X_{ij}<br />
=<br />
\begin{cases}<br />
1 & \text{if }h(x_i)=h(x_j),\\<br />
0 & \text{otherwise.}<br />
\end{cases}<br />
</math><br />
The total number of collision pairs among the <math>n</math> items <math>x_1,x_2,\ldots,x_n</math> is <br />
:<math>X=\sum_{i<j} X_{ij}.\,</math><br />
<br />
Since <math>\mathcal{H}</math> is 2-universal, for any <math>i\neq j</math>,<br />
:<math><br />
\Pr[X_{ij}=1]=\Pr[h(x_i)=h(x_j)]\le\frac{1}{M}.<br />
</math><br />
<br />
The expected number of collision pairs is <br />
:<math>\mathbf{E}[X]=\mathbf{E}\left[\sum_{i<j}X_{ij}\right]=\sum_{i<j}\mathbf{E}[X_{ij}]=\sum_{i<j}\Pr[X_{ij}=1]\le{n\choose 2}\frac{1}{M}<\frac{n^2}{2M}.<br />
</math><br />
<br />
In particular, for <math>n=M</math>, i.e. <math>n</math> items are mapped to <math>n</math> hash values by a pairwise independent hash function, the expected collision number is <math>\mathbf{E}[X]<\frac{n^2}{2M}=\frac{n}{2}</math>.<br />
<br />
The above analysis gives us an estimation on the expected number of collision pairs, such that <math>\mathbf{E}[X]<\frac{n^2}{2M}</math>. Apply the Markov's inequality, for <math>0<\epsilon<1</math>, we have<br />
:<math><br />
\Pr\left[X\ge \frac{n^2}{2\epsilon M}\right]\le\Pr\left[X\ge \frac{1}{\epsilon}\mathbf{E}[X]\right]\le\epsilon.<br />
</math><br />
<br />
When <math>n\le\sqrt{2\epsilon M}</math>, the number of collision pairs is <math>X\ge1</math> with probability at most <math>\epsilon</math>, therefore with probability at least <math>1-\epsilon</math>, there is no collision at all. Therefore, we have the following theorem.<br />
{{Theorem<br />
|Theorem|<br />
:If <math>h</math> is chosen uniformly from a 2-universal family of hash functions mapping the universe <math>[N]</math> to <math>[M]</math> where <math>N\ge M</math>, then for any set <math>S\subset [N]</math> of <math>n</math> items, where <math>n\le\sqrt{2\epsilon M}</math>, the probability that there exits a collision pair is<br />
::<math><br />
\Pr[\mbox{collision occurs}]\le\epsilon.<br />
</math><br />
}}<br />
<br />
Recall that for mutually independent choices of bins, for some <math>n=\sqrt{2M\ln(1/\epsilon)}</math>, the probability that a collision occurs is about <math>\epsilon</math>. For constant <math>\epsilon</math>, this gives an essentially same bound as the pairwise independent setting. Therefore, <br />
the behavior of pairwise independent hash function is essentially the same as the uniform random hash function for the birthday problem. This is easy to understand, because birthday problem is about the behavior of collisions, and the definition of 2-universal hash function can be interpreted as "functions that the probability of collision is as low as a uniform random function".<br />
<br />
= Set Membership=<br />
A basic question in Computer Science is:<br />
:"<math>\mbox{Is }x\in S?</math>"<br />
for a set <math>S</math> and an element <math>x</math>. This is the '''set membership''' problem.<br />
<br />
Formally, given an arbitrary set <math>S</math> of <math>n</math> elements from a universe <math>U</math>, we want to use a succinct '''data structure''' to represent this set <math>S</math>, so that upon each '''query''' of any element <math>x</math> from the universe <math>[N]</math>, the question of whether <math>x\in S</math> is efficiently answered. The complexity of such data structure is measured in two-fold:<br />
* '''space cost''': size of the data structure to represent a set <math>S</math> of size <math>n</math>;<br />
* '''time cost''': time complexity of answering each query by accessing to the data structure.<br />
<br />
Suppose that the universe <math>U</math> is of size <math>N</math>. Clearly, the membership problem can be solved by a '''dictionary data structure''', e.g.:<br />
* '''sorted table / balanced search tree''': with space cost <math>O(n\log N)</math> bits and time cost <math>O(\log n)</math>.<br />
<br />
Note that <math>\log{N\choose n}=\Theta\left(n\log \frac{N}{n}\right)</math> is the entropy of sets <math>S</math> of <math>n</math> elements from a universe <math>U</math> of size <math>N</math>. Therefore it is necessary to use so many bits to represent a set without losing any information. <br />
With hashing, we can solve this fundamental problem with asymptotic optimal space cost and time cost at the same time.<br />
<br />
== Perfect hashing using quadratic space==<br />
The idea of perfect hashing is that we use a hash function <math>h</math> to map the <math>n</math> items to distinct entries of the table; store every item <math>x\in S</math> in the entry <math>h(x)</math>; and also store the hash function <math>h</math> in a fixed location in the table (usually the beginning of the table). The algorithm for searching for an item is as follows:<br />
<br />
:search for <math>x</math> in table <math>T</math>:<br />
# retrieve <math>h</math> from a fixed location in the table;<br />
# if <math>x=T[h(x)]</math> return <math>h(x)</math>; else return NOT_FOUND;<br />
<br />
This scheme works as long as that the hash function satisfies the following two conditions:<br />
* The description of <math>h</math> is sufficiently short, so that <math>h</math> can be stored in one entry (or in constant many entries) of the table.<br />
* <math>h</math> has no collisions on <math>S</math>, i.e. there is no pair of items <math>x_1,x_2\in S</math> that are mapped to the same value by <math>h</math>.<br />
<br />
The first condition is easy to guarantee for 2-universal hash families. As shown by Carter-Wegman construction, a 2-universal hash function can be uniquely represented by two integers <math>a</math> and <math>b</math>, which can be stored in two entries (or just one, if the word length is sufficiently large) of the table.<br />
<br />
Our discussion is now focused on the second condition. We find that it relies on the ''perfectness'' of the hash function for a data set <math>S</math>.<br />
<br />
A hash function <math>h</math> is '''perfect''' for a set <math>S</math> of items if <math>h</math> maps all items in <math>S</math> to different values, i.e. there is no collision.<br />
<br />
We have shown by the birthday problem for 2-universal hashing that when <math>n</math> items are mapped to <math>n^2</math> values, for an <math>h</math> chosen uniformly from a 2-universal family of hash functions, the probability that a collision occurs is at most 1/2. Thus<br />
:<math><br />
\Pr[h\mbox{ is perfect for }S]\ge\frac{1}{2}<br />
</math><br />
for a table of <math>n^2</math> entries.<br />
<br />
The construction of perfect hashing is straightforward then:<br />
:For a set <math>S</math> of <math>n</math> elements:<br />
# uniformly choose an <math>h</math> from a 2-universal family <math>\mathcal{H}</math>; (for Carter-Wegman's construction, it means uniformly choose two integer <math>1\le a\le p-1</math> and <math>b\in[p]</math> for a sufficiently large prime <math>p</math>.)<br />
# check whether <math>h</math> is perfect for <math>S</math>;<br />
# if <math>h</math> is NOT perfect for <math>S</math>, start over again; otherwise, construct the table;<br />
<br />
This is a Las Vegas randomized algorithm, which construct a perfect hashing for a fixed set <math>S</math> with expectedly at most two trials (due to geometric distribution). The resulting data structure is a <math>O(n^2)</math>-size static dictionary of <math>n</math> elements which answers every search in deterministic <math>O(1)</math> time.<br />
<br />
== FKS perfect hashing ==<br />
In the last section we see how to use <math>O(n^2)</math> space and constant time for answering search in a set. Now we see how to do it with linear space and constant time. This solves the problem of searching asymptotically optimal for both time and space.<br />
<br />
This was once seemingly impossible, until Yao's seminal paper:<br />
*Yao. Should tables be sorted? ''Journal of the ACM (JACM)'', 1981.<br />
<br />
Yao's paper shows a possibility of achieving linear space and constant time at the same time by exploiting the power of hashing, but assumes an unrealistically large universe. <br />
<br />
Inspired by Yao's work, Fredman, Komlós, and Szemerédi discover the first linear-space and constant-time static dictionary in a realistic setting: <br />
* Fredman, Komlós, and Szemerédi. Storing a sparse table with O(1) worst case access time. ''Journal of the ACM (JACM)'', 1984.<br />
<br />
The idea of FKS hashing is to arrange hash table in two levels:<br />
* In the first level, <math>n</math> items are hashed to <math>n</math> ''buckets'' by a 2-universal hash function <math>h</math>. <br />
: Let <math>B_i</math> be the set of items hashed to the <math>i</math>th bucket.<br />
* In the second level, construct a <math>|B_i|^2</math>-size perfect hashing for each bucket <math>B_i</math>.<br />
<br />
The data structure can be stored in a table. The first few entries are reserved to store the primary hash function <math>h</math>. To help the searching algorithm locate a bucket, we use the next <math>n</math> entries of the table as the "pointers" to the bucket: each entry stores the address of the first entry of the space to store a bucket. In the rest of table, the <math>n</math> buckets are stored in order, each using a <math>|B_i|^2</math> space as required by perfect hashing.<br />
<br />
::[[File:FKS.png|600px]]<br />
<br />
It is easy to see that the search time is constant. To search for an item <math>x</math>, the algorithm does the followings:<br />
* Retrieve <math>h</math>.<br />
* Retrieve the address for bucket <math>h(x)</math>.<br />
* Search by perfect hashing within bucket <math>h(x)</math>.<br />
Each line takes constant time. So the worst-case search time is constant.<br />
<br />
We then need to guarantee that the space is linear to <math>n</math>. At the first glance, this seems impossible because each instance of perfect hashing for a bucket costs a square-size of space. We will prove that although the individual buckets use square-sized spaces, the sum of the them is still linear.<br />
<br />
For a fixed set <math>S</math> of <math>n</math> items, for a hash function <math>h</math> chosen uniformly from a 2-universe family which maps the items to <math>[n]</math>, called <math>n</math> ''buckets'', let <math>Y_i=|B_i|</math> be the number of items in <math>S</math> mapped to the <math>i</math>th bucket.<br />
We are going to bound the following quantity:<br />
:<math><br />
Y=\sum_{i=1}^n Y_i^2.<br />
</math><br />
Since each bucket <math>B_i</math> use a space of <math>Y_i^2</math> for perfect hashing. <math>Y</math> gives the size of the space for storing the buckets. <br />
<br />
We will show that <math>Y</math> is related to the total number of collision pairs. (Indeed, the number of collision pairs can be computed by a degree-2 polynomial, just like <math>Y</math>.)<br />
<br />
Note that a bucket of <math>Y_i</math> items contributes <math>{Y_i\choose 2}</math> collision pairs. Let <math>X</math> be the total number of collision pairs.<br />
<math>X</math> can be computed by summing over the collision pairs in every bucket:<br />
:<math><br />
X=\sum_{i=1}^n{Y_i\choose 2}=\sum_{i=1}^n\frac{Y_i(Y_i-1)}{2}=\frac{1}{2}\left(\sum_{i=1}^nY_i^2-\sum_{i=1}^nY_i\right)=\frac{1}{2}\left(\sum_{i=1}^nY_i^2-n\right).<br />
</math><br />
<br />
Therefore, the sum of squares of the sizes of buckets is related to collision number by:<br />
:<math><br />
\sum_{i=1}^nY_i^2=2X+n.<br />
</math><br />
By our analysis of the collision number, we know that for <math>n</math> items mapped to <math>n</math> buckets, the expected number of collision pairs is: <math>\mathbf{E}[X]\le \frac{n}{2}</math>.<br />
Thus,<br />
:<math><br />
\mathbf{E}\left[\sum_{i=1}^nY_i^2\right]=\mathbf{E}[2X+n]\le 2n.<br />
</math><br />
Due to Markov's inequality, <math>\sum_{i=1}^nY_i^2=O(n)</math> with a constant probability. For any set <math>S</math>, we can find a suitable <math>h</math> after expected constant number of trials, and FKS can be constructed with guaranteed (instead of expected) linear-size which answers each search in constant time.<br />
<br />
== Bloom filter ==<br />
Now we consider the lossy representation of the original data set <math>S</math>, to further save the space usage. Such lossy data structure is sometimes called a '''''sketch'''''.<br />
<br />
The Bloom filter is such a lossy data structure. It is a space-efficient hash table that solves the '''approximate membership''' problem with one-sided error (''false positive'').<br />
<br />
Given a set <math>S</math> of <math>n</math> elements from a universe <math>U</math>, a Bloom filter consists of an array <math>A</math> of <math>cn</math> bits, and <math>k</math> hash functions <math>h_1,h_2,\ldots,h_k</math> map <math>U</math> to <math>[cn]</math>, where both <math>c</math> and <math>k</math> are parameters that we can try to optimize later.<br />
<br />
As before, we assume the '''Uniform Hash Assumption (UHA)''': <math>h_1,h_2,\ldots,h_k</math> are mutually independent hash function where each <math>h_i</math> is a uniform random hash function <math>h_i:U\to[cn]</math>.<br />
<br />
The Bloom filter works as follows:<br />
{{Theorem|''Bloom filter'' (Bloom 1970)|<br />
:Suppose <math>h_1,h_2,\ldots,h_k:U\to[cn]</math> are uniform and independent random hash functions.<br />
-----<br />
:'''Data structure construction:''' Given a set <math>S\subset U</math> of size <math>n=|S|</math>, the data structure is a Boolean array <math>A</math> of <math>cn</math> bits constructed as<br />
:* initialize all <math>cn</math> bits of the Boolean array <math>A</math> to 0;<br />
:* for each <math>x\in S</math>, let <math>A[h_i(x)]=1</math> for all <math>1\le i\le k</math>.<br />
----<br />
:'''Query resolution:''' Upon each query of an arbitrary <math>x\in U</math>,<br />
:* answer "yes" if <math>A[h_i(x)]=1</math> for all <math>1\le i\le k</math> and "no" if otherwise.<br />
}}<br />
The Boolean array is our data structure, whose size is <math>cn</math> bits. With Uniform Hash Assumption (UHA), the time cost of the data structure for answering each query is <math>O(k)</math>.<br />
<br />
When the answer returned by the algorithm is "no", it holds that <math>A[h_i(x)]=0</math> for some <math>1\le i\le k</math>, in which case the query <math>x</math> must not belong to the set <math>S</math>. Thus, the Bloom filter has no false negatives.<br />
<br />
On the other hand, when the answer returned by the algorithm is "yes", <math>A[h_i(x)]=1</math> for all <math>1\le i\le k</math>. It is still possible for some <math>x\not\in S</math> that all bits <math>A[h_i(x)]</math> are set by elements in <math>S</math>. We want to bound such false positive, that is, the following probability for an <math>x\not\in S</math>:<br />
:<math>\Pr[\,\forall 1\le i\le k, A[h_i(x)]=1\,]</math>,<br />
which by independence between different hash functions and by symmetry is equal to:<br />
:<math>\Pr[\, A[h_1(x)]=1\,]^k=(1-\Pr[\, A[h_1(x)]=0\,])^k</math>.<br />
For an element <math>x\not\in S</math>, its hash value <math>h_1(x)</math> is independent of all hash values <math>h_i(y)</math> for all <math>1\le i\le k</math> and all <math>y\in S</math>. This is due to the Uniform Hash Assumption. The hash value <math>h_1(x)</math> of <math>x\not\in S</math> is then independent of the content of the array <math>A</math>. Therefore, the probability of this position <math>A[h_1(x)]</math> missed by all <math>kn</math> updates to the Boolean array <math>A</math> caused by all <math>n</math> elements in <math>S</math> is:<br />
:<math><br />
\Pr[\, A[h_1(x)]=0\,]=\left(1-\frac{1}{cn}\right)^{kn}\approx e^{-k/c}.<br />
</math><br />
<br />
Putting everything together, for any <math>x\not\in S</math>, the false positive is bounded as:<br />
:<math><br />
\begin{align}<br />
\Pr[\,\text{wrongly answer ''yes''}\,]<br />
&=\Pr[\,\forall 1\le i\le k, A[h_i(x)]=1\,]\\<br />
&=\Pr[\, A[h_1(x)]=1\,]^k=(1-\Pr[\, A[h_1(x)]=0\,])^k\\<br />
&=\left(1-\left(1-\frac{1}{cn}\right)^{kn}\right)^k\\<br />
&\approx \left(1- e^{-k/c}\right)^k<br />
\end{align}<br />
</math><br />
which is <math>(0.6185)^c</math> when <math>k=c\ln 2</math>.<br />
<br />
Bloom filter solves the membership query with a small constant error of false positives using a data structure of <math>O(n)</math> bits which answers each query with <math>O(1)</math> time cost.<br />
<br />
=Distinct Elements=<br />
Consider the following problem of '''counting distinct elements''': Suppose that <math>U</math> is a sufficiently large universe.<br />
*'''Input:''' a sequence of (not necessarily distinct) elements <math>x_1,x_2,\ldots,x_n\in U</math>;<br />
*'''Output:''' an estimation of the total number of distinct elements <math>z=|\{x_1,x_2,\ldots,x_n\}|</math>.<br />
<br />
A straightforward way of solving this problem is to maintain a dictionary data structure, which costs at least linear (<math>O(n)</math>) space. For ''big data'', where <math>n</math> is very large, this is still too expensive. However, due to an information-theoretical argument, linear space is necessary if you want to compute the ''exact'' value of <math>z</math>.<br />
<br />
Our goal is to relax the problem a little bit to significantly reduce the space cost by tolerating ''approximate'' answers. The form of approximation we consider is '''<math>(\epsilon,\delta)</math>-estimator'''.<br />
{{Theorem|<math>(\epsilon,\delta)</math>-estimator|<br />
: A random variable <math>\widehat{Z}</math> is an '''<math>(\epsilon,\delta)</math>-estimator''' of a quantity <math>z</math> if<br />
::<math>\Pr[\,(1-\epsilon)z\le \widehat{Z}\le (1+\epsilon)z\,]\ge 1-\delta</math>.<br />
: <math>\widehat{Z}</math> is said to be an '''unbiased estimator''' of <math>z</math> if <math>\mathbb{E}[\widehat{Z}]=z</math>.<br />
}}<br />
Usually <math>\epsilon</math> is called '''approximation error''' and <math>\delta</math> is called '''confidence error'''.<br />
<br />
We now present an elegant algorithm. The algorithm can be implemented in [https://en.wikipedia.org/wiki/Streaming_algorithm '''data stream model''']: The input elements <math>x_1,x_2,\ldots,x_n</math> is presented to the algorithm one at a time, where the size of data <math>n</math> is unknown to the algorithm. The algorithm maintains a value <math>\widehat{Z}</math> which is an <math>(\epsilon,\delta)</math>-estimator of the total number of distinct elements <math>z=|\{x_1,x_2,\ldots,x_n\}|</math>, using only a small amount of memory space to memorize (with loss) the data set <math>\{x_1,x_2,\ldots,x_n\}</math>.<br />
<br />
A famous quotation of Flajolet describes the performance of this algorithm as:<br />
<br />
"Using only memory equivalent to 5 lines of printed text, you can estimate with a typical accuracy of 5% and in a single pass the total vocabulary of Shakespeare."<br />
<br />
== The <math>\min</math>-sketch ==<br />
Suppose that we can access to an idealized random hash function <math>h:U\to[0,1]</math> which is uniformly distributed over all mappings from the universe <math>U</math> to unit interval <math>[0,1]</math>. <br />
<br />
Recall that the input sequence <math>x_1,x_2,\ldots,x_n\in U</math> consists of <math>z=|\{x_1,x_2,\ldots,x_n\}|</math> distinct elements. These elements are mapped by the random function <math>h</math> to <math>z</math> hash values uniformly and independently distributed in <math>[0,1]</math>. We could maintain these hash values instead of the original elements, but this would still be too expensive because in the worst case we still have up to <math>n</math> distinct values to maintain. However, due to the idealized random hash function, the unit interval <math>[0,1]</math> will be partitioned into <math>z+1</math> subintervals by these <math>z</math> uniform and independent hash values. The typical length of the subinterval gives an estimation of the number <math>z</math>.<br />
<br />
{{Theorem|Proposition|<br />
:<math>\mathbb{E}\left[\min_{1\le i\le n}h(x_i)\right]=\frac{1}{z+1}</math>.<br />
}}<br />
{{Proof|<br />
The input sequence <math>x_1,x_2,\ldots,x_n\in U</math> consisting of <math>z</math> distinct elements are mapped to <math>z</math> random hash values uniformly and independently distributed in <math>[0,1]</math>. These <math>z</math> hash values partition the unit interval <math>[0,1]</math> into <math>z+1</math> subintervals <math>[0,v_1],[v_1,v_2],[v_2,v_3]\ldots,[v_{z-1},v_z],[v_z,1]</math>, where <math>v_i</math> denotes the <math>i</math>-th smallest value among all hash values <math>\{h(x_1),h(x_2),\ldots,h(x_n)\}</math>. Clearly we have <br />
:<math>v_1=\min_{1\le i\le n}h(x_i)</math>. <br />
Meanwhile, since all hash values are uniformly and independently distributed in <math>[0,1]</math>, the lengths of all subintervals <math>v_1, v_2-v_1, v_3-v_2,\ldots, v_z-v_{z-1}, 1-v_z</math> are identically distributed. By symmetry, they have the same expectation, therefore<br />
:<math><br />
(z+1)\mathbb{E}[v_1]=<br />
\mathbb{E}[v_1]+\sum_{i=1}^{z-1}\mathbb{E}[v_{i+1}-v_i]+\mathbb{E}[1-v_z]<br />
=\mathbb{E}\left[v_1+(v_2-v_1)+(v_3-v_2)+\cdots+(v_{z}-v_{z-1})+1-v_z\right]<br />
=1,<br />
</math><br />
which implies that<br />
:<math>\mathbb{E}\left[\min_{1\le i\le n}h(x_i)\right]=\mathbb{E}[v_1]=\frac{1}{z+1}</math>.<br />
}}<br />
<br />
The quantity <math>\min_{1\le i\le n}h(x_i)</math> can be computed with small space cost (for storing the current smallest hash value) by scan the input sequence in a single pass. Because as we proved its expectation is <math>\frac{1}{z+1}</math>, the smallest hash value <math>Y=\min_{1\le i\le n}h(x_i)</math> gives an unbiased estimator for <math>\frac{1}{z+1}</math>. However, <math>\frac{1}{Y}-1</math> is not necessarily a good estimator for <math>z</math>. Actually, it is a rather poor estimator. Consider for example when <math>z=1</math>, all input elements are the same. In this case, there is only one hash value and <math>Y=\min_{1\le i\le n}h(x_i)</math> is distributed uniformly over <math>[0,1]</math>, thus <math>\frac{1}{Y}-1</math> fails to be close enough to the correct answer 1 with high probability.<br />
<br />
==Apply the mean trick to the <math>\min</math>-sketch==<br />
The reason that the above estimator of a single hash function performs poorly is that the unbiased estimator <math>\min_{1\le i\le n}h(x_i)</math> has large variance. So a natural way to reduce this variance is to have multiple independent hash functions and take the average. This generic approach for reducing the variance is called '''the mean trick'''. <br />
<br />
Suppose that we can access to <math>k</math> independent random hash functions <math>h_1,h_2,\ldots,h_k</math>, where each <math>h_j: U\to[0,1]</math> is uniformly and independently distributed over all functions mapping <math>U</math> to <math>[0,1]</math>. Here <math>k</math> is a parameter to be fixed by the desired approximation error <math>\epsilon</math> and confidence error <math>\delta</math>. The ''<math>\min</math>-sketch algorithm'' (using the mean trick) is given by the following pseudocode.<br />
<br />
{{Theorem|The <math>\min</math>-sketch|<br />
:Suppose that <math>h_1,h_2,\ldots,h_k: U\to[0,1]</math> are <math>k</math> uniform and independent random hash functions, where <math>k</math> is a parameter to be fixed later.<br />
-----<br />
:Scan the input sequence <math>x_1,x_2,\ldots,x_n\in U</math> in a single pass to compute:<br />
::* <math>Y_j=\min_{1\le i\le n}h_j(x_i)</math> for every <math>j=1,2,\ldots,k</math>;<br />
::* average value <math>\overline{Y}=\frac{1}{k}\sum_{j=1}^kY_j</math>;<br />
:return <math>\widehat{Z}=\frac{1}{\overline{Y}}-1</math> as the estimator.<br />
}}<br />
<br />
The algorithm is easy to implement in data stream model, with a space cost of storing <math>k</math> hash values. The following theorem guarantees that the algorithm returns an <math>(\epsilon,\delta)</math>-estimator of the total number of distinct elements for a suitable <math>k=O\left(\frac{1}{\epsilon^2\delta}\right)</math>.<br />
{{Theorem|Theorem|<br />
:For any <math>\epsilon,\delta<1/2</math>, if <math>k\ge\left\lceil\frac{4}{\epsilon^2\delta}\right\rceil</math> then the output <math>\widehat{Z}</math> always gives an <math>(\epsilon,\delta)</math>-estimator of the correct answer <math>z</math>.<br />
}}<br />
<br />
In the following we prove this main theorem for <math>\min</math>-sketch algorithm. <br />
<br />
An obstacle to analyze the estimator <math>\widehat{Z}=\frac{1}{\overline{Y}}-1</math> is that it is a nonlinear function of <math>\overline{Y}</math> who is easier to analyze. Nevertheless, we observe that <math>\widehat{Z}</math> is an <math>(\epsilon,\delta)</math>-estimator of <math>z</math> as long as <math>\overline{Y}</math> is an <math>(\epsilon/2,\delta)</math>-estimator of <math>\frac{1}{z+1}</math>. This can be deduced by just verifying the following:<br />
:<math>\frac{1-\epsilon/2}{z+1}\le \overline{Y}\le \frac{1+\epsilon/2}{z+1} \implies (1-\epsilon)z\le\frac{1}{\overline{Y}}-1\le (1+\epsilon)z</math>,<br />
for <math>\epsilon<\frac{1}{2}</math>. Therefore,<br />
:<math>\Pr\left[\,(1-\epsilon)z\le \widehat{Z} \le (1+\epsilon)z\,\right]\ge \Pr\left[\,\frac{1-\epsilon/2}{z+1}\le \overline{Y}\le \frac{1+\epsilon/2}{z+1}\,\right]<br />
=\Pr\left[\,\left|\overline{Y}-\frac{1}{z+1}\right|\le \frac{\epsilon/2}{z+1}\,\right]</math>.<br />
It is then sufficient to show that <math>\Pr\left[\,\left|\overline{Y}-\frac{1}{z+1}\right|\le \frac{\epsilon/2}{z+1}\,\right]\ge 1-\delta</math> for proving the main theorem above. We will see that this is equivalent to show the concentration inequality <br />
:<math>\Pr\left[\,\left|\overline{Y}-\mathbb{E}\left[\overline{Y}\right]\right|\le \frac{\epsilon/2}{z+1}\,\right]\ge 1-\delta\quad\qquad({\color{red}*})</math>.<br />
<br />
{{Theorem|Lemma|<br />
:The followings hold for each <math>Y_j</math>, <math>j=1,2\ldots,k</math>, and <math>\overline{Y}=\frac{1}{k}\sum_{j=1}^kY_j</math>:<br />
:*<math>\mathbb{E}\left[\overline{Y}\right]=\mathbb{E}\left[Y_j\right]=\frac{1}{z+1}</math>;<br />
:*<math>\mathbf{Var}\left[Y_j\right]\le\frac{1}{(z+1)^2}</math>, and consequently <math>\mathbf{Var}\left[\overline{Y}\right]\le\frac{1}{k(z+1)^2}</math>.<br />
}}<br />
{{Proof|<br />
As in the case of single hash function, by symmetry it holds that <math>\mathbb{E}[Y_j]=\frac{1}{z+1}</math> for every <math>j=1,2,\ldots,k</math>. Therefore,<br />
:<math>\mathbb{E}\left[\overline{Y}\right]=\frac{1}{k}\sum_{j=1}^k\mathbb{E}[Y_j]=\frac{1}{z+1}</math>.<br />
Recall that each <math>Y_j</math> is the minimum of <math>z</math> random hash values uniformly and independently distributed over <math>[0,1]</math>. By geometry probability, it holds that for any <math>y\in[0,1]</math>,<br />
:<math>\Pr[Y_j>y]=(1-y)^z</math>,<br />
which means <math>\Pr[Y_j\le y]=1-(1-y)^z</math>. Taking the derivative with respect to <math>y</math>, we obtain the probability density function of random variable <math>Y_j</math>, which is <math>z(1-y)^{z-1}</math>.<br />
<br />
We then compute the second moment.<br />
:<math>\mathbb{E}[Y_j^2]=\int^{1}_0y^2z(1-y)^{z-1}\,\mathrm{d}y=\frac{2}{(z+1)(z+2)}</math>.<br />
The variance is bounded as<br />
:<math>\mathbf{Var}\left[Y_j\right]=\mathbb{E}\left[Y_j^2\right]-\mathbb{E}\left[Y_j\right]^2=\frac{2}{(z+1)(z+2)}-\frac{1}{(z+1)^2}\le\frac{1}{(z+1)^2}</math>.<br />
Due to the (pairwise) independence between <math>Y_j</math>'s,<br />
::<math>\mathbf{Var}\left[\overline{Y}\right]=\mathbf{Var}\left[\frac{1}{k}\sum_{j=1}^kY_j\right]=\frac{1}{k^2}\sum_{j=1}^k\mathbf{Var}\left[Y_j\right]\le \frac{1}{k(z+1)^2}</math>.<br />
}}<br />
<br />
We resume to prove the inequality <math>({\color{red}*})</math>. By [[高级算法_(Fall 2023)/Basic_deviation_inequalities#Chebyshev.27s_inequality|Chebyshev's inequality]], it holds that <br />
:<math>\Pr\left[\,\left|\overline{Y}-\mathbb{E}\left[\overline{Y}\right]\right|> \frac{\epsilon/2}{z+1}\,\right]<br />
\le\frac{4}{\epsilon^2}(z+1)^2\mathbf{Var}\left[\overline{Y}\right]<br />
\le\frac{4}{\epsilon^2k}</math>.<br />
When <math>k\ge\left\lceil\frac{4}{\epsilon^2\delta}\right\rceil</math>, this probability is at most <math>\delta</math>. The inequality <math>({\color{red}*})</math> is proved. As we discussed above, this proves the above main theorem <math>\min</math>-sketch algorithm improved by the mean trick.<br />
<br />
= Frequency Estimation=<br />
Suppose that <math>U</math> is the data universe. The '''frequency estimation''' problem is defined as follows.<br />
*'''Data:''' a sequence of (not necessarily distinct) elements <math>x_1,x_2,\ldots,x_n\in U</math>;<br />
*'''Query:''' an element <math>x\in U</math>;<br />
*'''Output:''' an estimation <math>\hat{f}_x</math> of the frequency <math>f_x\triangleq|\{i\mid x_i=x\}|</math> of <math>x</math> in input data.<br />
<br />
We still want to give an algorithm in the data stream model: the algorithm scan the input sequence <math>x_1,x_2,\ldots,x_n</math> to construct a succinct data structure, such that upon each query of <math>x\in U</math>, the algorithm returns an estimation of the frequency <math>f_x</math>.<br />
<br />
Clearly this problem can always be solved by storing all appeared distinct elements along with their frequencies. However, the space cost of this straightforward solution is rather high. Instead, we want to use a lossy representation (a ''sketch'') of input data which uses significantly less space but can still answer queries with tolarable accuracy. <br />
<br />
Formally, upon each query of <math>x\in U</math>, the algorithm should return an answer <math>\hat{f}_x</math> satisfying:<br />
:<math>\Pr\left[\,\left|\hat{f}_x-f_x\right|\le \epsilon n\,\right]\ge 1-\delta</math>.<br />
Note that this notion of approximation is with bounded ''additive'' error which is weaker than the notion of <math>(\epsilon,\delta)</math>-estimator, whose error bound is ''multiplicative''. <br />
<br />
With such weak accuracy guarantee, its is possible to give a succinct data structure whose size is determined only by the error bounds <math>\epsilon</math> and <math>\delta</math> but independent of <math>n</math>, because only the frequencies of those '''heavy hitters''' (elements <math>x</math> with high frequencies <math>f_x>\epsilon n</math>) need to be memorized, and there are at most <math>1/\epsilon</math> many such heavy hitters.<br />
<br />
== Count-min sketch==<br />
The [https://en.wikipedia.org/wiki/Count–min_sketch count-min sketch] given by Cormode and Muthukrishnan is an elegant data structure for frequency estimation.<br />
<br />
The data structure is a two-dimensional <math>k\times m</math> integer array, where <math>k</math> and <math>m</math> are two parameters to be determined by the error bounds <math>\epsilon</math> and <math>\delta</math>. We still adopt the Uniform Hash Assumption to assume that we have access to <math>k</math> mutually independent uniform random hash functions <math>h_1,h_2,\ldots,h_k: U\to[m]</math>.<br />
{{Theorem|''Count-min sketch'' (Cormode and Muthukrishnan 2003)|<br />
:Suppose <math>h_1,h_2,\ldots,h_k: U\to[m]</math> are uniform and independent random hash functions.<br />
-----<br />
:'''Data structure construction:''' Given a sequence <math>x_1,x_2,\ldots,x_n\in U</math>, the data structure is a two-dimensional <math>k\times m</math> integer array <math>CMS[k][m]</math> constructed as<br />
:*initialize all entries of <math>CMS[k][m]</math> to 0;<br />
:*for <math>i=1,2,\ldots,n</math>, upon receiving <math>x_i</math>:<br />
::: for every <math>1\le j\le k</math>, evaluate <math>h_j(x_i)</math> and <math>CMS[j][h_j(x_i)]++</math>.<br />
----<br />
:'''Query resolution:''' Upon each query of an arbitrary <math>x\in U</math>,<br />
:* return <math>\hat{f}_x=\min_{1\le j\le k}CMS[j][h_j(x)]</math>.<br />
}}<br />
<br />
It is easy to see that the space cost of count-min sketch is <math>O(km)</math> memory words, or <math>O(km\log n)</math> bits. Each query is answered within time cost <math>O(k)</math>, assuming that an evaluation of hash function can be done in unit or constant time. We then analyze the error bounds.<br />
<br />
First, it is easy to observe that for any query <math>x\in U</math> and every hash function <math>1\le j\le k</math>, it always holds for the corresponding entry in the count-min sketch<br />
:<math>CMS[j][h_j(x)]\ge f_x</math>,<br />
because the appearances of element <math>x</math> in the input sequence contribute at least <math>f_x</math> to the value of <math>CMS[j][h_j(x)]</math>.<br />
<br />
Therefore, for any query <math>x\in U</math> it always holds for the answer <math>\hat{f}_x=\min_{1\le j\le k}CMS[j][h_j(x)]\ge f_x</math>, which means<br />
:<math>\Pr\left[\,\left|\hat{f}_x- f_x\right|\ge\epsilon n\,\right]=\Pr\left[\,\hat{f}_x- f_x\ge\epsilon n\,\right]=\prod_{j=1}^k\Pr[\,CMS[j][h_j(x)]-f_x\ge\epsilon n\,],\quad\qquad({\color{red}\diamondsuit})</math><br />
where the second equation is due to the mutual independence of random hash functions <math>h_1,h_2,\ldots,h_k</math>.<br />
<br />
It remains to upper bound the probability <math>\Pr[\,CMS[j][h_j(x)]-f_x\ge\epsilon n\,]</math>, which can be done by calculating the expectation of <math>CMS[j][h_j(x)]</math>.<br />
{{Theorem|Proposition|<br />
:For any <math>x\in U</math> and every <math>1\le j\le k</math>, it holds that <math>\mathbb{E}\left[CMS[j][h_j(x)]\right]\le f_x+\frac{n}{m}</math>.<br />
}}<br />
{{Proof|<br />
The value of <math>CMS[j][h_j(x)]</math> is constituted by the frequency <math>f_x</math> of <math>x</math> and the frequencies <math>f_y</math> of all other elements <math>y\neq x</math> among <math>x_1,x_2,\ldots,x_n</math>, thus<br />
:<math><br />
\begin{align}<br />
CMS[j][h_j(x)]<br />
&=f_x+\sum_{\scriptstyle y\in\{x_1,\ldots,x_n\}\setminus\{x\}\atop\scriptstyle h_j(y)=h_j(x)} f_y\\<br />
&=f_x+\sum_{y\in\{x_1,\ldots,x_n\}\setminus\{x\}} f_y \cdot I[h_j(y)=h_j(x)]<br />
\end{align}<br />
</math><br />
where <math>I[h_j(y)=h_j(x)]</math> denotes the Boolean random variable that indicates the occurrence of event <math>h_j(y)=h_j(x)</math>.<br />
<br />
By linearity of expectation,<br />
:<math>\mathbb{E}[CMS[j][h_j(x)]]=f_x+\sum_{y\in\{x_1,x_2,\ldots,x_n\}\setminus\{x\}} f_y \cdot \Pr[h_j(y)=h_j(x)]</math>.<br />
Due to Uniform Hash Assumption (UHA), <math>h_j: U\to[m]</math> is a uniform random function. For any <math>y\neq x</math>, the probability of hash collision is<br />
:<math>\Pr[h_j(y)=h_j(x)]=\frac{1}{m}</math>.<br />
Therefore,<br />
:<math><br />
\begin{align}<br />
\mathbb{E}[CMS[j][h_j(x)]]<br />
&=f_x+\frac{1}{m}\sum_{y\in\{x_1,\ldots,x_n\}\setminus\{x\}} f_y \\<br />
&\le f_x+\frac{1}{m}\sum_{y\in\{x_1,\ldots,x_n\}} f_y\\<br />
&=f_x+\frac{n}{m},<br />
\end{align}<br />
</math><br />
where the last equation is due to the obvious identity <math>\sum_{y\in\{x_1,\ldots,x_n\}}f_y=n</math>.<br />
}}<br />
The above proposition shows that for any <math>x\in U</math> and every <math>1\le j\le k</math><br />
:<math>\mathbb{E}\left[CMS[j][h_j(x)]-f_x\right]\le \frac{n}{m}</math>.<br />
Recall that <math>CMS[j][h_j(x)]\ge f_x</math> always holds, thus <math>CMS[j][h_j(x)]-f_x</math> is a positive random variable. By Markov's inequality, we have<br />
:<math>\Pr[\,CMS[j][h_j(x)]-f_x\ge\epsilon n\,]\le \frac{1}{\epsilon m}</math>.<br />
<br />
Combining with above equation <math>({\color{red}\diamondsuit})</math>, we have<br />
:<math>\Pr\left[\,\left|\hat{f}_x- f_x\right|\ge\epsilon n\,\right]=(\Pr[\,CMS[j][h_j(x)]-f_x\ge\epsilon n\,])^k\le \frac{1}{(\epsilon m)^k}</math>.<br />
By setting <math>m=\left\lceil\frac{\mathrm{e}}{\epsilon}\right\rceil</math> and <math>k=\left\lceil\ln\frac{1}{\delta}\right\rceil</math>, the above error probability is bounded as <math>\frac{1}{(\epsilon m)^k}\le\delta</math>. <br />
<br />
For any positive <math>\epsilon</math> and <math>\delta</math>, the count-min sketch gives a data structure of size <math>O(km)=O\left(\frac{1}{\epsilon}\log\frac{1}{\delta}\right)</math> (in memory words) and answering each query <math>x\in U</math> in time <math>O(k)=O\left(\frac{1}{\epsilon}\right)</math> with the following accuracy guarantee:<br />
:<math>\Pr\left[\,\left|\hat{f}_x- f_x\right|\le\epsilon n\,\right]\ge 1-\delta</math>.</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=%E9%AB%98%E7%BA%A7%E7%AE%97%E6%B3%95_(Fall_2026)&diff=13903高级算法 (Fall 2026)2026-09-07T10:12:32Z<p>Etone: /* Lecture Notes */</p>
<hr />
<div>{{Infobox<br />
|name = Infobox<br />
|bodystyle = <br />
|title = <font size=3>高级算法 <br />
<br>Advanced Algorithms</font><br />
|titlestyle = <br />
<br />
|image = <br />
|imagestyle = <br />
|caption = <br />
|captionstyle = <br />
|headerstyle = background:#ccf;<br />
|labelstyle = background:#ddf;<br />
|datastyle = <br />
<br />
|header1 =Instructor<br />
|label1 = <br />
|data1 = <br />
|header2 = <br />
|label2 = <br />
|data2 = '''尹一通'''<br />
|header3 = <br />
|label3 = Email<br />
|data3 = yinyt@nju.edu.cn <br />
|header4 =<br />
|label4= office<br />
|data4= 计算机系 804<br />
|header5 = <br />
|label5 = <br />
|data5 = '''栗师'''<br />
|header6 = <br />
|label6 = Email<br />
|data6 = shili@nju.edu.cn <br />
|header7 =<br />
|label7= office<br />
|data7= 计算机系 605<br />
|header8 = <br />
|label8 = <br />
|data8 = '''刘景铖'''<br />
|header9 = <br />
|label9 = Email<br />
|data9 = liu@nju.edu.cn <br />
|header10 =<br />
|label10= office<br />
|data10= 计算机系 516<br />
|header11 = Class<br />
|label11 = <br />
|data11 = <br />
|header12 =<br />
|label12 = Class meetings<br />
|data12 = 周一 5-6节 (单) 仙Ⅰ-319<br />
周三 5-6节 仙Ⅰ-319<br />
|header13 =<br />
|label13 = Place<br />
|data13 = <br />
|header14 =<br />
|label14 = Office hours<br />
|data14 = 周一4-5pm（尹一通 804）<br/><br />
周四4-5pm（刘景铖 516）<br/><br />
By appointment<br />
|header15 = Textbooks<br />
|label15 = <br />
|data15 = <br />
|header16 =<br />
|label16 = <br />
|data16 = [[File:MR-randomized-algorithms.png|border|100px]]<br />
|header17 =<br />
|label17 = <br />
|data17 = Motwani and Raghavan. <br>''Randomized Algorithms''.<br> Cambridge Univ Press, 1995.<br />
|header18 =<br />
|label18 = <br />
|data18 = [[File:Approximation_Algorithms.jpg|border|100px]]<br />
|header19 =<br />
|label19 = <br />
|data19 = Vazirani. <br>''Approximation Algorithms''. <br> Springer-Verlag, 2001.<br />
|belowstyle = background:#ddf;<br />
|below = <br />
}}<br />
<br />
This is the webpage for the ''Advanced Algorithms'' class of fall 2026. Students who take this class should check this page periodically for content updates and new announcements. <br />
<br />
= Announcement =<br />
<br />
= Course info =<br />
* '''Instructor ''': <br />
:* [http://tcs.nju.edu.cn/yinyt/ 尹一通]：[mailto:yinyt@nju.edu.cn <yinyt@nju.edu.cn>]，计算机系 804 <br />
:*[https://tcs.nju.edu.cn/shili/ 栗师]：[mailto:shili@nju.edu.cn <shili@nju.edu.cn>]，计算机系 605<br />
:* [https://liuexp.github.io 刘景铖]：[mailto:liu@nju.edu.cn <liu@nju.edu.cn>]，计算机系 516 <br />
* '''Teaching Assistant''': <br />
** 于逸潇：[mailto:yixiaoyu@smail.nju.edu.cn <yixiaoyu@smail.nju.edu.cn>]<br />
** 张弈垚：[mailto:zhangyiyao@smail.nju.edu.cn <zhangyiyao@smail.nju.edu.cn>]<br />
* '''Class meeting''': <br />
** 周一 5-6节 1-17周(单) 仙Ⅰ-319<br />
** 周三 5-6节 1-18周 仙Ⅰ-319<br />
* '''Office hour''': <br />
** 周一4-5pm（尹一通 804）<br />
** 周四4-5pm（刘景铖 516）<br />
** By appointment<br />
* '''QQ群''': 1098567018<br />
<br />
= Syllabus =<br />
随着计算机算法理论的不断发展，现代计算机算法的设计与分析大量地使用非初等的数学工具以及非传统的算法思想。“高级算法”这门课程就是面向计算机算法的这一发展趋势而设立的。课程将针对传统算法课程未系统涉及、却在计算机科学各领域的科研和实践中扮演重要角色的高等算法设计思想和算法分析工具进行系统讲授。<br />
<br />
=== 先修课程 Prerequisites ===<br />
* 必须：离散数学，概率论，线性代数。<br />
* 推荐：算法设计与分析。<br />
<br />
=== Course materials ===<br />
* [[高级算法 (Fall 2026) / Course materials|<font size=3>教材和参考书</font>]]<br />
<br />
=== 成绩 Grades ===<br />
* 课程成绩：本课程将会有若干次作业和一次期末考试。最终成绩将由平时作业成绩和期末考试成绩综合得出。<br />
* 迟交：如果有特殊的理由，无法按时完成作业，请提前联系授课老师，给出正当理由。否则迟交的作业将不被接受。<br />
<br />
=== <font color=red> 学术诚信 Academic Integrity </font>===<br />
学术诚信是所有从事学术活动的学生和学者最基本的职业道德底线，本课程将不遗余力的维护学术诚信规范，违反这一底线的行为将不会被容忍。<br />
<br />
作业完成的原则：署你名字的工作必须是你个人的贡献。在完成作业的过程中，允许讨论，前提是讨论的所有参与者均处于同等完成度。但关键想法的执行、以及作业文本的写作必须独立完成，并在作业中致谢（acknowledge）所有参与讨论的人。不允许其他任何形式的合作——尤其是与已经完成作业的同学“讨论”。<br />
<br />
本课程将对剽窃行为采取零容忍的态度。在完成作业过程中，对他人工作（出版物、互联网资料、其他人的作业等）直接的文本抄袭和对关键思想、关键元素的抄袭，按照 [http://www.acm.org/publications/policies/plagiarism_policy ACM Policy on Plagiarism]的解释，都将视为剽窃。剽窃者成绩将被取消。如果发现互相抄袭行为，<font color=red> 抄袭和被抄袭双方的成绩都将被取消</font>。因此请主动防止自己的作业被他人抄袭。<br />
<br />
学术诚信影响学生个人的品行，也关乎整个教育系统的正常运转。为了一点分数而做出学术不端的行为，不仅使自己沦为一个欺骗者，也使他人的诚实努力失去意义。让我们一起努力维护一个诚信的环境。<br />
<br />
= Assignments =<br />
Late policy: In general, we will accomodate late submission requests ONLY IF you made such requests ahead of time. <br />
<br />
<br />
= Lecture Notes =<br />
# [[高级算法 (Fall 2025)/Min Cut, Max Cut, and Spectral Cut|Min Cut, Max Cut, and Spectral Cut]] ([http://tcs.nju.edu.cn/slides/aa2026/Cut.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-cuts-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-cuts-zh.pdf AI生成讲义])<br />
#* [[高级算法 (Fall 2025)/Probability Basics|Probability basics]]<br />
# [[高级算法 (Fall 2026)/Fingerprinting| Fingerprinting]] ([http://tcs.nju.edu.cn/slides/aa2026/Fingerprinting.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-fingerprinting-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-fingerprinting-zh.pdf AI生成讲义])<br />
#* [[高级算法 (Fall 2026)/Finite Field Basics|Finite field basics]]<br />
# [[高级算法 (Fall 2026)/Hashing and Sketching|Hashing and Sketching]] <br />
#* [[高级算法 (Fall 2026)/Limited independence|Limited independence]]<br />
#* [[高级算法 (Fall 2026)/Basic deviation inequalities|Basic deviation inequalities]]<br />
<br />
= Related Online Courses=<br />
* [https://www.cs.cmu.edu/~15850/ Advanced Algorithms] by Anupam Gupta at CMU.<br />
* [http://people.csail.mit.edu/moitra/854.html Advanced Algorithms] by Ankur Moitra at MIT.<br />
* [http://courses.csail.mit.edu/6.854/current/ Advanced Algorithms] by David Karger and Aleksander Mądry at MIT.<br />
* [http://web.stanford.edu/class/cs168/index.html The Modern Algorithmic Toolbox] by Tim Roughgarden and Gregory Valiant at Stanford.<br />
* [https://www.cs.princeton.edu/courses/archive/fall18/cos521/ Advanced Algorithm Design] by Pravesh Kothari and Christopher Musco at Princeton.<br />
* [http://www.cs.cmu.edu/afs/cs.cmu.edu/academic/class/15859-f11/www/ Linear and Semidefinite Programming (Advanced Algorithms)] by Anupam Gupta and Ryan O'Donnell at CMU.<br />
* [https://www.cs.cmu.edu/~odonnell/papers/cs-theory-toolkit-lecture-notes.pdf CS Theory Toolkit] by Ryan O'Donnell at CMU.<br />
* [https://cs.uwaterloo.ca/~lapchi/cs860/index.html Eigenvalues and Polynomials] by Lap Chi Lau at University of Waterloo.<br />
* The [https://www.cs.cornell.edu/jeh/book.pdf "Foundations of Data Science" book] by Avrim Blum, John Hopcroft, and Ravindran Kannan.</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=%E9%AB%98%E7%BA%A7%E7%AE%97%E6%B3%95_(Fall_2026)/Fingerprinting&diff=13895高级算法 (Fall 2026)/Fingerprinting2026-09-02T16:03:28Z<p>Etone: /* Freivalds Algorithm */</p>
<hr />
<div>= Checking Matrix Multiplication=<br />
[[File: matrix_multiplication.png|thumb|360px|right|The evolution of time complexity <math>O(n^{\omega})</math> for matrix multiplication.]]<br />
<br />
Let <math>\mathbb{F}</math> be a feild (you may think of it as the filed <math>\mathbb{Q}</math> of rational numbers, or the finite field <math>\mathbb{Z}_p</math> of integers modulo prime <math>p</math>). We suppose that each field operation (addition, subtraction, multiplication, division) has unit cost. This model is called the '''unit-cost RAM''' model, which is an ideal abstraction of a computer.<br />
<br />
Consider the following problem:<br />
* '''Input''': Three <math>n\times n</math> matrices <math>A</math>, <math>B</math>, and <math>C</math> over the field <math>\mathbb{F}</math>.<br />
* '''Output''': "yes" if <math>C=AB</math> and "no" if otherwise.<br />
<br />
A naive way to solve this is to multiply <math>A</math> and <math>B</math> and compare the result with <math>C</math>. <br />
The straightforward algorithm for matrix multiplication takes <math>O(n^3)</math> time, assuming that each arithmetic operation takes unit time.<br />
The [http://en.wikipedia.org/wiki/Strassen_algorithm Strassen's algorithm] discovered in 1969 now implemented by many numerical libraries runs in time <math>O(n^{\log_2 7})\approx O(n^{2.81})</math>. Strassen's algorithm starts the search for fast matrix multiplication algorithms. The [http://en.wikipedia.org/wiki/Coppersmith%E2%80%93Winograd_algorithm Coppersmith–Winograd algorithm] discovered in 1987 runs in time <math>O(n^{2.376})</math> but is only faster than Strassens' algorithm on extremely large matrices due to the very large constant coefficient. This has been the best known for decades, until recently Stothers got an <math>O(n^{2.374})</math> algorithm in his PhD thesis in 2010, and independently Vassilevska Williams got an <math>O(n^{2.373})</math> algorithm in 2012. Both these improvements are based on generalization of Coppersmith–Winograd algorithm. It is unknown whether the matrix multiplication can be done in time <math>O(n^{2+o(1)})</math>.<br />
<br />
== Freivalds Algorithm ==<br />
The following is a very simple randomized algorithm due to Freivalds, running in <math>O(n^2)</math> time:<br />
<br />
{{Theorem|Algorithm (Freivalds, 1977)|<br />
*pick a vector <math>r \in\{0, 1\}^n</math> uniformly at random;<br />
*if <math>A(Br) = Cr</math> then return "yes" else return "no";<br />
}}<br />
The product <math>A(Br)</math> is computed by first multiplying <math>Br</math> and then <math>A(Br)</math>.<br />
The running time of Freivalds algorithm is <math>O(n^2)</math> because the algorithm computes 3 matrix-vector multiplications. <br />
<br />
If <math>AB=C</math> then <math>A(Br) = Cr</math> for any <math>r \in\{0, 1\}^n</math>, thus the algorithm will return a "yes" for any positive instance (<math>AB=C</math>). <br />
But if <math>AB \neq C</math> then the algorithm will make a mistake if it chooses such an <math>r</math> that <math>ABr = Cr</math>. However, the following lemma states that the probability of this event is bounded.<br />
<br />
{{Theorem|Lemma|<br />
:If <math>AB\neq C</math> then for a uniformly random <math>r \in\{0, 1\}^n</math>,<br />
::<math>\Pr[ABr = Cr]\le \frac{1}{2}</math>.<br />
}}<br />
{{Proof| Let <math>D=AB-C</math>. The event <math>ABr=Cr</math> is equivalent to that <math>Dr=0</math>. It is then sufficient to show that for a <math>D\neq \boldsymbol{0}</math>, it holds that <math>\Pr[Dr = \boldsymbol{0}]\le \frac{1}{2}</math>.<br />
<br />
Since <math>D\neq \boldsymbol{0}</math>, it must have at least one non-zero entry. Suppose that <math>D_{ij}\neq 0</math>.<br />
<br />
We assume the event that <math>Dr=\boldsymbol{0}</math>. In particular, the <math>i</math>-th entry of <math>Dr</math> is <br />
:<math>(Dr)_{i}=\sum_{k=1}^n D_{ik}r_k=0.</math> <br />
The <math>r_j</math> can be calculated by<br />
:<math>r_j=-\frac{1}{D_{ij}}\sum_{k\neq j}^n D_{ik}r_k.</math><br />
Once all other entries <math>r_k</math> with <math>k\neq j</math> are fixed, there is a unique solution of <math>r_j</math>. Therefore, the number of <math>r\in\{0,1\}^n</math> satisfying <math>Dr=\boldsymbol{0}</math> is at most <math>2^{n-1}</math>. The probability that <math>ABr=Cr</math> is bounded as<br />
:<math>\Pr[ABr=Cr]=\Pr[Dr=\boldsymbol{0}]\le\frac{2^{n-1}}{2^n}=\frac{1}{2}</math>.<br />
}}<br />
<br />
When <math>AB=C</math>, Freivalds algorithm always returns "yes"; and when <math>AB\neq C</math>, Freivalds algorithm returns "no" with probability at least 1/2.<br />
<br />
To improve its accuracy, we can run Freivalds algorithm for <math>k</math> times, each time with an ''independent'' <math>r\in\{0,1\}^n</math>, and return "yes" if and only if all running instances returns "yes".<br />
<br />
{{Theorem|Freivalds' Algorithm (multi-round)|<br />
*pick <math>k</math> vectors <math>r_1,r_2,\ldots,r_k \in\{0, 1\}^n</math> uniformly and independently at random;<br />
*if <math>A(Br_i) = Cr_i</math> for all <math>i=1,\ldots,k</math> then return "yes" else return "no";<br />
}}<br />
<br />
If <math>AB=C</math>, then the algorithm returns a "yes" with probability 1. If <math>AB\neq C</math>, then due to the independence, the probability that all <math>r_i</math> have <math>ABr_i=C_i</math> is at most <math>2^{-k}</math>, so the algorithm returns "no" with probability at least <math>1-2^{-k}</math>. For any <math>0<\epsilon<1</math>, choose <math>k=\log_2 \frac{1}{\epsilon}</math>. The algorithm runs in time <math>O(n^2\log_2\frac{1}{\epsilon})</math> and has a one-sided error (false positive) bounded by <math>\epsilon</math>.<br />
<br />
=Polynomial Identity Testing (PIT) =<br />
The '''Polynomial Identity Testing (PIT)''' is such a problem: given as input two polynomials, determine whether they are identical. It plays a fundamental role in ''Identity Testing'' problems.<br />
<br />
First, let's consider the univariate ("one variable") case:<br />
* '''Input:''' two polynomials <math>f, g\in\mathbb{F}[x]</math> of degree <math>d</math>.<br />
* Determine whether <math>f\equiv g</math> (<math>f</math> and <math>g</math> are identical).<br />
Here the <math>\mathbb{F}[x]</math> denotes the [http://en.wikipedia.org/wiki/Polynomial_ring ring of univariate polynomials] on a field <math>\mathbb{F}</math>. More precisely, a polynomial <math>f\in\mathbb{F}[x]</math> is<br />
:<math>f(x)=\sum_{i=0}^\infty a_ix^i</math>,<br />
where the coefficients <math>a_i</math> are taken from the field <math>\mathbb{F}</math>, and the addition and multiplication are also defined over the field <math>\mathbb{F}</math>. <br />
And:<br />
* the '''degree''' of <math>f</math> is the highest <math>i</math> with non-zero <math>a_i</math>;<br />
* a polynomial <math>f</math> is a '''zero-polynomial''', denoted as <math>f\equiv 0</math>, if all coefficients <math>a_i=0</math>.<br />
<br />
Alternatively, we can consider the following equivalent problem by comparing the polynomial <math>f-g</math> (whose degree is at most <math>d</math>) with the zero-polynomial:<br />
* '''Input:''' a polynomial <math>f\in\mathbb{F}[x]</math> of degree <math>d</math>.<br />
* Determine whether <math>f\equiv 0</math> (<math>f</math> is the 0 polynomial).<br />
<br />
The problem is trivial if the input polynomial <math>f</math> is given explicitly: one can trivially solve the problem by checking whether all <math>d+1</math> coefficients are <math>0</math>. To make the problem nontrivial, we assume that the input polynomial is given implicitly as a ''black box'' (also called an ''oracle''): the only way the algorithm can access to <math>f</math> is to evaluate <math>f(x)</math> over some <math>x</math> from the field <math>\mathbb{F}</math>, <math>x</math> chosen by the algorithm.<br />
<br />
A straightforward deterministic algorithm is to evaluate <math>f(x_1),f(x_2),\ldots,f(x_{d+1})</math> over <math>d+1</math> ''distinct'' elements <math>x_1,x_2,\ldots,x_{d+1}</math> from the field <math>\mathbb{F}</math> and check whether they are all zero. By the [https://en.wikipedia.org/wiki/Fundamental_theorem_of_algebra fundamental theorem of algebra], also known as polynomial interpolations, this guarantees to verify whether a degree-<math>d</math> univariate polynomial <math>f\equiv 0</math>.<br />
{{Theorem|Fundamental Theorem of Algebra|<br />
:Any non-zero univariate polynomial of degree <math>d</math> has at most <math>d</math> roots.<br />
}}<br />
The reason for this fundamental theorem holding generally over any field <math>\mathbb{F}</math> is that any univariate polynomial of degree <math>d</math> factors uniquely into at most <math>d</math> irreducible polynomials, each of which has at most one root.<br />
<br />
The following simple randomized algorithm is natural:<br />
{{Theorem|Algorithm for PIT|<br />
*suppose we have a finite subset <math>S\subseteq\mathbb{F}</math> (to be specified later);<br />
*pick <math>r\in S</math> ''uniformly'' at random;<br />
*if <math>f(r) = 0</math> then return “yes” else return “no”;<br />
}}<br />
<br />
This algorithm evaluates <math>f</math> at one point chosen uniformly at random from a finite subset <math>S\subseteq\mathbb{F}</math>. It is easy to see the followings:<br />
* If <math>f\equiv 0</math>, the algorithm always returns "yes", so it is always correct.<br />
* If <math>f\not\equiv 0</math>, the algorithm may wrongly return "yes" (a '''false positive'''). But this happens only when the random <math>r</math> is a root of <math>f</math>. By the fundamental theorem of algebra, <math>f</math> has at most <math>d</math> roots, so the probability that the algorithm is wrong is bounded as<br />
::<math>\Pr[f(r)=0]\le\frac{d}{|S|}.</math><br />
<br />
By fixing <math>S\subseteq\mathbb{F}</math> to be an arbitrary subset of size <math>|S|=2d</math>, this probability of false positive is at most <math>1/2</math>. We can reduce it to an arbitrarily small constant <math>\delta</math> by repeat the above testing ''independently'' for <math>\log_2 \frac{1}{\delta}</math> times, since the error probability decays geometrically as we repeat the algorithm independently. <br />
<br />
== Communication Complexity of Equality ==<br />
The [http://en.wikipedia.org/wiki/Communication_complexity communication complexity] is introduced by Andrew Chi-Chih Yao as a model of computation with more than one entities, each with partial information about the input.<br />
<br />
Assume that there are two entities, say Alice and Bob. Alice has a private input <math>a</math> and Bob has a private input <math>b</math>. Together they want to compute a function <math>f(a,b)</math> by communicating with each other. The communication follows a predefined '''communication protocol''' (the "algorithm" in this model). The complexity of a communication protocol is measured by the number of bits communicated between Alice and Bob in the worst case.<br />
<br />
The problem of checking identity is formally defined by the function EQ as follows: <math>\mathrm{EQ}:\{0,1\}^n\times\{0,1\}^n\rightarrow\{0,1\}</math> and for any <math>a,b\in\{0,1\}^n</math>,<br />
:<math><br />
\mathrm{EQ}(a,b)=<br />
\begin{cases}<br />
1& \mbox{if } a=b,\\<br />
0& \mbox{otherwise.}<br />
\end{cases}</math><br />
<br />
A trivial way to solve EQ is to let Bob send his entire input string <math>b</math> to Alice and let Alice check whether <math>a=b</math>. This costs <math>n</math> bits of communications.<br />
<br />
It is known that for deterministic communication protocols, this is the best we can get for computing EQ.<br />
<br />
{{Theorem|Theorem (Yao 1979)|<br />
:Any deterministic communication protocol computing EQ on two <math>n</math>-bit strings costs <math>n</math> bits of communication in the worst-case.<br />
}}<br />
<br />
This theorem is much more nontrivial to prove than it looks, because Alice and Bob are allowed to interact with each other in arbitrary ways. The proof of this theorem is in Yao's [http://math.ucla.edu/~znorwood/290d.2.14s/papers/yao.pdf celebrated paper in 1979 with a humble title]. It pioneered the field of communication complexity.<br />
<br />
If we allow randomness in protocols, and also tolerate a small probabilistic error, the problem can be solved with significantly less communications. To present this randomized protocol, we need a few preparations:<br />
* We represent the inputs <math>a,b \in\{0,1\}^{n}</math> of Alice and Bob as two univariate polynomials of degree at most <math>n-1</math>, respectively <br />
::<math>f(x)=\sum_{i=0}^{n-1}a_ix^{i}</math> and <math>g(x)=\sum_{i=0}^{n-1}b_ix^{i}</math>.<br />
* The two polynomials <math>f</math> and <math>g</math> are defined over finite field <math>\mathbb{Z}_p=\{0,1,\ldots,p-1\}</math> for some suitable prime <math>p</math> (to be specified later), which means the additions and multiplications are modulo <math>p</math>.<br />
The randomized communication protocol is then as follows:<br />
{{Theorem|A randomized protocol for EQ|<br />
'''Bob does''':<br />
:* pick <math>r\in\mathbb{Z}_p</math> uniformly at random;<br />
:* send <math>r</math> and <math>g(r)</math> to Alice;<br />
'''Upon receiving''' <math>r</math> and <math>g(r)</math> '''Alice does''':<br />
:* compute <math>f(r)</math>;<br />
:* If <math>f(r)= g(r)</math> return "'''yes'''"; else return "'''no'''".<br />
}}<br />
The communication complexity of the protocol is given by the number of bits used to represent the values of <math>r</math> and <math>g(r)</math>.<br />
Since the polynomials are defined over finite field <math>\mathbb{Z}_p</math> and the random number <math>f</math> is also chosen from <math>\mathbb{Z}_p</math>, this is bounded by <math>O(\log p)</math>. <br />
<br />
On the other hand the protocol makes mistakes only when <math>a\neq b</math> but wrongly answers "yes". This happens only when <math>f\not\equiv g</math> but <math>f(r)=g(r)</math>. The degrees of <math>f, g</math> are at most <math>n-1</math>, and <math>r</math> is chosen among <math>p</math> distinct values, we have<br />
:<math>\Pr[f(r)=g(r)]\le \frac{n-1}{p}</math>.<br />
<br />
By choosing <math>p</math> to be a prime in the interval <math>[n^2, 2n^2]</math> (by [https://en.wikipedia.org/wiki/Bertrand%27s_postulate Chebyshev's theorem], such prime <math>p</math> always exists), the above randomized communication protocol solves the Equality function EQ with an error probability of false positive at most <math>O(1/n)</math>, with communication complexity <math>O(\log n)</math>, an EXPONENTIAL improvement to ANY deterministic communication protocol!<br />
<br />
== Schwartz-Zippel Theorem ==<br />
Now let's see the the true form of '''Polynomial Identity Testing (PIT)''', for multivariate polynomials:<br />
* '''Input:''' two <math>n</math>-variate polynomials <math>f, g\in\mathbb{F}[x_1,x_2,\ldots,x_n]</math> of degree <math>d</math>.<br />
* Determine whether <math>f\equiv g</math>.<br />
The <math>\mathbb{F}[x_1,x_2,\ldots,x_n]</math> is the [http://en.wikipedia.org/wiki/Polynomial_ring#The_polynomial_ring_in_several_variables ring of multivariate polynomials] over field <math>\mathbb{F}</math>. An <math>n</math>-variate polynomial of degree <math>d</math>, written as a sum of monomials, is:<br />
:<math>f(x_1,x_2,\ldots,x_n)=\sum_{i_1,i_2,\ldots,i_n\ge 0\atop i_1+i_2+\cdots+i_n\le d}a_{i_1,i_2,\ldots,i_n}x_{1}^{i_1}x_2^{i_2}\cdots x_{n}^{i_n}</math>.<br />
The '''degree''' or '''total degree''' of a monomial <math>a_{i_1,i_2,\ldots,i_n}x_{1}^{i_1}x_2^{i_2}\cdots x_{n}^{i_n}</math> is given by <math>i_1+i_2+\cdots+i_n</math> and the degree of a polynomial <math>f</math> is the maximum degree of monomials of nonzero coefficients.<br />
<br />
As before, we also consider the following equivalent problem:<br />
* '''Input:''' a polynomial <math>f\in\mathbb{F}[x_1,x_2,\ldots,x_n]</math> of degree <math>d</math>.<br />
* Determine whether <math>f\equiv 0</math>.<br />
<br />
If <math>f</math> is written explicitly as a sum of monomials, then the problem can be solved by checking whether all coefficients, and there at most <math>{n+d\choose d}\le (n+d)^{d}</math> coefficients in an <math>n</math>-variate polynomial of degree at most <math>d</math>. <br />
<br />
A multivariate polynomial <math>f</math> can also be presented in its '''product form''', for example:<br />
{{Theorem|Example|<br />
The [http://en.wikipedia.org/wiki/Vandermonde_matrix Vandermonde matrix] <math>M=M(x_1,x_2,\ldots,x_n)</math> is defined as that <math>M_{ij}=x_i^{j-1}</math>, that is<br />
:<math>M=\begin{bmatrix}<br />
1 & x_1 & x_1^2 & \dots & x_1^{n-1}\\<br />
1 & x_2 & x_2^2 & \dots & x_2^{n-1}\\<br />
1 & x_3 & x_3^2 & \dots & x_3^{n-1}\\<br />
\vdots & \vdots & \vdots & \ddots &\vdots \\<br />
1 & x_n & x_n^2 & \dots & x_n^{n-1}<br />
\end{bmatrix}</math>.<br />
Let <math>f</math> be the polynomial defined as<br />
:<math><br />
f(x_1,\ldots,x_n)=\det(M)=\prod_{j<i}(x_i-x_j).<br />
</math><br />
}}<br />
For polynomials in product form, it is quite efficient to '''evaluate''' the polynomial at any specific point from the field over which the polynomial is defined, however, '''expanding''' the polynomial to a sum of monomials can be very expensive. <br />
<br />
The following is a simple randomized algorithm for testing identity of multivariate polynomials:<br />
{{Theorem|Randomized algorithm for multivariate PIT|<br />
*suppose we have a finite subset <math>S\subseteq\mathbb{F}</math> (to be specified later);<br />
* pick <math>r_1,r_2,\ldots,r_n\in S</math> ''uniformly'' and ''independently'' at random;<br />
* if <math>f(\vec{r})=f(r_1,r_2,\ldots,r_n) = 0</math> then return “yes” else return “no”;<br />
}}<br />
<br />
This algorithm evaluates <math>f</math> at one point chosen uniformly from an <math>n</math>-dimensional cube <math>S^n</math>, where <math>S\subseteq\mathbb{F}</math> is a finite subset. And:<br />
* If <math>f\equiv 0</math>, the algorithm always returns "yes", so it is always correct.<br />
* If <math>f\not\equiv 0</math>, the algorithm may wrongly return "yes" (a '''false positive'''). But this happens only when the random <math>\vec{r}=(r_1,r_2,\ldots,r_n)</math> is a root of <math>f</math>. The probability of this bad event is upper bounded by the following famous result due to [https://pdfs.semanticscholar.org/b913/cf330852035f49b4ec5fe2db86c47d8a98fd.pdf Schwartz (1980)] and [http://www.cecm.sfu.ca/~monaganm/teaching/TopicsinCA15/zippel79.pdf Zippel (1979)]. <br />
<br />
{{Theorem|Schwartz-Zippel Theorem|<br />
: Let <math>f\in\mathbb{F}[x_1,x_2,\ldots,x_n]</math> be a multivariate polynomial of degree <math>d</math> over a field <math>\mathbb{F}</math>. If <math>f\not\equiv 0</math>, then for any finite set <math>S\subset\mathbb{F}</math>, and <math>r_1,r_2\ldots,r_n\in S</math> chosen uniformly and independently at random, <br />
::<math>\Pr[f(r_1,r_2,\ldots,r_n)=0]\le\frac{d}{|S|}.</math><br />
}}<br />
The Schwartz-Zippel Theorem states that for any nonzero <math>n</math>-variate polynomial of degree at most <math>d</math>, the number of roots in any cube <math>S^n</math> is at most <math>d\cdot |S|^{n-1}</math>.<br />
<br />
Dana Moshkovitz gave a surprisingly simply and elegant [http://eccc.hpi-web.de/report/2010/096/ proof] of Schwartz-Zippel Theorem, using some advanced ideas. Now we introduce the standard proof by induction.<br />
<br />
{{Proof| <br />
The theorem is proved by induction on <math>n</math>.<br />
<br />
;Induction basis''':'''<br />
For <math>n=1</math>, this is the univariate case. Assume that <math>f\not\equiv 0</math>. Due to the fundamental theorem of algebra, any polynomial <math>f(x)</math> of degree at most <math>d</math> must have at most <math>d</math> roots, thus <br />
:<math>\Pr[f(r)=0]\le\frac{d}{|S|}.<br />
</math><br />
<br />
;Induction hypothesis''':''' <br />
Assume the theorem holds for any <math>m</math>-variate polynomials for all <math>m<n</math>.<br />
<br />
;Induction step''':'''<br />
For any <math>n</math>-variate polynomial <math>f(x_1,x_2,\ldots,x_n)</math> of degree at most <math>d</math>, we write <math>f</math> as<br />
:<math>f(x_1,x_2,\ldots,x_n)=\sum_{i=0}^kx_n^{i}f_i(x_1,x_2,\ldots,x_{n-1})</math>,<br />
where <math>k</math> is the highest degree of <math>x_n</math>, which means the degree of <math>f_k</math> is at most <math>d-k</math> and <math>f_k\not\equiv 0</math>.<br />
<br />
In particular, we write <math>f</math> as a sum of two parts:<br />
:<math>f(x_1,x_2,\ldots,x_n)=x_n^k f_k(x_1,x_2,\ldots,x_{n-1})+\bar{f}(x_1,x_2,\ldots,x_n)</math>,<br />
where both <math>f_k</math> and <math>\bar{f}</math> are polynomials, such that <br />
* <math>f_k\not\equiv 0</math> is as above, whose degree is at most <math>d-k</math>;<br />
* <math>\bar{f}(x_1,x_2,\ldots,x_n)=\sum_{i=0}^{k-1}x_n^i f_i(x_1,x_2,\ldots,x_{n-1})</math>, thus <math>\bar{f}(x_1,x_2,\ldots,x_n)</math> has no <math>x_n^{k}</math> factor in any term.<br />
<br />
By the law of total probability, we have<br />
:<math><br />
\begin{align}<br />
&\Pr[f(r_1,r_2,\ldots,r_n)=0]\\<br />
=<br />
&\Pr[f(\vec{r})=0\mid f_k(r_1,r_2,\ldots,r_{n-1})=0]\cdot\Pr[f_k(r_1,r_2,\ldots,r_{n-1})=0]\\<br />
&+\Pr[f(\vec{r})=0\mid f_k(r_1,r_2,\ldots,r_{n-1})\neq0]\cdot\Pr[f_k(r_1,r_2,\ldots,r_{n-1})\neq0].<br />
\end{align}<br />
</math><br />
Note that <math>f_k(r_1,r_2,\ldots,r_{n-1})</math> is a polynomial on <math>n-1</math> variables of degree <math>d-k</math> such that <math>f_k\not\equiv 0</math>.<br />
By the induction hypothesis, we have <br />
:<math><br />
\begin{align}<br />
(*)<br />
&\qquad<br />
&\Pr[f_k(r_1,r_2,\ldots,r_{n-1})=0]\le\frac{d-k}{|S|}.<br />
\end{align}<br />
</math><br />
<br />
Now we look at the case conditioning on <math>f_k(r_1,r_2,\ldots,r_{n-1})\neq0</math>. Recall that <math>\bar{f}(x_1,\ldots,x_n)</math> has no <math>x_n^k</math> factor in any term, thus the condition <math>f_k(r_1,r_2,\ldots,r_{n-1})\neq0</math> guarantees that <br />
:<math>f(r_1,\ldots,r_{n-1},x_n)=x_n^k f_k(r_1,r_2,\ldots,r_{n-1})+\bar{f}(r_1,r_2,\ldots,r_{n-1},x_n)=g_{r_1,\ldots,r_{n-1}}(x_n)</math><br />
is a nonzero univariate polynomial of <math>x_n</math> such that the degree of <math>g_{r_1,\ldots,r_{n-1}}(x_n)</math> is <math>k</math> and <math>g_{r_1,\ldots,r_{n-1}}\not\equiv 0</math>, for which we already known that the probability <math>g_{r_1,\ldots,r_{n-1}}(r_n)=0</math> is at most <math>\frac{k}{|S|}</math>.<br />
Therefore,<br />
:<math><br />
\begin{align}<br />
(**)<br />
&\qquad<br />
&\Pr[f(\vec{r})=0\mid f_k(r_1,r_2,\ldots,r_{n-1})\neq0]=\Pr[g_{r_1,\ldots,r_{n-1}}(r_n)=0\mid f_k(r_1,r_2,\ldots,r_{n-1})\neq0]\le\frac{k}{|S|}<br />
\end{align}<br />
</math>.<br />
Substituting both <math>(*)</math> and <math>(**)</math> back in the total probability, we have<br />
:<math><br />
\Pr[f(r_1,r_2,\ldots,r_n)=0]<br />
\le\frac{d-k}{|S|}+\frac{k}{|S|}=\frac{d}{|S|},<br />
</math><br />
which proves the theorem.<br />
}}<br />
<br />
== Detecting perfect matching ==<br />
<br />
TBA<br />
<br />
;Edmonds matrix<br />
<br />
= Fingerprinting =<br />
The polynomial identity testing algorithm in the Schwartz-Zippel theorem can be abstracted as the following framework:<br />
Suppose we want to compare two objects <math>X</math> and <math>Y</math>. Instead of comparing them directly, we compute random '''fingerprints''' <math>\mathrm{FING}(X)</math> and <math>\mathrm{FING}(Y)</math> of them and compare the fingerprints. <br />
<br />
The fingerprints has the following properties:<br />
* <math>\mathrm{FING}(\cdot)</math> is a function, meaning that if <math>X= Y</math> then <math>\mathrm{FING}(X)=\mathrm{FING}(Y)</math>.<br />
* It is much easier to compute and compare the fingerprints.<br />
* Ideally, the domain of fingerprints is much smaller than the domain of original objects, so storing and comparing fingerprints are easy. This means the fingerprint function <math>\mathrm{FING}(\cdot)</math> cannot be an injection (one-to-one mapping), so it's possible that different <math>X</math> and <math>Y</math> are mapped to the same fingerprint. We resolve this by making fingerprint function ''randomized'', and for <math>X\neq Y</math>, we want the probability <math>\Pr[\mathrm{FING}(X)=\mathrm{FING}(Y)]</math> to be small.<br />
<br />
In Schwartz-Zippel theorem, the objects to compare are polynomials from <math>\mathbb{F}[x_1,\ldots,x_n]</math>. Given a polynomial <math>f\in \mathbb{F}[x_1,\ldots,x_n]</math>, its fingerprint is computed as <math>\mathrm{FING}(f)=f(r_1,\ldots,r_n)</math> for <math>r_i</math> chosen independently and uniformly at random from some fixed set <math>S\subseteq\mathbb{F}</math>.<br />
<br />
With this generic framework, for various identity testing problems, we may design different fingerprints <math>\mathrm{FING}(\cdot)</math>.<br />
<br />
== Communication protocols for Equality ==<br />
Now consider again the communication model where the two players Alice with a private input <math>x\in\{0,1\}^n</math> and Bob with a private input <math>y\in\{0,1\}^n</math> together compute a function <math>f(x,y)</math> by running a communication protocol. <br />
<br />
We still consider the communication protocols for the equality function EQ<br />
:<math><br />
\mathrm{EQ}(x,y)=<br />
\begin{cases}<br />
1& \mbox{if } x=y,\\<br />
0& \mbox{otherwise.}<br />
\end{cases}</math><br />
<br />
With the language of fingerprinting, this communication problem can be solved by the following generic scheme:<br />
{{Theorem|Communication protocol for EQ by fingerprinting|<br />
'''Bob does''':<br />
:* choose a random fingerprint function <math>\mathrm{FING}(\cdot)</math> and compute the fingerprint of her input <math>\mathrm{FING}(y)</math>;<br />
:* sends both the description of <math>\mathrm{FING}(\cdot)</math> and the value of <math>\mathrm{FING}(y)</math> to Alice;<br />
'''Upon receiving''' the description of <math>\mathrm{FING}(\cdot)</math> and the value of <math>\mathrm{FING}(y)</math>, '''Alice does''':<br />
:* computes <math>\mathrm{FING}(x)</math> and check whether <math>\mathrm{FING}(x)=\mathrm{FING}(y)</math>.<br />
}}<br />
In this way we have a randomized communication protocol for the equality function EQ with false positive. The communication cost as well as the error probability are reduced to the question of how to design this random fingerprint function <math>\mathrm{FING}(\cdot)</math> to guarantee:<br />
# A random fingerprint function <math>\mathrm{FING}(\cdot)</math> can be described succinctly.<br />
# The range of <math>\mathrm{FING}(\cdot)</math> is small, so the fingerprints are succinct.<br />
# If <math>x\neq y</math>, the probability <math>\Pr[\mathrm{FING}(x)=\mathrm{FING}(y)]</math> is small.<br />
<br />
=== Fingerprinting by PIT===<br />
As before, we can define the fingerprint function as: for any bit-string <math>x\in\{0,1\}^n</math>, its random fingerprint is <math>\mathrm{FING}(x)=\sum_{i=1}^n x_i r^{i}</math>, where the additions and multiplications are defined over a finite field <math>\mathbb{Z}_p</math>, and <math>r</math> is chosen uniformly at random from <math>\mathbb{Z}_p</math>, where <math>p</math> is some suitable prime which can be represented in <math>\Theta(\log n)</math> bits. More specifically, we can choose <math>p</math> to be any prime from the interval <math>[n^2, 2n^2]</math>. Due to Chebyshev's theorem, such prime must exist. <br />
<br />
As we have shown before, it takes <math>O(\log p)=O(\log n)</math> bits to represent <math>\mathrm{FING}(y)</math> and to describe the random function <math>\mathrm{FING}(\cdot)</math> (since it a random function <math>\mathrm{FING}(\cdot)</math> from this family is uniquely identified by a random <math>r\in\mathbb{Z}_p</math>, which can be represented within <math>\log p=O(\log n)</math> bits). And it follows easily from the fundamental theorem of algebra that for any distinct <math>x, y\in\{0,1\}^n</math>,<br />
:<math><br />
\Pr[\mathrm{FING}(x)=\mathrm{FING}(y)] \le \frac{n-1}{p}\le \frac{1}{n}.<br />
</math><br />
<br />
=== Fingerprinting by randomized checksum===<br />
Now we consider a new fingerprint function: We treat each input string <math>x\in\{0,1\}^n</math> as the binary representation of a number, and let <math>\mathrm{FING}(x)=x\bmod p</math> for some random prime <math>p</math> chosen from <math>[k]=\{0,1,\ldots,k-1\}</math>, for some <math>k</math> to be specified later. <br />
<br />
Now a random fingerprint function <math>\mathrm{FING}(\cdot)</math> can be uniquely identified by this random prime <math>p</math>. The new communication protocol for EQ with this fingerprint is as follows:<br />
{{Theorem|Communication protocol for EQ by random checksum|<br />
'''Bob does''':<br />
:for some parameter <math>k</math> (to be specified), <br />
:* choose a prime <math>p\in[k]</math> uniformly at random;<br />
:* send <math>p</math> and <math>x\bmod p</math> to Alice;<br />
'''Upon receiving''' <math>p</math> and <math>x\bmod p</math>, '''Alice does''':<br />
:* check whether <math>x\bmod p=y\bmod p</math>.<br />
}}<br />
The number of bits to be communicated is obviously <math>O(\log k)</math>. When <math>x\neq y</math>, we want to upper bound the error probability <math>\Pr[x\bmod p=y\bmod p]</math>.<br />
<br />
Suppose without loss of generality <math>x>y</math>. Let <math>z=x-y</math>. Then <math>z<2^n</math> since <math>x,y\in[2^n]</math>, and <br />
<math>z\neq 0</math> for <math>x\neq y</math>. It holds that <math>x\equiv y\pmod p</math> if and only if <math>p\mid z</math>. Therefore, we only need to upper bound the probability<br />
:<math>\Pr[z\bmod p=0]</math><br />
for an arbitrarily fixed <math>0<z<2^n</math>, and a uniform random prime <math>p\in[k]</math>.<br />
<br />
The probability <math>\Pr[z\bmod p=0]</math> is computed directly as<br />
:<math>\Pr[z\bmod p=0]\le\frac{\mbox{the number of prime divisors of }z}{\mbox{the number of primes in }[k]}</math>.<br />
<br />
For the numerator, any positive <math>z<2^n</math> has at most <math>n</math> prime factors. To see this, by contradiction assume that <math>z</math> has more than <math>n</math> prime factors. Note that any prime number is at least 2. Then <math>z</math> must be greater than <math>2^n</math>, contradicting the fact that <math>z<2^n</math>.<br />
<br />
For the denominator, we need to lower bound the number of primes in <math>[k]</math>. This is given by the celebrated [http://en.wikipedia.org/wiki/Prime_number_theorem '''Prime Number Theorem (PNT)'''].<br />
<br />
{{Theorem|Prime Number Theorem|<br />
:Let <math>\pi(k)</math> denote the number of primes less than <math>k</math>. Then <math>\pi(k)\sim\frac{k}{\ln k}</math> as <math>k\rightarrow\infty</math>.<br />
}}<br />
<br />
Therefore, by choosing <math>k=2n^2\ln n</math>, we have that for a <math>0<z<2^n</math>, and a random prime <math>p\in[k]</math>,<br />
:<math>\Pr[z\bmod p=0]\le\frac{n}{\pi(k)}\sim\frac{1}{n}</math>,<br />
which means the for any inputs <math>x,y\in\{0,1\}^n</math>, if <math>x\neq y</math>, then the false positive is bounded as<br />
:<math>\Pr[\mathrm{FING}(x)=\mathrm{FING}(y)]\le\Pr[|x-y|\bmod p=0]\le \frac{1}{n}</math>.<br />
Moreover, by this choice of parameter <math>k=2n^2\ln n</math>, the communication complexity of the protocol is bounded by <math>O(\log k)=O(\log n)</math>.<br />
<br />
== Pattern matching ==<br />
Consider the following problem of pattern matching, which has nothing to do with communication complexity.<br />
<br />
*Input: a string <math>x\in\{0,1\}^n</math> and a "pattern" <math>y\in\{0,1\}^m</math>.<br />
*Determine whether the pattern <math>y</math> is a contiguous substring of <math>x</math>. Usually, we are also asked to find the location of the substring.<br />
<br />
A naive algorithm trying every possible match runs in <math>O(nm)</math> time. The more sophisticated KMP algorithm inspired by automaton theory runs in <math>O(n+m)</math> time.<br />
<br />
A simple randomized algorithm, due to Karp and Rabin, uses the idea of fingerprinting and also runs in <math>O(n + m)</math> time.<br />
<br />
Let <math>X(j)=x_jx_{j+1}\cdots x_{j+m-1}</math> denote the substring of <math>x</math> of length <math>m</math> starting at position <math>j</math>.<br />
<br />
{{Theorem|Algorithm (Karp-Rabin)|<br />
:pick a random prime <math>p\in[k]</math>;<br />
:'''for''' <math>j = 1</math> to <math>n -m + 1</math> '''do'''<br />
::'''if''' <math>X(j)\bmod p = y \bmod p</math> then report a match;<br />
:'''return''' "no match";<br />
}}<br />
<br />
So the algorithm just compares the <math>\mathrm{FING}(X(j))</math> and <math>\mathrm{FING}(y)</math> for every <math>j</math>, with the same definition of fingerprint function <math>\mathrm{FING}(\cdot)</math> as in the communication protocol for EQ.<br />
<br />
By the same analysis, by choosing <math>k=n^2m\ln (n^2m)</math>, the probability of a single false match is <br />
:<math>\Pr[X(j)\bmod p=y\bmod p\mid X(j)\neq y ]=O\left(\frac{1}{n^2}\right)</math>.<br />
By the union bound, the probability that a false match occurs is <math>O\left(\frac{1}{n}\right)</math>.<br />
<br />
The algorithm runs in linear time if we assume that we can compute <math>X(j)\bmod p </math> for each <math>j</math> in constant time. This outrageous assumption can be made realistic by the following observation.<br />
<br />
{{Theorem|Lemma|<br />
:Let <math>\mathrm{FING}(a)=a\bmod p</math>.<br />
::<math>\mathrm{FING}(X(j+1))\equiv2(\mathrm{FING}(X(j))-2^{m-1}x_j)+x_{j+m}\pmod p\,</math>.<br />
}}<br />
{{Proof| It holds that <br />
:<math>X(j+1)=2(X(j)-2^{m-1}x_j)+x_{j+m}\,</math>.<br />
So the equation holds on the finite field modulo <math>p</math>.<br />
}}<br />
<br />
Due to this lemma, each fingerprint <math>\mathrm{FING}(X(j))</math> can be computed in an incremental way, each in constant time. The running time of the algorithm is <math>O(n+m)</math>.<br />
<br />
== Checking distinctness ==<br />
Consider the following problem of '''checking distinctness''':<br />
*Given a sequence <math>x_1,x_2,\ldots,x_n\in\{1,2,\ldots,n\}</math>, check whether every element of <math>\{1,2,\ldots,n\}</math> appears '''exactly''' once.<br />
<br />
Obviously this problem can be solved in linear time and linear space (in addition to the space for storing the input) by maintaining a <math>n</math>-bit vector that indicates which numbers among <math>\{1,2,\ldots,n\}</math> have appeared.<br />
<br />
When this <math>n</math> is enormously large, <math>\Omega(n)</math> space cost is too expensive. We wonder whether we could solve this problem with a space cost (in addition to the space for storing the input) much less than <math>O(n)</math>. This can be done by fingerprinting if we tolerate a certain degree of inaccuracy.<br />
<br />
We consider the following more generalized problem, '''checking identity of multisets''':<br />
* '''Input:''' two multisets <math>A=\{a_1,a_2,\ldots, a_n\}</math> and <math>B=\{b_1,b_2,\ldots, b_n\}</math> where <math>a_1,a_2,\ldots,b_1,b_2,\ldots,b_n\in \{1,2,\ldots,n\}</math>.<br />
* Determine whether <math>A=B</math> (multiset equivalence).<br />
<br />
Here for a '''multiset''' <math>A=\{a_1,a_2,\ldots, a_n\}</math>, its elements <math>a_i</math> are not necessarily distinct. The '''multiplicity''' of an element <math>a_i</math> in a multiset <math>A</math> is the number of times <math>a_i</math> appears in <math>A</math>. Two multisets <math>A</math> and <math>B</math> are equivalent if they contain the same set of elements and the multiplicities of every element in <math>A</math> and <math>B</math> are equal. <br />
<br />
Obviously the above problem of checking distinctness can be treated as a special case of checking identity of multisets: by checking the identity of the multiset <math>A</math> and set <math>\{1,2,\ldots, n\}</math>.<br />
<br />
The following fingerprinting function for multisets was introduced by Lipton for solving multiset identity testing.<br />
<br />
{{Theorem|Fingerprint for multiset|<br />
:Let <math>p</math> be a uniform random prime chosen from the interval <math>[(n\log n)^2,2(n\log n)^2]</math>. By Chebyshev's theorem, such prime must exist. And consider the the finite field <math>\mathbb{Z}_p=[p]</math>.<br />
<br />
:Given a multiset <math>A=\{a_1,a_2,\ldots,a_n\}</math>, we define a univariate polynomial <math>f_A\in\mathbb{Z}_p[x]</math> over the finite field <math>\mathbb{Z}_p</math> as follows:<br />
::<math>f_A(x)=\prod_{i=1}^n(x-a_i)</math>,<br />
:where <math>+</math> and <math>\cdot</math> are defined over the finite field <math>\mathbb{Z}_p</math>.<br />
<br />
:We then define the random fingerprinting function as:<br />
::<math>\mathrm{FING}(A)=f_A(r)=\prod_{i=1}^n(r-a_i)</math>,<br />
:where <math>r</math> is chosen uniformly at random from <math>\mathbb{Z}_p</math>.<br />
}}<br />
<br />
Since all computations of <math>\mathrm{FING}(A)=\prod_{i=1}^n(r-a_i)</math> are over the finite field <math>\mathbb{Z}_p</math>, the space cost for computing the fingerprint <math>\mathrm{FING}(A)</math> is only <math>O(\log p)=O(\log n)</math>.<br />
<br />
Moreover, the fingerprinting function <math>\mathrm{FING}(A)</math> is invariant under permutation of elements of the multiset <math>A=\{a_1,a_2,\ldots,a_n\}</math>, thus it is indeed a function of multisets (meaning every multiset has only one fingerprint). Therefore, if <math>A=B</math> then <math>\mathrm{FING}(A)=\mathrm{FING}(B)</math>.<br />
<br />
For two distinct multisets <math>A\neq B</math>, it is possible that <math>\mathrm{FING}(A)=\mathrm{FING}(B)</math>, but the following theorem due to Lipton bounds this error probability of false positive.<br />
<br />
{{Theorem|Theorem (Lipton 1989)|<br />
:Let <math>A=\{a_1,a_2,\ldots,a_n\}</math> and <math>B=\{b_1,b_2,\ldots,b_n\}</math> be two multisets whose elements are from <math>\{1,2,\ldots,n\}</math>. If <math>A\neq B</math>, then<br />
::<math>\Pr[\mathrm{FING}(A)= \mathrm{FING}(B)]=O\left(\frac{1}{n}\right)</math>.<br />
}}<br />
{{Proof| <br />
Let <math>\tilde{f}_A(x)=\prod_{i=1}^n(x-a_i)</math> and <math>\tilde{f}_B(x)=\prod_{i=1}^n(x-b_i)</math> be two univariate polynomials defined over reals <math>\mathbb{R}</math>. Note that in contrast to <math>f_A(x)</math> and <math>f_B(x)</math>, the <math>+</math> and <math>\cdot</math> in <math>\tilde{f}_A(x), \tilde{f}_B(x)</math> do not modulo <math>p</math>. It is easy to verify that the polynomials <math>\tilde{f}_A(x), \tilde{f}_B(x)</math> have the following properties:<br />
*<math>\tilde{f}_A\equiv \tilde{f}_B</math> if and only if <math>A=B</math>. Here <math>A=B</math> means the multiset equivalence.<br />
*By the properties of finite field, for any value <math>r\in\mathbb{Z}_p</math>, it holds that <math>f_A(r)=\tilde{f}_A(r)\bmod p</math> and <math>f_B(r)=\tilde{f}_B(r)\bmod p</math>.<br />
<br />
Therefore, assuming that <math>A\neq B</math>, we must have <math>\tilde{f}_A(x)\not\equiv \tilde{f}_B(x)</math>. Then by the law of total probability:<br />
:<math><br />
\begin{align}<br />
\Pr[\mathrm{FING}(A)= \mathrm{FING}(B)]<br />
&=<br />
\Pr\left[f_A(r)=f_B(r)\mid f_A\not\equiv f_B\right]\Pr[f_A\not\equiv f_B]\\<br />
&\quad\,\,+\Pr\left[f_A(r)=f_B(r)\mid f_A\equiv f_B\right]\Pr[f_A\equiv f_B]\\<br />
&\le \Pr\left[f_A(r)=f_B(r)\mid f_A\not\equiv f_B\right]+\Pr[f_A\equiv f_B].<br />
\end{align}<br />
</math><br />
Note that the degrees of <math>f_A,f_B</math> are at most <math>n</math> and <math>r</math> is chosen uniformly from <math>[p]</math>. By the Schwartz-Zippel theorem for univariate polynomials, the first probability<br />
:<math><br />
\Pr\left[f_A(r)=f_B(r)\mid f_A\not\equiv f_B\right]\le \frac{n}{p}=o\left(\frac{1}{n}\right),<br />
</math><br />
since <math>p</math> is chosen from the interval <math>[(n\log n)^2,2(n\log n)^2]</math>.<br />
<br />
For the second probability <math>\Pr[f_A\equiv f_B]</math>, recall that <math>\tilde{f}_A\not\equiv \tilde{f}_B</math>, therefore there is at least a non-zero coefficient <math>c\le n^n</math> in <math>\tilde{f}_A-\tilde{f}_B</math>. The event <math>f_A\equiv f_B</math> occurs only if <math>c\bmod p=0</math>, which means<br />
:<math><br />
\begin{align}<br />
\Pr[f_A\equiv f_B]<br />
&\le \Pr[c\bmod p=0]\\<br />
&=\frac{\text{number of prime factors of }c}{\text{number of primes in }[(n\log n)^2,2(n\log n)^2]}\\<br />
&\le \frac{n\log_2n}{\pi(2(n\log n)^2)-\pi((n\log n)^2)}.<br />
\end{align}<br />
</math><br />
By the prime number theorem, <math>\pi(N)\rightarrow \frac{N}{\ln N}</math> as <math>N\to\infty</math>. Therefore,<br />
:<math><br />
\Pr[f_A\equiv f_B]=O\left(\frac{n\log n}{n^2\log n}\right)=O\left(\frac{1}{n}\right).<br />
</math><br />
Combining everything together, we have<br />
<math>\Pr[\mathrm{FING}(A)= \mathrm{FING}(B)]=O\left(\frac{1}{n}\right)</math>.<br />
}}</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=%E9%AB%98%E7%BA%A7%E7%AE%97%E6%B3%95_(Fall_2026)&diff=13894高级算法 (Fall 2026)2026-09-02T16:01:20Z<p>Etone: /* Lecture Notes */</p>
<hr />
<div>{{Infobox<br />
|name = Infobox<br />
|bodystyle = <br />
|title = <font size=3>高级算法 <br />
<br>Advanced Algorithms</font><br />
|titlestyle = <br />
<br />
|image = <br />
|imagestyle = <br />
|caption = <br />
|captionstyle = <br />
|headerstyle = background:#ccf;<br />
|labelstyle = background:#ddf;<br />
|datastyle = <br />
<br />
|header1 =Instructor<br />
|label1 = <br />
|data1 = <br />
|header2 = <br />
|label2 = <br />
|data2 = '''尹一通'''<br />
|header3 = <br />
|label3 = Email<br />
|data3 = yinyt@nju.edu.cn <br />
|header4 =<br />
|label4= office<br />
|data4= 计算机系 804<br />
|header5 = <br />
|label5 = <br />
|data5 = '''栗师'''<br />
|header6 = <br />
|label6 = Email<br />
|data6 = shili@nju.edu.cn <br />
|header7 =<br />
|label7= office<br />
|data7= 计算机系 605<br />
|header8 = <br />
|label8 = <br />
|data8 = '''刘景铖'''<br />
|header9 = <br />
|label9 = Email<br />
|data9 = liu@nju.edu.cn <br />
|header10 =<br />
|label10= office<br />
|data10= 计算机系 516<br />
|header11 = Class<br />
|label11 = <br />
|data11 = <br />
|header12 =<br />
|label12 = Class meetings<br />
|data12 = 周一 5-6节 (单) 仙Ⅰ-319<br />
周三 5-6节 仙Ⅰ-319<br />
|header13 =<br />
|label13 = Place<br />
|data13 = <br />
|header14 =<br />
|label14 = Office hours<br />
|data14 = 周一4-5pm（尹一通 804）<br/><br />
周四4-5pm（刘景铖 516）<br/><br />
By appointment<br />
|header15 = Textbooks<br />
|label15 = <br />
|data15 = <br />
|header16 =<br />
|label16 = <br />
|data16 = [[File:MR-randomized-algorithms.png|border|100px]]<br />
|header17 =<br />
|label17 = <br />
|data17 = Motwani and Raghavan. <br>''Randomized Algorithms''.<br> Cambridge Univ Press, 1995.<br />
|header18 =<br />
|label18 = <br />
|data18 = [[File:Approximation_Algorithms.jpg|border|100px]]<br />
|header19 =<br />
|label19 = <br />
|data19 = Vazirani. <br>''Approximation Algorithms''. <br> Springer-Verlag, 2001.<br />
|belowstyle = background:#ddf;<br />
|below = <br />
}}<br />
<br />
This is the webpage for the ''Advanced Algorithms'' class of fall 2026. Students who take this class should check this page periodically for content updates and new announcements. <br />
<br />
= Announcement =<br />
<br />
= Course info =<br />
* '''Instructor ''': <br />
:* [http://tcs.nju.edu.cn/yinyt/ 尹一通]：[mailto:yinyt@nju.edu.cn <yinyt@nju.edu.cn>]，计算机系 804 <br />
:*[https://tcs.nju.edu.cn/shili/ 栗师]：[mailto:shili@nju.edu.cn <shili@nju.edu.cn>]，计算机系 605<br />
:* [https://liuexp.github.io 刘景铖]：[mailto:liu@nju.edu.cn <liu@nju.edu.cn>]，计算机系 516 <br />
* '''Teaching Assistant''': <br />
** 于逸潇：[mailto:yixiaoyu@smail.nju.edu.cn <yixiaoyu@smail.nju.edu.cn>]<br />
** 张弈垚：[mailto:zhangyiyao@smail.nju.edu.cn <zhangyiyao@smail.nju.edu.cn>]<br />
* '''Class meeting''': <br />
** 周一 5-6节 1-17周(单) 仙Ⅰ-319<br />
** 周三 5-6节 1-18周 仙Ⅰ-319<br />
* '''Office hour''': <br />
** 周一4-5pm（尹一通 804）<br />
** 周四4-5pm（刘景铖 516）<br />
** By appointment<br />
* '''QQ群''': 1098567018<br />
<br />
= Syllabus =<br />
随着计算机算法理论的不断发展，现代计算机算法的设计与分析大量地使用非初等的数学工具以及非传统的算法思想。“高级算法”这门课程就是面向计算机算法的这一发展趋势而设立的。课程将针对传统算法课程未系统涉及、却在计算机科学各领域的科研和实践中扮演重要角色的高等算法设计思想和算法分析工具进行系统讲授。<br />
<br />
=== 先修课程 Prerequisites ===<br />
* 必须：离散数学，概率论，线性代数。<br />
* 推荐：算法设计与分析。<br />
<br />
=== Course materials ===<br />
* [[高级算法 (Fall 2026) / Course materials|<font size=3>教材和参考书</font>]]<br />
<br />
=== 成绩 Grades ===<br />
* 课程成绩：本课程将会有若干次作业和一次期末考试。最终成绩将由平时作业成绩和期末考试成绩综合得出。<br />
* 迟交：如果有特殊的理由，无法按时完成作业，请提前联系授课老师，给出正当理由。否则迟交的作业将不被接受。<br />
<br />
=== <font color=red> 学术诚信 Academic Integrity </font>===<br />
学术诚信是所有从事学术活动的学生和学者最基本的职业道德底线，本课程将不遗余力的维护学术诚信规范，违反这一底线的行为将不会被容忍。<br />
<br />
作业完成的原则：署你名字的工作必须是你个人的贡献。在完成作业的过程中，允许讨论，前提是讨论的所有参与者均处于同等完成度。但关键想法的执行、以及作业文本的写作必须独立完成，并在作业中致谢（acknowledge）所有参与讨论的人。不允许其他任何形式的合作——尤其是与已经完成作业的同学“讨论”。<br />
<br />
本课程将对剽窃行为采取零容忍的态度。在完成作业过程中，对他人工作（出版物、互联网资料、其他人的作业等）直接的文本抄袭和对关键思想、关键元素的抄袭，按照 [http://www.acm.org/publications/policies/plagiarism_policy ACM Policy on Plagiarism]的解释，都将视为剽窃。剽窃者成绩将被取消。如果发现互相抄袭行为，<font color=red> 抄袭和被抄袭双方的成绩都将被取消</font>。因此请主动防止自己的作业被他人抄袭。<br />
<br />
学术诚信影响学生个人的品行，也关乎整个教育系统的正常运转。为了一点分数而做出学术不端的行为，不仅使自己沦为一个欺骗者，也使他人的诚实努力失去意义。让我们一起努力维护一个诚信的环境。<br />
<br />
= Assignments =<br />
Late policy: In general, we will accomodate late submission requests ONLY IF you made such requests ahead of time. <br />
<br />
<br />
= Lecture Notes =<br />
# [[高级算法 (Fall 2025)/Min Cut, Max Cut, and Spectral Cut|Min Cut, Max Cut, and Spectral Cut]] ([http://tcs.nju.edu.cn/slides/aa2026/Cut.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-cuts-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-cuts-zh.pdf AI生成讲义])<br />
#* [[高级算法 (Fall 2025)/Probability Basics|Probability basics]]<br />
# [[高级算法 (Fall 2026)/Fingerprinting| Fingerprinting]] ([http://tcs.nju.edu.cn/slides/aa2026/Fingerprinting.pdf slides]) ([http://tcs.nju.edu.cn/slides/aa2026/AA26-note-fingerprinting-en.pdf AI-generated lecture note], [http://tcs.nju.edu.cn/slides/aa2026/AA26-note-fingerprinting-zh.pdf AI生成讲义])<br />
#* [[高级算法 (Fall 2026)/Finite Field Basics|Finite field basics]]<br />
<br />
= Related Online Courses=<br />
* [https://www.cs.cmu.edu/~15850/ Advanced Algorithms] by Anupam Gupta at CMU.<br />
* [http://people.csail.mit.edu/moitra/854.html Advanced Algorithms] by Ankur Moitra at MIT.<br />
* [http://courses.csail.mit.edu/6.854/current/ Advanced Algorithms] by David Karger and Aleksander Mądry at MIT.<br />
* [http://web.stanford.edu/class/cs168/index.html The Modern Algorithmic Toolbox] by Tim Roughgarden and Gregory Valiant at Stanford.<br />
* [https://www.cs.princeton.edu/courses/archive/fall18/cos521/ Advanced Algorithm Design] by Pravesh Kothari and Christopher Musco at Princeton.<br />
* [http://www.cs.cmu.edu/afs/cs.cmu.edu/academic/class/15859-f11/www/ Linear and Semidefinite Programming (Advanced Algorithms)] by Anupam Gupta and Ryan O'Donnell at CMU.<br />
* [https://www.cs.cmu.edu/~odonnell/papers/cs-theory-toolkit-lecture-notes.pdf CS Theory Toolkit] by Ryan O'Donnell at CMU.<br />
* [https://cs.uwaterloo.ca/~lapchi/cs860/index.html Eigenvalues and Polynomials] by Lap Chi Lau at University of Waterloo.<br />
* The [https://www.cs.cornell.edu/jeh/book.pdf "Foundations of Data Science" book] by Avrim Blum, John Hopcroft, and Ravindran Kannan.</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=%E9%AB%98%E7%BA%A7%E7%AE%97%E6%B3%95_(Fall_2026)&diff=13893高级算法 (Fall 2026)2026-09-02T09:19:28Z<p>Etone: /* Lecture Notes */</p>
<hr />
<div>{{Infobox<br />
|name = Infobox<br />
|bodystyle = <br />
|title = <font size=3>高级算法 <br />
<br>Advanced Algorithms</font><br />
|titlestyle = <br />
<br />
|image = <br />
|imagestyle = <br />
|caption = <br />
|captionstyle = <br />
|headerstyle = background:#ccf;<br />
|labelstyle = background:#ddf;<br />
|datastyle = <br />
<br />
|header1 =Instructor<br />
|label1 = <br />
|data1 = <br />
|header2 = <br />
|label2 = <br />
|data2 = '''尹一通'''<br />
|header3 = <br />
|label3 = Email<br />
|data3 = yinyt@nju.edu.cn <br />
|header4 =<br />
|label4= office<br />
|data4= 计算机系 804<br />
|header5 = <br />
|label5 = <br />
|data5 = '''栗师'''<br />
|header6 = <br />
|label6 = Email<br />
|data6 = shili@nju.edu.cn <br />
|header7 =<br />
|label7= office<br />
|data7= 计算机系 605<br />
|header8 = <br />
|label8 = <br />
|data8 = '''刘景铖'''<br />
|header9 = <br />
|label9 = Email<br />
|data9 = liu@nju.edu.cn <br />
|header10 =<br />
|label10= office<br />
|data10= 计算机系 516<br />
|header11 = Class<br />
|label11 = <br />
|data11 = <br />
|header12 =<br />
|label12 = Class meetings<br />
|data12 = 周一 5-6节 (单) 仙Ⅰ-319<br />
周三 5-6节 仙Ⅰ-319<br />
|header13 =<br />
|label13 = Place<br />
|data13 = <br />
|header14 =<br />
|label14 = Office hours<br />
|data14 = 周一4-5pm（尹一通 804）<br/><br />
周四4-5pm（刘景铖 516）<br/><br />
By appointment<br />
|header15 = Textbooks<br />
|label15 = <br />
|data15 = <br />
|header16 =<br />
|label16 = <br />
|data16 = [[File:MR-randomized-algorithms.png|border|100px]]<br />
|header17 =<br />
|label17 = <br />
|data17 = Motwani and Raghavan. <br>''Randomized Algorithms''.<br> Cambridge Univ Press, 1995.<br />
|header18 =<br />
|label18 = <br />
|data18 = [[File:Approximation_Algorithms.jpg|border|100px]]<br />
|header19 =<br />
|label19 = <br />
|data19 = Vazirani. <br>''Approximation Algorithms''. <br> Springer-Verlag, 2001.<br />
|belowstyle = background:#ddf;<br />
|below = <br />
}}<br />
<br />
This is the webpage for the ''Advanced Algorithms'' class of fall 2026. Students who take this class should check this page periodically for content updates and new announcements. <br />
<br />
= Announcement =<br />
<br />
= Course info =<br />
* '''Instructor ''': <br />
:* [http://tcs.nju.edu.cn/yinyt/ 尹一通]：[mailto:yinyt@nju.edu.cn <yinyt@nju.edu.cn>]，计算机系 804 <br />
:*[https://tcs.nju.edu.cn/shili/ 栗师]：[mailto:shili@nju.edu.cn <shili@nju.edu.cn>]，计算机系 605<br />
:* [https://liuexp.github.io 刘景铖]：[mailto:liu@nju.edu.cn <liu@nju.edu.cn>]，计算机系 516 <br />
* '''Teaching Assistant''': <br />
** 于逸潇：[mailto:yixiaoyu@smail.nju.edu.cn <yixiaoyu@smail.nju.edu.cn>]<br />
** 张弈垚：[mailto:zhangyiyao@smail.nju.edu.cn <zhangyiyao@smail.nju.edu.cn>]<br />
* '''Class meeting''': <br />
** 周一 5-6节 1-17周(单) 仙Ⅰ-319<br />
** 周三 5-6节 1-18周 仙Ⅰ-319<br />
* '''Office hour''': <br />
** 周一4-5pm（尹一通 804）<br />
** 周四4-5pm（刘景铖 516）<br />
** By appointment<br />
* '''QQ群''': 1098567018<br />
<br />
= Syllabus =<br />
随着计算机算法理论的不断发展，现代计算机算法的设计与分析大量地使用非初等的数学工具以及非传统的算法思想。“高级算法”这门课程就是面向计算机算法的这一发展趋势而设立的。课程将针对传统算法课程未系统涉及、却在计算机科学各领域的科研和实践中扮演重要角色的高等算法设计思想和算法分析工具进行系统讲授。<br />
<br />
=== 先修课程 Prerequisites ===<br />
* 必须：离散数学，概率论，线性代数。<br />
* 推荐：算法设计与分析。<br />
<br />
=== Course materials ===<br />
* [[高级算法 (Fall 2026) / Course materials|<font size=3>教材和参考书</font>]]<br />
<br />
=== 成绩 Grades ===<br />
* 课程成绩：本课程将会有若干次作业和一次期末考试。最终成绩将由平时作业成绩和期末考试成绩综合得出。<br />
* 迟交：如果有特殊的理由，无法按时完成作业，请提前联系授课老师，给出正当理由。否则迟交的作业将不被接受。<br />
<br />
=== <font color=red> 学术诚信 Academic Integrity </font>===<br />
学术诚信是所有从事学术活动的学生和学者最基本的职业道德底线，本课程将不遗余力的维护学术诚信规范，违反这一底线的行为将不会被容忍。<br />
<br />
作业完成的原则：署你名字的工作必须是你个人的贡献。在完成作业的过程中，允许讨论，前提是讨论的所有参与者均处于同等完成度。但关键想法的执行、以及作业文本的写作必须独立完成，并在作业中致谢（acknowledge）所有参与讨论的人。不允许其他任何形式的合作——尤其是与已经完成作业的同学“讨论”。<br />
<br />
本课程将对剽窃行为采取零容忍的态度。在完成作业过程中，对他人工作（出版物、互联网资料、其他人的作业等）直接的文本抄袭和对关键思想、关键元素的抄袭，按照 [http://www.acm.org/publications/policies/plagiarism_policy ACM Policy on Plagiarism]的解释，都将视为剽窃。剽窃者成绩将被取消。如果发现互相抄袭行为，<font color=red> 抄袭和被抄袭双方的成绩都将被取消</font>。因此请主动防止自己的作业被他人抄袭。<br />
<br />
学术诚信影响学生个人的品行，也关乎整个教育系统的正常运转。为了一点分数而做出学术不端的行为，不仅使自己沦为一个欺骗者，也使他人的诚实努力失去意义。让我们一起努力维护一个诚信的环境。<br />
<br />
= Assignments =<br />
Late policy: In general, we will accomodate late submission requests ONLY IF you made such requests ahead of time. <br />
<br />
<br />
= Lecture Notes =<br />
# [[高级算法 (Fall 2025)/Min Cut, Max Cut, and Spectral Cut|Min Cut, Max Cut, and Spectral Cut]] ([http://tcs.nju.edu.cn/slides/aa2026/Cut.pdf slides])<br />
#* [[高级算法 (Fall 2025)/Probability Basics|Probability basics]]<br />
# [[高级算法 (Fall 2026)/Fingerprinting| Fingerprinting]] ([http://tcs.nju.edu.cn/slides/aa2026/Fingerprinting.pdf slides]) <br />
#* [[高级算法 (Fall 2026)/Finite Field Basics|Finite field basics]]<br />
<br />
= Related Online Courses=<br />
* [https://www.cs.cmu.edu/~15850/ Advanced Algorithms] by Anupam Gupta at CMU.<br />
* [http://people.csail.mit.edu/moitra/854.html Advanced Algorithms] by Ankur Moitra at MIT.<br />
* [http://courses.csail.mit.edu/6.854/current/ Advanced Algorithms] by David Karger and Aleksander Mądry at MIT.<br />
* [http://web.stanford.edu/class/cs168/index.html The Modern Algorithmic Toolbox] by Tim Roughgarden and Gregory Valiant at Stanford.<br />
* [https://www.cs.princeton.edu/courses/archive/fall18/cos521/ Advanced Algorithm Design] by Pravesh Kothari and Christopher Musco at Princeton.<br />
* [http://www.cs.cmu.edu/afs/cs.cmu.edu/academic/class/15859-f11/www/ Linear and Semidefinite Programming (Advanced Algorithms)] by Anupam Gupta and Ryan O'Donnell at CMU.<br />
* [https://www.cs.cmu.edu/~odonnell/papers/cs-theory-toolkit-lecture-notes.pdf CS Theory Toolkit] by Ryan O'Donnell at CMU.<br />
* [https://cs.uwaterloo.ca/~lapchi/cs860/index.html Eigenvalues and Polynomials] by Lap Chi Lau at University of Waterloo.<br />
* The [https://www.cs.cornell.edu/jeh/book.pdf "Foundations of Data Science" book] by Avrim Blum, John Hopcroft, and Ravindran Kannan.</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=%E9%AB%98%E7%BA%A7%E7%AE%97%E6%B3%95_(Fall_2026)/Finite_Field_Basics&diff=13892高级算法 (Fall 2026)/Finite Field Basics2026-09-02T09:19:10Z<p>Etone: Created page with "=Field= Let <math>S</math> be a set, '''closed''' under binary operations <math>+</math> (addition) and <math>\cdot</math> (multiplication). It gives us the following algebraic structures if the corresponding set of axioms are satisfied. {|class="wikitable" !colspan="7"|Structures !Axioms !Operations |- |rowspan="9" style="background-color:#ffffcc;text-align:center;"|'''''field''''' |rowspan="8" style="background-color:#ffffcc;text-align:center;"|'''''commutative<br>rin..."</p>
<hr />
<div>=Field=<br />
Let <math>S</math> be a set, '''closed''' under binary operations <math>+</math> (addition) and <math>\cdot</math> (multiplication). It gives us the following algebraic structures if the corresponding set of axioms are satisfied.<br />
{|class="wikitable"<br />
!colspan="7"|Structures <br />
!Axioms<br />
!Operations<br />
|-<br />
|rowspan="9" style="background-color:#ffffcc;text-align:center;"|'''''field'''''<br />
|rowspan="8" style="background-color:#ffffcc;text-align:center;"|'''''commutative<br>ring'''''<br />
|rowspan="7" style="background-color:#ffffcc;text-align:center;"|'''''ring'''''<br />
|rowspan="4" style="background-color:#ffffcc;text-align:center;"|'''''abelian<br>group'''''<br />
|rowspan="3" style="background-color:#ffffcc;text-align:center;"|'''''group'''''<br />
| rowspan="2" style="background-color:#ffffcc;text-align:center;"|'''''monoid'''''<br />
|style="background-color:#ffffcc;text-align:center;"|'''''semigroup'''''<br />
|1. '''Addition''' is '''associative''': <math>\forall x,y,z\in S, (x+y)+z= x+(y+z).</math><br />
|rowspan="4" style="text-align:center;"|<math>+</math><br />
|-<br />
|<br />
|2. Existence of '''additive identity 0''': <math>\forall x\in S, x+0= 0+x=x.</math><br />
|-<br />
|colspan="2"|<br />
|3. Everyone has an '''additive inverse''': <math>\forall x\in S, \exists -x\in S, \text{ s.t. } x+(-x)= (-x)+x=0.</math><br />
|-<br />
|colspan="3"|<br />
|4. '''Addition''' is '''commutative''': <math>\forall x,y\in S, x+y= y+x.</math><br />
|-<br />
|colspan="4" rowspan="3"|<br />
|5. Multiplication '''distributes''' over addition: <math>\forall x,y,z\in S, x\cdot(y+z)= x\cdot y+x\cdot z</math> and <math>(y+z)\cdot x= y\cdot x+z\cdot x.</math><br />
|style="text-align:center;"|<math>+,\cdot</math><br />
|-<br />
|6. '''Multiplication''' is '''associative''': <math>\forall x,y,z\in S, (x\cdot y)\cdot z= x\cdot (y\cdot z).</math><br />
|rowspan="4" style="text-align:center;"|<math>\cdot</math><br />
|-<br />
|7. Existence of '''multiplicative identity 1''': <math>\forall x\in S, x\cdot 1= 1\cdot x=x.</math><br />
|-<br />
|colspan="5"|<br />
|8. '''Multiplication''' is '''commutative''': <math>\forall x,y\in S, x\cdot y= y\cdot x.</math><br />
|-<br />
|colspan="6"|<br />
|9. Every non-zero element has a '''multiplicative inverse''': <math>\forall x\in S\setminus\{0\}, \exists x^{-1}\in S, \text{ s.t. } x\cdot x^{-1}= x^{-1}\cdot x=1.</math><br />
|}<br />
The semigroup, monoid, group and abelian group are given by <math>(S,+)</math>, and the ring, commutative ring, and field are given by <math>(S,+,\cdot)</math>.<br />
<br />
Examples:<br />
* '''Infinite fields''': <math>\mathbb{Q}</math>, <math>\mathbb{R}</math>, <math>\mathbb{C}</math> are fields. The integer set <math>\mathbb{Z}</math> is a commutative ring but is not a field.<br />
* '''Finite fields''': Finite fields are called '''Galois fields'''. The number of elements of a finite field is called its '''order'''. A finite field of order <math>q</math>, is usually denoted as <math>\mathsf{GF}(q)</math> or <math>\mathbb{F}_q</math>.<br />
** '''Prime field''' <math>{\mathbb{Z}_p}</math>: For any integer <math>n>1</math>, <math>\mathbb{Z}_n=\{0,1,\ldots,n-1\}</math> under modulo-<math>p</math> addition <math>+</math> and multiplication <math>\cdot</math> forms a commutative ring. It is called '''quotient ring''', and is sometimes denoted as <math>\mathbb{Z}/n\mathbb{Z}</math>. In particular, for '''prime''' <math>p</math>, <math>\mathbb{Z}_p</math> is a field. This can be verified by [http://en.wikipedia.org/wiki/Fermat%27s_little_theorem Fermat's little theorem].<br />
** '''Boolean arithmetics''' <math>\mathsf{GF}(2)</math>: The finite field of order 2 <math>\mathsf{GF}(2)</math> contains only two elements 0 and 1, with bit-wise XOR as addition and bit-wise AND as multiplication. <math>\mathsf{GF}(2^n)</math><br />
** Other examples: There are other examples of finite fields, for instance <math>\{a+bi\mid a,b\in \mathbb{Z}_3\}</math> where <math>i=\sqrt{-1}</math>. This field is isomorphic to <math>\mathsf{GF}(9)</math>. In fact, the following theorem holds for finite fields of given order.<br />
<br />
{{Theorem|Theorem|<br />
:A finite field of order <math>q</math> exists if and only if <math>q=p^k</math> for some prime number <math>p</math> and positive integer <math>k</math>. Moreover, all fields of a given order are isomorphic.<br />
}}<br />
<br />
=Polynomial over a field=<br />
Given a field <math>\mathbb{F}</math>, the '''polynomial ring''' <math>\mathbb{F}[x]</math> consists of all polynomials in the variable <math>x</math> with coefficients in <math>\mathbb{F}</math>. Addition and multiplication of polynomials are naturally defined by applying the distributive law and combining like terms.<br />
<br />
{{Theorem|Proposition (polynomial ring)|<br />
:<math>\mathbb{F}[x]</math> is a ring.<br />
}}<br />
The '''degree''' <math>\mathrm{deg}(f)</math> of a polynomial <math>f\in \mathbb{F}[x]</math> is the exponent on the '''leading term''', the term with a nonzero coefficient that has the largest exponent.<br />
<br />
Because <math>\mathbb{F}[x]</math> is a ring, we cannot do division the way we do it in a field like <math>\mathbb{R}</math>, but we can do division the way we do it in a ring like <math>\mathbb{Z}</math>, leaving a '''remainder'''. The equivalent of the '''integer division''' for <math>\mathbb{Z}</math> is as follows.<br />
{{Theorem|Proposition (division for polynomials)|<br />
:Given a polynomial <math>f</math> and a nonzero polynomial <math>g</math> in <math>\mathbb{F}[x]</math>, there are unique polynomials <math>q</math> and <math>r</math> such that <math>f =q\cdot g+r</math> and <math>\mathrm{deg}(r)<\mathrm{deg}(g)</math>.<br />
}}<br />
The proof of this is by induction on <math>\mathrm{deg}(f)</math>, with the basis <math>\mathrm{deg}(f)<\mathrm{deg}(g)</math>, in which case the theorem holds trivially by letting <math>q=0</math> and <math>r=f</math>.<br />
<br />
As we turn <math>\mathbb{Z}</math> (a ring) into a finite field <math>\mathbb{Z}_p</math> by taking quotients <math>\bmod p</math>, we can turn a polynomial ring <math>\mathbb{F}[x]</math> into a finite field by taking <math>\mathbb{F}[x]</math> modulo a "prime-like" polynomial, using the division of polynomials above.<br />
<br />
{{Theorem|Definition (irreducible polynomial)|<br />
:An '''irreducible polynomial''', or a '''prime polynomial''', is a non-constant polynomial <math>f</math> that ''cannot'' be factored as <math>f=g\cdot h</math> for any non-constant polynomials <math>g</math> and <math>h</math>.<br />
}}</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=%E9%AB%98%E7%BA%A7%E7%AE%97%E6%B3%95_(Fall_2026)/Fingerprinting&diff=13891高级算法 (Fall 2026)/Fingerprinting2026-09-02T09:18:54Z<p>Etone: Created page with "= Checking Matrix Multiplication= The evolution of time complexity <math>O(n^{\omega})</math> for matrix multiplication. Let <math>\mathbb{F}</math> be a feild (you may think of it as the filed <math>\mathbb{Q}</math> of rational numbers, or the finite field <math>\mathbb{Z}_p</math> of integers modulo prime <math>p</math>). We suppose that each field operation (addition, subtraction, multiplication, division) has u..."</p>
<hr />
<div>= Checking Matrix Multiplication=<br />
[[File: matrix_multiplication.png|thumb|360px|right|The evolution of time complexity <math>O(n^{\omega})</math> for matrix multiplication.]]<br />
<br />
Let <math>\mathbb{F}</math> be a feild (you may think of it as the filed <math>\mathbb{Q}</math> of rational numbers, or the finite field <math>\mathbb{Z}_p</math> of integers modulo prime <math>p</math>). We suppose that each field operation (addition, subtraction, multiplication, division) has unit cost. This model is called the '''unit-cost RAM''' model, which is an ideal abstraction of a computer.<br />
<br />
Consider the following problem:<br />
* '''Input''': Three <math>n\times n</math> matrices <math>A</math>, <math>B</math>, and <math>C</math> over the field <math>\mathbb{F}</math>.<br />
* '''Output''': "yes" if <math>C=AB</math> and "no" if otherwise.<br />
<br />
A naive way to solve this is to multiply <math>A</math> and <math>B</math> and compare the result with <math>C</math>. <br />
The straightforward algorithm for matrix multiplication takes <math>O(n^3)</math> time, assuming that each arithmetic operation takes unit time.<br />
The [http://en.wikipedia.org/wiki/Strassen_algorithm Strassen's algorithm] discovered in 1969 now implemented by many numerical libraries runs in time <math>O(n^{\log_2 7})\approx O(n^{2.81})</math>. Strassen's algorithm starts the search for fast matrix multiplication algorithms. The [http://en.wikipedia.org/wiki/Coppersmith%E2%80%93Winograd_algorithm Coppersmith–Winograd algorithm] discovered in 1987 runs in time <math>O(n^{2.376})</math> but is only faster than Strassens' algorithm on extremely large matrices due to the very large constant coefficient. This has been the best known for decades, until recently Stothers got an <math>O(n^{2.374})</math> algorithm in his PhD thesis in 2010, and independently Vassilevska Williams got an <math>O(n^{2.373})</math> algorithm in 2012. Both these improvements are based on generalization of Coppersmith–Winograd algorithm. It is unknown whether the matrix multiplication can be done in time <math>O(n^{2+o(1)})</math>.<br />
<br />
== Freivalds Algorithm ==<br />
The following is a very simple randomized algorithm due to Freivalds, running in <math>O(n^2)</math> time:<br />
<br />
{{Theorem|Algorithm (Freivalds, 1979)|<br />
*pick a vector <math>r \in\{0, 1\}^n</math> uniformly at random;<br />
*if <math>A(Br) = Cr</math> then return "yes" else return "no";<br />
}}<br />
The product <math>A(Br)</math> is computed by first multiplying <math>Br</math> and then <math>A(Br)</math>.<br />
The running time of Freivalds algorithm is <math>O(n^2)</math> because the algorithm computes 3 matrix-vector multiplications. <br />
<br />
If <math>AB=C</math> then <math>A(Br) = Cr</math> for any <math>r \in\{0, 1\}^n</math>, thus the algorithm will return a "yes" for any positive instance (<math>AB=C</math>). <br />
But if <math>AB \neq C</math> then the algorithm will make a mistake if it chooses such an <math>r</math> that <math>ABr = Cr</math>. However, the following lemma states that the probability of this event is bounded.<br />
<br />
{{Theorem|Lemma|<br />
:If <math>AB\neq C</math> then for a uniformly random <math>r \in\{0, 1\}^n</math>,<br />
::<math>\Pr[ABr = Cr]\le \frac{1}{2}</math>.<br />
}}<br />
{{Proof| Let <math>D=AB-C</math>. The event <math>ABr=Cr</math> is equivalent to that <math>Dr=0</math>. It is then sufficient to show that for a <math>D\neq \boldsymbol{0}</math>, it holds that <math>\Pr[Dr = \boldsymbol{0}]\le \frac{1}{2}</math>.<br />
<br />
Since <math>D\neq \boldsymbol{0}</math>, it must have at least one non-zero entry. Suppose that <math>D_{ij}\neq 0</math>.<br />
<br />
We assume the event that <math>Dr=\boldsymbol{0}</math>. In particular, the <math>i</math>-th entry of <math>Dr</math> is <br />
:<math>(Dr)_{i}=\sum_{k=1}^n D_{ik}r_k=0.</math> <br />
The <math>r_j</math> can be calculated by<br />
:<math>r_j=-\frac{1}{D_{ij}}\sum_{k\neq j}^n D_{ik}r_k.</math><br />
Once all other entries <math>r_k</math> with <math>k\neq j</math> are fixed, there is a unique solution of <math>r_j</math>. Therefore, the number of <math>r\in\{0,1\}^n</math> satisfying <math>Dr=\boldsymbol{0}</math> is at most <math>2^{n-1}</math>. The probability that <math>ABr=Cr</math> is bounded as<br />
:<math>\Pr[ABr=Cr]=\Pr[Dr=\boldsymbol{0}]\le\frac{2^{n-1}}{2^n}=\frac{1}{2}</math>.<br />
}}<br />
<br />
When <math>AB=C</math>, Freivalds algorithm always returns "yes"; and when <math>AB\neq C</math>, Freivalds algorithm returns "no" with probability at least 1/2.<br />
<br />
To improve its accuracy, we can run Freivalds algorithm for <math>k</math> times, each time with an ''independent'' <math>r\in\{0,1\}^n</math>, and return "yes" if and only if all running instances returns "yes".<br />
<br />
{{Theorem|Freivalds' Algorithm (multi-round)|<br />
*pick <math>k</math> vectors <math>r_1,r_2,\ldots,r_k \in\{0, 1\}^n</math> uniformly and independently at random;<br />
*if <math>A(Br_i) = Cr_i</math> for all <math>i=1,\ldots,k</math> then return "yes" else return "no";<br />
}}<br />
<br />
If <math>AB=C</math>, then the algorithm returns a "yes" with probability 1. If <math>AB\neq C</math>, then due to the independence, the probability that all <math>r_i</math> have <math>ABr_i=C_i</math> is at most <math>2^{-k}</math>, so the algorithm returns "no" with probability at least <math>1-2^{-k}</math>. For any <math>0<\epsilon<1</math>, choose <math>k=\log_2 \frac{1}{\epsilon}</math>. The algorithm runs in time <math>O(n^2\log_2\frac{1}{\epsilon})</math> and has a one-sided error (false positive) bounded by <math>\epsilon</math>.<br />
<br />
<br />
=Polynomial Identity Testing (PIT) =<br />
The '''Polynomial Identity Testing (PIT)''' is such a problem: given as input two polynomials, determine whether they are identical. It plays a fundamental role in ''Identity Testing'' problems.<br />
<br />
First, let's consider the univariate ("one variable") case:<br />
* '''Input:''' two polynomials <math>f, g\in\mathbb{F}[x]</math> of degree <math>d</math>.<br />
* Determine whether <math>f\equiv g</math> (<math>f</math> and <math>g</math> are identical).<br />
Here the <math>\mathbb{F}[x]</math> denotes the [http://en.wikipedia.org/wiki/Polynomial_ring ring of univariate polynomials] on a field <math>\mathbb{F}</math>. More precisely, a polynomial <math>f\in\mathbb{F}[x]</math> is<br />
:<math>f(x)=\sum_{i=0}^\infty a_ix^i</math>,<br />
where the coefficients <math>a_i</math> are taken from the field <math>\mathbb{F}</math>, and the addition and multiplication are also defined over the field <math>\mathbb{F}</math>. <br />
And:<br />
* the '''degree''' of <math>f</math> is the highest <math>i</math> with non-zero <math>a_i</math>;<br />
* a polynomial <math>f</math> is a '''zero-polynomial''', denoted as <math>f\equiv 0</math>, if all coefficients <math>a_i=0</math>.<br />
<br />
Alternatively, we can consider the following equivalent problem by comparing the polynomial <math>f-g</math> (whose degree is at most <math>d</math>) with the zero-polynomial:<br />
* '''Input:''' a polynomial <math>f\in\mathbb{F}[x]</math> of degree <math>d</math>.<br />
* Determine whether <math>f\equiv 0</math> (<math>f</math> is the 0 polynomial).<br />
<br />
The problem is trivial if the input polynomial <math>f</math> is given explicitly: one can trivially solve the problem by checking whether all <math>d+1</math> coefficients are <math>0</math>. To make the problem nontrivial, we assume that the input polynomial is given implicitly as a ''black box'' (also called an ''oracle''): the only way the algorithm can access to <math>f</math> is to evaluate <math>f(x)</math> over some <math>x</math> from the field <math>\mathbb{F}</math>, <math>x</math> chosen by the algorithm.<br />
<br />
A straightforward deterministic algorithm is to evaluate <math>f(x_1),f(x_2),\ldots,f(x_{d+1})</math> over <math>d+1</math> ''distinct'' elements <math>x_1,x_2,\ldots,x_{d+1}</math> from the field <math>\mathbb{F}</math> and check whether they are all zero. By the [https://en.wikipedia.org/wiki/Fundamental_theorem_of_algebra fundamental theorem of algebra], also known as polynomial interpolations, this guarantees to verify whether a degree-<math>d</math> univariate polynomial <math>f\equiv 0</math>.<br />
{{Theorem|Fundamental Theorem of Algebra|<br />
:Any non-zero univariate polynomial of degree <math>d</math> has at most <math>d</math> roots.<br />
}}<br />
The reason for this fundamental theorem holding generally over any field <math>\mathbb{F}</math> is that any univariate polynomial of degree <math>d</math> factors uniquely into at most <math>d</math> irreducible polynomials, each of which has at most one root.<br />
<br />
The following simple randomized algorithm is natural:<br />
{{Theorem|Algorithm for PIT|<br />
*suppose we have a finite subset <math>S\subseteq\mathbb{F}</math> (to be specified later);<br />
*pick <math>r\in S</math> ''uniformly'' at random;<br />
*if <math>f(r) = 0</math> then return “yes” else return “no”;<br />
}}<br />
<br />
This algorithm evaluates <math>f</math> at one point chosen uniformly at random from a finite subset <math>S\subseteq\mathbb{F}</math>. It is easy to see the followings:<br />
* If <math>f\equiv 0</math>, the algorithm always returns "yes", so it is always correct.<br />
* If <math>f\not\equiv 0</math>, the algorithm may wrongly return "yes" (a '''false positive'''). But this happens only when the random <math>r</math> is a root of <math>f</math>. By the fundamental theorem of algebra, <math>f</math> has at most <math>d</math> roots, so the probability that the algorithm is wrong is bounded as<br />
::<math>\Pr[f(r)=0]\le\frac{d}{|S|}.</math><br />
<br />
By fixing <math>S\subseteq\mathbb{F}</math> to be an arbitrary subset of size <math>|S|=2d</math>, this probability of false positive is at most <math>1/2</math>. We can reduce it to an arbitrarily small constant <math>\delta</math> by repeat the above testing ''independently'' for <math>\log_2 \frac{1}{\delta}</math> times, since the error probability decays geometrically as we repeat the algorithm independently. <br />
<br />
== Communication Complexity of Equality ==<br />
The [http://en.wikipedia.org/wiki/Communication_complexity communication complexity] is introduced by Andrew Chi-Chih Yao as a model of computation with more than one entities, each with partial information about the input.<br />
<br />
Assume that there are two entities, say Alice and Bob. Alice has a private input <math>a</math> and Bob has a private input <math>b</math>. Together they want to compute a function <math>f(a,b)</math> by communicating with each other. The communication follows a predefined '''communication protocol''' (the "algorithm" in this model). The complexity of a communication protocol is measured by the number of bits communicated between Alice and Bob in the worst case.<br />
<br />
The problem of checking identity is formally defined by the function EQ as follows: <math>\mathrm{EQ}:\{0,1\}^n\times\{0,1\}^n\rightarrow\{0,1\}</math> and for any <math>a,b\in\{0,1\}^n</math>,<br />
:<math><br />
\mathrm{EQ}(a,b)=<br />
\begin{cases}<br />
1& \mbox{if } a=b,\\<br />
0& \mbox{otherwise.}<br />
\end{cases}</math><br />
<br />
A trivial way to solve EQ is to let Bob send his entire input string <math>b</math> to Alice and let Alice check whether <math>a=b</math>. This costs <math>n</math> bits of communications.<br />
<br />
It is known that for deterministic communication protocols, this is the best we can get for computing EQ.<br />
<br />
{{Theorem|Theorem (Yao 1979)|<br />
:Any deterministic communication protocol computing EQ on two <math>n</math>-bit strings costs <math>n</math> bits of communication in the worst-case.<br />
}}<br />
<br />
This theorem is much more nontrivial to prove than it looks, because Alice and Bob are allowed to interact with each other in arbitrary ways. The proof of this theorem is in Yao's [http://math.ucla.edu/~znorwood/290d.2.14s/papers/yao.pdf celebrated paper in 1979 with a humble title]. It pioneered the field of communication complexity.<br />
<br />
If we allow randomness in protocols, and also tolerate a small probabilistic error, the problem can be solved with significantly less communications. To present this randomized protocol, we need a few preparations:<br />
* We represent the inputs <math>a,b \in\{0,1\}^{n}</math> of Alice and Bob as two univariate polynomials of degree at most <math>n-1</math>, respectively <br />
::<math>f(x)=\sum_{i=0}^{n-1}a_ix^{i}</math> and <math>g(x)=\sum_{i=0}^{n-1}b_ix^{i}</math>.<br />
* The two polynomials <math>f</math> and <math>g</math> are defined over finite field <math>\mathbb{Z}_p=\{0,1,\ldots,p-1\}</math> for some suitable prime <math>p</math> (to be specified later), which means the additions and multiplications are modulo <math>p</math>.<br />
The randomized communication protocol is then as follows:<br />
{{Theorem|A randomized protocol for EQ|<br />
'''Bob does''':<br />
:* pick <math>r\in\mathbb{Z}_p</math> uniformly at random;<br />
:* send <math>r</math> and <math>g(r)</math> to Alice;<br />
'''Upon receiving''' <math>r</math> and <math>g(r)</math> '''Alice does''':<br />
:* compute <math>f(r)</math>;<br />
:* If <math>f(r)= g(r)</math> return "'''yes'''"; else return "'''no'''".<br />
}}<br />
The communication complexity of the protocol is given by the number of bits used to represent the values of <math>r</math> and <math>g(r)</math>.<br />
Since the polynomials are defined over finite field <math>\mathbb{Z}_p</math> and the random number <math>f</math> is also chosen from <math>\mathbb{Z}_p</math>, this is bounded by <math>O(\log p)</math>. <br />
<br />
On the other hand the protocol makes mistakes only when <math>a\neq b</math> but wrongly answers "yes". This happens only when <math>f\not\equiv g</math> but <math>f(r)=g(r)</math>. The degrees of <math>f, g</math> are at most <math>n-1</math>, and <math>r</math> is chosen among <math>p</math> distinct values, we have<br />
:<math>\Pr[f(r)=g(r)]\le \frac{n-1}{p}</math>.<br />
<br />
By choosing <math>p</math> to be a prime in the interval <math>[n^2, 2n^2]</math> (by [https://en.wikipedia.org/wiki/Bertrand%27s_postulate Chebyshev's theorem], such prime <math>p</math> always exists), the above randomized communication protocol solves the Equality function EQ with an error probability of false positive at most <math>O(1/n)</math>, with communication complexity <math>O(\log n)</math>, an EXPONENTIAL improvement to ANY deterministic communication protocol!<br />
<br />
== Schwartz-Zippel Theorem ==<br />
Now let's see the the true form of '''Polynomial Identity Testing (PIT)''', for multivariate polynomials:<br />
* '''Input:''' two <math>n</math>-variate polynomials <math>f, g\in\mathbb{F}[x_1,x_2,\ldots,x_n]</math> of degree <math>d</math>.<br />
* Determine whether <math>f\equiv g</math>.<br />
The <math>\mathbb{F}[x_1,x_2,\ldots,x_n]</math> is the [http://en.wikipedia.org/wiki/Polynomial_ring#The_polynomial_ring_in_several_variables ring of multivariate polynomials] over field <math>\mathbb{F}</math>. An <math>n</math>-variate polynomial of degree <math>d</math>, written as a sum of monomials, is:<br />
:<math>f(x_1,x_2,\ldots,x_n)=\sum_{i_1,i_2,\ldots,i_n\ge 0\atop i_1+i_2+\cdots+i_n\le d}a_{i_1,i_2,\ldots,i_n}x_{1}^{i_1}x_2^{i_2}\cdots x_{n}^{i_n}</math>.<br />
The '''degree''' or '''total degree''' of a monomial <math>a_{i_1,i_2,\ldots,i_n}x_{1}^{i_1}x_2^{i_2}\cdots x_{n}^{i_n}</math> is given by <math>i_1+i_2+\cdots+i_n</math> and the degree of a polynomial <math>f</math> is the maximum degree of monomials of nonzero coefficients.<br />
<br />
As before, we also consider the following equivalent problem:<br />
* '''Input:''' a polynomial <math>f\in\mathbb{F}[x_1,x_2,\ldots,x_n]</math> of degree <math>d</math>.<br />
* Determine whether <math>f\equiv 0</math>.<br />
<br />
If <math>f</math> is written explicitly as a sum of monomials, then the problem can be solved by checking whether all coefficients, and there at most <math>{n+d\choose d}\le (n+d)^{d}</math> coefficients in an <math>n</math>-variate polynomial of degree at most <math>d</math>. <br />
<br />
A multivariate polynomial <math>f</math> can also be presented in its '''product form''', for example:<br />
{{Theorem|Example|<br />
The [http://en.wikipedia.org/wiki/Vandermonde_matrix Vandermonde matrix] <math>M=M(x_1,x_2,\ldots,x_n)</math> is defined as that <math>M_{ij}=x_i^{j-1}</math>, that is<br />
:<math>M=\begin{bmatrix}<br />
1 & x_1 & x_1^2 & \dots & x_1^{n-1}\\<br />
1 & x_2 & x_2^2 & \dots & x_2^{n-1}\\<br />
1 & x_3 & x_3^2 & \dots & x_3^{n-1}\\<br />
\vdots & \vdots & \vdots & \ddots &\vdots \\<br />
1 & x_n & x_n^2 & \dots & x_n^{n-1}<br />
\end{bmatrix}</math>.<br />
Let <math>f</math> be the polynomial defined as<br />
:<math><br />
f(x_1,\ldots,x_n)=\det(M)=\prod_{j<i}(x_i-x_j).<br />
</math><br />
}}<br />
For polynomials in product form, it is quite efficient to '''evaluate''' the polynomial at any specific point from the field over which the polynomial is defined, however, '''expanding''' the polynomial to a sum of monomials can be very expensive. <br />
<br />
The following is a simple randomized algorithm for testing identity of multivariate polynomials:<br />
{{Theorem|Randomized algorithm for multivariate PIT|<br />
*suppose we have a finite subset <math>S\subseteq\mathbb{F}</math> (to be specified later);<br />
* pick <math>r_1,r_2,\ldots,r_n\in S</math> ''uniformly'' and ''independently'' at random;<br />
* if <math>f(\vec{r})=f(r_1,r_2,\ldots,r_n) = 0</math> then return “yes” else return “no”;<br />
}}<br />
<br />
This algorithm evaluates <math>f</math> at one point chosen uniformly from an <math>n</math>-dimensional cube <math>S^n</math>, where <math>S\subseteq\mathbb{F}</math> is a finite subset. And:<br />
* If <math>f\equiv 0</math>, the algorithm always returns "yes", so it is always correct.<br />
* If <math>f\not\equiv 0</math>, the algorithm may wrongly return "yes" (a '''false positive'''). But this happens only when the random <math>\vec{r}=(r_1,r_2,\ldots,r_n)</math> is a root of <math>f</math>. The probability of this bad event is upper bounded by the following famous result due to [https://pdfs.semanticscholar.org/b913/cf330852035f49b4ec5fe2db86c47d8a98fd.pdf Schwartz (1980)] and [http://www.cecm.sfu.ca/~monaganm/teaching/TopicsinCA15/zippel79.pdf Zippel (1979)]. <br />
<br />
{{Theorem|Schwartz-Zippel Theorem|<br />
: Let <math>f\in\mathbb{F}[x_1,x_2,\ldots,x_n]</math> be a multivariate polynomial of degree <math>d</math> over a field <math>\mathbb{F}</math>. If <math>f\not\equiv 0</math>, then for any finite set <math>S\subset\mathbb{F}</math>, and <math>r_1,r_2\ldots,r_n\in S</math> chosen uniformly and independently at random, <br />
::<math>\Pr[f(r_1,r_2,\ldots,r_n)=0]\le\frac{d}{|S|}.</math><br />
}}<br />
The Schwartz-Zippel Theorem states that for any nonzero <math>n</math>-variate polynomial of degree at most <math>d</math>, the number of roots in any cube <math>S^n</math> is at most <math>d\cdot |S|^{n-1}</math>.<br />
<br />
Dana Moshkovitz gave a surprisingly simply and elegant [http://eccc.hpi-web.de/report/2010/096/ proof] of Schwartz-Zippel Theorem, using some advanced ideas. Now we introduce the standard proof by induction.<br />
<br />
{{Proof| <br />
The theorem is proved by induction on <math>n</math>.<br />
<br />
;Induction basis''':'''<br />
For <math>n=1</math>, this is the univariate case. Assume that <math>f\not\equiv 0</math>. Due to the fundamental theorem of algebra, any polynomial <math>f(x)</math> of degree at most <math>d</math> must have at most <math>d</math> roots, thus <br />
:<math>\Pr[f(r)=0]\le\frac{d}{|S|}.<br />
</math><br />
<br />
;Induction hypothesis''':''' <br />
Assume the theorem holds for any <math>m</math>-variate polynomials for all <math>m<n</math>.<br />
<br />
;Induction step''':'''<br />
For any <math>n</math>-variate polynomial <math>f(x_1,x_2,\ldots,x_n)</math> of degree at most <math>d</math>, we write <math>f</math> as<br />
:<math>f(x_1,x_2,\ldots,x_n)=\sum_{i=0}^kx_n^{i}f_i(x_1,x_2,\ldots,x_{n-1})</math>,<br />
where <math>k</math> is the highest degree of <math>x_n</math>, which means the degree of <math>f_k</math> is at most <math>d-k</math> and <math>f_k\not\equiv 0</math>.<br />
<br />
In particular, we write <math>f</math> as a sum of two parts:<br />
:<math>f(x_1,x_2,\ldots,x_n)=x_n^k f_k(x_1,x_2,\ldots,x_{n-1})+\bar{f}(x_1,x_2,\ldots,x_n)</math>,<br />
where both <math>f_k</math> and <math>\bar{f}</math> are polynomials, such that <br />
* <math>f_k\not\equiv 0</math> is as above, whose degree is at most <math>d-k</math>;<br />
* <math>\bar{f}(x_1,x_2,\ldots,x_n)=\sum_{i=0}^{k-1}x_n^i f_i(x_1,x_2,\ldots,x_{n-1})</math>, thus <math>\bar{f}(x_1,x_2,\ldots,x_n)</math> has no <math>x_n^{k}</math> factor in any term.<br />
<br />
By the law of total probability, we have<br />
:<math><br />
\begin{align}<br />
&\Pr[f(r_1,r_2,\ldots,r_n)=0]\\<br />
=<br />
&\Pr[f(\vec{r})=0\mid f_k(r_1,r_2,\ldots,r_{n-1})=0]\cdot\Pr[f_k(r_1,r_2,\ldots,r_{n-1})=0]\\<br />
&+\Pr[f(\vec{r})=0\mid f_k(r_1,r_2,\ldots,r_{n-1})\neq0]\cdot\Pr[f_k(r_1,r_2,\ldots,r_{n-1})\neq0].<br />
\end{align}<br />
</math><br />
Note that <math>f_k(r_1,r_2,\ldots,r_{n-1})</math> is a polynomial on <math>n-1</math> variables of degree <math>d-k</math> such that <math>f_k\not\equiv 0</math>.<br />
By the induction hypothesis, we have <br />
:<math><br />
\begin{align}<br />
(*)<br />
&\qquad<br />
&\Pr[f_k(r_1,r_2,\ldots,r_{n-1})=0]\le\frac{d-k}{|S|}.<br />
\end{align}<br />
</math><br />
<br />
Now we look at the case conditioning on <math>f_k(r_1,r_2,\ldots,r_{n-1})\neq0</math>. Recall that <math>\bar{f}(x_1,\ldots,x_n)</math> has no <math>x_n^k</math> factor in any term, thus the condition <math>f_k(r_1,r_2,\ldots,r_{n-1})\neq0</math> guarantees that <br />
:<math>f(r_1,\ldots,r_{n-1},x_n)=x_n^k f_k(r_1,r_2,\ldots,r_{n-1})+\bar{f}(r_1,r_2,\ldots,r_{n-1},x_n)=g_{r_1,\ldots,r_{n-1}}(x_n)</math><br />
is a nonzero univariate polynomial of <math>x_n</math> such that the degree of <math>g_{r_1,\ldots,r_{n-1}}(x_n)</math> is <math>k</math> and <math>g_{r_1,\ldots,r_{n-1}}\not\equiv 0</math>, for which we already known that the probability <math>g_{r_1,\ldots,r_{n-1}}(r_n)=0</math> is at most <math>\frac{k}{|S|}</math>.<br />
Therefore,<br />
:<math><br />
\begin{align}<br />
(**)<br />
&\qquad<br />
&\Pr[f(\vec{r})=0\mid f_k(r_1,r_2,\ldots,r_{n-1})\neq0]=\Pr[g_{r_1,\ldots,r_{n-1}}(r_n)=0\mid f_k(r_1,r_2,\ldots,r_{n-1})\neq0]\le\frac{k}{|S|}<br />
\end{align}<br />
</math>.<br />
Substituting both <math>(*)</math> and <math>(**)</math> back in the total probability, we have<br />
:<math><br />
\Pr[f(r_1,r_2,\ldots,r_n)=0]<br />
\le\frac{d-k}{|S|}+\frac{k}{|S|}=\frac{d}{|S|},<br />
</math><br />
which proves the theorem.<br />
}}<br />
<br />
== Detecting perfect matching ==<br />
<br />
TBA<br />
<br />
;Edmonds matrix<br />
<br />
= Fingerprinting =<br />
The polynomial identity testing algorithm in the Schwartz-Zippel theorem can be abstracted as the following framework:<br />
Suppose we want to compare two objects <math>X</math> and <math>Y</math>. Instead of comparing them directly, we compute random '''fingerprints''' <math>\mathrm{FING}(X)</math> and <math>\mathrm{FING}(Y)</math> of them and compare the fingerprints. <br />
<br />
The fingerprints has the following properties:<br />
* <math>\mathrm{FING}(\cdot)</math> is a function, meaning that if <math>X= Y</math> then <math>\mathrm{FING}(X)=\mathrm{FING}(Y)</math>.<br />
* It is much easier to compute and compare the fingerprints.<br />
* Ideally, the domain of fingerprints is much smaller than the domain of original objects, so storing and comparing fingerprints are easy. This means the fingerprint function <math>\mathrm{FING}(\cdot)</math> cannot be an injection (one-to-one mapping), so it's possible that different <math>X</math> and <math>Y</math> are mapped to the same fingerprint. We resolve this by making fingerprint function ''randomized'', and for <math>X\neq Y</math>, we want the probability <math>\Pr[\mathrm{FING}(X)=\mathrm{FING}(Y)]</math> to be small.<br />
<br />
In Schwartz-Zippel theorem, the objects to compare are polynomials from <math>\mathbb{F}[x_1,\ldots,x_n]</math>. Given a polynomial <math>f\in \mathbb{F}[x_1,\ldots,x_n]</math>, its fingerprint is computed as <math>\mathrm{FING}(f)=f(r_1,\ldots,r_n)</math> for <math>r_i</math> chosen independently and uniformly at random from some fixed set <math>S\subseteq\mathbb{F}</math>.<br />
<br />
With this generic framework, for various identity testing problems, we may design different fingerprints <math>\mathrm{FING}(\cdot)</math>.<br />
<br />
== Communication protocols for Equality ==<br />
Now consider again the communication model where the two players Alice with a private input <math>x\in\{0,1\}^n</math> and Bob with a private input <math>y\in\{0,1\}^n</math> together compute a function <math>f(x,y)</math> by running a communication protocol. <br />
<br />
We still consider the communication protocols for the equality function EQ<br />
:<math><br />
\mathrm{EQ}(x,y)=<br />
\begin{cases}<br />
1& \mbox{if } x=y,\\<br />
0& \mbox{otherwise.}<br />
\end{cases}</math><br />
<br />
With the language of fingerprinting, this communication problem can be solved by the following generic scheme:<br />
{{Theorem|Communication protocol for EQ by fingerprinting|<br />
'''Bob does''':<br />
:* choose a random fingerprint function <math>\mathrm{FING}(\cdot)</math> and compute the fingerprint of her input <math>\mathrm{FING}(y)</math>;<br />
:* sends both the description of <math>\mathrm{FING}(\cdot)</math> and the value of <math>\mathrm{FING}(y)</math> to Alice;<br />
'''Upon receiving''' the description of <math>\mathrm{FING}(\cdot)</math> and the value of <math>\mathrm{FING}(y)</math>, '''Alice does''':<br />
:* computes <math>\mathrm{FING}(x)</math> and check whether <math>\mathrm{FING}(x)=\mathrm{FING}(y)</math>.<br />
}}<br />
In this way we have a randomized communication protocol for the equality function EQ with false positive. The communication cost as well as the error probability are reduced to the question of how to design this random fingerprint function <math>\mathrm{FING}(\cdot)</math> to guarantee:<br />
# A random fingerprint function <math>\mathrm{FING}(\cdot)</math> can be described succinctly.<br />
# The range of <math>\mathrm{FING}(\cdot)</math> is small, so the fingerprints are succinct.<br />
# If <math>x\neq y</math>, the probability <math>\Pr[\mathrm{FING}(x)=\mathrm{FING}(y)]</math> is small.<br />
<br />
=== Fingerprinting by PIT===<br />
As before, we can define the fingerprint function as: for any bit-string <math>x\in\{0,1\}^n</math>, its random fingerprint is <math>\mathrm{FING}(x)=\sum_{i=1}^n x_i r^{i}</math>, where the additions and multiplications are defined over a finite field <math>\mathbb{Z}_p</math>, and <math>r</math> is chosen uniformly at random from <math>\mathbb{Z}_p</math>, where <math>p</math> is some suitable prime which can be represented in <math>\Theta(\log n)</math> bits. More specifically, we can choose <math>p</math> to be any prime from the interval <math>[n^2, 2n^2]</math>. Due to Chebyshev's theorem, such prime must exist. <br />
<br />
As we have shown before, it takes <math>O(\log p)=O(\log n)</math> bits to represent <math>\mathrm{FING}(y)</math> and to describe the random function <math>\mathrm{FING}(\cdot)</math> (since it a random function <math>\mathrm{FING}(\cdot)</math> from this family is uniquely identified by a random <math>r\in\mathbb{Z}_p</math>, which can be represented within <math>\log p=O(\log n)</math> bits). And it follows easily from the fundamental theorem of algebra that for any distinct <math>x, y\in\{0,1\}^n</math>,<br />
:<math><br />
\Pr[\mathrm{FING}(x)=\mathrm{FING}(y)] \le \frac{n-1}{p}\le \frac{1}{n}.<br />
</math><br />
<br />
=== Fingerprinting by randomized checksum===<br />
Now we consider a new fingerprint function: We treat each input string <math>x\in\{0,1\}^n</math> as the binary representation of a number, and let <math>\mathrm{FING}(x)=x\bmod p</math> for some random prime <math>p</math> chosen from <math>[k]=\{0,1,\ldots,k-1\}</math>, for some <math>k</math> to be specified later. <br />
<br />
Now a random fingerprint function <math>\mathrm{FING}(\cdot)</math> can be uniquely identified by this random prime <math>p</math>. The new communication protocol for EQ with this fingerprint is as follows:<br />
{{Theorem|Communication protocol for EQ by random checksum|<br />
'''Bob does''':<br />
:for some parameter <math>k</math> (to be specified), <br />
:* choose a prime <math>p\in[k]</math> uniformly at random;<br />
:* send <math>p</math> and <math>x\bmod p</math> to Alice;<br />
'''Upon receiving''' <math>p</math> and <math>x\bmod p</math>, '''Alice does''':<br />
:* check whether <math>x\bmod p=y\bmod p</math>.<br />
}}<br />
The number of bits to be communicated is obviously <math>O(\log k)</math>. When <math>x\neq y</math>, we want to upper bound the error probability <math>\Pr[x\bmod p=y\bmod p]</math>.<br />
<br />
Suppose without loss of generality <math>x>y</math>. Let <math>z=x-y</math>. Then <math>z<2^n</math> since <math>x,y\in[2^n]</math>, and <br />
<math>z\neq 0</math> for <math>x\neq y</math>. It holds that <math>x\equiv y\pmod p</math> if and only if <math>p\mid z</math>. Therefore, we only need to upper bound the probability<br />
:<math>\Pr[z\bmod p=0]</math><br />
for an arbitrarily fixed <math>0<z<2^n</math>, and a uniform random prime <math>p\in[k]</math>.<br />
<br />
The probability <math>\Pr[z\bmod p=0]</math> is computed directly as<br />
:<math>\Pr[z\bmod p=0]\le\frac{\mbox{the number of prime divisors of }z}{\mbox{the number of primes in }[k]}</math>.<br />
<br />
For the numerator, any positive <math>z<2^n</math> has at most <math>n</math> prime factors. To see this, by contradiction assume that <math>z</math> has more than <math>n</math> prime factors. Note that any prime number is at least 2. Then <math>z</math> must be greater than <math>2^n</math>, contradicting the fact that <math>z<2^n</math>.<br />
<br />
For the denominator, we need to lower bound the number of primes in <math>[k]</math>. This is given by the celebrated [http://en.wikipedia.org/wiki/Prime_number_theorem '''Prime Number Theorem (PNT)'''].<br />
<br />
{{Theorem|Prime Number Theorem|<br />
:Let <math>\pi(k)</math> denote the number of primes less than <math>k</math>. Then <math>\pi(k)\sim\frac{k}{\ln k}</math> as <math>k\rightarrow\infty</math>.<br />
}}<br />
<br />
Therefore, by choosing <math>k=2n^2\ln n</math>, we have that for a <math>0<z<2^n</math>, and a random prime <math>p\in[k]</math>,<br />
:<math>\Pr[z\bmod p=0]\le\frac{n}{\pi(k)}\sim\frac{1}{n}</math>,<br />
which means the for any inputs <math>x,y\in\{0,1\}^n</math>, if <math>x\neq y</math>, then the false positive is bounded as<br />
:<math>\Pr[\mathrm{FING}(x)=\mathrm{FING}(y)]\le\Pr[|x-y|\bmod p=0]\le \frac{1}{n}</math>.<br />
Moreover, by this choice of parameter <math>k=2n^2\ln n</math>, the communication complexity of the protocol is bounded by <math>O(\log k)=O(\log n)</math>.<br />
<br />
== Pattern matching ==<br />
Consider the following problem of pattern matching, which has nothing to do with communication complexity.<br />
<br />
*Input: a string <math>x\in\{0,1\}^n</math> and a "pattern" <math>y\in\{0,1\}^m</math>.<br />
*Determine whether the pattern <math>y</math> is a contiguous substring of <math>x</math>. Usually, we are also asked to find the location of the substring.<br />
<br />
A naive algorithm trying every possible match runs in <math>O(nm)</math> time. The more sophisticated KMP algorithm inspired by automaton theory runs in <math>O(n+m)</math> time.<br />
<br />
A simple randomized algorithm, due to Karp and Rabin, uses the idea of fingerprinting and also runs in <math>O(n + m)</math> time.<br />
<br />
Let <math>X(j)=x_jx_{j+1}\cdots x_{j+m-1}</math> denote the substring of <math>x</math> of length <math>m</math> starting at position <math>j</math>.<br />
<br />
{{Theorem|Algorithm (Karp-Rabin)|<br />
:pick a random prime <math>p\in[k]</math>;<br />
:'''for''' <math>j = 1</math> to <math>n -m + 1</math> '''do'''<br />
::'''if''' <math>X(j)\bmod p = y \bmod p</math> then report a match;<br />
:'''return''' "no match";<br />
}}<br />
<br />
So the algorithm just compares the <math>\mathrm{FING}(X(j))</math> and <math>\mathrm{FING}(y)</math> for every <math>j</math>, with the same definition of fingerprint function <math>\mathrm{FING}(\cdot)</math> as in the communication protocol for EQ.<br />
<br />
By the same analysis, by choosing <math>k=n^2m\ln (n^2m)</math>, the probability of a single false match is <br />
:<math>\Pr[X(j)\bmod p=y\bmod p\mid X(j)\neq y ]=O\left(\frac{1}{n^2}\right)</math>.<br />
By the union bound, the probability that a false match occurs is <math>O\left(\frac{1}{n}\right)</math>.<br />
<br />
The algorithm runs in linear time if we assume that we can compute <math>X(j)\bmod p </math> for each <math>j</math> in constant time. This outrageous assumption can be made realistic by the following observation.<br />
<br />
{{Theorem|Lemma|<br />
:Let <math>\mathrm{FING}(a)=a\bmod p</math>.<br />
::<math>\mathrm{FING}(X(j+1))\equiv2(\mathrm{FING}(X(j))-2^{m-1}x_j)+x_{j+m}\pmod p\,</math>.<br />
}}<br />
{{Proof| It holds that <br />
:<math>X(j+1)=2(X(j)-2^{m-1}x_j)+x_{j+m}\,</math>.<br />
So the equation holds on the finite field modulo <math>p</math>.<br />
}}<br />
<br />
Due to this lemma, each fingerprint <math>\mathrm{FING}(X(j))</math> can be computed in an incremental way, each in constant time. The running time of the algorithm is <math>O(n+m)</math>.<br />
<br />
== Checking distinctness ==<br />
Consider the following problem of '''checking distinctness''':<br />
*Given a sequence <math>x_1,x_2,\ldots,x_n\in\{1,2,\ldots,n\}</math>, check whether every element of <math>\{1,2,\ldots,n\}</math> appears '''exactly''' once.<br />
<br />
Obviously this problem can be solved in linear time and linear space (in addition to the space for storing the input) by maintaining a <math>n</math>-bit vector that indicates which numbers among <math>\{1,2,\ldots,n\}</math> have appeared.<br />
<br />
When this <math>n</math> is enormously large, <math>\Omega(n)</math> space cost is too expensive. We wonder whether we could solve this problem with a space cost (in addition to the space for storing the input) much less than <math>O(n)</math>. This can be done by fingerprinting if we tolerate a certain degree of inaccuracy.<br />
<br />
We consider the following more generalized problem, '''checking identity of multisets''':<br />
* '''Input:''' two multisets <math>A=\{a_1,a_2,\ldots, a_n\}</math> and <math>B=\{b_1,b_2,\ldots, b_n\}</math> where <math>a_1,a_2,\ldots,b_1,b_2,\ldots,b_n\in \{1,2,\ldots,n\}</math>.<br />
* Determine whether <math>A=B</math> (multiset equivalence).<br />
<br />
Here for a '''multiset''' <math>A=\{a_1,a_2,\ldots, a_n\}</math>, its elements <math>a_i</math> are not necessarily distinct. The '''multiplicity''' of an element <math>a_i</math> in a multiset <math>A</math> is the number of times <math>a_i</math> appears in <math>A</math>. Two multisets <math>A</math> and <math>B</math> are equivalent if they contain the same set of elements and the multiplicities of every element in <math>A</math> and <math>B</math> are equal. <br />
<br />
Obviously the above problem of checking distinctness can be treated as a special case of checking identity of multisets: by checking the identity of the multiset <math>A</math> and set <math>\{1,2,\ldots, n\}</math>.<br />
<br />
The following fingerprinting function for multisets was introduced by Lipton for solving multiset identity testing.<br />
<br />
{{Theorem|Fingerprint for multiset|<br />
:Let <math>p</math> be a uniform random prime chosen from the interval <math>[(n\log n)^2,2(n\log n)^2]</math>. By Chebyshev's theorem, such prime must exist. And consider the the finite field <math>\mathbb{Z}_p=[p]</math>.<br />
<br />
:Given a multiset <math>A=\{a_1,a_2,\ldots,a_n\}</math>, we define a univariate polynomial <math>f_A\in\mathbb{Z}_p[x]</math> over the finite field <math>\mathbb{Z}_p</math> as follows:<br />
::<math>f_A(x)=\prod_{i=1}^n(x-a_i)</math>,<br />
:where <math>+</math> and <math>\cdot</math> are defined over the finite field <math>\mathbb{Z}_p</math>.<br />
<br />
:We then define the random fingerprinting function as:<br />
::<math>\mathrm{FING}(A)=f_A(r)=\prod_{i=1}^n(r-a_i)</math>,<br />
:where <math>r</math> is chosen uniformly at random from <math>\mathbb{Z}_p</math>.<br />
}}<br />
<br />
Since all computations of <math>\mathrm{FING}(A)=\prod_{i=1}^n(r-a_i)</math> are over the finite field <math>\mathbb{Z}_p</math>, the space cost for computing the fingerprint <math>\mathrm{FING}(A)</math> is only <math>O(\log p)=O(\log n)</math>.<br />
<br />
Moreover, the fingerprinting function <math>\mathrm{FING}(A)</math> is invariant under permutation of elements of the multiset <math>A=\{a_1,a_2,\ldots,a_n\}</math>, thus it is indeed a function of multisets (meaning every multiset has only one fingerprint). Therefore, if <math>A=B</math> then <math>\mathrm{FING}(A)=\mathrm{FING}(B)</math>.<br />
<br />
For two distinct multisets <math>A\neq B</math>, it is possible that <math>\mathrm{FING}(A)=\mathrm{FING}(B)</math>, but the following theorem due to Lipton bounds this error probability of false positive.<br />
<br />
{{Theorem|Theorem (Lipton 1989)|<br />
:Let <math>A=\{a_1,a_2,\ldots,a_n\}</math> and <math>B=\{b_1,b_2,\ldots,b_n\}</math> be two multisets whose elements are from <math>\{1,2,\ldots,n\}</math>. If <math>A\neq B</math>, then<br />
::<math>\Pr[\mathrm{FING}(A)= \mathrm{FING}(B)]=O\left(\frac{1}{n}\right)</math>.<br />
}}<br />
{{Proof| <br />
Let <math>\tilde{f}_A(x)=\prod_{i=1}^n(x-a_i)</math> and <math>\tilde{f}_B(x)=\prod_{i=1}^n(x-b_i)</math> be two univariate polynomials defined over reals <math>\mathbb{R}</math>. Note that in contrast to <math>f_A(x)</math> and <math>f_B(x)</math>, the <math>+</math> and <math>\cdot</math> in <math>\tilde{f}_A(x), \tilde{f}_B(x)</math> do not modulo <math>p</math>. It is easy to verify that the polynomials <math>\tilde{f}_A(x), \tilde{f}_B(x)</math> have the following properties:<br />
*<math>\tilde{f}_A\equiv \tilde{f}_B</math> if and only if <math>A=B</math>. Here <math>A=B</math> means the multiset equivalence.<br />
*By the properties of finite field, for any value <math>r\in\mathbb{Z}_p</math>, it holds that <math>f_A(r)=\tilde{f}_A(r)\bmod p</math> and <math>f_B(r)=\tilde{f}_B(r)\bmod p</math>.<br />
<br />
Therefore, assuming that <math>A\neq B</math>, we must have <math>\tilde{f}_A(x)\not\equiv \tilde{f}_B(x)</math>. Then by the law of total probability:<br />
:<math><br />
\begin{align}<br />
\Pr[\mathrm{FING}(A)= \mathrm{FING}(B)]<br />
&=<br />
\Pr\left[f_A(r)=f_B(r)\mid f_A\not\equiv f_B\right]\Pr[f_A\not\equiv f_B]\\<br />
&\quad\,\,+\Pr\left[f_A(r)=f_B(r)\mid f_A\equiv f_B\right]\Pr[f_A\equiv f_B]\\<br />
&\le \Pr\left[f_A(r)=f_B(r)\mid f_A\not\equiv f_B\right]+\Pr[f_A\equiv f_B].<br />
\end{align}<br />
</math><br />
Note that the degrees of <math>f_A,f_B</math> are at most <math>n</math> and <math>r</math> is chosen uniformly from <math>[p]</math>. By the Schwartz-Zippel theorem for univariate polynomials, the first probability<br />
:<math><br />
\Pr\left[f_A(r)=f_B(r)\mid f_A\not\equiv f_B\right]\le \frac{n}{p}=o\left(\frac{1}{n}\right),<br />
</math><br />
since <math>p</math> is chosen from the interval <math>[(n\log n)^2,2(n\log n)^2]</math>.<br />
<br />
For the second probability <math>\Pr[f_A\equiv f_B]</math>, recall that <math>\tilde{f}_A\not\equiv \tilde{f}_B</math>, therefore there is at least a non-zero coefficient <math>c\le n^n</math> in <math>\tilde{f}_A-\tilde{f}_B</math>. The event <math>f_A\equiv f_B</math> occurs only if <math>c\bmod p=0</math>, which means<br />
:<math><br />
\begin{align}<br />
\Pr[f_A\equiv f_B]<br />
&\le \Pr[c\bmod p=0]\\<br />
&=\frac{\text{number of prime factors of }c}{\text{number of primes in }[(n\log n)^2,2(n\log n)^2]}\\<br />
&\le \frac{n\log_2n}{\pi(2(n\log n)^2)-\pi((n\log n)^2)}.<br />
\end{align}<br />
</math><br />
By the prime number theorem, <math>\pi(N)\rightarrow \frac{N}{\ln N}</math> as <math>N\to\infty</math>. Therefore,<br />
:<math><br />
\Pr[f_A\equiv f_B]=O\left(\frac{n\log n}{n^2\log n}\right)=O\left(\frac{1}{n}\right).<br />
</math><br />
Combining everything together, we have<br />
<math>\Pr[\mathrm{FING}(A)= \mathrm{FING}(B)]=O\left(\frac{1}{n}\right)</math>.<br />
}}</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=%E9%AB%98%E7%BA%A7%E7%AE%97%E6%B3%95_(Fall_2026)&diff=13890高级算法 (Fall 2026)2026-09-02T09:18:27Z<p>Etone: /* Lecture Notes */</p>
<hr />
<div>{{Infobox<br />
|name = Infobox<br />
|bodystyle = <br />
|title = <font size=3>高级算法 <br />
<br>Advanced Algorithms</font><br />
|titlestyle = <br />
<br />
|image = <br />
|imagestyle = <br />
|caption = <br />
|captionstyle = <br />
|headerstyle = background:#ccf;<br />
|labelstyle = background:#ddf;<br />
|datastyle = <br />
<br />
|header1 =Instructor<br />
|label1 = <br />
|data1 = <br />
|header2 = <br />
|label2 = <br />
|data2 = '''尹一通'''<br />
|header3 = <br />
|label3 = Email<br />
|data3 = yinyt@nju.edu.cn <br />
|header4 =<br />
|label4= office<br />
|data4= 计算机系 804<br />
|header5 = <br />
|label5 = <br />
|data5 = '''栗师'''<br />
|header6 = <br />
|label6 = Email<br />
|data6 = shili@nju.edu.cn <br />
|header7 =<br />
|label7= office<br />
|data7= 计算机系 605<br />
|header8 = <br />
|label8 = <br />
|data8 = '''刘景铖'''<br />
|header9 = <br />
|label9 = Email<br />
|data9 = liu@nju.edu.cn <br />
|header10 =<br />
|label10= office<br />
|data10= 计算机系 516<br />
|header11 = Class<br />
|label11 = <br />
|data11 = <br />
|header12 =<br />
|label12 = Class meetings<br />
|data12 = 周一 5-6节 (单) 仙Ⅰ-319<br />
周三 5-6节 仙Ⅰ-319<br />
|header13 =<br />
|label13 = Place<br />
|data13 = <br />
|header14 =<br />
|label14 = Office hours<br />
|data14 = 周一4-5pm（尹一通 804）<br/><br />
周四4-5pm（刘景铖 516）<br/><br />
By appointment<br />
|header15 = Textbooks<br />
|label15 = <br />
|data15 = <br />
|header16 =<br />
|label16 = <br />
|data16 = [[File:MR-randomized-algorithms.png|border|100px]]<br />
|header17 =<br />
|label17 = <br />
|data17 = Motwani and Raghavan. <br>''Randomized Algorithms''.<br> Cambridge Univ Press, 1995.<br />
|header18 =<br />
|label18 = <br />
|data18 = [[File:Approximation_Algorithms.jpg|border|100px]]<br />
|header19 =<br />
|label19 = <br />
|data19 = Vazirani. <br>''Approximation Algorithms''. <br> Springer-Verlag, 2001.<br />
|belowstyle = background:#ddf;<br />
|below = <br />
}}<br />
<br />
This is the webpage for the ''Advanced Algorithms'' class of fall 2026. Students who take this class should check this page periodically for content updates and new announcements. <br />
<br />
= Announcement =<br />
<br />
= Course info =<br />
* '''Instructor ''': <br />
:* [http://tcs.nju.edu.cn/yinyt/ 尹一通]：[mailto:yinyt@nju.edu.cn <yinyt@nju.edu.cn>]，计算机系 804 <br />
:*[https://tcs.nju.edu.cn/shili/ 栗师]：[mailto:shili@nju.edu.cn <shili@nju.edu.cn>]，计算机系 605<br />
:* [https://liuexp.github.io 刘景铖]：[mailto:liu@nju.edu.cn <liu@nju.edu.cn>]，计算机系 516 <br />
* '''Teaching Assistant''': <br />
** 于逸潇：[mailto:yixiaoyu@smail.nju.edu.cn <yixiaoyu@smail.nju.edu.cn>]<br />
** 张弈垚：[mailto:zhangyiyao@smail.nju.edu.cn <zhangyiyao@smail.nju.edu.cn>]<br />
* '''Class meeting''': <br />
** 周一 5-6节 1-17周(单) 仙Ⅰ-319<br />
** 周三 5-6节 1-18周 仙Ⅰ-319<br />
* '''Office hour''': <br />
** 周一4-5pm（尹一通 804）<br />
** 周四4-5pm（刘景铖 516）<br />
** By appointment<br />
* '''QQ群''': 1098567018<br />
<br />
= Syllabus =<br />
随着计算机算法理论的不断发展，现代计算机算法的设计与分析大量地使用非初等的数学工具以及非传统的算法思想。“高级算法”这门课程就是面向计算机算法的这一发展趋势而设立的。课程将针对传统算法课程未系统涉及、却在计算机科学各领域的科研和实践中扮演重要角色的高等算法设计思想和算法分析工具进行系统讲授。<br />
<br />
=== 先修课程 Prerequisites ===<br />
* 必须：离散数学，概率论，线性代数。<br />
* 推荐：算法设计与分析。<br />
<br />
=== Course materials ===<br />
* [[高级算法 (Fall 2026) / Course materials|<font size=3>教材和参考书</font>]]<br />
<br />
=== 成绩 Grades ===<br />
* 课程成绩：本课程将会有若干次作业和一次期末考试。最终成绩将由平时作业成绩和期末考试成绩综合得出。<br />
* 迟交：如果有特殊的理由，无法按时完成作业，请提前联系授课老师，给出正当理由。否则迟交的作业将不被接受。<br />
<br />
=== <font color=red> 学术诚信 Academic Integrity </font>===<br />
学术诚信是所有从事学术活动的学生和学者最基本的职业道德底线，本课程将不遗余力的维护学术诚信规范，违反这一底线的行为将不会被容忍。<br />
<br />
作业完成的原则：署你名字的工作必须是你个人的贡献。在完成作业的过程中，允许讨论，前提是讨论的所有参与者均处于同等完成度。但关键想法的执行、以及作业文本的写作必须独立完成，并在作业中致谢（acknowledge）所有参与讨论的人。不允许其他任何形式的合作——尤其是与已经完成作业的同学“讨论”。<br />
<br />
本课程将对剽窃行为采取零容忍的态度。在完成作业过程中，对他人工作（出版物、互联网资料、其他人的作业等）直接的文本抄袭和对关键思想、关键元素的抄袭，按照 [http://www.acm.org/publications/policies/plagiarism_policy ACM Policy on Plagiarism]的解释，都将视为剽窃。剽窃者成绩将被取消。如果发现互相抄袭行为，<font color=red> 抄袭和被抄袭双方的成绩都将被取消</font>。因此请主动防止自己的作业被他人抄袭。<br />
<br />
学术诚信影响学生个人的品行，也关乎整个教育系统的正常运转。为了一点分数而做出学术不端的行为，不仅使自己沦为一个欺骗者，也使他人的诚实努力失去意义。让我们一起努力维护一个诚信的环境。<br />
<br />
= Assignments =<br />
Late policy: In general, we will accomodate late submission requests ONLY IF you made such requests ahead of time. <br />
<br />
<br />
= Lecture Notes =<br />
# [[高级算法 (Fall 2025)/Min Cut, Max Cut, and Spectral Cut|Min Cut, Max Cut, and Spectral Cut]] ([[http://tcs.nju.edu.cn/slides/aa2026/Cut.pdf slides]])<br />
#* [[高级算法 (Fall 2025)/Probability Basics|Probability basics]]<br />
# [[高级算法 (Fall 2026)/Fingerprinting| Fingerprinting]] ([http://tcs.nju.edu.cn/slides/aa2026/Fingerprinting.pdf slides]) <br />
#* [[高级算法 (Fall 2026)/Finite Field Basics|Finite field basics]]<br />
<br />
= Related Online Courses=<br />
* [https://www.cs.cmu.edu/~15850/ Advanced Algorithms] by Anupam Gupta at CMU.<br />
* [http://people.csail.mit.edu/moitra/854.html Advanced Algorithms] by Ankur Moitra at MIT.<br />
* [http://courses.csail.mit.edu/6.854/current/ Advanced Algorithms] by David Karger and Aleksander Mądry at MIT.<br />
* [http://web.stanford.edu/class/cs168/index.html The Modern Algorithmic Toolbox] by Tim Roughgarden and Gregory Valiant at Stanford.<br />
* [https://www.cs.princeton.edu/courses/archive/fall18/cos521/ Advanced Algorithm Design] by Pravesh Kothari and Christopher Musco at Princeton.<br />
* [http://www.cs.cmu.edu/afs/cs.cmu.edu/academic/class/15859-f11/www/ Linear and Semidefinite Programming (Advanced Algorithms)] by Anupam Gupta and Ryan O'Donnell at CMU.<br />
* [https://www.cs.cmu.edu/~odonnell/papers/cs-theory-toolkit-lecture-notes.pdf CS Theory Toolkit] by Ryan O'Donnell at CMU.<br />
* [https://cs.uwaterloo.ca/~lapchi/cs860/index.html Eigenvalues and Polynomials] by Lap Chi Lau at University of Waterloo.<br />
* The [https://www.cs.cornell.edu/jeh/book.pdf "Foundations of Data Science" book] by Avrim Blum, John Hopcroft, and Ravindran Kannan.</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=%E7%BB%84%E5%90%88%E6%95%B0%E5%AD%A6_(Fall_2026)/Matching_theory&diff=13833组合数学 (Fall 2026)/Matching theory2026-06-17T04:05:18Z<p>Etone: Created page with "== Systems of Distinct Representatives (SDR)== A '''system of distinct representatives (SDR)''' (also called a '''transversal''') for a sequence of (not necessarily distinct) sets <math>S_1,S_2,\ldots,S_m</math> is a sequence of <font color=red>''distinct''</font> elements <math>x_1,x_2,\ldots,x_m</math> such that <math>x_i\in S_i</math> for all <math>i=1,2,\ldots,m</math>. === Hall's marriage theorem === If the sets <math>S_1,S_2,\ldots,S_m</math> have a system of dist..."</p>
<hr />
<div>== Systems of Distinct Representatives (SDR)==<br />
A '''system of distinct representatives (SDR)''' (also called a '''transversal''') for a sequence of (not necessarily distinct) sets <math>S_1,S_2,\ldots,S_m</math> is a sequence of <font color=red>''distinct''</font> elements <math>x_1,x_2,\ldots,x_m</math> such that <math>x_i\in S_i</math> for all <math>i=1,2,\ldots,m</math>.<br />
<br />
=== Hall's marriage theorem ===<br />
If the sets <math>S_1,S_2,\ldots,S_m</math> have a system of distinct representatives <math>x_1\in S_1,x_2\in S_2,\ldots,x_m\in S_m</math>, then it is obvious that <math>\left|S_1\cup S_2\cup\cdots\cup S_m\right|\ge |\{x_1,x_2,\ldots,x_m\}|=m</math>. Moreover, for any subset <math>I\subseteq\{1,2,\ldots,m\}</math>,<br />
:<math>\left|\bigcup_{i\in I}S_i\right|\ge |\{x_i\mid i\in I\}|=|I|</math><br />
because <math>x_1,x_2,\ldots,x_m</math> are distinct.<br />
<br />
Surprisingly, this obvious necessary condition for the existence of SDR is also sufficient, which is stated by the Hall's theorem, also called the '''mariage theorem'''.<br />
{{Theorem|Hall's Theorem|<br />
:The sets <math>S_1,S_2,\ldots,S_m</math> have a system of distinct representatives (SDR) if and only if<br />
::<math>\left|\bigcup_{i\in I}S_i\right|\ge |I|</math> for all <math>I\subseteq\{1,2,\ldots,m\}</math>.<br />
}}<br />
<br />
The condition that <math>\left|\bigcup_{i\in I}S_i\right|\ge |I|</math> for all <math>I\subseteq\{1,2,\ldots,m\}</math> is also called the '''Hall's condition'''.<br />
<br />
{{Proof|<br />
We only need to prove the sufficiency of Hall's condition for the existence of SDR. We do it by induction on <math>m</math>. When <math>m=1</math>, the theorem trivially holds. Now assume the theorem hold for any integer smaller than <math>m</math>. <br />
<br />
A subcollection of sets <math>\{S_i\mid i\in I\}</math>, <math>|I|<m\,</math>, is called a '''critical family''' if <math>\left|\bigcup_{i\in I}S_i\right|=|I|</math>.<br />
<br />
'''Case.1:''' There is no critical family, i.e. for each <math>I\subset\{1,2,\ldots,m\}</math>, <math>\left|\bigcup_{i\in I}S_i\right|>|I|</math>.<br />
<br />
Take <math>S_m</math> and choose an arbitrary <math>x\in S_m</math> as its representative. Remove <math>x</math> from all other sets by letting <math>S'_i=S_i\setminus \{x\}</math> for <math>1\le i\le m-1</math>. Then for all <math>I\subseteq\{1,2,\ldots,m-1\}</math>, <br />
:<math>\left|\bigcup_{i\in I}S_i'\right|\ge \left|\bigcup_{i\in I}S_i\right|-1\ge |I|</math>.<br />
Due to the induction hypothesis, <math>S_1,\ldots,S_m</math> have an SDR, say <math>x_1\in S_1,\ldots,x_{m-1}\in S_{m-1}</math>. It is obvious that none of them equals <math>x</math> because <math>x</math> is removed. Thus, <math>x_1,\ldots,x_{m-1}</math> and <math>x\,</math> form an SDR for <math>S_1,S_2,\ldots,S_m</math>.<br />
<br />
'''Case.2:''' There is a critical family, i.e. <math>\exists I\subset\{1,2,\ldots,m\}, |I|<m</math>, such that <math>\left|\bigcup_{i\in I}S_i\right|=|I|</math>.<br />
<br />
Suppose <math>S_{m-k+1},\ldots, S_m</math> are such a collection of <math>k<m</math> sets. Hall's condition certainly holds for these sets. Since <math>k<m</math>, due to the induction hypothesis, there is an SDR for the sets, say <math>x_{m-k+1}\in S_{m-k+1},\ldots,x_m\in S_m</math>.<br />
<br />
Again, remove the <math>k</math> elements from the remaining sets by letting <math>S'_i=S_i\setminus\{x_{m-k+1},\ldots,x_m\}</math> for <math>1\le i\le m-k</math>. By the Hall's condition, <br />
for any <math>I\subseteq\{1,2,\ldots,m-k\}</math>, writing that <math>S=\bigcup_{i\in I}S_i\cup\bigcup_{i=m-k+1}^m S_i</math>,<br />
:<math>\left|S\right|\ge |I|+k</math>,<br />
thus <br />
:<math>\left|\bigcup_{i\in I}S_i'\right|\ge |S|-\left|\bigcup_{i=m-k+1}^m S_i\right|\ge |I|</math>.<br />
Due to the induction hypothesis, there is an SDR for <math>S_1,\ldots, S_{m-k}</math>, say <math>x_1\in S_1,\ldots, x_{m-k}\in S_{m-k}</math>. Combining it with the SDR <math>x_{m-k+1},\ldots, x_{m}</math> for <math>S_{m-k+1},\ldots, S_{m}</math>, we have an SDR for <math>S_{1},\ldots, S_{m}</math>.<br />
}}<br />
<br />
Hall's theorem is usually stated as a theorem for the existence of matching in a bipartite graph.<br />
<br />
In a graph <math>G(V,E)</math>, a '''matching''' <math>M\subseteq E</math> is an independent set for edges, that is, for any <math>e_1,e_2\in M</math> that <math>e_1\neq e_2</math>, <math>e_1</math> and <math>e_2</math> are not adjacent to the same vertex.<br />
<br />
In a bipartite graph <math>G(U,V,E)</math>, we say <math>M</math> is a matching of <math>U</math> (or a matching of <math>V</math>), if every vertex in <math>U</math> (or <math>V</math>) is adjacent to some edge in <math>M</math>, i.e., all vertices in <math>U</math> (or <math>V</math>) are matched.<br />
<br />
In a graph <math>G(V,E)</math>, for any vertex <math>v\in V</math>, let <math>N(v)</math> denote the set of neighbors of <math>v</math> in <math>G</math>; and for any vertex set <math>S\subseteq V</math>, we override the notation as <math>N(S)=\bigcup_{v\in S}N(v)</math>, i.e. the set of vertices that are adjacent to one of the vertices in <math>S</math>.<br />
<br />
{{Theorem|Hall's Theorem (graph theory form)|<br />
:A bipartite graph <math>G(U,V,E)</math> has a matching of <math>U</math> if and only if<br />
::<math>\left|N(S)\right|\ge |S|</math> for all <math>S\subseteq U</math>.<br />
}}<br />
<br />
Consider the collection of sets <math>N(u), u\in U</math>. A matching of <math>U</math> is an SDR for these sets. Then clearly the theorem is equivalent to Hall's theorem.<br />
<br />
=== Min-max theorems ===<br />
In combinatorics (and also in other branches of mathematics), there is a family of theorems which relate the minimum of one thing to the maximum of something else. The following are some examples.<br />
*'''König-Egerváry theorem''' (König 1931; Egerváry 1931): in a bipartite graph, the maximum number of edges in a matching equals the minimum number of vertices in a vertex cover.<br />
*'''Menger's theorem''' (Menger 1927): the minimum number of vertices separating two given vertices in a graph equals the maximum number of vertex-disjoint paths between the two vertices.<br />
*'''Dilworth's theorem''' (Dilworth 1950): the minimum number of chains which cover a partially ordered set equals the maximum number of elements in an antichain.<br />
<br />
== König-Egerváry theorem==<br />
A [http://en.wikipedia.org/wiki/Matching_(graph_theory) '''matching'''] in a graph <math>G(V,E)</math> is a set <math>M\subseteq E</math> of edges such that no two edges <math>e_1,e_2\in M</math> shares a vertex. In other words, a matching is just an edge version of independent set.<br />
<br />
A [http://en.wikipedia.org/wiki/Vertex_cover '''vertex cover'''] in a graph <math>G(V,E)</math> is a vertex set <math>C\subseteq V</math> such that every edge <math>e\in E</math> is adjacent to some <math>u\in C</math>, that is, all edges in the graph are "covered" by some vertex in the vertex cover <math>C</math>.<br />
<br />
The '''König-Egerváry theorem''' (also called the '''König's theorem''') states the equality of the sizes of maximum matching and minimum vertex cover.<br />
<br />
{{Theorem|König-Egerváry Theorem (graph theory form)|<br />
:In any bipartite graph, the size of a ''maximum'' matching equals the size of a ''minimum'' vertex cover.<br />
}}<br />
<br />
The König-Egerváry theorem can be reformulated in its matrix form. A bipartite graph <math>G(U,V,E)</math> can be represented as a <math>|U|\times |V|</math> matrix <math>A</math> with 0-1 entries. For any <math>u\in U</math> and <math>v\in V</math>, <math>A(u,v)=1</math> if and only if <math>uv\in E</math>. (Note that this definition is different from adjacency matrix for graphs.)<br />
<br />
Then, a matching in <math>G(U,V,E)</math> corresponds to a set of '''independent 1's''' in <math>A</math>: a set of 1's that do not share rows or columns. A vertex cover corresponds to a set of rows and columns '''covering''' all 1's in <math>A</math>: a set of rows and columns that every 1-entry in <math>A</math> belongs to at least one of these rows or columns.<br />
<br />
It is easy to see the König-Egerváry theorem for bipartite graphs can be equivalently described as follows:<br />
<br />
{{Theorem|König-Egerváry Theorem (matrix form)|<br />
:Let <math>A</math> be an <math>m\times n</math> 0-1 matrix. The ''maximum'' number of independent 1's is equal to the ''minimum'' number of rows and columns required to cover all the 1's in <math>A</math>.<br />
}}<br />
<br />
We give a proof by the Hall's theorem.<br />
<br />
{{Proof|<br />
Let <math>r</math> denote the maximum number of independent 1's in <math>A</math> and <math>s</math> be the minimum number of rows and columns to cover all 1's in <math>A</math>. Clearly, <math>r\le s</math>, since any set of <math>r</math> independent 1's requires together <math>r</math> rows and columns to cover. <br />
<br />
We now prove <math>r\ge s</math>. Assume that some <math>a</math> rows and <math>b</math> columns cover all the 1's in <math>A</math>, and <math>s=a+b</math>, i.e. the covering is minimum. Because permuting the rows or the columns change neither <math>r</math> nor <math>s</math> (as reordering the vertices on either side in a bipartite graph changes nothing to the size of matchings and vertex covers), we may assume that the first <math>a</math> rows and the first <math>b</math> columns cover the 1's. Write <math>A</math> in the form<br />
:<math>A=\begin{bmatrix}<br />
B_{a\times b} &C_{a\times (n-b)}\\<br />
D_{(m-a)\times b} &E_{(m-a)\times (n-b)}<br />
\end{bmatrix}</math>,<br />
where the submatrix <math>E</math> has only zero entries. We will show that there are <math>a</math> independent 1's in <math>C</math> and <math>b</math> independent 1's in <math>D</math>, thus together <math>A</math> has <math>a+b=s</math> independent 1's, which will imply that <math>r\ge s</math>, as desired.<br />
<br />
Define<br />
:<math>S_i=\{j\mid c_{ij}=1\}</math><br />
It is obvious that <math>S_i\subseteq\{1,2,\ldots,n-b\}</math>. We claim that the sequence <math>S_1,S_2,\ldots, S_a</math> has a system of distinct representatives, i.e., we can choose a 1 from each row, no two in the same column. Otherwise, Hall's theorem tells us that there exists some <math>I\subseteq\{1,2,\ldots,a\}</math>, such that <math>\left|\bigcup_{i\in I}S_i\right|<|I|</math>, that is, the 1's in the rows contained by <math>I</math> can be covered by less than <math>|I|</math> columns. Thus, the 1's in <math>C</math> can be covered by <math>a-|I|</math> and less than <math>|I|</math> columns, altogether less than <math>a</math> rows and columns. Therefore, the 1's in <math>A</math> can be covered by less than <math>a+b</math> rows and columns, which contradicts the assumption that the size of the minimum covering of all 1's in <math>A</math> is <math>a+b</math>. Therefore, we show that <math>C</math> has <math>a</math> independent 1's.<br />
<br />
By the same argument, we can show that <math>D</math> has <math>b</math> independent 1's. Since <math>C</math> and <math>D</math> share no rows or columns, the number of independent 1's in <math>A</math> is <math>r\ge a+b=s</math>.<br />
}}<br />
<br />
== Dilworth's theorem ==<br />
Recall that a [http://en.wikipedia.org/wiki/Partially_ordered_set '''partially ordered set'''] (or '''poset''') consists of a set <math>P</math> and a binary relation <math>\le</math> defined on <math>P</math>, satisfying<br />
*'''reflexivity''': <math>x\le x</math>;<br />
*'''antisymmetry''': <math>x\le y</math> and <math>y\le x</math> only if <math>x=y</math>;<br />
*'''transitivity''': if <math>x\le y</math> and <math>y\le z</math>, then <math>x\le z</math>.<br />
<br />
Two elements <math>x,y\in P</math> are said to be '''comparable''' if <math>x\le y</math> or <math>y\le x</math>; and <math>x,y\in P</math> are '''incomparable''' if otherwise.<br />
<br />
A poset <math>P</math> is a '''totally ordered set''', or a '''chain''', if all pairs of elements in <math>P</math> are comparable. A poset <math>P</math> is an '''antichain''' if all pairs of elements in <math>P</math> are incomparable.<br />
<br />
Given a poset <math>P</math>, we can partition it into chains. What is the minimum number of chains that we can break <math>P</math> into? Dilworth's theorem tells us that it is equal to the size of the maximum antichain.<br />
<br />
{{Theorem|Dilworth's Theorem|<br />
:Suppose that the largest antichain in the poset <math>P</math> has size <math>r</math>. Then <math>P</math> can be partitioned into <math>r</math> disjoint chains.<br />
}}<br />
<br />
{{Proof|<br />
Suppose <math>P</math> has an antichain <math>|A|</math>, and <math>P</math> can be partitioned into disjoint chains <math>C_1,C_2,\ldots,C_s</math>. Then <math>|A|\le s</math>, since every chain can pass though an antichain at most once, that is, <math>|C_i\cap A|\le 1</math> for all <math>i=1,2,\ldots,s</math>.<br />
<br />
Therefore, we only need to prove that there exist an antichain <math>A\subseteq P</math> of size <math>r</math>, and a partition of <math>P</math> into at most <math>r</math> chains.<br />
<br />
Define a bipartite graph <math>G(U,V,E)</math> where <math>U=V=P</math>, and for any <math>u\in U</math> and <math>v\in v</math>, <math>uv\in E</math> if and only if <math>u<v</math> in the poset <math>P</math>. By König-Egerváry theorem, there is a matching <math>M\subseteq E</math> and a vertex set <math>C</math> such that every edge in <math>E</math> is adjacent to at least a vertex in <math>C</math>, and <math>|M|=|C|</math>.<br />
<br />
Denote <math>|M|=|C|=m</math>. Let <math>A</math> be the set of '''uncovered''' elements in poset <math>P</math>, i.e., the elements of <math>P</math> that do not correspond to any vertex in <math>C</math>. Clearly, <math>|A|\ge n-m</math>. We claim that <math>A\subseteq P</math> is an antichain. By contradiction, assume there exists <math>x,y\in A</math> such that <math>x<y</math>. Then, by the definition of <math>G</math>, there exist <math>u_x\in U,v_x\in V</math> which corresponds to <math>x</math>, and <math>u_y\in U,v_y\in V</math> which corresponds to <math>y</math>, such that <math>u_xv_y\in E</math>. But since <math>A</math> includes only those elements whose corresponding vertices are not in <math>C</math>, none of <math>u_x,v_x,u_y,v_y</math> can be in <math>C</math>, which contradicts the fact that <math>C</math> is a vertex cover of <math>G</math> that every edges in <math>G</math> are adjacent to at least a vertex in <math>C</math>.<br />
<br />
Let <math>B</math> be a family of chains formed by including <math>u</math> and <math>v</math> in the same chain whenever <math>uv\in M</math>. A moment thought would tell us that the number of chains in <math>B</math> is equal to the '''unmatched''' vertices in <math>U</math> (or <math>V</math>). Thus, <math>|B|=n-m</math>.<br />
<br />
Altogether, we construct an antichain of size <math>|A|\ge n-m</math> and partition the poset <math>P</math> into <math>|B|=n-m</math> disjoint chains. The theorem is proved.<br />
}}<br />
<br />
=== Application: Erdős-Szekeres Theorem ===<br />
Let <math>(a_1,a_2,\ldots,a_n)</math> be a sequence of <math>n</math> distinct real numbers.A '''subsequence''' of <math>(a_1,a_2,\ldots,a_n)</math> is an <math>(a_{i_1},a_{i_2},\ldots,a_{i_k})</math>, with <math>i_1<i_2<\cdots<i_k</math>.<br />
<br />
A sequence <math>(a_1,a_2,\ldots,a_n)</math> is '''increasing''' if <math>a_1<a_2<\cdots<a_n</math>, and '''decreasing''' if <math>a_1>a_2>\cdots>a_n</math>.<br />
<br />
Recall that the Erdős-Szekeres theorem states the existence of long increasing subsequence or decreasing subsequence. Last time we prove this by the pigeonhole principle. Now we use the Dilworth's theorem to prove it, which is also the original proof due to Erdős-Szekeres.<br />
{{Theorem|Erdős-Szekeres Theorem|<br />
:A sequence of more than <math>mn</math> different real numbers must contain either an increasing subsequence of length <math>m+1</math>, or a decreasing subsequence of length <math>n+1</math>.<br />
}}<br />
<br />
{{Prooftitle|Proof by Dilworth's theorem|(Original proof of Erdős-Szekeres)<br />
Let <math>(a_1,a_2,\ldots,a_N)</math> be the sequence of <math>N>mn</math> distinct real numbers. Define the poset as<br />
:<math>P=\{(i,a_i)\mid i=1,2,\ldots,N\}</math><br />
and <math>(i,a_i)\le (j,a_j)</math> if and only if <math>i\le j</math> and <math>a_i\le a_j</math>.<br />
<br />
A chain <math>(i_1,a_{i_1})<(i_2,a_{i_2})<\cdots<(i_k,a_{i_k})</math> must have <math>i_1<i_2<\cdots<i_k</math> and <math>a_{i_1}<a_{i_2}<\cdots<a_{i_k}</math>. Thus, each chain correspond to an increasing subsequence.<br />
<br />
Let <math>(j_1,a_{j_1}),(j_2,a_{j_2}),\cdots,(j_k,a_{j_k})</math> be an antichain. Without loss of generality, we can assume that <math>j_1<j_2<\cdots<j_k</math>. The only case that these elements are non-comparable is that <math>a_{j_1}>a_{j_2}>\cdots>a_{j_k}</math>, otherwise if <math>a_{j_s}< a_{j_t}</math> for some <math>s<t</math>, then <math>(j_s,a_{j_s})<(j_t,a_{j_t})</math>, which contradicts the fact that it is an antichain. Thus, each antichain corresponds to a decreasing subsequence.<br />
<br />
If <math>P</math> has an antichain of size <math>n+1</math>, then <math>(a_1,a_2,\ldots,a_N)</math> has a decreasing subsequence of size <math>n+1</math>, and we are done.<br />
<br />
Alternatively, if the largest antichain in <math>P</math> is of size at most <math>n</math>, then by Dilworth's theorem, <math>P</math> can be partitioned into no more than <math>n</math> disjoint chains, due to pigeonhole principle, one of which must be of length <math>m+1</math>, which means <math>(a_1,a_2,\ldots,a_N)</math> has an increasing subsequence of size <math>m+1</math>.<br />
}}<br />
<br />
=== Application: Hall's Theorem ===<br />
To recognize the power of Dilworth's theorem, we show that it contains Hall's theorem as a special case!<br />
{{Theorem|Hall's Theorem |<br />
:The sets <math>S_1,S_2,\ldots,S_m</math> have a system of distinct representatives (SDR) if and only if<br />
::<math>\left|\bigcup_{i\in I}S_i\right|\ge |I|</math> for all <math>I\subseteq\{1,2,\ldots,m\}</math>.<br />
}}<br />
<br />
{{Prooftitle|Proof by Dilworth's theorem|<br />
As we discussed before, the necessity of Hall's condition for the existence of SDR is easy. We prove its sufficiency by Dilworth's theorem.<br />
<br />
Denote <math>X=\bigcup_{i=1}^mS_i</math>. Construct a poset <math>P</math> by letting <math>P=X\cup\{S_1,S_2,\ldots,S_m\}</math> and <math>x<S_{i}</math> for all <math>x\in S_{i}</math>. There are no other comparabilities.<br />
<br />
It is obvious that <math>X</math> is an antichain. We claim it is also the largest one. To prove this, let <math>A</math> be an arbitrary antichain, and let <math>I=\{i\mid S_i\in A\}</math>. Then <math>A</math> contains no elements of <math>\bigcup_{i\in I}S_i</math>, since if <math>x\in A</math> and <math>x\in S_i\in A</math>, then <math>x<S_i</math> and <math>A</math> cannot be an antichain. Thus,<br />
:<math>|A|\le |I|+|X|-\left|\bigcup_{i\in I}S_i\right|</math><br />
and by Hall's condition <math>\left|\bigcup_{i\in I}S_i\right|\ge|I|</math>, thus <math>|A|\le |X|</math>, as claimed.<br />
<br />
Now, Dilworth's theorem implies that <math>P</math> can be partitioned into <math>|X|</math> chains. Since <math>X</math> is an antichain and each chain can pass though an antichain on at most one element, each of the <math>|X|</math> chains contain precisely one <math>x\in X</math>. And since <math>\{S_1,\ldots,S_m\}</math> is also an antichain, each of these <math>|X|</math> chains contain at most one <math>S_i</math>. Altogether, the <math>|X|</math> chains are in the form:<br />
:<math>\{x_1,S_1\},\{x_2,S_2\},\ldots,\{x_m,S_m\},\{x_{m+1}\}\ldots,\{x_{|X|}\}</math>.<br />
Since the only comparabilities in our posets are <math>x\in S_i</math> and the above chains are disjoint, we have <math>x_1\in S_1, x_2\in S_2,\ldots,x_m\in S_m</math> as an SDR.<br />
}}<br />
<br />
== Flow and Cut==<br />
<br />
=== Flows ===<br />
An instance of the maximum flow problem consists of:<br />
* a directed graph <math>G(V,E)</math>;<br />
* two distinguished vertices <math>s</math> (the '''source''') and <math>t</math> (the '''sink'''), where the in-degree of <math>s</math> and the out-degree of <math>t</math> are both 0;<br />
* the '''capacity function''' <math>c:E\rightarrow\mathbb{R}^+</math> which associates each directed edge <math>(u,v)\in E</math> a nonnegative real number <math>c_{uv}</math> called the '''capacity''' of the edge.<br />
<br />
The quadruple <math>(G,c,s,t)</math> is called a '''flow network'''.<br />
<br />
A function <math>f:E\rightarrow\mathbb{R}^+</math> is called a '''flow''' (to be specific an '''<math>s</math>-<math>t</math> flow''') in the network <math>G(V,E)</math> if it satisfies:<br />
* '''Capacity constraint:''' <math>f_{uv}\le c_{uv}</math> for all <math>(u,v)\in E</math>.<br />
* '''Conservation constraint:''' <math>\sum_{u:(u,v)\in E}f_{uv}=\sum_{w:(v,w)\in E}f_{vw}</math> for all <math>v\in V\setminus\{s,t\}</math>.<br />
<br />
The '''value''' of the flow <math>f</math> is <math>\sum_{v:(s,v)\in E}f_{sv}</math>. <br />
<br />
Given a flow network, the maximum flow problem asks to find the flow of the maximum value.<br />
<br />
The maximum flow problem can be described as the following linear program.<br />
:<math>\text{maximize} \quad \sum_{v:(s,v)\in E}f_{sv}</math><br />
:<math><br />
\begin{align}<br />
\text{s.t.} <br />
&&f_{uv} &\le c_{uv} &\quad& \forall (u,v)\in E\\<br />
&&\sum_{u:(u,v)\in E}f_{uv}-\sum_{w:(v,w)\in E}f_{vw} &=0 &\quad& \forall v\in V\setminus\{s,t\}\\<br />
&&f_{uv} &\ge 0 &\quad& \forall (u,v)\in E<br />
\end{align}<br />
</math><br />
<br />
=== Cuts ===<br />
{{Theorem|Definition|<br />
:Let <math>(G(V,E),c,s,t)</math> be a flow network. Let <math>S\subset V</math>. We call <math>(S,\bar{S})</math> an '''<math>s</math>-<math>t</math> cut''' if <math>s\in S</math> and <math>t\not\in S</math>.<br />
:The '''value''' of the cut (also called the '''capacity''' of the cut) is defined as <math>\sum_{u\in S,v\not\in S\atop (u,v)\in E}c_{uv}</math>.<br />
}}<br />
<br />
A fundamental fact in flow theory is that cuts always upper bound flows.<br />
<br />
{{Theorem|Lemma|<br />
:Let <math>(G(V,E),c,s,t)</math> be a flow network. Let <math>f</math> be an arbitrary flow in <math>G</math>, and let <math>(S,\bar{S})</math> be an arbitrary <math>s</math>-<math>t</math> cut. Then<br />
::<math>\sum_{v:(s,v)}f_{sv}\le \sum_{u\in S,v\not\in S\atop (u,v)\in E}c_{uv}</math>,<br />
:that is, the value of any flow is no greater than the value of any cut.<br />
}}<br />
{{Proof|By the definition of <math>s</math>-<math>t</math> cut, <math>s\in S</math> and <math>t\not\in S</math>. <br />
<br />
Due to the conservation of flow, <br />
:<math>\sum_{u\in S}\left(\sum_{v:(u,v)\in E}f_{uv}-\sum_{v:(v,u)\in E}f_{vu}\right)=\sum_{v:(s,v)\in E}f_{sv}+\sum_{u\in S\setminus\{s\}}\left(\sum_{v:(u,v)\in E}f_{uv}-\sum_{v:(v,u)\in E}f_{vu}\right)=\sum_{v:(s,v)\in E}f_{sv}\,.</math><br />
On the other hand, summing flow over edges,<br />
:<math>\sum_{v\in S}\left(\sum_{u:(u,v)\in E}f_{uv}-\sum_{u:(v,u)\in E}f_{vu}\right)=\sum_{u\in S,v\in S\atop (u,v)\in E}\left(f_{uv}-f_{uv}\right)+\sum_{u\in S,v\not\in S\atop (u,v)\in E}f_{uv}-\sum_{u\in S,v\not\in S\atop (v,u)\in E}f_{vu}=\sum_{u\in S,v\not\in S\atop (u,v)\in E}f_{uv}-\sum_{u\in S,v\not\in S\atop (v,u)\in E}f_{vu}\,.</math><br />
Therefore,<br />
:<math>\sum_{v:(s,v)\in E}f_{sv}=\sum_{u\in S,v\not\in S\atop (u,v)\in E}f_{uv}-\sum_{u\in S,v\not\in S\atop (v,u)\in E}f_{vu}\le\sum_{u\in S,v\not\in S\atop (u,v)\in E}f_{uv}\le \sum_{u\in S,v\not\in S\atop (u,v)\in E}c_{uv}\,,</math><br />
}}<br />
<br />
=== Augmenting paths ===<br />
{{Theorem|Definition (Augmenting path)|<br />
:Let <math>f</math> be a flow in <math>G</math>. An '''augmenting path to <math>u_k</math>''' is a sequence of distinct vertices <math>P=(u_0,u_1,\cdots, u_k)</math>, such that <br />
:* <math>u_0=s\,</math>;<br />
:and each pair of consecutive vertices <math>u_{i}u_{i+1}\,</math> in <math>P</math> corresponds to either a '''forward edge''' <math>(u_{i},u_{i+1})\in E</math> or a '''reverse edge''' <math>(u_{i+1},u_{i})\in E</math>, and <br />
:* <math>f(u_i,u_{i+1})<c(u_i,u_{i+1})\,</math> when <math>u_{i}u_{i+1}\,</math> corresponds to a forward edge <math>(u_{i},u_{i+1})\in E</math>, and <br />
:* <math>f(u_{i+1},u_i)>0\,</math> when <math>u_{i}u_{i+1}\,</math> corresponds to a reverse edge <math>(u_{i+1},u_{i})\in E</math>.<br />
:If <math>u_k=t\,</math>, we simply call <math>P</math> an '''augmenting path'''.<br />
}}<br />
<br />
Let <math>f</math> be a flow in <math>G</math>. Suppose there is an augmenting path <math>P=u_0u_1\cdots u_k</math>, where <math>u_0=s</math> and <math>u_k=t</math>. Let <math>\epsilon>0</math> be a positive constant satisfying <br />
*<math>\epsilon \le c(u_{i},u_{i+1})-f(u_i,u_{i+1})</math> for all forward edges <math>(u_{i},u_{i+1})\in E</math> in <math>P</math>;<br />
*<math>\epsilon \le f(u_{i+1},u_i)</math> for all reverse edges <math>(u_{i+1},u_i)\in E</math> in <math>P</math>.<br />
Due to the definition of augmenting path, we can always find such a positive <math>\epsilon</math>. <br />
<br />
Increase <math>f(u_i,u_{i+1})</math> by <math>\epsilon</math> for all forward edges <math>(u_{i},u_{i+1})\in E</math> in <math>P</math> and decrease <math>f(u_{i+1},u_i)</math> by <math>\epsilon</math> for all reverse edges <math>(u_{i+1},u_i)\in E</math> in <math>P</math>. Denote the modified flow by <math>f'</math>. It can be verified that <math>f'</math> satisfies the capacity constraint and conservation constraint thus is still a valid flow. On the other hand, the value of the new flow <math>f'</math><br />
:<math>\sum_{v:(s,v)\in E}f_{sv}'=\epsilon+\sum_{v:(s,v)\in E}f_{sv}>\sum_{v:(s,v)\in E}f_{sv}</math>.<br />
Therefore, the value of the flow can be "augmented" by adjusting the flow on the augmenting path. This immediately implies that if a flow is maximum, then there is no augmenting path. Surprisingly, the converse is also true, thus maximum flows are "characterized" by augmenting paths.<br />
<br />
{{Theorem|Lemma|<br />
:A flow <math>f</math> is maximum if and only if there are no augmenting paths.<br />
}}<br />
{{Proof|We have already proved the "only if" direction above. Now we prove the "if" direction.<br />
<br />
Let <math>S=\{u\in V\mid \exists\text{an augmenting path to }u\}</math>. Clearly <math>s\in S</math>, and since there is no augmenting path <math>t\not\in S</math>. Therefore, <math>(S,\bar{S})</math> defines an <math>s</math>-<math>t</math> cut. <br />
<br />
We claim that<br />
:<math>\sum_{v:(s,v)}f_{sv}= \sum_{u\in S,v\not\in S\atop (u,v)\in E}c_{uv}</math>,<br />
that is, the value of flow <math>f</math> approach the value of the cut <math>(S,\bar{S})</math> defined above. By the above lemma, this will imply that the current flow <math>f</math> is maximum.<br />
<br />
To prove this claim, we first observe that<br />
:<math>\sum_{v:(s,v)}f_{sv}= \sum_{u\in S,v\not\in S\atop (u,v)\in E}f_{uv}-\sum_{u\in S,v\not\in S\atop (v,u)\in E}f_{vu}</math>.<br />
This identity is implied by the flow conservation constraint, and holds for any <math>s</math>-<math>t</math> cut <math>(S,\bar{S})</math>.<br />
<br />
We then claim that <br />
*<math>f_{uv}=c_{uv}</math> for all <math>u\in S,v\not\in S, (u,v)\in E</math>; and <br />
*<math>f_{vu}=0</math> for all <math>u\in S,v\not\in S, (v,u)\in E</math>.<br />
If otherwise, then the augmenting path to <math>u</math> apending <math>uv</math> becomes a new augmenting path to <math>v</math>, which contradicts that <math>S</math> includes all vertices to which there exist augmenting paths.<br />
<br />
Therefore,<br />
:<math>\sum_{v:(s,v)}f_{sv}= \sum_{u\in S,v\not\in S\atop (u,v)\in E}f_{uv}-\sum_{u\in S,v\not\in S\atop (v,u)\in E}f_{vu} = \sum_{u\in S,v\not\in S\atop (u,v)\in E}c_{uv}</math>.<br />
As discussed above, this proves the theorem.<br />
}}<br />
<br />
== Max-Flow Min-Cut ==<br />
<br />
=== The max-flow min-cut theorem ===<br />
{{Theorem|Max-Flow Min-Cut Theorem|<br />
:In a flow network, the maximum value of any <math>s</math>-<math>t</math> flow equals the minimum value of any <math>s</math>-<math>t</math> cut.<br />
}}<br />
<br />
{{Proof|<br />
Let <math>f</math> be a flow with maximum value, so there is no augmenting path.<br />
<br />
Again, let <br />
:<math>S=\{u\in V\mid \exists\text{an augmenting path to }u\}</math>. <br />
As proved above, <math>(S,\bar{S})</math> forms an <math>s</math>-<math>t</math> cut, and<br />
:<math>\sum_{v:(s,v)}f_{sv}= \sum_{u\in S,v\not\in S\atop (u,v)\in E}c_{uv}</math>,<br />
that is, the value of flow <math>f</math> equals the value of cut <math>(S,\bar{S})</math>. <br />
<br />
Since we know that all <math>s</math>-<math>t</math> flows are not greater than any <math>s</math>-<math>t</math> cut, the value of flow <math>f</math> equals the minimum value of any <math>s</math>-<math>t</math> cut.<br />
}}<br />
<br />
=== Flow Integrality Theorem ===<br />
{{Theorem|Flow Integrality Theorem|<br />
:Let <math>(G,c,s,t)</math> be a flow network with integral capacity <math>c</math>. There exists an integral flow which is maximum.<br />
}}<br />
{{Proof|<br />
Let <math>f</math> be an integral flow of maximum value. If there is an augmenting path, since both <math>c</math> and <math>f</math> are integral, a new flow can be constructed of value 1+the value of <math>f</math>, contradicting that <math>f</math> is maximum over all integral flows. Therefore, there is no augmenting path, which means that <math>f</math> is maximum over all flows, integral or not.<br />
}}<br />
<br />
=== Applications: Menger's theorem ===<br />
Given an undirected graph <math>G(V,E)</math> and two distinct vertices <math>s,t\in V</math>, a set of edges <math>C\subseteq E</math> is called an '''<math>s</math>-<math>t</math> cut''', if deleting edges in <math>C</math> disconnects <math>s</math> and <math>t</math>.<br />
<br />
A simple path from <math>s</math> to <math>t</math> is called an '''<math>s</math>-<math>t</math> path'''. Two paths are said to be '''edge-disjoint''' if they do not share any edge.<br />
<br />
{{Theorem|Theorem (Menger 1927)|<br />
:Let <math>G(V,E)</math> be an arbitrary undirected graph and <math>s,t\in V</math> be two distinct vertices. The minimum size of any <math>s</math>-<math>t</math> cut equals the maximum number of edge-disjoint <math>s</math>-<math>t</math> paths.<br />
}}<br />
{{proof|<br />
Construct a directed graph <math>G'(V,E')</math> from <math>G(V,E)</math> as follows: replace every undirected edge <math>uv\in E</math> that <math>s,t\not\in\{u,v\}</math> by two directed edges <math>(u,v)</math> and <math>(v,u)</math>; replace every undirected edge <math>sv\in E</math> by <math>(s,v)</math>, and very undirected edge <math>vt\in E</math> by <math>(v,t)</math>. Then assign every directed edge with capacity 1.<br />
<br />
It is easy to verify that in the flow network constructed as above, the followings hold:<br />
*An integral <math>s</math>-<math>t</math> flow corresponds to a set of <math>s</math>-<math>t</math> paths in the undirected graph <math>G</math>, where the value of the flow is the number of <math>s</math>-<math>t</math> paths.<br />
*An <math>s</math>-<math>t</math> cut in the flow network corresponds to an <math>s</math>-<math>t</math> cut in the undirected graph <math>G</math> with the same value.<br />
The Menger's theorem follows as a direct consequence of the max-flow min-cut theorem.<br />
}}<br />
<br />
=== Applications: König-Egerváry theorem ===<br />
Let <math>G(V,E)</math> be a graph. An edge set <math>M\subseteq E</math> is called a '''matching''' if no edge in <math>M</math> shares any vertex. A vertex set <math>C\subseteq V</math> is called a '''vertex cover''' if for any edge <math>uv\in E</math>, either <math>u\in C</math> or <math>v\in C</math>.<br />
<br />
{{Theorem|Theorem (König 1936)|<br />
:In any bipartite graph <math>G(V_1,V_2,E)</math>, the size of a ''maximum'' matching equals the size of a ''minimum'' vertex cover.<br />
}}<br />
We now show how a reduction of bipartite matchings to flows. <br />
<br />
Construct a flow network <math>(G'(V,E'),c,s,t)</math> as follows:<br />
* <math>V=V_1\cup V_2\cup\{s,t\}</math> where <math>s</math> and <math>t</math> are two new vertices.<br />
* For ever <math>uv\in E</math>, add <math>(u,v)</math> to <math>E'</math>; for every <math>u\in V_1</math>, add <math>(s,u)</math> to <math>E'</math>; and for every <math>v\in V_2</math>, add <math>(v,t)</math> to <math>E'</math>.<br />
* Let <math>c_{su}=1</math> for every <math>u\in V_1</math> and <math>c_{vt}=1</math> for every <math>v\in V_2</math>. Let <math>c_{uv}=\infty</math> for every bipartite edges <math>(u,v)</math>.<br />
<br />
{{Theorem|Lemma|<br />
:The size of a maximum matching in <math>G</math> is equal to the value of a maximum <math>s</math>-<math>t</math> flow in <math>G'</math>.<br />
}}<br />
{{proof|<br />
Given an integral <math>s</math>-<math>t</math> flow <math>f</math> in <math>G'</math>, let <math>M=\{uv\in E\mid f_{uv}=1\}</math>. Then <math>M</math> must be a matching since for every <math>u\in V_1</math>. To see this, observe that there is at most one <math>v\in V_2</math> that <math>f_{uv}=1</math>, because of that <math>f_{su}\le c_{su}=1</math> and conservation of flows. The same holds for vertices in <math>V_2</math> by the same argument. Therefore, each flow corresponds to a matching.<br />
<br />
Given a matching <math>M</math> in bipartite graph <math>G</math>, define an integral flow <math>f</math> as such: for <math>uv\in E</math>, <math>f_{uv}=1</math> if <math>uv\in M</math> and <math>f_{uv}=0</math> if otherwise; for <math>u\in V_1</math>, <math>f_{su}=1</math> if <math>uv\in M</math> for some <math>v</math> and <math>f_{su}=0</math> if otherwise; for <math>v\in V_2</math>, <math>f_{vt}=1</math> if <math>uv\in M</math> for some <math>u</math> and <math>f_{vt}=0</math> if otherwise.<br />
<br />
It is easy to check that <math>f</math> is valid <math>s</math>-<math>t</math> flow in <math>G'</math>. Therefore, there is an one-one correspondence between flows in <math>G'</math> and matchings in <math>G</math>. The lemma follows naturally.<br />
}}<br />
<br />
We then establish a correspondence between <math>s</math>-<math>t</math> cuts in <math>G'</math> and vertex covers in <math>G</math>.<br />
<br />
Suppose <math>(S,\bar{S})</math> is an <math>s</math>-<math>t</math> cut in <math>G'</math>.<br />
{{Theorem|Lemma|<br />
:The size of a minimum vertex cover in <math>G</math> is equal to the value of a minimum <math>s</math>-<math>t</math> cut in <math>G'</math>.<br />
}}<br />
{{proof|<br />
Let <math>(S,\bar{S})</math> be an <math>s</math>-<math>t</math> cut of minimum capacity in <math>G'</math>. Then <math>\sum_{u\in S, v\not\in S\atop (u,v)\in E'}c_{uv}</math> must be finite since <math>S=\{s\}</math> gives us an <math>s</math>-<math>t</math> cut whose capacity is <math>|V_1|</math> which is finite. Therefore, no edge <math>uv\in E</math> has <math>u\in V_1\cap S</math> and <math>v\in V_2\setminus S</math>, i.e., for all <math>uv\in E</math>, either <math>u\in V_1\setminus S</math> or <math>v\in V_2\cap S</math>. Therefore, <math>(V_1\setminus S)\cup(V_2\cap S)</math> is a vertex cover in <math>G</math>, whose size is<br />
:<math>|(V_1\setminus S)\cup(V_2\cap S)|=|V_1\setminus S|+|V_2\cap S|=\sum_{u\in V_1\setminus S}c_{su}+\sum_{v\in V_2\cap S}c_{ut}=\sum_{u\in S,v\not\in S\atop (u,v)\in E'}c_{uv}</math>.<br />
The last term is the capacity of the minimum <math>s</math>-<math>t</math> cut <math>(S,\bar{S})</math>.<br />
}}<br />
<br />
The König-Egerváry theorem then holds as a consequence of the max-flow min-cut theorem.</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=%E7%BB%84%E5%90%88%E6%95%B0%E5%AD%A6_(Spring_2026)&diff=13832组合数学 (Spring 2026)2026-06-17T04:04:50Z<p>Etone: /* Lecture Notes */</p>
<hr />
<div>{{Infobox<br />
|name = Infobox<br />
|bodystyle = <br />
|title = <font size=3>组合数学 <br><br />
Combinatorics</font><br />
|titlestyle = <br />
<br />
|image = <br />
|imagestyle = <br />
|caption = <br />
|captionstyle = <br />
|headerstyle = background:#ccf;<br />
|labelstyle = background:#ddf;<br />
|datastyle = <br />
<br />
|header1 =Instructor<br />
|label1 = <br />
|data1 = <br />
|header2 = <br />
|label2 = <br />
|data2 = 尹一通<br />
|header3 = <br />
|label3 = Email<br />
|data3 = yinyt@nju.edu.cn <br />
|header4 =<br />
|label4= office<br />
|data4= 计算机系 804<br />
|header5 = Class<br />
|label5 = <br />
|data5 = <br />
|header6 =<br />
|label6 = Class meetings<br />
|data6 = Wednesday, 2pm-4pm <br> 逸B-313<br />
|header7 =<br />
|label7 = Place<br />
|data7 = <br />
|header8 =<br />
|label8 = Office hours<br />
|data8 = Tuesday, 2-3pm <br>计算机系 804<br />
|header9 = Textbook<br />
|label9 = <br />
|data9 = <br />
|header10 =<br />
|label10 = <br />
|data10 = [[File:LW-combinatorics.jpeg|border|100px]]<br />
|header11 =<br />
|label11 = <br />
|data11 = van Lint and Wilson. <br> ''A course in Combinatorics, 2nd ed.'', <br> Cambridge Univ Press, 2001.<br />
|header12 =<br />
|label12 = <br />
|data12 = [[File:Jukna_book.jpg|border|100px]]<br />
|header13 =<br />
|label13 = <br />
|data13 = Jukna. ''Extremal Combinatorics: <br> With Applications in Computer Science,<br>2nd ed.'', Springer, 2011.<br />
|belowstyle = background:#ddf;<br />
|below = <br />
}}<br />
<br />
This is the webpage for the ''Combinatorics'' class of Spring 2026. Students who take this class should check this page periodically for content updates and new announcements. <br />
<br />
= Announcement =<br />
* '''(2026/03/25)'''<font color=red size=4> 第一次作业已发布</font>，请在 2026/04/08 上课之前提交到 [mailto:njucomb26@163.com njucomb26@163.com] (文件名为'学号_姓名_A1.pdf')<br />
* '''(2026/04/21)'''<font color=red size=4> 第二次作业已发布</font>，请在 2026/05/13 上课之前提交到 [mailto:njucomb26@163.com njucomb26@163.com] (文件名为'学号_姓名_A2.pdf')<br />
* '''(2026/05/22)'''<font color=red size=4> 第三次作业已发布</font>，请在 2026/06/03 上课之前提交到 [mailto:njucomb26@163.com njucomb26@163.com] (文件名为'学号_姓名_A3.pdf')<br />
<br />
= Course info =<br />
* '''Instructor ''': 尹一通 ([http://tcs.nju.edu.cn/yinyt/ homepage])<br />
:*'''email''': yinyt@nju.edu.cn<br />
:*'''office''': 计算机系 804 <br />
* '''Teaching assistant''':<br />
** 丁天行([mailto:652024330006@smail.nju.edu.cn 652024330006@smail.nju.edu.cn])<br />
** 周灿<br />
** 方子伊<br />
* '''Class meeting''': Wednesday, 2pm-4pm, 逸A-313.<br />
* '''Office hour''': TBA<br />
:* '''QQ群''': 1090691552 (加入时需报姓名、专业、学号)<br />
<br />
= Syllabus =<br />
<br />
=== 先修课程 Prerequisites ===<br />
* 离散数学（Discrete Mathematics）<br />
* 线性代数（Linear Algebra）<br />
* 概率论（Probability Theory）<br />
<br />
=== Course materials ===<br />
* [[组合数学 (Spring 2025)/Course materials|<font size=3>教材和参考书清单</font>]]<br />
<br />
=== 成绩 Grades ===<br />
* 课程成绩：本课程将会有若干次作业和一次期末考试。最终成绩将由平时作业成绩 (≥ 60%) 和期末考试成绩 (≤ 40%) 综合得出。<br />
* 迟交：如果有特殊的理由，无法按时完成作业，请提前联系授课老师，给出正当理由。否则迟交的作业将不被接受。<br />
<br />
=== <font color=red> 学术诚信 Academic Integrity </font>===<br />
学术诚信是所有从事学术活动的学生和学者最基本的职业道德底线，本课程将不遗余力的维护学术诚信规范，违反这一底线的行为将不会被容忍。<br />
<br />
作业完成的原则：署你名字的工作必须是你个人的贡献。在完成作业的过程中，允许讨论，前提是讨论的所有参与者均处于同等完成度。但关键想法的执行、以及作业文本的写作必须独立完成，并在作业中致谢（acknowledge）所有参与讨论的人。不允许其他任何形式的合作——尤其是与已经完成作业的同学“讨论”。<br />
<br />
本课程将对剽窃行为采取零容忍的态度。在完成作业过程中，对他人工作（出版物、互联网资料、其他人的作业等）直接的文本抄袭和对关键思想、关键元素的抄袭，按照 [http://www.acm.org/publications/policies/plagiarism_policy ACM Policy on Plagiarism]的解释，都将视为剽窃。剽窃者成绩将被取消。如果发现互相抄袭行为，<font color=red> 抄袭和被抄袭双方的成绩都将被取消</font>。因此请主动防止自己的作业被他人抄袭。<br />
<br />
学术诚信影响学生个人的品行，也关乎整个教育系统的正常运转。为了一点分数而做出学术不端的行为，不仅使自己沦为一个欺骗者，也使他人的诚实努力失去意义。让我们一起努力维护一个诚信的环境。<br />
<br />
= Assignments =<br />
* [[组合数学 (Spring 2026)/Problem Set 1|Problem Set 1]]<br />
* [[组合数学 (Spring 2026)/Problem Set 2|Problem Set 2]]<br />
* [[组合数学 (Spring 2026)/Problem Set 3|Problem Set 3]]<br />
<br />
= Lecture Notes =<br />
# [[组合数学 (Spring 2026)/Basic enumeration|Basic enumeration | 基本计数]] ([http://tcs.nju.edu.cn/slides/comb2026/BasicEnumeration.pdf slides])<br />
# [[组合数学 (Spring 2026)/Generating functions|Generating functions | 生成函数]] ([http://tcs.nju.edu.cn/slides/comb2026/GeneratingFunction.pdf slides])<br />
# [[组合数学 (Fall 2026)/Sieve methods|Sieve methods | 筛法]] ([http://tcs.nju.edu.cn/slides/comb2026/PIE.pdf slides])<br />
# Guest lecture by Prof. Penghui Yao on entropy and counting ([http://tcs.nju.edu.cn/slides/comb2026/entropy.pdf notes]) <br />
# [[组合数学 (Fall 2026)/Cayley's formula|Cayley's formula | Cayley公式]] ([http://tcs.nju.edu.cn/slides/comb2026/Cayley.pdf slides])<br />
# [[组合数学 (Fall 2026)/Existence problems|Existence problems | 存在性问题]]<br />
# [[组合数学 (Fall 2026)/The probabilistic method|The probabilistic method | 概率法]] ([http://tcs.nju.edu.cn/slides/comb2026/ProbMethod.pdf slides])<br />
# [[组合数学 (Fall 2026)/Extremal graph theory|Extremal graph theory | 极值图论]] ([http://tcs.nju.edu.cn/slides/comb2026/ExtremalGraphs.pdf slides])<br />
# [[组合数学 (Fall 2026)/Extremal set theory|Extremal set theory | 极值集合论]]（[http://tcs.nju.edu.cn/slides/comb2026/ExtremalSets.pdf slides]）<br />
#* [https://mathweb.ucsd.edu/~ronspubs/90_03_erdos_ko_rado.pdf Old and new proofs of the Erdős–Ko–Rado theorem] by Frankl and Graham<br />
#* An [http://tcs.nju.edu.cn/slides/comb2026/sunflower-note.pdf LLM-generated lecture note] on Alweiss-Lovet-Wu-Zhang's improvement over the sunflower lemma, with simplified proofs by Rao-Tao<br />
# [[组合数学 (Fall 2026)/Ramsey theory|Ramsey theory | Ramsey理论]]（[http://tcs.nju.edu.cn/slides/comb2026/Ramsey.pdf slides]）<br />
# [[组合数学 (Fall 2026)/Matching theory|Matching theory | 匹配论]]（[http://tcs.nju.edu.cn/slides/comb2026/Matchings.pdf slides]）<br />
<br />
= Resources =<br />
* [http://math.mit.edu/~fox/MAT307.html Combinatorics course] by Jacob Fox<br />
* [https://yufeizhao.com/pm/ Probabilistic Methods in Combinatorics] and [https://yufeizhao.com/gtacbook/ Graph Theory and Additive Combinatorics] by Yufei Zhao<br />
* [https://www.math.uvic.ca/~noelj/combinatoricsLectures.html Combinatorics Lecture Videos online]<br />
* [https://www.math.ucla.edu/~pak/lectures/Math-Videos/comb-videos.htm Collection of Combinatorics Videos]<br />
<br />
= Concepts =<br />
* [http://en.wikipedia.org/wiki/Binomial_coefficient Binomial coefficient]<br />
* [http://en.wikipedia.org/wiki/Twelvefold_way The twelvefold way]<br />
* [http://en.wikipedia.org/wiki/Composition_(number_theory) Composition of a number]<br />
* [http://en.wikipedia.org/wiki/Multiset#Formal_definition Multiset]<br />
* [http://en.wikipedia.org/wiki/Combination#Number_of_combinations_with_repetition Combinations with repetition], [http://en.wikipedia.org/wiki/Multiset#Counting_multisets <math>k</math>-multisets on a set]<br />
* [http://en.wikipedia.org/wiki/Multinomial_theorem#Multinomial_coefficients Multinomial coefficients]<br />
* [http://en.wikipedia.org/wiki/Stirling_numbers_of_the_second_kind Stirling number of the second kind]<br />
* [http://en.wikipedia.org/wiki/Partition_(number_theory) Partition of a number]<br />
** [http://en.wikipedia.org/wiki/Young_tableau Young tableau]<br />
* [http://en.wikipedia.org/wiki/Fibonacci_number Fibonacci number]<br />
* [http://en.wikipedia.org/wiki/Catalan_number Catalan number]<br />
* [http://en.wikipedia.org/wiki/Generating_function Generating function] and [http://en.wikipedia.org/wiki/Formal_power_series formal power series]<br />
* [http://en.wikipedia.org/wiki/Binomial_series Newton's formula]<br />
* [http://en.wikipedia.org/wiki/Inclusion-exclusion_principle The principle of inclusion-exclusion] (and more generally the [http://en.wikipedia.org/wiki/Sieve_theory sieve method])<br />
* [http://en.wikipedia.org/wiki/M%C3%B6bius_inversion_formula Möbius inversion formula]<br />
* [http://en.wikipedia.org/wiki/Derangement Derangement], and [http://en.wikipedia.org/wiki/M%C3%A9nage_problem Problème des ménages]<br />
* [http://en.wikipedia.org/wiki/Ryser%27s_formula#Ryser_formula Ryser's formula]<br />
* [http://en.wikipedia.org/wiki/Euler_totient Euler totient function]<br />
* [https://en.wikipedia.org/wiki/Burnside%27s_lemma Burnside's lemma]<br />
** [https://en.wikipedia.org/wiki/Group_action Group action]<br />
** [https://en.wikipedia.org/wiki/Group_action#Orbits_and_stabilizers Orbits]<br />
* [https://en.wikipedia.org/wiki/P%C3%B3lya_enumeration_theorem Pólya enumeration theorem]<br />
** [https://en.wikipedia.org/wiki/Permutation_group Permutation group]<br />
** [https://en.wikipedia.org/wiki/Cycle_index Cycle index]<br />
* [http://en.wikipedia.org/wiki/Cayley_formula Cayley's formula]<br />
** [http://en.wikipedia.org/wiki/Prüfer_sequence Prüfer code for trees]<br />
** [http://en.wikipedia.org/wiki/Kirchhoff%27s_matrix_tree_theorem Kirchhoff's matrix-tree theorem]<br />
* [http://en.wikipedia.org/wiki/Double_counting_(proof_technique) Double counting] and the [http://en.wikipedia.org/wiki/Handshaking_lemma handshaking lemma]<br />
* [http://en.wikipedia.org/wiki/Sperner's_lemma Sperner's lemma] and [http://en.wikipedia.org/wiki/Brouwer_fixed_point_theorem Brouwer fixed point theorem]<br />
* [http://en.wikipedia.org/wiki/Pigeonhole_principle Pigeonhole principle]<br />
:* [http://en.wikipedia.org/wiki/Erd%C5%91s%E2%80%93Szekeres_theorem Erdős–Szekeres theorem]<br />
:* [http://en.wikipedia.org/wiki/Dirichlet's_approximation_theorem Dirichlet's approximation theorem]<br />
* [http://en.wikipedia.org/wiki/Probabilistic_method The Probabilistic Method]<br />
* [http://en.wikipedia.org/wiki/Lov%C3%A1sz_local_lemma Lovász local lemma]<br />
* [http://en.wikipedia.org/wiki/Erd%C5%91s%E2%80%93R%C3%A9nyi_model Erdős–Rényi model for random graphs]<br />
* [http://en.wikipedia.org/wiki/Extremal_graph_theory Extremal graph theory]<br />
* [http://en.wikipedia.org/wiki/Turan_theorem Turán's theorem], [http://en.wikipedia.org/wiki/Tur%C3%A1n_graph Turán graph]<br />
* Two analytic inequalities: <br />
:*[http://en.wikipedia.org/wiki/Cauchy%E2%80%93Schwarz_inequality Cauchy–Schwarz inequality]<br />
:* the [http://en.wikipedia.org/wiki/Inequality_of_arithmetic_and_geometric_means inequality of arithmetic and geometric means]<br />
* [http://en.wikipedia.org/wiki/Erd%C5%91s%E2%80%93Stone_theorem Erdős–Stone theorem] (fundamental theorem of extremal graph theory)<br />
* [https://en.wikipedia.org/wiki/Sunflower_(mathematics) Sunflower lemma and conjecture]<br />
* [https://en.wikipedia.org/wiki/Erd%C5%91s%E2%80%93Ko%E2%80%93Rado_theorem Erdős–Ko–Rado theorem]<br />
* [https://en.wikipedia.org/wiki/Sperner%27s_theorem Sperner's theorem]<br />
** [https://en.wikipedia.org/wiki/Sperner_family Sperner system] or '''antichain'''<br />
* [https://en.wikipedia.org/wiki/Sauer%E2%80%93Shelah_lemma Sauer–Shelah lemma]<br />
** [https://en.wikipedia.org/wiki/Vapnik%E2%80%93Chervonenkis_dimension Vapnik–Chervonenkis dimension]<br />
* [https://en.wikipedia.org/wiki/Kruskal%E2%80%93Katona_theorem Kruskal–Katona theorem]<br />
* [http://en.wikipedia.org/wiki/Ramsey_theory Ramsey theory]<br />
:*[http://en.wikipedia.org/wiki/Ramsey's_theorem Ramsey's theorem]<br />
:*[http://en.wikipedia.org/wiki/Happy_Ending_problem Happy Ending problem]<br />
:*[https://en.wikipedia.org/wiki/Van_der_Waerden%27s_theorem Van der Waerden's theorem]<br />
:*[https://en.wikipedia.org/wiki/Hales%E2%80%93Jewett_theorem Hales–Jewett theorem]<br />
* [https://en.wikipedia.org/wiki/Hall%27s_marriage_theorem Hall's theorem ] (the marriage theorem)<br />
:* [https://en.wikipedia.org/wiki/Doubly_stochastic_matrix Birkhoff–Von Neumann theorem]<br />
* [http://en.wikipedia.org/wiki/K%C3%B6nig's_theorem_(graph_theory) König-Egerváry theorem]<br />
* [http://en.wikipedia.org/wiki/Dilworth's_theorem Dilworth's theorem]<br />
:* [http://en.wikipedia.org/wiki/Erd%C5%91s%E2%80%93Szekeres_theorem Erdős–Szekeres theorem]<br />
* The [http://en.wikipedia.org/wiki/Max-flow_min-cut_theorem Max-Flow Min-Cut Theorem]<br />
:* [https://en.wikipedia.org/wiki/Menger%27s_theorem Menger's theorem]<br />
:* [http://en.wikipedia.org/wiki/Maximum_flow_problem Maximum flow]<br />
* [https://en.wikipedia.org/wiki/Linear_programming Linear programming]<br />
** [https://en.wikipedia.org/wiki/Dual_linear_program Duality] <br />
** [https://en.wikipedia.org/wiki/Unimodular_matrix Unimodularity]<br />
* [https://en.wikipedia.org/wiki/Matroid Matroid]</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=%E7%BB%84%E5%90%88%E6%95%B0%E5%AD%A6_(Fall_2026)/Ramsey_theory&diff=13754组合数学 (Fall 2026)/Ramsey theory2026-05-20T13:33:09Z<p>Etone: Created page with "== Ramsey's Theorem == === Ramsey's theorem for graph === {{Theorem|Ramsey's Theorem| :Let <math>k,\ell</math> be positive integers. Then there exists an integer <math>R(k,\ell)</math> satisfying: :If <math>n\ge R(k,\ell)</math>, for any coloring of edges of <math>K_n</math> with two colors red and blue, there exists a red <math>K_k</math> or a blue <math>K_\ell</math>. }} {{Proof| We show that <math>R(k,\ell)</math> is finite by induction on <math>k+\ell</math>. For the..."</p>
<hr />
<div>== Ramsey's Theorem ==<br />
=== Ramsey's theorem for graph ===<br />
{{Theorem|Ramsey's Theorem|<br />
:Let <math>k,\ell</math> be positive integers. Then there exists an integer <math>R(k,\ell)</math> satisfying:<br />
:If <math>n\ge R(k,\ell)</math>, for any coloring of edges of <math>K_n</math> with two colors red and blue, there exists a red <math>K_k</math> or a blue <math>K_\ell</math>.<br />
}}<br />
{{Proof|<br />
We show that <math>R(k,\ell)</math> is finite by induction on <math>k+\ell</math>. For the base case, it is easy to verify that<br />
:<math>R(k,1)=R(1,\ell)=1</math>.<br />
For general <math>k</math> and <math>\ell</math>, we will show that <br />
:<math>R(k,\ell)\le R(k,\ell-1)+R(k-1,\ell)</math>.<br />
Suppose we have a two coloring of <math>K_n</math>, where <math>n=R(k,\ell-1)+R(k-1,\ell)</math>. Take an arbitrary vertex <math>v</math>, and split <math>V\setminus\{v\}</math> into two subsets <math>S</math> and <math>T</math>, where<br />
:<math>\begin{align}<br />
S&=\{u\in V\setminus\{v\}\mid uv \text{ is blue }\}\\<br />
T&=\{u\in V\setminus\{v\}\mid uv \text{ is red }\}<br />
\end{align}</math><br />
Since <br />
:<math>|S|+|T|+1=n=R(k,\ell-1)+R(k-1,\ell)</math>,<br />
we have either <math>|S|\ge R(k,\ell-1)</math> or <math>|T|\ge R(k-1,\ell)</math>. By symmetry, suppose <math>|S|\ge R(k,\ell-1)</math>. By induction hypothesis, the complete subgraph defined on <math>S</math> has either a red <math>K_k</math>, in which case we are done; or a blue <math>K_{\ell-1}</math>, in which case the complete subgraph defined on <math>S\cup{v}</math> must have a blue <math>K_\ell</math> since all edges from <math>v</math> to vertices in <math>S</math> are blue.<br />
}}<br />
<br />
{{Theorem|Ramsey's Theorem (graph, multicolor)|<br />
:Let <math>r, k_1,k_2,\ldots,k_r</math> be positive integers. Then there exists an integer <math>R(r;k_1,k_2,\ldots,k_r)</math> satisfying:<br />
:For any <math>r</math>-coloring of a complete graph of <math>n\ge R(r;k_1,k_2,\ldots,k_r)</math> vertices, there exists a monochromatic <math>k_i</math>-clique with the <math>i</math>th color for some <math>i\in\{1,2,\ldots,r\}</math>.<br />
}}<br />
<br />
{{Theorem|Lemma (the "mixing color" trick)|<br />
:<math>R(r;k_1,k_2,\ldots,k_r)\le R(r-1;k_1,k_2,\ldots,k_{r-2},R(2;k_{r-1},k_r))</math><br />
}}<br />
{{Proof|<br />
We transfer the <math>r</math>-coloring to <math>(r-1)</math>-coloring by identifying the <math>(r-1)</math>th and the <math>r</math>th colors. <br />
<br />
If <math>n\ge R(r-1;k_1,k_2,\ldots,k_{r-2},R(2;k_{r-1},k_r))</math>, then for any <math>r</math>-coloring of <math>K_n</math>, there either exist an <math>i\in\{1,2,\ldots,r-2\}</math> and a <math>k_i</math>-clique which is monochromatically colored with the <math>i</math>th color; or exists clique of <math>R(2;k_{r-1},k_r)</math> vertices which is monochromatically colored with the mixed color of the original <math>(r-1)</math>th and <math>r</math>th colors, which again implies that there exists either a <math>k</math>-clique which is monochromatically colored with the original <math>(r-1)</math>th color, or a <math>\ell</math>-clique which is monochromatically colored with the original <math>r</math>th color. This implies the recursion.<br />
}}<br />
<br />
=== Ramsey number ===<br />
The smallest number <math>R(k,\ell)</math> satisfying the condition in the Ramsey theory is called the '''Ramsey number'''. <br />
<br />
Alternatively, we can define <math>R(k,\ell)</math> as the smallest <math>N</math> such that if <math>n\ge N</math>, for any 2-coloring of <math>K_n</math> in red and blue, there is either a red <math>K_k</math> or a blue <math>K_\ell</math>. The Ramsey theorem is stated as:<br />
:"''<math>R(k,\ell)</math> is finite for any positive integers <math>k</math> and <math>\ell</math>.''"<br />
<br />
The core of the inductive proof of the Ramsey theorem is the following recursion<br />
:<math>\begin{align}<br />
R(k,1) &=R(1,\ell)=1\\<br />
R(k,\ell) &\le R(k,\ell-1)+R(k-1,\ell).<br />
\end{align}</math><br />
<br />
From this recursion, we can deduce an upper bound for the Ramsey number.<br />
{{Theorem|Theorem|<br />
:<math>R(k,\ell)\le{k+\ell-2\choose k-1}</math>.<br />
}}<br />
{{Proof|It is easy to verify the bound by induction.<br />
}}<br />
<br />
------<br />
<br />
The following theorem is due to Spencer in 1975, which is the best known lower bound for diagonal Ramsey number.<br />
<br />
{{Theorem|Theorem (Spencer 1975)|<br />
:<math>R(k,k)\ge Ck2^{k/2}</math> for some constant <math>C>0</math>.<br />
}}<br />
<br />
Its proof uses the Lovász local lemma in the probabilistic method.<br />
{{Theorem<br />
|Lovász Local Lemma (symmetric case)|<br />
:Let <math>A_1,A_2,\ldots,A_n</math> be a set of events, and assume that the following hold:<br />
:#for all <math>1\le i\le n</math>, <math>\Pr[A_i]\le p</math>;<br />
:# each event <math>A_i</math> is independent of all but at most <math>d</math> other events, and<br />
:::<math>ep(d+1)\le 1</math>.<br />
:Then<br />
::<math>\Pr\left[\bigwedge_{i=1}^n\overline{A_i}\right]>0</math>.<br />
}}<br />
<br />
We can use the local lemma to prove a lower bound for the diagonal Ramsey number.<br />
{{Proof|<br />
To prove a lower bound <math>R(k,k)>n</math>, it is sufficient to show that there exists a 2-coloring of <math>K_n</math> without a monochromatic <math>K_k</math>. We prove this by the probabilistic method.<br />
<br />
Pick a random 2-coloring of <math>K_n</math> by coloring each edge uniformly and independently with one of the two colors. For any set <math>S</math> of <math>k</math> vertices, let <math>A_S</math> denote the event that <math>S</math> forms a monochromatic <math>K_k</math>. It is easy to see that <math>\Pr[A_s]=2^{1-{k\choose 2}}=p</math>.<br />
<br />
For any <math>k</math>-subset <math>T</math> of vertices, <math>A_S</math> and <math>A_T</math> are dependent if and only if <math>|S\cap T|\ge 2</math>. For each <math>S</math>, the number of <math>T</math> that <math>|S\cap T|\ge 2</math> is at most <math>{k\choose 2}{n\choose k-2}</math>, so the max degree of the dependency graph is <math>d\le{k\choose 2}{n\choose k-2}</math>.<br />
<br />
Take <math>n=Ck2^{k/2}</math> for some appropriate constant <math>C>0</math>.<br />
:<math><br />
\begin{align}<br />
\mathrm{e}p(d+1)<br />
&\le \mathrm{e}2^{1-{k\choose 2}}\left({k\choose 2}{n\choose k-2}+1\right)\\<br />
&\le 2^{3-{k\choose 2}}{k\choose 2}{n\choose k-2}\\<br />
&\le 1<br />
\end{align}<br />
</math><br />
Applying the local lemma, the probability that there is no monochromatic <math>K_k</math> is <br />
:<math>\Pr\left[\bigwedge_{S\in{[n]\choose k}}\overline{A_S}\right]>0</math>.<br />
Therefore, there exists a 2-coloring of <math>K_n</math> which has no monochromatic <math>K_k</math>, which means<br />
:<math>R(k,k)>n=Ck2^{k/2}</math>.<br />
}}<br />
<br />
{{Theorem|Theorem|<br />
:<math>\Omega\left(k2^{k/2}\right)\le R(k,k)\le{2k-2\choose k-1}=O\left(k^{-1/2}4^{k}\right)</math>.<br />
}}<br />
<br />
{| class="wikitable"<br />
! ''<math>k</math>'',''<math>l</math>''<br />
! 1<br />
! 2<br />
! 3<br />
! 4<br />
! 5<br />
! 6<br />
! 7<br />
! 8<br />
! 9<br />
! 10<br />
|-<br />
! 1<br />
| 1<br />
| 1<br />
| 1<br />
| 1<br />
| 1<br />
| 1<br />
| 1<br />
| 1<br />
| 1<br />
| 1<br />
|-<br />
! 2<br />
| 1<br />
| 2<br />
| 3<br />
| 4<br />
| 5<br />
| 6<br />
| 7<br />
| 8<br />
| 9<br />
| 10<br />
|-<br />
! 3<br />
| 1<br />
| 3<br />
| 6<br />
| 9<br />
| 14<br />
| 18<br />
| 23<br />
| 28<br />
| 36<br />
| 40–43<br />
|-<br />
! 4<br />
| 1<br />
| 4<br />
| 9<br />
| 18<br />
| 25<br />
| 35–41<br />
| 49–61<br />
| 56–84<br />
| 73–115<br />
| 92–149<br />
|-<br />
! 5<br />
| 1<br />
| 5<br />
| 14<br />
| 25<br />
| 43–49<br />
| 58–87<br />
| 80–143<br />
| 101–216<br />
| 125–316<br />
| 143–442<br />
|-<br />
! 6<br />
| 1<br />
| 6<br />
| 18<br />
| 35–41<br />
| 58–87<br />
| 102–165<br />
| 113–298<br />
| 127–495<br />
| 169–780<br />
| 179–1171<br />
|-<br />
! 7<br />
| 1<br />
| 7<br />
| 23<br />
| 49–61<br />
| 80–143<br />
| 113–298<br />
| 205–540<br />
| 216–1031<br />
| 233–1713<br />
| 289–2826<br />
|-<br />
! 8<br />
| 1<br />
| 8<br />
| 28<br />
| 56–84<br />
| 101–216<br />
| 127–495<br />
| 216–1031<br />
| 282–1870<br />
| 317–3583<br />
| 317-6090<br />
|-<br />
! 9<br />
| 1<br />
| 9<br />
| 36<br />
| 73–115<br />
| 125–316<br />
| 169–780<br />
| 233–1713<br />
| 317–3583<br />
| 565–6588<br />
| 580–12677<br />
|-<br />
! 10<br />
| 1<br />
| 10<br />
| 40–43<br />
| 92–149<br />
| 143–442<br />
| 179–1171<br />
| 289–2826<br />
| 317-6090<br />
| 580–12677<br />
| 798–23556<br />
|}<br />
<br />
=== Ramsey's theorem for hypergraph ===<br />
{{Theorem|Ramsey's Theorem (hypergraph, multicolor)|<br />
:Let <math>r, t, k_1,k_2,\ldots,k_r</math> be positive integers. Then there exists an integer <math>R_t(r;k_1,k_2,\ldots,k_r)</math> satisfying:<br />
:For any <math>r</math>-coloring of <math>{[n]\choose t}</math> with <math>n\ge R_t(r;k_1,k_2,\ldots,k_r)</math>, there exist an <math>i\in\{1,2,\ldots,r\}</math> and a subset <math>X\subseteq [n]</math> with <math>|X|\ge k_i</math> such that all members of <math>{X\choose t}</math> are colored with the <math>i</math>th color.<br />
}}<br />
<br />
<math>n\rightarrow(k_1,k_2,\ldots,k_r)^t</math><br />
<br />
{{Theorem|Lemma (the "mixing color" trick)|<br />
:<math>R_t(r;k_1,k_2,\ldots,k_r)\le R_t(r-1;k_1,k_2,\ldots,k_{r-2},R_t(2;k_{r-1},k_r))</math><br />
}}<br />
<br />
It is then sufficient to prove the Ramsey's theorem for the two-coloring of a hypergraph, that is, to prove <math>R_t(k,\ell)=R_t(2;k,\ell)</math> is finite.<br />
<br />
{{Theorem|Lemma|<br />
:<math>R_t(k,\ell)\le R_{t-1}(R_t(k-1,\ell),R_t(k,\ell-1))+1</math><br />
}}<br />
{{Proof|<br />
Let <math>n=R_{t-1}(R_t(k-1,\ell),R_t(k,\ell-1))+1</math>. Denote <math>[n]=\{1,2,\ldots,n\}</math>.<br />
<br />
Let <math>f:{[n]\choose t}\rightarrow\{{\color{red}\text{red}},{\color{blue}\text{blue}}\}</math> be an arbitrary 2-coloring of <math>{[n]\choose t}</math>. It is then sufficient to show that there either exists an <math>X\subseteq[n]</math> with <math>|X|=k</math> such that all members of <math>{X\choose t}</math> are colored red by <math>f</math>; or exists an <math>X\subseteq[n]</math> with <math>|X|=\ell</math> such that all members of <math>{X\choose t}</math> are colored blue by <math>f</math>.<br />
<br />
We remove <math>n</math> from <math>[n]</math> and define a new coloring <math>f'</math> of <math>{[n-1]\choose t-1}</math> by<br />
:<math>f'(A)=f(A\cup\{n\})</math> for any <math>A\in{[n-1]\choose t-1}</math>.<br />
By the choice of <math>n</math> and by symmetry, there exists a subset <math>S\subseteq[n-1]</math> with <math>|X|=R_t(k-1,\ell)</math> such that all members of <math>{S\choose t-1}</math> are colored with red by <math>f'</math>. Then there either exists an <math>X\subseteq S</math> with <math>|X|=\ell</math> such that <math>{X\choose t}</math> is colored all blue by <math>f</math>, in which case we are done; or exists an <math>X\subseteq S</math> with <math>|X|=k-1</math> such that <math>{X\choose t}</math> is colored all red by <math>f</math>. Next we prove that in the later case <math>{X\cup{n}\choose t}</math> is all red, which will close our proof. Since all <math>A\in{S\choose t-1}</math> are colored with red by <math>f'</math>, then by our definition of <math>f'</math>, <math>f(A\cup\{n\})={\color{red}\text{red}}</math> for all <math>A\in {X\choose t-1}\subseteq{S\choose t-1}</math>. Recalling that <math>{X\choose t}</math> is colored all red by <math>f</math>, <math>{X\cup\{n\}\choose t}</math> is colored all red by <math>f</math> and we are done.<br />
}}<br />
<br />
== Applications of Ramsey Theorem ==<br />
=== The "Happy Ending" problem ===<br />
{{Theorem|The happy ending problem|<br />
:Any set of 5 points in the plane, no three on a line, has a subset of 4 points that form the vertices of a convex quadrilateral.<br />
}}<br />
See the article<br />
[http://www.maa.org/mathland/mathtrek_10_3_00.html] for the proof.<br />
<br />
We say a set of points in the plane in [http://en.wikipedia.org/wiki/General_position '''general positions'''] if no three of the points are on the same line.<br />
<br />
{{Theorem|Theorem (Erdős-Szekeres 1935)|<br />
:For any positive integer <math>m\ge 3</math>, there is an <math>N(m)</math> such that any set of at least <math>N(m)</math> points in general position in the plane (i.e., no three of the points are on a line) contains <math>m</math> points that are the vertices of a convex <math>m</math>-gon.<br />
}}<br />
{{Proof|<br />
Let <math>N(m)=R_3(m,m)</math>. For <math>n\ge N(m)</math>, let <math>X</math> be an arbitrary set of <math>n</math> points in the plane, no three of which are on a line. Define a 2-coloring of the 3-subsets of points <math>f:{X\choose 3}\rightarrow\{0,1\}</math> as follows: for any <math>\{a,b,c\}\in{X\choose 3}</math>, let <math>\triangle_{abc}\subset X</math> be the set of points covered by the triangle <math>abc</math>; and <math>f(\{a,b,c\})=|\triangle_{abc}|\bmod 2</math>, that is, <math>f(\{a,b,c\})</math> indicates the oddness of the number of points covered by the triangle <math>abc</math>.<br />
<br />
Since <math>|X|\ge R_3(m,m)</math>, there exists a <math>Y\subseteq X</math> such that <math>|Y|=m</math> and all members of <math>{Y\choose 3}</math> are colored with the same value by <math>f</math>. <br />
<br />
We claim that the <math>m</math> points in <math>Y</math> are the vertices of a convex <math>m</math>-gon. If otherwise, by the definition of convexity, there exist <math>\{a,b,c,d\}\subseteq Y</math> such that <math>d\in\triangle_{abc}</math>. Since no three points are in the same line, <br />
:<math>\triangle_{abc}=\triangle_{abd}\cup\triangle_{acd}\cup\triangle_{bcd}\cup\{d\}</math>,<br />
where all unions are disjoint. Then <math>|\triangle_{abc}|=|\triangle_{abd}|+|\triangle_{acd}|+|\triangle_{bcd}|+1</math>, which implies that <math>f(\{a,b,c\}), f(\{a,b,d\}), f(\{a,c,d\}), f(\{b,c,d\})\,</math> cannot be equal, contradicting that all members of <math>{Y\choose 3}</math> have the same color.<br />
}}<br />
<br />
=== Yao's lower bound for implicit data structures ===<br />
We consider the following fundamental problem of '''membership query'''.<br />
{{Theorem|Membership Query|<br />
:'''Input''': A data set <math>S\subset U</math> of size <math>n</math>, where <math>U</math> is a data universe of size <math>N</math>.<br />
:'''Query''': a data item (also called a '''key''') <math>x\in U</math>.<br />
:'''Answer''': Whether <math>x\in S</math>.<br />
}}<br />
This is a basic problem for data structures. People want to design efficient data structures to store the data set <math>S</math> so that the query "Is <math>x\in S</math>?" can be efficiently answered by accessing the data structure as little as possible in the worst case.<br />
<br />
A '''sorted table''' for a data set <math>S\subset [N]</math> is a natural data structure in which the elements of <math>S=\{x_1,x_2,\ldots,x_n\}</math>, where <math>x_1<x_2<\cdots<x_n</math>, are stored in an array, one element <math>x_i</math> in each entry, in the increasing order.<br />
For a sorted table, the membership query problem can be solved via '''binary search''' within <math>\Omega(\log_2 n)</math> memory accesses in the worst case. The following [https://dl.acm.org/doi/pdf/10.1145/322261.322274 fundamental result of Andrew Chi-Chih Yao (姚期智)] shows that this is the best possible for sorted tables. The proof is an elegant application of the '''adversarial argument'''(对手论证).<br />
<br />
{{Theorem|Lemma (Yao 1981)|<br />
:Suppose that <math>n\ge 2</math>, <math>N\ge 2n-1</math>, the data universe is <math>U=[N]</math>, and the size of the data set is <math>n</math>.<br />
:If the data structure is a '''sorted table''', any search algorithm requires at least <math>\lceil\log_2 (n+1)\rceil</math> accesses to the data structure in the worst case.<br />
}}<br />
{{Proof|<br />
We will show by an adversarial argument that <math>\lceil\log_2 (n+1)\rceil</math> accesses are required to search for the key value <math>x=n</math> in the universe <math>[N]=\{1,2,\ldots,N\}</math>. The construction of the adversarial data set <math>S</math> is by induction on <math>n</math>.<br />
<br />
For <math>n=2</math> and <math>N\ge 2n-1=3</math> it is easy to see that 2 memory accesses are required to make sure whether the key value <math>n=2</math> presents in a sorted table containing 2 keys out of a data universe of size 3, in the worst case.<br />
<br />
Let <math>n>2</math>. Assume the induction hypothesis for all smaller <math>n</math>. We will prove it for the size of data set <math>n</math>, size of universe <math>N\ge 2n-1</math> and the search key <math>x=n</math>.<br />
<br />
Suppose that the first access position is <math>k</math>. The adversary chooses the table content <math>T[k]</math>. The adversary's strategy is:<br />
:<math><br />
\begin{align}<br />
T[k]=<br />
\begin{cases}<br />
k & k\le \frac{n}{2},\\<br />
N-(n-k) & k> \frac{n}{2}.<br />
\end{cases}<br />
\end{align}<br />
</math><br />
By symmetry, suppose it is the first case that <math>k\le \frac{n}{2}</math>. Then the key <math>x=n</math> may be in any position <math>i</math>, where <math>\frac{n}{2}+1\le i\le n</math>. In fact, <math>T\left[ \left\lceil \frac{n}{2}\right\rceil +1\right]</math> through <math>T[n]</math> is a sorted table of size <math>n'=\left\lfloor \frac{n}{2}\right\rfloor</math> which may contain any <math>n'</math>-subset of <math>\left\{\left\lceil \frac{n}{2}\right\rceil+1, \left\lceil \frac{n}{2}\right\rceil+2,\ldots,N\right\}</math>, and hence, in particular, any <math>n'</math>-subset of the new universe<br />
:<math>U'=\left\{\left\lceil \frac{n}{2}\right\rceil+1, \left\lceil \frac{n}{2}\right\rceil+2,\ldots,N-\left\lceil \frac{n}{2}\right\rceil\right\}</math>.<br />
The size <math>N'</math> of <math>U'</math> satisfies<br />
:<math>N'=N-2\left\lceil \frac{n}{2}\right\rceil\ge 2(n-1)-2\left\lceil \frac{n}{2}\right\rceil \ge 2\left\lfloor \frac{n}{2}\right\rfloor-1= 2n'-1</math>,<br />
and the desired key <math>n</math> has the relative value <math>x'=n- \left\lceil \frac{n}{2}\right\rceil=\left\lfloor \frac{n}{2}\right\rfloor=n'</math> in the new universe <math>U'</math>. <br />
<br />
By the induction hypothesis, <math>\lceil\log_2 (n'+1)\rceil</math> more memory accesses will be required. Hence the total number of memory accesses is at least <br />
:<math>1+\lceil\log_2 (n'+1)\rceil=1+\left\lceil\log_2 \left(\left\lfloor \frac{n}{2}\right\rfloor+1\right)\right\rceil\ge \lceil\log_2 (n+1)\rceil</math>.<br />
<br />
If the first access is <math>k> \frac{n}{2}</math>, we symmetrically get that <math>T[1]</math> through <math>T\left[\left\lfloor \frac{n}{2}\right\rfloor\right]</math> is a sorted table of size <math>n'=\left\lfloor \frac{n}{2}\right\rfloor</math> which may contain any <math>n'</math>-subset of the universe<br />
:<math>U'=\left\{\left\lceil \frac{n}{2}\right\rceil+1, \left\lceil \frac{n}{2}\right\rceil+2,\ldots,N-\left\lceil \frac{n}{2}\right\rceil\right\}</math>.<br />
The rest is the same as before. This completes the induction.<br />
}}<br />
<br />
We have seen that on a sorted table, there is no search algorithm outperforming the binary search in the worst case.<br />
Our question is:<br />
:''Is there any other order than the increasing order, on which there is a better search algorithm?''<br />
<br />
An '''implicit data structure''' use no extra space in addition to the original data set, thus a data structure can only be represented ''implicitly'' by the order of the data items in the table. That is, each data set is stored as a permutation of the set. Formally, an implicit data structure is described by a function<br />
:<math>f:{U\choose n}\rightarrow[n!]</math>,<br />
where each <math>\pi\in[n!]</math> specify a permutation of the sorted table, and a data set <math>S=\{x_1,x_2,\ldots,x_n\}</math> where <math>x_1<x_2<\cdots<x_n</math> is stored as an array <math>(x_{\pi(1)},x_{\pi(2)},\ldots,x_{\pi(n)}\}</math> where <math>\pi=f(S)</math>. <br />
<br />
Thus, the sorted table is the simplest implicit data structure, in which <math>f(S)</math> always gives the identity permutation for all <math>S\in{U\choose n}</math>.<br />
We observe that if <math>f</math> maps all data sets <math>S\in{U\choose n}</math> to the same permutation <math>\pi</math>, then the data structure is equivalent to the sorted table, under the bijection that the <math>i</math>th entry in the sorted table corresponds to the <math>\pi(i)</math>th entry of the actual array, where the same <math>\Omega(\log_2 n)</math> lower bound applies.<br />
<br />
This observation can be generalized and made formal as follows.<br />
<br />
{{Theorem|Observation|<br />
:If there is a sub-universe <math>X\subseteq U</math> such that for every data set <math>S\in {X\choose n}</math>, the implicit data structure <math>f(S)=\pi</math> stores the data set using the the same permutation <math>\pi</math>, i.e.<br />
::<math>f\left({X\choose n}\right)=\{\pi\}</math><br />
:then this implicit data structure is equivalent to the sorted table for all data sets from the new universe <math>X</math>, under the bijection that the <math>i</math>th entry in the sorted table corresponds to the <math>\pi(i)</math>th entry of the array.<br />
:Therefore, if <math>|X|\ge 2n</math>, then the same <math>\Omega(\log_2 n)</math> lower bound for searching in a sorted table applies.<br />
}}<br />
<br />
Due to Ramsey theorem, for sufficiently large <math>N</math> which satisfies <math>N\ge R_{n}(n!;2n)</math>, for any <math>f:{U\choose n}\rightarrow[n!]</math>, there is an <math>X\subseteq U</math> of size <math>|X|\ge 2n</math>, such that <math>\left|f\left({X\choose n}\right)\right|=1</math>, which guarantees the existence of the sub-universe <math>X\subseteq U</math> required in the above observation for (wildly) large universe sizes <math>N</math>, which implies the following lower bound for implicit data structures.<br />
<br />
{{Theorem|Theorem (Yao 1981)|<br />
:Suppose that <math>n\ge 2</math>, and <math>N\ge 2n</math>, the data universe is <math>U=[N]</math>, and the size of the data set is <math>n</math>.<br />
:For any '''implicit data structure''', if <math>N</math> is sufficiently large, then any search algorithm requires at least <math>\lfloor\log_2 n\rfloor</math> accesses to the data structure in the worst case.<br />
}}<br />
<br />
=== Linial's lower bound for local computation===<br />
In the studies of '''local computation''' (initiated by [https://www.cs.huji.ac.il/~nati/PAPERS/locality_dist_graph_algs.pdf Linial] and [https://www.wisdom.weizmann.ac.il/~naor/PAPERS/lcl.pdf Naor and Stockmeyer]), people wants to answer questions like:<br />
::''Can locally defined problems be computed locally?''<br />
In general, the answer is no to the above question. A famous example is Linial's lower bound for '''maximal independent set''' ('''MIS''') in a ring.<br />
<br />
Consider a very simple distributed network, a ring that contains <math>n</math> nodes, where each node is assigned a unique ID from <math>[n]=\{1,2,\ldots, n\}</math>. The labeled network is then described by a tuple <math>(a_1,a_2,\ldots,a_n)</math> of IDs, which is a permutation of <math>[n]</math>, where <math>a_i\in [n]</math> gives the ID of the <math>i</math>th node in the ring.<br />
<br />
In a distributed algorithm, in each round, every node communicates with its 2 neighbors in the ring, and when the algorithm terminates, each node returns its local output. For example, in the MIS problem, the goal of the algorithm is to construct a maximal independent set: upon termination, each node returns a bit to indicate whether the node is in the constructed independent set. And the output gives a correct MIS as long as it satisfies both the followings: <br />
* there are no two consecutive nodes in the ring both outputting 1;<br />
* there are no three consecutive nodes in the ring all outputting 0.<br />
This is clearly a locally defined problem. In fact, it is a constraint satisfaction problem (CSP) where each constraint only involves 1-local or 2-local neighborhood.<br />
<br />
We are interested in the distributed algorithms that can always produce the correct answer, and want to prove a lower bound for the number of rounds required in the worst case by such distributed algorithms.<br />
<br />
As a local distributed algorithm, each node <math>i</math> initially does not know anything beyond its local information, which is just its own ID <math>a_i\in [n]</math>. <br />
And after <math>t</math> rounds, information-theoretically, each node <math>i</math> can at best know all information within its <math>t</math>-local neighborhood, which is represented by the <math>(2t-1)</math>-tuple <math>(a_{i-t},\ldots,a_{i-1},a_i,a_{i+1},\ldots a_{i+t})</math>, with the addition/subtraction modulo <math>n</math> along the ring.<br />
<br />
This suggests us to define such a computational model for local distributed algorithms: any <math>t</math>-round local algorithm is described by a function<br />
:<math>\mathcal{L}:[n]^{2t+1}\to\{0,1\}</math>.<br />
For the <math>i</math>th node in the ring, its output is given by <math>\mathcal{L}(a_{i-t},\ldots,a_{i-1},a_i,a_{i+1},\ldots a_{i+t})</math>, where <math>a_j</math> represents the ID of the <math>j</math>th node in the ring, with the addition/subtraction modulo <math>n</math>.<br />
<br />
As a correct algorithm for constructing MIS, it must hold that any three consecutive nodes can never output the same value. We then have the following lower bound for <math>t</math> for such algorithms.<br />
<br />
{{Theorem|Theorem (Linial 1992)|<br />
:For any <math>t</math>-round local algorithm for maximal independent set (MIS) on a ring of <math>n</math> nodes, it holds that<br />
:: <math>t=\Omega(\log^*n)</math><br />
:where <math>\log^*n</math> represents the [https://en.wikipedia.org/wiki/Iterated_logarithm iterated logarithm], which is the number of times the logarithm function must be iteratively applied to <math>n</math> before the result is less than or equal to 1.<br />
}}<br />
This lower bound shows that even on very simple network like ring, some very basic locally defined problem (MIS) cannot be computed locally (within constant locality).<br />
<br />
The original proof of Linial relies on chromatic number of so-called neighborhood graphs. Here we give an alternative proof based on Ramsey theorem found by Baruch Awerbuch.<br />
{{Proof|<br />
As we discussed earlier, any <math>t</math>-round local algorithm can be represented by a mapping<br />
:<math>\mathcal{L}:[n]^{2t+1}\to\{0,1\}</math>.<br />
It can naturally defines a 2-coloring:<br />
:<math>f:{[n]\choose {2t+1}}\to\{0,1\}</math><br />
by the following construction: for any <math>\{a_1,a_2,\ldots,a_{2t+1}\}\subseteq [n]</math>, where <math>a_1<a_2<\cdots<a_{2t+1}</math>, we define<br />
:<math>f(\{a_1,a_2,\ldots,a_{2t+1}\})=\mathcal{L}(a_1,a_2,\ldots,a_{2t+1})</math>.<br />
By Ramsey theorem, for <math>n\ge R_{2t+1}(2;2t+3,2t+3)</math>, there exists a subset <math>\{a_1,a_2,\ldots,a_{2t+3}\}\subseteq [n]</math>, where <math>a_1<a_2<\cdots<a_{2t+3}</math>, such that <br />
:<math>\left|f\left({S\choose {2t+1}}\right)\right|=1</math>.<br />
By our construction of <math>f</math>, this means<br />
:<math>\mathcal{L}(a_1,a_2,\ldots,a_{2t+1})=\mathcal{L}(a_2,a_3,\ldots,a_{2t+2})=\mathcal{L}(a_3,a_4,\ldots,a_{2t+3})</math>,<br />
which contradicts to that the output of <math>\mathcal{L}</math> should indicate an MIS, on any ring with <math>2t+3</math> consecutive nodes labeled as <math>(a_1,a_2,\ldots,a_{2t+3})</math>, because on such rings, there would be 3 consecutive nodes with the same output bit.<br />
<br />
Therefore, any <math>t</math>-round local algorithm that can always correctly produce an MIS on a ring, must satisfies that<br />
:<math>n<R_{2t+1}(2;2t+3,2t+3)\le \underbrace{2^{2^{\unicode{x22F0}^{2}}}}_{ct}</math>,<br />
for some constant <math>c>0</math>, whose inverse function gives the lower bound<br />
:<math>t=\Omega(\log^*n)</math>.<br />
}}</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=%E7%BB%84%E5%90%88%E6%95%B0%E5%AD%A6_(Spring_2026)&diff=13753组合数学 (Spring 2026)2026-05-20T13:32:42Z<p>Etone: /* Lecture Notes */</p>
<hr />
<div>{{Infobox<br />
|name = Infobox<br />
|bodystyle = <br />
|title = <font size=3>组合数学 <br><br />
Combinatorics</font><br />
|titlestyle = <br />
<br />
|image = <br />
|imagestyle = <br />
|caption = <br />
|captionstyle = <br />
|headerstyle = background:#ccf;<br />
|labelstyle = background:#ddf;<br />
|datastyle = <br />
<br />
|header1 =Instructor<br />
|label1 = <br />
|data1 = <br />
|header2 = <br />
|label2 = <br />
|data2 = 尹一通<br />
|header3 = <br />
|label3 = Email<br />
|data3 = yinyt@nju.edu.cn <br />
|header4 =<br />
|label4= office<br />
|data4= 计算机系 804<br />
|header5 = Class<br />
|label5 = <br />
|data5 = <br />
|header6 =<br />
|label6 = Class meetings<br />
|data6 = Wednesday, 2pm-4pm <br> 逸B-313<br />
|header7 =<br />
|label7 = Place<br />
|data7 = <br />
|header8 =<br />
|label8 = Office hours<br />
|data8 = Tuesday, 2-3pm <br>计算机系 804<br />
|header9 = Textbook<br />
|label9 = <br />
|data9 = <br />
|header10 =<br />
|label10 = <br />
|data10 = [[File:LW-combinatorics.jpeg|border|100px]]<br />
|header11 =<br />
|label11 = <br />
|data11 = van Lint and Wilson. <br> ''A course in Combinatorics, 2nd ed.'', <br> Cambridge Univ Press, 2001.<br />
|header12 =<br />
|label12 = <br />
|data12 = [[File:Jukna_book.jpg|border|100px]]<br />
|header13 =<br />
|label13 = <br />
|data13 = Jukna. ''Extremal Combinatorics: <br> With Applications in Computer Science,<br>2nd ed.'', Springer, 2011.<br />
|belowstyle = background:#ddf;<br />
|below = <br />
}}<br />
<br />
This is the webpage for the ''Combinatorics'' class of Spring 2026. Students who take this class should check this page periodically for content updates and new announcements. <br />
<br />
= Announcement =<br />
* '''(2026/03/25)'''<font color=red size=4> 第一次作业已发布</font>，请在 2026/04/08 上课之前提交到 [mailto:njucomb26@163.com njucomb26@163.com] (文件名为'学号_姓名_A1.pdf')<br />
* '''(2026/04/21)'''<font color=red size=4> 第二次作业已发布</font>，请在 2026/05/13 上课之前提交到 [mailto:njucomb26@163.com njucomb26@163.com] (文件名为'学号_姓名_A2.pdf')<br />
<br />
= Course info =<br />
* '''Instructor ''': 尹一通 ([http://tcs.nju.edu.cn/yinyt/ homepage])<br />
:*'''email''': yinyt@nju.edu.cn<br />
:*'''office''': 计算机系 804 <br />
* '''Teaching assistant''':<br />
** 丁天行([mailto:652024330006@smail.nju.edu.cn 652024330006@smail.nju.edu.cn])<br />
** 周灿<br />
** 方子伊<br />
* '''Class meeting''': Wednesday, 2pm-4pm, 逸A-313.<br />
* '''Office hour''': TBA<br />
:* '''QQ群''': 1090691552 (加入时需报姓名、专业、学号)<br />
<br />
= Syllabus =<br />
<br />
=== 先修课程 Prerequisites ===<br />
* 离散数学（Discrete Mathematics）<br />
* 线性代数（Linear Algebra）<br />
* 概率论（Probability Theory）<br />
<br />
=== Course materials ===<br />
* [[组合数学 (Spring 2025)/Course materials|<font size=3>教材和参考书清单</font>]]<br />
<br />
=== 成绩 Grades ===<br />
* 课程成绩：本课程将会有若干次作业和一次期末考试。最终成绩将由平时作业成绩 (≥ 60%) 和期末考试成绩 (≤ 40%) 综合得出。<br />
* 迟交：如果有特殊的理由，无法按时完成作业，请提前联系授课老师，给出正当理由。否则迟交的作业将不被接受。<br />
<br />
=== <font color=red> 学术诚信 Academic Integrity </font>===<br />
学术诚信是所有从事学术活动的学生和学者最基本的职业道德底线，本课程将不遗余力的维护学术诚信规范，违反这一底线的行为将不会被容忍。<br />
<br />
作业完成的原则：署你名字的工作必须是你个人的贡献。在完成作业的过程中，允许讨论，前提是讨论的所有参与者均处于同等完成度。但关键想法的执行、以及作业文本的写作必须独立完成，并在作业中致谢（acknowledge）所有参与讨论的人。不允许其他任何形式的合作——尤其是与已经完成作业的同学“讨论”。<br />
<br />
本课程将对剽窃行为采取零容忍的态度。在完成作业过程中，对他人工作（出版物、互联网资料、其他人的作业等）直接的文本抄袭和对关键思想、关键元素的抄袭，按照 [http://www.acm.org/publications/policies/plagiarism_policy ACM Policy on Plagiarism]的解释，都将视为剽窃。剽窃者成绩将被取消。如果发现互相抄袭行为，<font color=red> 抄袭和被抄袭双方的成绩都将被取消</font>。因此请主动防止自己的作业被他人抄袭。<br />
<br />
学术诚信影响学生个人的品行，也关乎整个教育系统的正常运转。为了一点分数而做出学术不端的行为，不仅使自己沦为一个欺骗者，也使他人的诚实努力失去意义。让我们一起努力维护一个诚信的环境。<br />
<br />
= Assignments =<br />
* [[组合数学 (Spring 2026)/Problem Set 1|Problem Set 1]]<br />
* [[组合数学 (Spring 2026)/Problem Set 2|Problem Set 2]]<br />
<br />
= Lecture Notes =<br />
# [[组合数学 (Spring 2026)/Basic enumeration|Basic enumeration | 基本计数]] ([http://tcs.nju.edu.cn/slides/comb2026/BasicEnumeration.pdf slides])<br />
# [[组合数学 (Spring 2026)/Generating functions|Generating functions | 生成函数]] ([http://tcs.nju.edu.cn/slides/comb2026/GeneratingFunction.pdf slides])<br />
# [[组合数学 (Fall 2026)/Sieve methods|Sieve methods | 筛法]] ([http://tcs.nju.edu.cn/slides/comb2026/PIE.pdf slides])<br />
# Guest lecture by Prof. Penghui Yao on entropy and counting ([http://tcs.nju.edu.cn/slides/comb2026/entropy.pdf notes]) <br />
# [[组合数学 (Fall 2026)/Cayley's formula|Cayley's formula | Cayley公式]] ([http://tcs.nju.edu.cn/slides/comb2026/Cayley.pdf slides])<br />
# [[组合数学 (Fall 2026)/Existence problems|Existence problems | 存在性问题]]<br />
# [[组合数学 (Fall 2026)/The probabilistic method|The probabilistic method | 概率法]] ([http://tcs.nju.edu.cn/slides/comb2026/ProbMethod.pdf slides])<br />
# [[组合数学 (Fall 2026)/Extremal graph theory|Extremal graph theory | 极值图论]] ([http://tcs.nju.edu.cn/slides/comb2026/ExtremalGraphs.pdf slides])<br />
# [[组合数学 (Fall 2026)/Extremal set theory|Extremal set theory | 极值集合论]]（[http://tcs.nju.edu.cn/slides/comb2026/ExtremalSets.pdf slides]）<br />
#* [https://mathweb.ucsd.edu/~ronspubs/90_03_erdos_ko_rado.pdf Old and new proofs of the Erdős–Ko–Rado theorem] by Frankl and Graham<br />
#* An [http://tcs.nju.edu.cn/slides/comb2026/sunflower-note.pdf LLM-generated lecture note] on Alweiss-Lovet-Wu-Zhang's improvement over the sunflower lemma, with simplified proofs by Rao-Tao<br />
# [[组合数学 (Fall 2026)/Ramsey theory|Ramsey theory | Ramsey理论]]（[http://tcs.nju.edu.cn/slides/comb2026/Ramsey.pdf slides]）<br />
<br />
= Resources =<br />
* [http://math.mit.edu/~fox/MAT307.html Combinatorics course] by Jacob Fox<br />
* [https://yufeizhao.com/pm/ Probabilistic Methods in Combinatorics] and [https://yufeizhao.com/gtacbook/ Graph Theory and Additive Combinatorics] by Yufei Zhao<br />
* [https://www.math.uvic.ca/~noelj/combinatoricsLectures.html Combinatorics Lecture Videos online]<br />
* [https://www.math.ucla.edu/~pak/lectures/Math-Videos/comb-videos.htm Collection of Combinatorics Videos]<br />
<br />
= Concepts =<br />
* [http://en.wikipedia.org/wiki/Binomial_coefficient Binomial coefficient]<br />
* [http://en.wikipedia.org/wiki/Twelvefold_way The twelvefold way]<br />
* [http://en.wikipedia.org/wiki/Composition_(number_theory) Composition of a number]<br />
* [http://en.wikipedia.org/wiki/Multiset#Formal_definition Multiset]<br />
* [http://en.wikipedia.org/wiki/Combination#Number_of_combinations_with_repetition Combinations with repetition], [http://en.wikipedia.org/wiki/Multiset#Counting_multisets <math>k</math>-multisets on a set]<br />
* [http://en.wikipedia.org/wiki/Multinomial_theorem#Multinomial_coefficients Multinomial coefficients]<br />
* [http://en.wikipedia.org/wiki/Stirling_numbers_of_the_second_kind Stirling number of the second kind]<br />
* [http://en.wikipedia.org/wiki/Partition_(number_theory) Partition of a number]<br />
** [http://en.wikipedia.org/wiki/Young_tableau Young tableau]<br />
* [http://en.wikipedia.org/wiki/Fibonacci_number Fibonacci number]<br />
* [http://en.wikipedia.org/wiki/Catalan_number Catalan number]<br />
* [http://en.wikipedia.org/wiki/Generating_function Generating function] and [http://en.wikipedia.org/wiki/Formal_power_series formal power series]<br />
* [http://en.wikipedia.org/wiki/Binomial_series Newton's formula]<br />
* [http://en.wikipedia.org/wiki/Inclusion-exclusion_principle The principle of inclusion-exclusion] (and more generally the [http://en.wikipedia.org/wiki/Sieve_theory sieve method])<br />
* [http://en.wikipedia.org/wiki/M%C3%B6bius_inversion_formula Möbius inversion formula]<br />
* [http://en.wikipedia.org/wiki/Derangement Derangement], and [http://en.wikipedia.org/wiki/M%C3%A9nage_problem Problème des ménages]<br />
* [http://en.wikipedia.org/wiki/Ryser%27s_formula#Ryser_formula Ryser's formula]<br />
* [http://en.wikipedia.org/wiki/Euler_totient Euler totient function]<br />
* [https://en.wikipedia.org/wiki/Burnside%27s_lemma Burnside's lemma]<br />
** [https://en.wikipedia.org/wiki/Group_action Group action]<br />
** [https://en.wikipedia.org/wiki/Group_action#Orbits_and_stabilizers Orbits]<br />
* [https://en.wikipedia.org/wiki/P%C3%B3lya_enumeration_theorem Pólya enumeration theorem]<br />
** [https://en.wikipedia.org/wiki/Permutation_group Permutation group]<br />
** [https://en.wikipedia.org/wiki/Cycle_index Cycle index]<br />
* [http://en.wikipedia.org/wiki/Cayley_formula Cayley's formula]<br />
** [http://en.wikipedia.org/wiki/Prüfer_sequence Prüfer code for trees]<br />
** [http://en.wikipedia.org/wiki/Kirchhoff%27s_matrix_tree_theorem Kirchhoff's matrix-tree theorem]<br />
* [http://en.wikipedia.org/wiki/Double_counting_(proof_technique) Double counting] and the [http://en.wikipedia.org/wiki/Handshaking_lemma handshaking lemma]<br />
* [http://en.wikipedia.org/wiki/Sperner's_lemma Sperner's lemma] and [http://en.wikipedia.org/wiki/Brouwer_fixed_point_theorem Brouwer fixed point theorem]<br />
* [http://en.wikipedia.org/wiki/Pigeonhole_principle Pigeonhole principle]<br />
:* [http://en.wikipedia.org/wiki/Erd%C5%91s%E2%80%93Szekeres_theorem Erdős–Szekeres theorem]<br />
:* [http://en.wikipedia.org/wiki/Dirichlet's_approximation_theorem Dirichlet's approximation theorem]<br />
* [http://en.wikipedia.org/wiki/Probabilistic_method The Probabilistic Method]<br />
* [http://en.wikipedia.org/wiki/Lov%C3%A1sz_local_lemma Lovász local lemma]<br />
* [http://en.wikipedia.org/wiki/Erd%C5%91s%E2%80%93R%C3%A9nyi_model Erdős–Rényi model for random graphs]<br />
* [http://en.wikipedia.org/wiki/Extremal_graph_theory Extremal graph theory]<br />
* [http://en.wikipedia.org/wiki/Turan_theorem Turán's theorem], [http://en.wikipedia.org/wiki/Tur%C3%A1n_graph Turán graph]<br />
* Two analytic inequalities: <br />
:*[http://en.wikipedia.org/wiki/Cauchy%E2%80%93Schwarz_inequality Cauchy–Schwarz inequality]<br />
:* the [http://en.wikipedia.org/wiki/Inequality_of_arithmetic_and_geometric_means inequality of arithmetic and geometric means]<br />
* [http://en.wikipedia.org/wiki/Erd%C5%91s%E2%80%93Stone_theorem Erdős–Stone theorem] (fundamental theorem of extremal graph theory)<br />
* [https://en.wikipedia.org/wiki/Sunflower_(mathematics) Sunflower lemma and conjecture]<br />
* [https://en.wikipedia.org/wiki/Erd%C5%91s%E2%80%93Ko%E2%80%93Rado_theorem Erdős–Ko–Rado theorem]<br />
* [https://en.wikipedia.org/wiki/Sperner%27s_theorem Sperner's theorem]<br />
** [https://en.wikipedia.org/wiki/Sperner_family Sperner system] or '''antichain'''<br />
* [https://en.wikipedia.org/wiki/Sauer%E2%80%93Shelah_lemma Sauer–Shelah lemma]<br />
** [https://en.wikipedia.org/wiki/Vapnik%E2%80%93Chervonenkis_dimension Vapnik–Chervonenkis dimension]<br />
* [https://en.wikipedia.org/wiki/Kruskal%E2%80%93Katona_theorem Kruskal–Katona theorem]<br />
* [http://en.wikipedia.org/wiki/Ramsey_theory Ramsey theory]<br />
:*[http://en.wikipedia.org/wiki/Ramsey's_theorem Ramsey's theorem]<br />
:*[http://en.wikipedia.org/wiki/Happy_Ending_problem Happy Ending problem]<br />
:*[https://en.wikipedia.org/wiki/Van_der_Waerden%27s_theorem Van der Waerden's theorem]<br />
:*[https://en.wikipedia.org/wiki/Hales%E2%80%93Jewett_theorem Hales–Jewett theorem]<br />
* [https://en.wikipedia.org/wiki/Hall%27s_marriage_theorem Hall's theorem ] (the marriage theorem)<br />
:* [https://en.wikipedia.org/wiki/Doubly_stochastic_matrix Birkhoff–Von Neumann theorem]<br />
* [http://en.wikipedia.org/wiki/K%C3%B6nig's_theorem_(graph_theory) König-Egerváry theorem]<br />
* [http://en.wikipedia.org/wiki/Dilworth's_theorem Dilworth's theorem]<br />
:* [http://en.wikipedia.org/wiki/Erd%C5%91s%E2%80%93Szekeres_theorem Erdős–Szekeres theorem]<br />
* The [http://en.wikipedia.org/wiki/Max-flow_min-cut_theorem Max-Flow Min-Cut Theorem]<br />
:* [https://en.wikipedia.org/wiki/Menger%27s_theorem Menger's theorem]<br />
:* [http://en.wikipedia.org/wiki/Maximum_flow_problem Maximum flow]<br />
* [https://en.wikipedia.org/wiki/Linear_programming Linear programming]<br />
** [https://en.wikipedia.org/wiki/Dual_linear_program Duality] <br />
** [https://en.wikipedia.org/wiki/Unimodular_matrix Unimodularity]<br />
* [https://en.wikipedia.org/wiki/Matroid Matroid]</div>Etonehttps://tcs.nju.edu.cn/wiki/index.php?title=%E7%BB%84%E5%90%88%E6%95%B0%E5%AD%A6_(Fall_2026)/Extremal_set_theory&diff=13736组合数学 (Fall 2026)/Extremal set theory2026-05-13T08:49:22Z<p>Etone: Created page with "== Sunflowers == An set system is a '''sunflower''' if all its member sets intersect at the same set of elements. {{Theorem|Definition (sunflower)| : A set family <math>\mathcal{F}\subseteq 2^X</math> is a '''sunflower''' of size <math>r</math> with a '''core''' <math>C\subseteq X</math> if ::<math>\forall S,T\in\mathcal{F}</math> that <math>S\neq T</math>, <math>S\cap T=C</math>. }} Note that we do not require the core to be nonempty, thus a family of disjoint sets is..."</p>
<hr />
<div>== Sunflowers ==<br />
An set system is a '''sunflower''' if all its member sets intersect at the same set of elements.<br />
{{Theorem|Definition (sunflower)|<br />
: A set family <math>\mathcal{F}\subseteq 2^X</math> is a '''sunflower''' of size <math>r</math> with a '''core''' <math>C\subseteq X</math> if <br />
::<math>\forall S,T\in\mathcal{F}</math> that <math>S\neq T</math>, <math>S\cap T=C</math>.<br />
}}<br />
Note that we do not require the core to be nonempty, thus a family of disjoint sets is also a sunflower (with the core <math>\emptyset</math>).<br />
<br />
The next result due to Erdős and Rado, called the sunflower lemma, is a famous result in extremal set theory, and has some important applications in Boolean circuit complexity.<br />
{{Theorem|Sunflower Lemma (Erdős-Rado)|<br />
:Let <math>\mathcal{F}\subseteq {X\choose k}</math>. If <math>|\mathcal{F}|>k!(r-1)^k</math>, then <math>\mathcal{F}</math> contains a sunflower of size <math>r</math>.<br />
}}<br />
{{Proof|<br />
We proceed by induction on <math>k</math>. For <math>k=1</math>, <math>\mathcal{F}\subseteq{X\choose 1}</math>, thus all sets in <math>\mathcal{F}</math> are disjoint. And since <math>|\mathcal{F}|>r-1</math>, we can choose <math>r</math> of these sets and form a sunflower.<br />
<br />
Now let <math>k\ge 2</math> and assume the lemma holds for all smaller <math>k</math>. Take a maximal family <math>\mathcal{G}\subseteq \mathcal{F}</math> whose members are disjoint, i.e. for any <math>S,T\in \mathcal{G}</math> that <math>S\neq T</math>, <math>S\cap T=\emptyset</math>.<br />
<br />
If <math>|\mathcal{G}|\ge r</math>, then <math>\mathcal{G}</math> is a sunflower of size at least <math>r</math> and we are done.<br />
<br />
Assume that <math>|\mathcal{G}|\le r-1</math>, and let <math>Y=\bigcup_{S\in\mathcal{G}}S</math>. Then <math>|Y|=k|\mathcal{G}|\le k(r-1)</math> (since all members of <math>\mathcal{G}</math>) are disjoint). We claim that <math>Y</math> intersets all members of <math>\mathcal{F}</math>, since if otherwise, there exists an <math>S\in\mathcal{F}</math> such that <math>S\cap Y=\emptyset</math>, then we can enlarge <math>\mathcal{G}</math> by adding <math>S</math> into <math>\mathcal{G}</math> and still have all members of <math>\mathcal{G}</math> disjoint, which contradicts the assumption that <math>\mathcal{G}</math> is the maximum of such families.<br />
<br />
By the pigeonhole principle, some elements <math>y\in Y</math> must contained in at least<br />
:<math>\frac{|\mathcal{F}|}{|Y|}>\frac{k!(r-1)^k}{k(r-1)}=(k-1)!(r-1)^{k-1}</math><br />
members of <math>\mathcal{F}</math>. We delete this <math>y</math> from these sets and consider the family <br />
:<math>\mathcal{H}=\{S\setminus\{y\}\mid S\in\mathcal{F}\wedge y\in S\}</math>.<br />
We have <math>\mathcal{H}\subseteq {X\choose k-1}</math> and <math>|\mathcal{H}|>(k-1)!(r-1)^{k-1}</math>, thus by the induction hypothesis, <math>\mathcal{H}</math>contains a sunflower of size <math>r</math>. Adding <math>y</math> to the members of this sunflower, we get the desired sunflower in the original family <math>\mathcal{F}</math>.<br />
}}<br />
<br />
==The Erdős–Ko–Rado Theorem ==<br />
A set family <math>\mathcal{F}\subseteq 2^X</math> is called '''intersecting''', if for any <math>S,T\in\mathcal{F}</math>, <math>S\cap T\neq\emptyset</math>. A natural question of extremal favor is: "how large can an intersecting family be?"<br />
<br />
Assume <math>|X|=n</math>. When <math>n<2k</math>, every pair of <math>k</math>-subsets of <math>X</math> intersects. So the non-trivial case is when <math>n\ge 2k</math>. The famous Erdős–Ko–Rado theorem gives the largest possible cardinality of a nontrivially intersecting family. <br />
<br />
According to Erdős, the theorem itself was proved in 1938, but was not published until 23 years later.<br />
{{Theorem|Erdős–Ko–Rado theorem (proved in 1938, published in 1961)|<br />
:Let <math>\mathcal{F}\subseteq {X\choose k}</math> where <math>|X|=n</math> and <math>n\ge 2k</math>. If <math>\mathcal{F}</math> is intersecting, then<br />
::<math>|\mathcal{F}|\le{n-1\choose k-1}</math>.<br />
}}<br />
<br />
=== Katona's proof ===<br />
We first introduce a proof discovered by Katona in 1972. The proof uses double counting.<br />
<br />
Let <math>\pi</math> be a '''cyclic permutation''' of <math>X</math>, that is, we think of assigning <math>X</math> in a circle and ignore the rotations of the circle. It is easy to see that there are <math>(n-1)!</math> cyclic permutations of an <math>n</math>-set (each cyclic permutation corresponds to <math>n</math> permutations).<br />
Let <br />
:<math>\mathcal{G}_\pi=\{\{\pi_{(i+j)\bmod n}\mid j\in[k]\}\mid i\in [n]\}</math>.<br />
<br />
The next lemma states the following observation: in a circle of <math>n</math> points, supposed <math>n\ge 2k</math>, there can be at most <math>k</math> arcs, each consisting of <math>k</math> points, such that every pair of arcs share at least one point.<br />
{{Theorem|Lemma|<br />
:Let <math>\mathcal{F}\subseteq {X\choose k}</math> where <math>|X|=n</math> and <math>n\ge 2k</math>. If <math>\mathcal{F}</math> is intersecting, then for any cyclic permutation <math>\pi</math> of <math>X</math>, it holds that <math>|\mathcal{G}_\pi\cap\mathcal{F}|\le k</math>.<br />
}}<br />
{{Proof|<br />
Fix a cyclic permutation <math>\pi</math> of <math>X</math>. Let <math>A_i=\{\pi_{(i+j+n)\bmod n}\mid j\in[k]\}</math>. Then <math>\mathcal{G}_\pi</math> can be written as <math>\mathcal{G}_\pi=\{A_i\mid i\in [n]\}</math>. <br />
<br />
Suppose that <math>A_t\in\mathcal{F}</math>. Since <math>\mathcal{F}</math> is intersecting, the only sets <math>A_i</math> that can be in <math>\mathcal{F}</math> other than <math>A_t</math> itself are the <math>2k-2</math> sets <math>A_i</math> with <math>t-(k-1)\le i\le t+k-1, i\neq t</math>. We partition these sets into <math>k-1</math> pairs <math>\{A_i,A_{i+k}\}</math>, where <math>t-(k-1)\le i\le t-1</math>.<br />
<br />
Note that for <math>n\ge 2k</math>, it holds that <math>A_i\cap C_{i+k}=\emptyset</math>. Since <math>\mathcal{F}</math> is intersecting, <math>\mathcal{F}</math> can contain at most one set of each such pair. The lemma follows.<br />
}}<br />
<br />
The Katona's proof of Erdős–Ko–Rado theorem is done by counting in two ways the pairs of member <math>S</math> of <math>\mathcal{F}</math> and cyclic permutation <math>\pi</math> which contain <math>S</math> as a continuous path on the circle (i.e., an arc).<br />
<br />
{{Prooftitle|Katona's proof of Erdős–Ko–Rado theorem|(double counting)<br />
Let <br />
:<math>\mathcal{R}=\{(S,\pi)\mid \pi \text{ is a cyclic permutation of }X, \text{and }S\in\mathcal{F}\cap\mathcal{G}_\pi\}</math>.<br />
We count <math>\mathcal{R}</math> in two ways.<br />
<br />
First, due to the lemma, <math>|\mathcal{F}\cap\mathcal{G}_\pi|\le k</math> for any cyclic permutation <math>\pi</math>. There are <math>(n-1)!</math> cyclic permutations in total. Thus,<br />
:<math>|\mathcal{R}|=\sum_{\text{cyclic }\pi}|\mathcal{F}\cap\mathcal{G}_\pi|\le k(n-1)!</math>.<br />
<br />
Next, for each <math>S\in\mathcal{F}</math>, the number of cyclic permutations <math>\pi</math> in which <math>S</math> is continuous is <math>|S|!(n-|S|)!=k!(n-k)!</math>. Thus,<br />
:<math>|\mathcal{R}|=\sum_{S\in\mathcal{F}}k!(n-k)!=|\mathcal{F}|k!(n-k)!</math>.<br />
<br />
Altogether, we have <br />
:<math>|\mathcal{F}|\le\frac{k(n-1)!}{k!(n-k)!}=\frac{(n-1)!}{(k-1)!(n-k)!}={n-1\choose k-1}</math>.<br />
}}<br />
<br />
=== Erdős' shifting technique ===<br />
We now introduce the original proof of the Erdős–Ko–Rado theorem, which uses a technique called '''shifting''' (originally called '''compression''').<br />
<br />
Without loss of generality, we assume <math>X=[n]</math>, and restate the Erdős–Ko–Rado theorem as follows.<br />
{{Theorem|Erdős–Ko–Rado theorem|<br />
:Let <math>\mathcal{F}\subseteq {[n]\choose k}</math> and <math>n\ge 2k</math>. If <math>\mathcal{F}</math> is intersecting, then <math>|\mathcal{F}|\le{n-1\choose k-1}</math>.<br />
}}<br />
<br />
We define a '''shift operator''' for the set family.<br />
{{Theorem|Definition (shift operator)|<br />
: Assume <math>\mathcal{F}\subseteq 2^{[n]}</math>, and <math>0\le i<j\le n-1</math>. Define the '''<math>(i,j)</math>-shift''' <math>S_{ij}</math> as an operator on <math>\mathcal{F}</math> as follows:<br />
:*for each <math>T\in\mathcal{F}</math>, write <math>T_{ij}=(T\setminus\{j\})\cup\{i\} </math>, and let<br />
::<math>S_{ij}(T)=<br />
\begin{cases}<br />
T_{ij} & \mbox{if }j\in T, i\not\in T, \mbox{ and }T_{ij} \not\in\mathcal{F},\\<br />
T & \mbox{otherwise;}<br />
\end{cases}</math><br />
:* let <math>S_{ij}(\mathcal{F})=\{S_{ij}(T)\mid T\in \mathcal{F}\}</math>.<br />
}}<br />
<br />
It is easy to verify the following propositions of shifts.<br />
<br />
{{Theorem|Proposition|<br />
# <math>|S_{ij}(T)|=|T|\,</math> and <math>|S_{ij}(\mathcal{F})|=\mathcal{F}</math>;<br />
# if <math>\mathcal{F}</math> is intersecting, then so is <math>S_{ij}(\mathcal{F})</math>.<br />
}}<br />
{{Proof|<br />
(1) is quite obvious. Now we prove (2).<br />
<br />
Consider any <math>A,B\in\mathcal{F}</math>. If <math>A\cap B</math> includes any element other than <math>j</math> then <math>A</math> and <math>B</math> are still intersecting after <math>(i,j)</math>-shift. Thus without loss of generality, we may consider the only unsafe case where <math>A\cap B=\{j\}</math>, and <math>B</math> is successfully shifted to <math>B_{ij}=(B\setminus\{j\})\cup\{i\}</math> but <math>A</math> fails to shift to <math>A_{ij}=(A\setminus\{j\})\cup\{i\}</math>.<br />
<br />
Since <math>B</math> is successfully shifted to <math>B_{ij}</math>, we know that it must hold that <math>i\not\in B</math> and <math>B_{ij}\not\in\mathcal{F}</math>. And the only two reasons for which <math>A</math> may fail to shift to <math>A_{ij}</math> are: (1) <math>i\in A</math> and (2) <math>i\not\in A</math> but <math>A_{ij}\in\mathcal{F}</math>.<br />
<br />
Case.1: <math>i\in A</math>. In this case, since <math>i\in B_{ij}</math>, we have <math>S_{ij}(A)\cap S_{ij}(B)=A\cap B_{ij}=\{i\}</math>, i.e. <math>B</math> is still intersecting with <math>A</math> after shifting.<br />
<br />
Case.2: <math>i\not\in A</math> but <math>A_{ij}\in\mathcal{F}</math>. In this case, it is easy to verify that <math>A_{ij}\cap B=(A\cap B)\setminus\{j\}=\emptyset</math>. Recall that we assume <math>A_{ij}\in\mathcal{F}</math>. This contradicts to that <math>\mathcal{F}</math> is intersecting.<br />
<br />
In conclusion, in all cases, <math>S_{ij}(\mathcal{F})</math> remains intersecting.<br />
}}<br />
<br />
Repeatedly applying <math>S_{ij}(\mathcal{F})</math> for any <math>0\le i<j\le n-1</math>, since we only replace elements by smaller elements, eventually <math>\mathcal{F}</math> will stop changing, that is, <math>S_{ij}(\mathcal{F})=\mathcal{F}</math> for all <math>0\le i<j\le n-1</math>. We call such an <math>\mathcal{F}</math> '''shifted'''.<br />
<br />
The idea behind the shifting technique is very natural: by applying shifting, all intersecting families are transformed to some ''special forms'', and we only need to prove the theorem for these special form of intersecting families.<br />
<br />
{{Prooftitle|Proof of Erdős-Ko-Rado theorem| (The original proof of Erdős-Ko-Rado by shifting)<br />
By the above lemma, it is sufficient to prove the Erdős-Ko-Rado theorem holds for shifted <math>\mathcal{F}</math>. We assume that <math>\mathcal{F}</math> is shifted.<br />
<br />
First, it is trivial to see that the theorem holds for <math>k=1</math> (no matter whether shifted).<br />
<br />
Next, we show that the theorem holds when <math>n=2k</math> (no matter whether shifted). For any <math>S\in{X\choose k}</math>, both <math>S</math> and <math>X\setminus S</math> are in <math>{X\choose k}</math>, but at most one of them can be in <math>\mathcal{F}</math>. Thus,<br />
:<math>|\mathcal{F}|\le\frac{1}{2}{n\choose k}=\frac{n!}{2k!(n-k)!}=\frac{(n-1)!}{(k-1)!(n-k)!}={n-1\choose k-1}</math>.<br />
<br />
We then apply the induction on <math>n</math>. For <math>n> 2k</math>, the induction hypothesis is stated as:<br />
* the Erdős-Ko-Rado theorem holds for any smaller <math>n</math>.<br />
Define<br />
:<math>\mathcal{F}_0=\{S\in\mathcal{F}\mid n\not\in S\}</math> and <math>\mathcal{F}_1=\{S\in\mathcal{F}\mid n\in S\}</math>.<br />
Clearly, <math>\mathcal{F}_0\subseteq{[n-1]\choose k}</math> and <math>\mathcal{F}_0</math> is intersecting. Due to the induction hypothesis, <math>|\mathcal{F}_0|\le{n-2\choose k-1}</math>. <br />
<br />
In order to apply the induction, we let<br />
:<math>\mathcal{F}_1'=\{S\setminus\{n\}\mid S\in\mathcal{F}_1\}</math>.<br />
Clearly, <math>\mathcal{F}_1'\subseteq{[n-1]\choose k-1}</math>. If only it is also intersecting, we can apply the induction hypothesis, and indeed it is. To see this, by contradiction we assume that <math>\mathcal{F}_1'</math> is not intersecting. Then there must exist <math>A,B\in\mathcal{F}</math> such that <math>A\cap B=\{n\}</math>, which means that <math>|A\cup B|\le 2k-1<n-1</math>. Thus, there is some <math>0\le i\le n-1</math> such that <math>i\not\in A\cup B</math>. Since <math>\mathcal{F}</math> is shifted, <math>A_{in}=A\setminus\{n\}\cup\{i\}\in\mathcal{F}</math>. On the other hand it can be verified that <math>A_{in}\cap B=\emptyset</math>, which contradicts that <math>\mathcal{F}</math> is intersecting. <br />
<br />
Thus, <math>\mathcal{F}_1'\subseteq{[n-1]\choose k-1}</math> and <math>\mathcal{F}_1'</math> is intersecting. Due to the induction hypothesis, <math>|\mathcal{F}_1'|\le{n-2\choose k-2}</math>. <br />
<br />
Combining these together,<br />
:<math>|\mathcal{F}|=|\mathcal{F}_0|+|\mathcal{F}_1|=|\mathcal{F}_0|+|\mathcal{F}_1'|\le {n-2\choose k-1}+{n-2\choose k-2}={n-1\choose k-1}</math>.<br />
}}<br />
<br />
== Sperner system ==<br />
A set family <math>\mathcal{F}\subseteq 2^X</math> with the relation <math>\subseteq</math> define a poset. Thus, a '''chain''' is a sequence <math>S_1\subseteq S_2\subseteq\cdots\subseteq S_k</math>.<br />
<br />
A set family <math>\mathcal{F}\subseteq 2^X</math> is an '''antichain''' (also called a '''Sperner system''') if for all <math>S,T\in\mathcal{F}</math> that <math>S\neq T</math>, we have <math>S\not\subseteq T</math>.<br />
<br />
The <math>k</math>-uniform <math>{X\choose k}</math> is an antichain. Let <math>n=|X|</math>. The size of <math>{X\choose k}</math> is maximized when <math>k=\lfloor n/2\rfloor</math>. We wonder whether this is also the largest possible size of any antichain <math>\mathcal{F}\subseteq 2^X</math>.<br />
<br />
In 1928, Emanuel Sperner proved a theorem saying that it is indeed the largest possible antichain. This result, called Sperner's theorem today, initiated the studies of extremal set theory.<br />
<br />
{{Theorem|Theorem (Sperner 1928)|<br />
:Let <math>\mathcal{F}\subseteq 2^X</math> where <math>|X|=n</math>. If <math>\mathcal{F}</math> is an antichain, then<br />
::<math>|\mathcal{F}|\le{n\choose \lfloor n/2\rfloor}</math>.<br />
}}<br />
<br />
=== First proof (shadows)===<br />
We first introduce the original proof by Sperner, which uses concepts called '''shadows''' and '''shades''' of set systems.<br />
<br />
{{Theorem|Definition|<br />
:Let <math>|X|=n\,</math> and <math>\mathcal{F}\subseteq {X\choose k}</math>, <math>k<n\,</math>. <br />
:The '''shade''' of <math>\mathcal{F}</math> is defined to be<br />
::<math>\nabla\mathcal{F}=\left\{T\in {X\choose k+1}\,\,\bigg|\,\, \exists S\in\mathcal{F}\mbox{ such that } S\subset T\right\}</math>.<br />
:Thus the shade <math>\nabla\mathcal{F}</math> of <math>\mathcal{F}</math> consists of all subsets of <math>X</math> which can be obtained by adding an element to a set in <math>\mathcal{F}</math>.<br />
:Similarly, the '''shadow''' of <math>\mathcal{F}</math> is defined to be<br />
::<math>\Delta\mathcal{F}=\left\{T\in {X\choose k-1}\,\,\bigg|\,\, \exists S\in\mathcal{F}\mbox{ such that } T\subset S\right\}</math>.<br />
:Thus the shadow <math>\Delta\mathcal{F}</math> of <math>\mathcal{F}</math> consists of all subsets of <math>X</math> which can be obtained by removing an element from a set in <math>\mathcal{F}</math>.<br />
}}<br />
<br />
Next lemma bounds the effects of shadows and shades on the sizes of set systems.<br />
<br />
{{Theorem|Lemma (Sperner)|<br />
:Let <math>|X|=n\,</math> and <math>\mathcal{F}\subseteq {X\choose k}</math>. Then<br />
::<math><br />
\begin{align}<br />
&|\nabla\mathcal{F}|\ge\frac{n-k}{k+1}|\mathcal{F}| &\text{ if } k<n\\<br />
&|\Delta\mathcal{F}|\ge\frac{k}{n-k+1}|\mathcal{F}| &\text{ if } k>0.<br />
\end{align}<br />
</math><br />
}}<br />
<br />
{{Proof|<br />
The lemma is proved by double counting. We prove the inequality of <math>|\nabla\mathcal{F}|</math>. Assume that <math>0\le k<n</math>.<br />
<br />
Define<br />
:<math>\mathcal{R}=\{(S,T)\mid S\in\mathcal{F}, T\in\nabla\mathcal{F}, S\subset T\}</math>.<br />
We estimate <math>|\mathcal{R}|</math> in two ways. <br />
<br />
For each <math>S\in\mathcal{F}</math>, there are <math>n-k</math> different <math>T\in\nabla\mathcal{F}</math> that <math>S\subset T</math>.<br />
:<math>|\mathcal{R}|=(n-k)|\mathcal{F}|</math>.<br />
For each <math>T\in\nabla\mathcal{F}</math>, there are <math>k+1</math> ways to choose an <math>S\subset T</math> with <math>|S|=k</math>, some of which may not be in <math>\mathcal{F}</math>.<br />
:<math>|\mathcal{R}|\le (k+1)|\nabla\mathcal{F}|</math>.<br />
<br />
Altogether, we show that <math>|\nabla\mathcal{F}|\ge\frac{n-k}{k+1}|\mathcal{F}|</math>.<br />
<br />
The inequality of <math>|\Delta\mathcal{F}|</math> can be proved in the same way.<br />
}}<br />
<br />
An immediate corollary of the previous lemma is as follows.<br />
<br />
{{Theorem|Proposition 1|<br />
:If <math>k\le \frac{n-1}{2}</math>, then <math>|\nabla\mathcal{F}|\ge|\mathcal{F}|</math>.<br />
:If <math>k\ge \frac{n-1}{2}</math>, then <math>|\Delta\mathcal{F}|\ge|\mathcal{F}|</math>.<br />
}}<br />
<br />
The idea of Sperner's proof is pretty clear: <br />
* we "push up" all the sets in <math>\mathcal{F}</math> of size <math><\frac{n-1}{2}</math> replacing them by their shades; <br />
* and also "push down" all the sets in <math>\mathcal{F}</math> of size <math>\ge\frac{n+1}{2}</math> replacing them by their shadows. <br />
Repeat this process we end up with a set system <math>\mathcal{F}\subseteq{X\choose \lfloor n/2\rfloor}</math>. We need to show that this process does not decrease the size of <math>\mathcal{F}</math>.<br />
<br />
{{Theorem|Proposition 2|<br />
:Suppose that <math>\mathcal{F}\subseteq2^X</math> where <math>|X|=n</math>. Let <math>\mathcal{F}_k=\mathcal{F}\cap{X\choose k}</math>. Let <math>k_\min</math> be the smallest <math>k</math> that <math>|\mathcal{F}_k|>0</math>, and let<br />
::<math><br />
\mathcal{F}'=\begin{cases}<br />
\mathcal{F}\setminus\mathcal{F}_{k_\min}\cup \nabla\mathcal{F}_{k_\min} & \mbox{if }k_\min<\frac{n-1}{2},\\<br />
\mathcal{F} & \mbox{otherwise.}<br />
\end{cases}<br />
</math><br />
:Similarly, let <math>k_\max</math> be the largest <math>k</math> that <math>|\mathcal{F}_k|>0</math>, and let<br />
::<math><br />
\mathcal{F}''=\begin{cases}<br />
\mathcal{F}\setminus\mathcal{F}_{k_\max}\cup \Delta\mathcal{F}_{k_\max} & \mbox{if }k_\max\ge\frac{n+1}{2},\\<br />
\mathcal{F} & \mbox{otherwise.}<br />
\end{cases}<br />
</math><br />
:If <math>\mathcal{F}</math> is an antichain, <math>\mathcal{F}'</math> and <math>\mathcal{F}''</math> are antichains, and we have <math>|\mathcal{F}'|\ge|\mathcal{F}|</math> and <math>|\mathcal{F}''|\ge|\mathcal{F}|</math>.<br />
}}<br />
{{Proof|<br />
We show that <math>\mathcal{F}'</math> is an antichain and <math>|\mathcal{F}'|\ge|\mathcal{F}|</math>. <br />
<br />
First, observe that <math>\nabla\mathcal{F}_k\cap\mathcal{F}=\emptyset</math>, otherwise <math>\mathcal{F}</math> cannot be an antichain, and due to Proposition 1, <math>|\nabla\mathcal{F}_k|\ge|\mathcal{F}_k|</math> when <math>k\le \frac{n-1}{2}</math>, so <math>|\mathcal{F}'|=|\mathcal{F}|-|\mathcal{F}_k|+|\nabla\mathcal{F}_k|\ge |\mathcal{F}|</math>. <br />
<br />
Now we prove that <math>\mathcal{F}'</math> is an antichain . By contradiction, assume that there are <math>S, T\in \mathcal{F}'</math>, such that <math>S\subset T</math>. One of the <math>S,T</math> must be in <math>\nabla\mathcal{F}_{k_\min}</math>, or otherwise <math>\mathcal{F}</math> cannot be an antichain. Recall that <math>k_\min</math> is the smallest <math>k</math> that <math>|\mathcal{F}_k|>0</math>, thus it must be <math>S\in \nabla\mathcal{F}_{k_\min}</math>, and <math>T\in\mathcal{F}</math>. This implies that there is an <math>R\in \mathcal{F}_{k_\min}\subseteq \mathcal{F}</math> such that <math>R\subset S\subset T</math>, which contradicts that <math>\mathcal{F}</math> is an antichain.<br />
<br />
The statement for <math>\mathcal{F}''</math> can be proved in the same way.<br />
}}<br />
<br />
Applying the above process, we prove the Sperner's theorem.<br />
{{Prooftitle|Proof of Sperner's theorem | (original proof of Sperner)<br />
Let <math>\mathcal{F}_k=\{S\in\mathcal{F}\mid |S|=k\}</math>, where <math>0\le k\le n</math>.<br />
<br />
We change <math>\mathcal{F}</math> as follows: <br />
* for the smallest <math>k</math> that <math>|\mathcal{F}_k|>0</math>, if <math>k<\frac{n-1}{2}</math>, replace <math>\mathcal{F}_k</math> by <math>\nabla\mathcal{F}_k</math>.<br />
<br />
Due to Proposition 2, this procedure preserves <math>\mathcal{F}</math> as an antichain and does not decrease <math>|\mathcal{F}|</math>. Repeat this procedure, until <math>|\mathcal{F}_k|=0</math> for all <math>k<\frac{n-1}{2}</math>, that is, there is no member set of <math>\mathcal{F}</math> has size less than <math>\frac{n-1}{2}</math>.<br />
<br />
We then define another symmetric procedure:<br />
* for the largest <math>k</math> that <math>|\mathcal{F}_k|>0</math>, if <math>k\ge\frac{n+1}{2}</math>, replace <math>\mathcal{F}_k</math> by <math>\Delta\mathcal{F}_k</math>.<br />
Also due to Proposition 2, this procedure preserves <math>\mathcal{F}</math> as an antichain and does not decrease <math>|\mathcal{F}|</math>. After repeatedly applying this procedure, <math>|\mathcal{F}_k|=0</math> for all <math>k\ge\frac{n+1}{2}</math>. <br />
<br />
The resulting <math>\mathcal{F}</math> has <math>\mathcal{F}\subseteq{X\choose \lfloor n/2\rfloor}</math>, and since <math>|\mathcal{F}|</math> is never decreased, for the original <math>\mathcal{F}</math>, we have<br />
:<math>|\mathcal{F}|\le {n\choose \lfloor n/2\rfloor}</math>.<br />
}}<br />
<br />
=== Second proof (counting)===<br />
We now introduce an elegant proof due to Lubell. The proof uses a counting argument, and tells more information than just the size of the set system.<br />
<br />
{{Prooftitle|Proof of Sperner's theorem | (Lubell 1966)<br />
Let <math>\pi</math> be a permutation of <math>X</math>. We say that an <math>S\subseteq X</math> '''prefixes''' <math>\pi</math>, if <math>S=\{\pi_1,\pi_2,\ldots, \pi_{|S|}\}</math>, that is, <math>S</math> is precisely the set of the first <math>|S|</math> elements in the permutation <math>\pi</math>.<br />
<br />
Fix an <math>S\subseteq X</math>. It is easy to see that the number of permutations <math>\pi</math> of <math>X</math> prefixed by <math>S</math> is <math>|S|!(n-|S|)!</math>. Also, since <math>\mathcal{F}</math> is an antichain, no permutation <math>\pi</math> of <math>X</math> can be prefixed by more than one members of <math>\mathcal{F}</math>, otherwise one of the member sets must contain the other, which contradicts that <math>\mathcal{F}</math> is an antichain. Thus, the number of permutations <math>\pi</math> prefixed by some <math>S\in\mathcal{F}</math> is <br />
:<math>\sum_{S\in\mathcal{F}}|S|!(n-|S|)!</math>,<br />
which cannot be larger than the total number of permutations, <math>n!</math>, therefore,<br />
:<math>\sum_{S\in\mathcal{F}}|S|!(n-|S|)!\le n!</math>.<br />
Dividing both sides by <math>n!</math>, we have<br />
:<math>\sum_{S\in\mathcal{F}}\frac{1}{{n\choose |S|}}=\sum_{S\in\mathcal{F}}\frac{|S|!(n-|S|)!}{n!}\le 1</math>,<br />
where <math>{n\choose |S|}\le {n\choose \lfloor n/2\rfloor}</math>, so<br />
:<math>\sum_{S\in\mathcal{F}}\frac{1}{{n\choose |S|}}\ge \frac{|\mathcal{F}|}{{n\choose \lfloor n/2\rfloor}}</math>.<br />
Combining this with the above inequality, we prove the Sperner's theorem.<br />
}}<br />
<br />
=== The LYM inequality ===<br />
Lubell's proof proves the following inequality:<br />
:<math>\sum_{S\in\mathcal{F}}\frac{1}{{n\choose |S|}}\le 1</math><br />
which is actually stronger than Sperner's original statement that <math>|\mathcal{F}|\le{n\choose \lfloor n/2\rfloor}</math>.<br />
<br />
This inequality is independently discovered by Lubell-Yamamoto, Meschalkin, and Bollobás, and is called the LYM inequality today.<br />
<br />
{{Theorem|Theorem (Lubell, Yamamoto 1954; Meschalkin 1963)|<br />
:Let <math>\mathcal{F}\subseteq 2^X</math> where <math>|X|=n</math>. If <math>\mathcal{F}</math> is an antichain, then<br />
::<math>\sum_{S\in\mathcal{F}}\frac{1}{{n\choose |S|}}\le 1</math>.<br />
}}<br />
<br />
In Lubell's counting argument proves the LYM inequality, which implies the Sperner's theorem. Here we give another proof of the LYM inequality by the probabilistic method, due to Noga Alon.<br />
<br />
{{Prooftitle|Third proof (the probabilistic method)| (Due to Alon.)<br />
Let <math>\pi</math> be a uniformly random permutation of <math>X</math>. Define a random maximal chain by<br />
:<math>\mathcal{C}_\pi=\{\{\pi_i\mid 1\le i\le k\}\mid 0\le k\le n\}</math>.<br />
For any <math>S\in\mathcal{F}</math>, let <math>X_S</math> be the 0-1 random variable which indicates whether <math>S\in\mathcal{C}_\pi</math>, that is<br />
:<math><br />
X_S=\begin{cases}<br />
1 & \mbox{if }S\in\mathcal{C}_\pi,\\<br />
0 & \mbox{otherwise.}<br />
\end{cases}<br />
</math><br />
Note that for a uniformly random <math>\pi</math>, <math>\mathcal{C}_\pi</math> has exact one member set of size <math>|S|</math>, uniformly distributed over <math>{X\choose |S|}</math>, thus<br />
:<math>\mathbf{E}[X_S]=\Pr[S\in\mathcal{C}_\pi]=\frac{1}{{n\choose |S|}}</math>.<br />
Let <math>X=\sum_{S\in\mathcal{F}}X_S</math>. Note that <math>X=|\mathcal{F}\cap\mathcal{C}_\pi|</math>. By the linearity of expectation,<br />
:<math>\mathbf{E}[X]=\sum_{S\in\mathcal{F}}\mathbf{E}[X_S]=\sum_{S\in\mathcal{F}}\frac{1}{{n\choose |S|}}</math>.<br />
On the other hand, since <math>\mathcal{F}</math> is an antichain, it can never intersect a chain at more than one elements, thus we always have <math>X=|\mathcal{F}\cap\mathcal{C}_\pi|\le 1</math>. Therefore,<br />
:<math>\sum_{S\in\mathcal{F}}\frac{1}{{n\choose |S|}}\le \mathbf{E}[X] \le 1</math>.<br />
}}<br />
<br />
The Sperner's theorem is an immediate consequence of the LYM inequality.<br />
<br />
{{Theorem|Proposition|<br />
:<math>\sum_{S\in\mathcal{F}}\frac{1}{{n\choose |S|}}\le 1</math> implies that <math>|\mathcal{F}|\le{n\choose \lfloor n/2\rfloor}</math>.<br />
}}<br />
{{Proof|<br />
It holds that <math>{n\choose k}\le {n\choose \lfloor n/2\rfloor}</math> for any <math>k</math>. Thus,<br />
:<math>1\ge \sum_{S\in\mathcal{F}}\frac{1}{{n\choose |S|}}\ge \frac{|\mathcal{F}|}{{n\choose \lfloor n/2\rfloor}}</math>,<br />
which implies that <math>|\mathcal{F}|\le {n\choose \lfloor n/2\rfloor}</math>.<br />
}}<br />
<br />
== Sauer's lemma and VC-dimension ==<br />
<br />
=== Shattering and the VC-dimension ===<br />
{{Theorem|Definition (shatter)|<br />
:Let <math>\mathcal{F}\subseteq 2^X</math> be set family and let <math>R\subseteq X</math> be a subset. The '''trace''' of <math>\mathcal{F}</math> on <math>R</math>, denoted <math>\mathcal{F}|_R</math> is defined as<br />
::<math>\mathcal{F}|_R=\{S\cap R\mid S\in\mathcal{F}\}</math>.<br />
:We say that <math>\mathcal{F}</math> '''shatters''' <math>R</math> if <math>\mathcal{F}|_R=2^R</math>, i.e. for all <math>T\subseteq R</math>, there exists an <math>S\in\mathcal{F}</math> such that <math>T=S\cap R</math>.<br />
}}<br />
<br />
The [http://en.wikipedia.org/wiki/VC_dimension '''VC dimension'''] is defined by the power of a family to shatter a set.<br />
<br />
{{Theorem|Definition (VC-dimension)|<br />
:The '''Vapnik–Chervonenkis dimension''' ('''VC-dimension''') of a set family <math>\mathcal{F}\subseteq 2^X</math>, denoted <math>\text{VC-dim}(\mathcal{F})</math>, is the size of the largest <math>R\subseteq X</math> shattered by <math>\mathcal{F}</math>.<br />
}}<br />
<br />
It is a core concept in [http://en.wikipedia.org/wiki/Computational_learning_theory computational learning theory].<br />
<br />
Each subset <math>S\subseteq X</math> can be equivalently represented by its characteristic function <math>f_S:X\rightarrow\{0,1\}</math>, such that for each <math>x\in X</math>,<br />
:<math>f_S(x)=\begin{cases}<br />
1 & x\in S\\<br />
0 & x\not\in S.<br />
\end{cases}</math><br />
<br />
Then a set family <math>\mathcal{F}\subseteq2^X</math> corresponds to a collection of boolean functions <math>\{f_S\mid S\in\mathcal{F}\}</math>, which is a subset of all Boolean functions in the form <math>f:X\rightarrow\{0,1\}</math>. We wonder on how large a subdomain <math>Y\subseteq X</math>, <math>\mathcal{F}</math> includes all the <math>2^{|Y|}</math> mappings <math>Y\rightarrow\{0,1\}</math>. The largest size of such subdomain is the VC-dimension. It measures how complicated a collection of boolean functions (or equivalently a set family) is.<br />
<br />
=== Sauer's Lemma ===<br />
The definition of the VC-dimension involves enumerating all subsets, thus is difficult to analyze in general. The following famous result state a very simple sufficient condition to lower bound the VC-dimension, regarding only the size of the family. The lemma is due to Sauer, and independently due to Shelah and Perles. A slightly weaker version is found by Vapnik and Chervonenkis, who use the framework to develop a theory of classifications.<br />
<br />
{{Theorem|Sauer's Lemma (Sauer; Shelah-Perles; Vapnik-Chervonenkis)|<br />
:Let <math>\mathcal{F}\subseteq 2^X</math> where <math>|X|=n</math>. If <math>|\mathcal{F}|>\sum_{1\le i<k}{n\choose i}</math>, then there exists an <math>R\in{X\choose k}</math> such that <math>\mathcal{F}</math> shatters <math>R</math>.<br />
}}<br />
In other words, for any set family <math>\mathcal{F}</math> with <math>|\mathcal{F}|>\sum_{1\le i<k}{n\choose i}</math>, its VC-dimension <math>\text{VC-dim}(\mathcal{F})\ge k</math>.<br />
<br />
=== Hereditary family ===<br />
We note the Sauer's lemma is especially easy to prove for a special type of set families, called the '''hereditary''' families.<br />
{{Theorem|Definition (hereditary family)|<br />
:A set system <math>\mathcal{F}\subseteq 2^X</math> is said to be '''hereditary''' (also called an '''ideal''' or an '''abstract simplicial complex'''), if<br />
::<math>S\subseteq T\in\mathcal{F}</math> implies <math>S\in\mathcal{F}</math>.<br />
}}<br />
In other words, for a hereditary family <math>\mathcal{F}</math>, if <math>R\in\mathcal{F}</math>, then all subsets of <math>R</math> are also in <math>\mathcal{F}</math>. An immediate consequence is the following proposition.<br />
{{Theorem|Proposition|<br />
:Let <math>\mathcal{F}</math> be a hereditary family. If <math>R\in\mathcal{F}</math> then <math>\mathcal{F}</math> shatters <math>R</math>.<br />
}}<br />
<br />
Therefore, it is very easy to prove the Sauer's lemma for hereditary families: <br />
{{Theorem|Lemma|<br />
:For <math>\mathcal{F}\subseteq 2^X</math>, <math>|X|=n</math>, if <math>\mathcal{F}</math> is hereditary and <math>|\mathcal{F}|>\sum_{1\le i<k}{n\choose i}</math> then there exists an <math>R\in{X\choose k}</math> such that <math>\mathcal{F}</math> shatters <math>R</math>.<br />
}}<br />
{{Proof|<br />
Since <math>\mathcal{F}</math> is hereditary, we only need to show that there exists an <math>R\in\mathcal{F}</math> of size <math>|R|\ge k</math>, which must be true, because if all members of <math>\mathcal{F}</math> are of sizes <math><k</math>, then <math>|\mathcal{F}|\le\left|\bigcup_{1\le i<k}{X\choose k}\right|=\sum_{1\le i<k}{n\choose i}</math>, a contradiction.<br />
}}<br />
<br />
To prove the Sauer's lemma for general non-hereditary families, we can use some way to reduce arbitrary families to hereditary families. Here we apply the shifting technique to achieve this.<br />
<br />
=== Down-shifts ===<br />
Note that we work on <math>\mathcal{F}\subseteq2^X</math>, instead of <math>\mathcal{F}\subseteq{X\choose k}</math> like in the Erdős–Ko–Rado theorem, so we do not need to preserve the size of member sets. Instead, we need to reduce an arbitrary family to a hereditary one, thus we use a shift operator which replaces a member set by a subset of it.<br />
{{Theorem|Definition (down-shifts)|<br />
: Assume <math>\mathcal{F}\subseteq 2^{[n]}</math>, and <math>i\in[n]</math>. Define the '''down-shift''' operator <math>S_{i}</math> as follows:<br />
:* for each <math>T\in\mathcal{F}</math>, let<br />
::<math>S_{i}(T)=<br />
\begin{cases}<br />
T\setminus\{i\} & \mbox{if }i\in T \mbox{ and }T\setminus\{i\} \not\in\mathcal{F},\\<br />
T & \mbox{otherwise;}<br />
\end{cases}</math><br />
:* let <math>S_{i}(\mathcal{F})=\{S_{i}(T)\mid T\in \mathcal{F}\}</math>.<br />
}}<br />
<br />
Repeatedly applying <math>S_i</math> to <math>\mathcal{F}</math> for all <math>i\in[n]</math>, due to the finiteness, eventually <math>\mathcal{F}</math> is not changed by any <math>S_i</math>. We call such a family '''down-shifted'''. A family <math>\mathcal{F}</math> is down-shifted if and only if <math>S_i(\mathcal{F})=\mathcal{F}</math> for all <math>i\in[n]</math>. It is then easy to see that a down-shifted <math>\mathcal{F}</math> must be hereditary.<br />
<br />
{{Theorem|Theorem|<br />
:If <math>\mathcal{F}\subseteq2^X</math> is down-shifted, then <math>\mathcal{F}</math> is hereditary.<br />
}}<br />
<br />
In order to use down-shift to prove the Sauer's lemma, we need to make sure that down-shift does not decrease <math>|\mathcal{F}|</math> and does not increase the VC-dimension <math>\text{VC-dim}(\mathcal{F})</math>.<br />
<br />
{{Theorem|Proposition|<br />
# <math>|S_{i}(\mathcal{F})|=\mathcal{F}</math>;<br />
# <math>|S_i(\mathcal{F})|_R|\le |\mathcal{F}|_R|</math>, thus if <math>S_{i}(\mathcal{F})</math> shatters an <math>R</math>, so does <math>\mathcal{F}</math>.<br />
}}<br />
(1) is immediate. (2) is proved by case analysis. We omit the proof.<br />
<br />
;Proof of Sauer's lemma<br />
Now we can prove the Sauer's lemma for arbitrary <math>\mathcal{F}\subseteq 2^{[n]}</math>. <br />
<br />
For any <math>\mathcal{F}\subseteq 2^{[n]}</math>, repeatedly apply <math>S_i</math> for all <math>i\in</math> till the family is down-shifted, which is denoted by <math>\mathcal{F}'</math>. We have proved that <math>|\mathcal{F}'|=|\mathcal{F}|>\sum_{1\le i<k}{n\choose i}</math> and <math>\mathcal{F}'</math> is hereditary, thus as argued before, here exists an <math>R</math> of size <math>k</math> shattered by <math>\mathcal{F}'</math>. By the above proposition, <math>|\mathcal{F}'|_R|\le |\mathcal{F}|_R|</math>, thus <math>\mathcal{F}</math> also shatters <math>R</math>. The lemma is proved.<br />
<br />
<math>\square</math><br />
<br />
<br />
== The Kruskal–Katona Theorem ==<br />
The '''shadow''' of a set system <math>\mathcal{F}</math>, denoted <math>\Delta\mathcal{F}</math>, consists of all sets which can be obtained by removing an element from a set in <math>\mathcal{F}</math>.<br />
<br />
{{Theorem|Definition|<br />
:Let <math>\mathcal{F}\subseteq {X\choose k}</math>. The '''shadow''' of <math>\mathcal{F}</math> is defined to be<br />
::<math>\Delta\mathcal{F}=\left\{T\in {X\choose k-1}\,\,\bigg|\,\, \exists S\in\mathcal{F}\mbox{ such that } T\subset S\right\}</math>.<br />
}}<br />
<br />
The shadow contains rich information about the set system. An extremal question is: for a system of fixed number of <math>k</math>-sets, how small can its shadow be? The Kruskal–Katona theorem gives an answer to this question.<br />
<br />
To state the result of the Kruskal–Katona theorem, we need to introduce the concepts of the <math>k</math>-cascade representation of numbers and the colex order of sets.<br />
<br />
=== <math>k</math>-cascade representation of a number ===<br />
{{Theorem|Theorem|<br />
:Given positive integers <math>m</math> and <math>k</math>, there exists a unique representation of <math>m</math> in the form<br />
::<math>m={m_k\choose k}+{m_{k-1}\choose k-1}+\cdots+{m_t\choose t}=\sum_{\ell=t}^k{m_\ell\choose \ell}</math>,<br />
:where <math>m_k>m_{k-1}>\cdots>m_t\ge t\ge 1</math>.<br />
:This representation of <math>m</math> is called a '''<math>k</math>-cascade''' (or a '''<math>k</math>-binomial''') representation of <math>m</math>. <br />
}}<br />
{{Proof|<br />
In fact, the <math>k</math>-cascade representation <math>(m_k,m_{k-1},\ldots,m_t)</math> of an <math>m</math> can be found by the following simple greedy algorithm:<br />
----<br />
:<math>\ell=k;</math><br />
:while (<math>m>0</math>) do<br />
::let <math>m_\ell</math> be the largest integer for which <math>{m_\ell\choose \ell}\le m;</math><br />
::<math>m=m-m_\ell;</math><br />
::<math>\ell=\ell-1;</math><br />
----<br />
We then show that the above algorithm constructs a sequence <math>(m_k,m_{k-1},\ldots,m_t)</math> with <math>m_k>m_{k-1}>\cdots>m_t\ge t\ge 1</math>.<br />
<br />
Suppose the current <math>\ell</math> and the current <math>m</math>. To see that <math>m_{\ell-1}< m_\ell</math>, we suppose otherwise <math>m_{\ell-1}\ge m_\ell</math>. Then <br />
:<math>m\ge {m_\ell\choose \ell}+{m_{\ell-1}\choose \ell-1}\ge{m_\ell\choose \ell}+{m_{\ell}\choose \ell-1}={1+m_{\ell}\choose \ell}</math><br />
contradicting the maximality of <math>m_\ell</math>. Therefore, <math>m_\ell>m_{\ell-1}</math>.<br />
<br />
The algorithm continues reducing <math>m</math> to smaller nonnegative values, and eventually reaches a stage where the choice of <math>m_t</math> for some <math>t\ge 2</math> where <math>{m_t\choose t}</math> equals the current <math>m</math>; or gets right down to choosing <math>m_1</math> as the integer such that <math>m_1={m_1\choose 1}</math> equals the current <math>m</math>.<br />
<br />
Therefore, <math>m={m_k\choose k}+{m_{k-1}\choose k-1}+\cdots+{m_t\choose t}</math> and <math>m_k>m_{k-1}>\cdots>m_t\ge t\ge 1</math>.<br />
<br />
The uniqueness of the <math>k</math>-cascade representation follows by the induction on <math>k</math>.<br />
<br />
When <math>k=1</math>, <math>m</math> has a unique <math>k</math>-cascade representation <math>m={m\choose 1}</math>.<br />
<br />
For general <math>k>1</math>, suppose that every nonnegative integer <math>m</math> has a unique <math>(k-1)</math>-cascade representation.<br />
<br />
Suppose then that <math>m</math> has two <math>k</math>-cascade representations:<br />
:<math>m={a_k\choose k}+{a_{k-1}\choose k-1}+\cdots+{a_t\choose t}={b_k\choose k}+{b_{k-1}\choose k-1}+\cdots+{b_r\choose r}</math>.<br />
We then show that it must hold that <math>a_k=b_k</math>.<br />
If <math>a_k\neq b_k</math>, WLOG, suppose that <math>a_k<b_k</math>. We obtain<br />
:<math><br />
\begin{align}<br />
m&<br />
={a_k\choose k}+{a_{k-1}\choose k-1}+\cdots+{a_t\choose t}\\<br />
&\le {a_k\choose k}+{a_{k}-1\choose k-1}+\cdots+{a_k-(k-t)\choose t}+\cdots+{a_k-k+1\choose 1}\\<br />
&={a_k+1\choose k}-1,<br />
\end{align}</math><br />
where the last equation is got by repeatedly applying the identity<br />
:<math>{n\choose k}+{n\choose k-1}={n+1\choose k}</math>.<br />
We then obtain <br />
:<math>m<{a_k+1\choose k}\le{b_k\choose k}\le m</math>, <br />
which is a contradiction. Therefore, <math>a_k=b_k</math>, and by the induction hypothesis, the remaining value <math>m-a_k=m-b_k</math> has a unique <math>(k-1)</math>-cascade representation, so <math>a_i=b_i</math> for all <math>i</math>.<br />
}}<br />
<br />
=== Co-lexicographic order of subsets ===<br />
The co-lexicographic order of sets plays a particularly important role in the investigation of the size of the shadow of a system of <math>k</math>-sets.<br />
{{Theorem|Definition|<br />
: The '''co-lexicographic (colex) order''' (also called the '''reverse lexicographic order''') of sets is defined as follows: for any <math>A,B\subseteq [n]</math>, <math>A\neq B</math>,<br />
::<math>A<B</math> if <math>\max A\setminus B < \max B\setminus A</math>.<br />
}}<br />
<br />
We can sort sets in colex order by first writing each set as a tuple, whose elements are in decreasing order, and then sorting the tuples in the lexicographic order of tuples.<br />
<br />
For example, <math>{[5]\choose 3}</math> in colex order is<br />
{3,2,1}<br />
{4,2,1}<br />
{4,3,1}<br />
{4,3,2}<br />
{5,2,1}<br />
{5,3,1}<br />
{5,3,2}<br />
{5,4,1}<br />
{5,4,2}<br />
{5,4,3}<br />
<br />
We find that the first <math>{n\choose 3}</math> sets in this order for <math>n=3,4,5</math>, form precisely <math>{[n]\choose 3}</math>. And if we write the colex order of <math>{[6]\choose 3}</math>, the above colex order of <math>{[5]\choose 3}</math> appears as a prefix of that order. Elaborating on this, we have:<br />
{{Theorem|Proposition|<br />
:Let <math>\mathcal{R}(m,k)</math> be the first <math>m</math> sets in the colex order of <math>{\mathbb{N}\choose k}</math>. Then <br />
::<math>\mathcal{R}\left({n\choose k},k\right)={[n]\choose k}</math>,<br />
:that is, the first <math>{n\choose k}</math> sets in the colex order of all <math>k</math>-sets of natural numbers is precisely <math>{[n]\choose k}</math>.<br />
}}<br />
<br />
This proposition says that the sets in <math>\mathcal{R}(m,k)</math> is highly overlapped, which suggests that <math>\mathcal{R}(m,k)</math> may have small shadow. The size of the shadow of <math>\mathcal{R}(m,k)</math> is closely related to the <math>k</math>-cascade representation of <math>m</math>.<br />
<br />
{{Theorem|Theorem|<br />
:Suppose the <math>k</math>-cascade representation of <math>m</math> is <br />
::<math>m={m_k\choose k}+{m_{k-1}\choose k-1}+\cdots+{m_t\choose t}</math>.<br />
:Then<br />
::<math>|\Delta\mathcal{R}(m,k)|={m_k\choose k-1}+{m_{k-1}\choose k-2}+\cdots+{m_t\choose t-1}</math>.<br />
}}<br />
{{Proof|<br />
Given <math>m</math> and its <math>k</math>-cascade representation <math>m={m_k\choose k}+{m_{k-1}\choose k-1}+\cdots+{m_t\choose t}</math>, the <math>\mathcal{R}(m,k)</math> is constructed as:<br />
* all sets in <math>{[m_k]\choose k}</math>;<br />
* all sets in <math>{[m_{k-1}]\choose k-1}</math>, unioned with <math>\{1+m_k\}\,</math>;<br />
::<math>\vdots</math><br />
* all sets in <math>{[m_{t}]\choose t}</math>, unioned with <math>\{1+m_k,1+m_{k-1},\ldots,1+m_{t+1}\}</math>.<br />
The shadow <math>\Delta\mathcal{R}(m,k)</math> is the collection of all <math>(k-1)</math>-sets contained by the above sets, which are<br />
* all sets in <math>{[m_k]\choose k-1}</math>;<br />
* all sets in <math>{[m_{k-1}]\choose k-2}</math>, unioned with <math>\{1+m_k\}\,</math>;<br />
::<math>\vdots</math><br />
* all sets in <math>{[m_{t}]\choose t-1}</math>, unioned with <math>\{1+m_k,1+m_{k-1},\ldots,1+m_{t+1}\}</math>.<br />
Thus,<br />
:<math>|\Delta\mathcal{R}(m,k)|={m_k\choose k-1}+{m_{k-1}\choose k-2}+\cdots+{m_t\choose t-1}</math>.<br />
}}<br />
<br />
=== The Kruskal–Katona theorem ===<br />
The Kruskal–Katona theorem states that among all systems of <math>m</math> <math>k</math>-sets, <math>\mathcal{R}(m,k)</math>, i.e., the first <math>m</math> <math>k</math>-sets in the colex order, has the smallest shadow.<br />
<br />
The theorem is proved independently by Joseph Kruskal in 1963 and G.O.H. Katona in 1966, and is a fundamental result in finite set theory and combinatorial topology.<br />
{{Theorem|Theorem (Kruskal 1963, Katona 1966)|<br />
:Let <math>\mathcal{F}\subseteq {X\choose k}</math> with <math>|\mathcal{F}|=m</math>, and suppose that<br />
::<math>m={m_k\choose k}+{m_{k-1}\choose k-1}+\cdots+{m_t\choose t}</math><br />
:where <math>m_k>m_{k-1}>\cdots>m_t\ge t\ge 1</math>. Then<br />
::<math>|\Delta\mathcal{F}|\ge {m_k\choose k-1}+{m_{k-1}\choose k-2}+\cdots+{m_t\choose t-1}</math>.<br />
}}<br />
<br />
The <math>f</math>-vector of a set system <math>\mathcal{F}\subseteq 2^X</math> is a vector <math>(|\mathcal{F}_0|,|\mathcal{F}_1|,\ldots,|\mathcal{F}_n|)</math> where<br />
:<math>\mathcal{F}_k=\{S\mid S\in\mathcal{F}, |S|=k\}</math>,<br />
i.e., the <math>f</math>-vector gives the number of member sets of each size.<br />
<br />
In a hereditary family (also called an [http://en.wikipedia.org/wiki/Abstract_simplicial_complex abstract simplicial complex]), <math>\mathcal{F}_k</math> is formed by the shadow of <math>\mathcal{F}_{k+1}</math> as well as some additional <math>k</math>-sets introduced in this level. The Kruskal-Katona theorem gives a lower bound on <math>|\mathcal{F}_i|</math>, given the <math>|\mathcal{F}_{i-1}|</math>.<br />
<br />
The original proof of the theorem is rather complicated. In the following years, several different proofs were discovered. Here we present a proof dueto Frankl by the shifting technique.<br />
<br />
;Frankl's shifting proof of Kruskal-Katonal theorem (Frankl 1984)<br />
<br />
We take the classic <math>(i,j)</math>-shift operator <math>S_{ij}</math> defined in the original proof of the Erdős-Ko-Rado theorem.<br />
{{Theorem|Definition (<math>(i,j)</math>-shift)|<br />
: Assume <math>\mathcal{F}\subseteq 2^{[n]}</math>, and <math>0\le i<j\le n-1</math>. Define the '''<math>(i,j)</math>-shift''' <math>S_{ij}</math> as an operator on <math>\mathcal{F}</math> as follows:<br />
:*for each <math>T\in\mathcal{F}</math>, write <math>T_{ij}=(T\setminus\{j\})\cup\{i\} </math>, and let<br />
::<math>S_{ij}(T)=<br />
\begin{cases}<br />
T_{ij} & \mbox{if }j\in T, i\not\in T, \mbox{ and }T_{ij} \not\in\mathcal{F},\\<br />
T & \mbox{otherwise;}<br />
\end{cases}</math><br />
:* let <math>S_{ij}(\mathcal{F})=\{S_{ij}(T)\mid T\in \mathcal{F}\}</math>.<br />
}}<br />
<br />
It is immediate that shifting does not change the size of the set or the size of the system, i.e., <math>|S_{ij}(T)|=|T|\,</math> and <math>|S_{ij}(\mathcal{F})|=\mathcal{F}</math>. And due to the finiteness, repeatedly applying <math>(i,j)</math>-shifts for any <math>1\le i<j\le n</math>, eventually <math>\mathcal{F}</math> does not changing any more. We called such an <math>\mathcal{F}</math> with <math>\mathcal{F}=S_{ij}(\mathcal{F})</math> for any <math>1\le i<j\le n</math> '''shifted'''.<br />
<br />
In order to make the shifting technique work for shadows, we have to prove that shifting does not increase the size of the shadow.<br />
<br />
{{Theorem|Proposition|<br />
:<math>\Delta S_{ij}(\mathcal{F})\subseteq S_{ij}(\Delta\mathcal{F})</math>.<br />
}}<br />
{{Proof|<br />
We abuse the notation <math>\Delta</math> and let <math>\Delta A=\Delta\{A\}</math> if <math>A</math> is a set instead of a set system.<br />
<br />
It is sufficient to show that for any <math>A\in\mathcal{F}</math>, <math>\Delta S_{ij}(A)\subseteq S_{ij}(\Delta\mathcal{F})</math>, which can be proved by case analysis. We omit the proof.<br />
}}<br />
<br />
An immediate corollary of the above proposition is that the <math>(i,j)</math>-shifts <math>S_{ij}</math> for any <math>1\le i<j\le n</math> do not increase the size of the shadow.<br />
<br />
{{Theorem|Corollary|<br />
:<math>|\Delta S_{ij}(\mathcal{F})|\le |\Delta\mathcal{F}|</math>.<br />
}}<br />
{{Proof|<br />
By the above proposition, <math>|\Delta S_{ij}(\mathcal{F})|\le|S_{ij}(\Delta\mathcal{F})|</math>, and we know that <math>S_{ij}</math> does not change the cardinality of a set family, that is, <math>|S_{ij}(\Delta\mathcal{F})|=|\Delta\mathcal{F}|</math>, therefore <math>\Delta S_{ij}(\mathcal{F})|\le|\Delta\mathcal{F}|</math>.<br />
}}<br />
<br />
;Proof of Kruskal-Katona theorem<br />
We know that shifts never enlarge the shadow, thus it is sufficient to prove the theorem for shifted <math>\mathcal{F}</math>. We then assume <math>\mathcal{F}</math> is shifted.<br />
<br />
Apply induction on <math>m</math> and for given <math>m</math> on <math>k</math>. The theorem holds trivially for the case that <math>k=1</math> and <math>m</math> is arbitrary.<br />
<br />
Define <br />
:<math>\mathcal{F}_0=\{A\in\mathcal{F}\mid 1\not\in A\}</math>, <br />
:<math>\mathcal{F}_1=\{A\in\mathcal{F}\mid 1\in A\}</math>.<br />
And let <math>\mathcal{F}_1'=\{A\setminus\{1\}\mid A\in\mathcal{F}_1\}</math>.<br />
<br />
Clearly <math>\mathcal{F}_0,\mathcal{F}_1\subseteq{X\choose k}</math>, <math>\mathcal{F}_1'\subseteq{X\choose k-1}</math>, and<br />
:<math>|\mathcal{F}|=|\mathcal{F}_0|+|\mathcal{F}_1|=|\mathcal{F}_0|+|\mathcal{F}_1'|</math>.<br />
<br />
Our induction is based on the following observation regarding the size of the shadow.<br />
{{Theorem|Lemma 1|<br />
:<math>|\Delta\mathcal{F}|\ge|\Delta\mathcal{F}_1'|+|\mathcal{F}_1'|</math>.<br />
}}<br />
{{Proof|<br />
Obviously <math>\mathcal{F}\supseteq\mathcal{F}_1</math> and <math>\Delta\mathcal{F}\supseteq\Delta\mathcal{F}_1</math>.<br />
:<math>\begin{align}<br />
\Delta\mathcal{F}_1<br />
&=\left\{A\in{X\choose k-1}\,\,\bigg|\,\, \exists B\in\mathcal{F}_1, A\subset B\right\}\\<br />
&=\left\{A\in{X\choose k-1}\,\,\bigg|\,\, 1\in A, \exists B\in\mathcal{F}_1, A\subset B\right\}\\<br />
&\quad\, \cup \left\{A\in{X\choose k-1}\,\,\bigg|\,\, 1\not\in A, \exists B\in\mathcal{F}_1, A\subset B\right\}\\<br />
&=\{S\cup\{1\}\mid S\in\Delta\mathcal{F}_1'\}\cup\mathcal{F}_1'.<br />
\end{align}</math><br />
The union is taken over two disjoint families. Therefore,<br />
:<math>|\Delta\mathcal{F}|\ge|\Delta\mathcal{F}_1|=|\Delta\mathcal{F}_1'|+|\mathcal{F}_1'|</math>.<br />
}}<br />
<br />
The following property of shifted <math>\mathcal{F}</math> is essential for our proof.<br />
{{Theorem|Lemma 2|<br />
:For shifted <math>\mathcal{F}</math>, it holds that <math>\Delta\mathcal{F}_0\subseteq \mathcal{F}_1'</math>.<br />
}}<br />
{{Proof|<br />
If <math>A\in\Delta\mathcal{F}_0</math> then <math>A\cup\{j\}\in\mathcal{F}_0\subseteq\mathcal{F}</math> for some <math>j>1</math> so that, since <math>\mathcal{F}</math> is shifted, applying the <math>(1,j)</math>-shift <math>S_{1j}</math>, <math>A\cup\{1\}\in\mathcal{F}</math>, thus, <math>A\in\mathcal{F}_1'</math>.<br />
}}<br />
<br />
We then bound the size of <math>\mathcal{F}_1'</math> as:<br />
<br />
{{Theorem|Lemma 3|<br />
:If <math>\mathcal{F}</math> is shifted, then<br />
::<math>|\mathcal{F}_1'|\ge{m_k-1\choose k-1}+{m_{k-1}-1\choose k-2}+\cdots+{m_t-1\choose t-1}</math>.<br />
}}<br />
{{Proof|<br />
By contradiction, assume that<br />
:<math>|\mathcal{F}_1'|<{m_k-1\choose k-1}+{m_{k-1}-1\choose k-2}+\cdots+{m_t-1\choose t-1}</math>.<br />
Then by <math>|\mathcal{F}|=|\mathcal{F}_0|+|\mathcal{F}_1'|</math>, it holds that<br />
:<math><br />
\begin{align}<br />
|\mathcal{F}_0| &=m-|\mathcal{F}_1'|\\<br />
&>\left\{{m_k\choose k}- {m_k-1\choose k-1}\right\}+\cdots+\left\{{m_t\choose t}- {m_t-1\choose t-1}\right\}\\<br />
&={m_k-1\choose k}+\cdots+{m_t-1\choose t},<br />
\end{align}</math><br />
so that, by the induction hypothesis, <br />
:<math>|\Delta\mathcal{F}_0|\ge{m_k-1\choose k-1}+{m_{k-1}-1\choose k-1}+\cdots+{m_t-1\choose t-1}>|\mathcal{F}_1'|</math>.<br />
On the other hand, by Lemma 2, <math>|\mathcal{F}_1'|\ge|\Delta\mathcal{F}_0|</math>. Thus <math>|\mathcal{F}_1'|\ge|\Delta\mathcal{F}_0|>|\mathcal{F}_1'|</math>, a contradiction.<br />
}}<br />
<br />
Now we officially apply the induction. By Lemma 1, <br />
:<math>|\Delta\mathcal{F}|\ge|\Delta\mathcal{F}_1'|+|\mathcal{F}_1'|</math>.<br />
Note that <math>\mathcal{F}_1'\subseteq{X\choose k-1}</math>, and due to Lemma 3, <br />
:<math>|\mathcal{F}_1'|\ge{m_k-1\choose k-1}+{m_{k-1}-1\choose k-2}+\cdots+{m_t-1\choose t-1}</math>, <br />
thus by the induction hypothesis, <br />
:<math>|\Delta\mathcal{F}_1'|\ge{m_k-1\choose k-2}+{m_{k-1}-1\choose k-3}+\cdots+{m_t-1\choose t-2}</math>. <br />
Combining them together, we have<br />
:<math><br />
\begin{align}<br />
|\Delta\mathcal{F}|<br />
&\ge |\Delta\mathcal{F}_1'|+|\mathcal{F}_1'|\\<br />
&\ge {m_k-1\choose k-2}+\cdots+{m_t-1\choose t-2}+{m_k-1\choose k-1}+\cdots+{m_t-1\choose t-1}\\<br />
&= {m_k\choose k-1}+{m_{k-1}\choose k-2}+\cdots+{m_t\choose t-1}.<br />
\end{align}</math><br />
<br />
:<math>\square</math><br />
<br />
=== Shadows of specific sizes ===<br />
The definition of shadow can be generalized to the subsets of any size.<br />
{{Theorem|Definition|<br />
:The '''<math>r</math>-shadow''' of <math>\mathcal{F}</math> is defined as<br />
::<math>\Delta_r\mathcal{F}=\left\{S\mid |S|=r\text{ and }\exists T\in\mathcal{F}, S\subseteq T\right\}</math>.<br />
}}<br />
And the general version of the Kruskal-Katona theorem can be deduced.<br />
{{Theorem|Kruskal-Katona Theorem (general version)|<br />
:Let <math>\mathcal{F}\subseteq {X\choose k}</math> with <math>|\mathcal{F}|=m</math>, and suppose that<br />
::<math>m={m_k\choose k}+{m_{k-1}\choose k-1}+\cdots+{m_t\choose t}</math><br />
:where <math>m_k>m_{k-1}>\cdots>m_t\ge t\ge 1</math>. Then for all <math>r</math>, <math>1\le r\le k</math>,<br />
::<math>\left|\Delta_r\mathcal{F}\right|\ge {m_k\choose r}+{m_{k-1}\choose r-1}+\cdots+{m_t\choose t-k+r}</math>.<br />
}}<br />
{{Proof|<br />
Note that for <math>\mathcal{F}\subseteq {X\choose k}</math>,<br />
:<math>\Delta_r\mathcal{F}=\underbrace{\Delta\cdots\Delta}_{k-r}\mathcal{F}</math>.<br />
The theorem follows by repeatedly applying the Kruskal-Katona theorem for <math>\Delta\mathcal{F}</math>.<br />
}}<br />
<br />
=== The Erdős–Ko–Rado theorem, revisited===<br />
To demonstrate the power of the Krulskal-Katona theorem, we show that it actually includes the Erdős–Ko–Rado theorem as a special case. The following elegant proof of the Erdős–Ko–Rado theorem is due to Daykin and Clements independently.<br />
{{Theorem|Erdős–Ko–Rado Theorem|<br />
:Let <math>\mathcal{F}\subseteq {X\choose k}</math> where <math>|X|=n</math> and <math>n\ge 2k</math>. If <math>S\cap T\neq\emptyset</math> for any <math>S,T\in\mathcal{F}</math>, then<br />
::<math>|\mathcal{F}|\le{n-1\choose k-1}</math>.<br />
}}<br />
<br />
{{Prooftitle|Proof by the Kruskal-Katona theorem|(Daykin 1974, Clements 1976)<br />
By contradiction, suppose that <math>|\mathcal{F}|>{n-1\choose k-1}</math>.<br />
<br />
We define the dual system <br />
:<math>\mathcal{G}=\{\bar{S}\mid S\in\mathcal{F}\}</math>, where <math>\bar{S}=X\setminus S</math>.<br />
For any <math>S,T\in\mathcal{F}</math>, the condition <math>S\cap T\neq \emptyset</math> is equivalent to <math>S\not\subseteq \bar{T}</math>, so <math>\mathcal{F}</math> and <math>\Delta_k\mathcal{G}</math> are disjoint, thus<br />
:<math>|\mathcal{F}|+|\Delta_k\mathcal{G}|\le{n\choose k}</math>.<br />
Clearly, <math>\mathcal{G}\subseteq {X\choose n-k}</math> and <br />
:<math>|\mathcal{G}|=|\mathcal{F}|>{n-1\choose k-1}={n-1\choose n-k}</math>.<br />
By the Kruskal-Katona theorem, <math>|\Delta_k\mathcal{G}|\ge{n-1\choose k}</math>.<br />
:<math>|\mathcal{F}|+|\Delta_k\mathcal{G}|>{n-1\choose k-1}+{n-1\choose k}={n\choose k}</math>,<br />
a contradiction.<br />
}}</div>Etone