随机算法 (Spring 2013) and 随机算法 (Spring 2013)/Conditional Probability: Difference between pages

From TCS Wiki
(Difference between pages)
Jump to navigation Jump to search
VisualWikitext
imported>Etone
 
imported>Etone
(Created page with "= Conditional Probability = In probability theory, the word "condition" is a verb. "Conditioning on the event ..." means that it is assumed that the event occurs. {{Theorem |De…")
 
Line 1: Line 1:
{{Infobox
= Conditional Probability =
|name        = Infobox
In probability theory, the word "condition" is a verb. "Conditioning on the event ..." means that it is assumed that the event occurs.
|bodystyle    =  
|title        = <font size=3>随机算法
<br>Randomized Algorithms</font>
|titlestyle  =


|image        =
{{Theorem
|imagestyle  =
|Definition (conditional probability)|
|caption      =
:The '''conditional probability''' that event <math>\mathcal{E}_1</math> occurs given that event <math>\mathcal{E}_2</math> occurs is
|captionstyle =
::<math>
|headerstyle  = background:#ccf;
\Pr[\mathcal{E}_1\mid \mathcal{E}_2]=\frac{\Pr[\mathcal{E}_1\wedge \mathcal{E}_2]}{\Pr[\mathcal{E}_2]}.
|labelstyle  = background:#ddf;
</math>
|datastyle    =
 
|header1 =Instructor
|label1  =
|data1  =
|header2 =
|label2  =
|data2  = 尹一通
|header3 =
|label3  = Email
|data3  = yitong.yin@gmail.com  yinyt@nju.edu.cn 
|header4 =
|label4= office
|data4= 计算机系 804
|header5 = Class
|label5  =
|data5  =
|header6 =
|label6  = Class meetings
|data6  = Tuesday, 10am-12pm <br> 仙逸B-207
|header7 =
|label7  = Place
|data7  =
|header8 =
|label8  = Office hours
|data8  = Wednesday, 2-4pm <br>计算机系 804
|header9 = Textbooks
|label9  =
|data9  =
|header10 =
|label10  =
|data10  = [[File:MR-randomized-algorithms.png|border|100px]]
|header11 =
|label11  =
|data11  = Motwani and Raghavan. <br>''Randomized Algorithms''.<br> Cambridge Univ Press, 1995.
|header12 =
|label12  =
|data12  = [[File:Probability_and_Computing.png|border|100px]]
|header13 =
|label13  =
|data13  =  Mitzenmacher and Upfal. <br>''Probability and Computing: Randomized Algorithms and Probabilistic Analysis''. <br> Cambridge Univ Press, 2005.
|belowstyle = background:#ddf;
|below =
}}
}}


This is the page for the class ''Randomized Algorithms'' for the Spring 2013 semester. Students who take this class should check this page periodically for content updates and new announcements.  
The conditional probability is well-defined only if <math>\Pr[\mathcal{E}_2]\neq0</math>.


= Announcement =
For independent events <math>\mathcal{E}_1</math> and <math>\mathcal{E}_2</math>, it holds that
* <font color=red size=4>The third [[随机算法 (Spring 2013)/Problem_Set_3|homework assignment]] is out, due in two weeks.</font>
:<math>
* The second [[随机算法 (Spring 2013)/Problem_Set_2|homework assignment]] is out, due in two weeks.
\Pr[\mathcal{E}_1\mid \mathcal{E}_2]=\frac{\Pr[\mathcal{E}_1\wedge \mathcal{E}_2]}{\Pr[\mathcal{E}_2]}
*  第1次作业第3题新增一问。由于是在作业发布之后修改，是否做这一问题不会影响分数，但增加此问会使该题目更有意义。
=\frac{\Pr[\mathcal{E}_1]\cdot\Pr[\mathcal{E}_2]}{\Pr[\mathcal{E}_2]}
*  The first [[随机算法 (Spring 2013)/Problem_Set_1|homework assignment]] is out, due in two weeks.
=\Pr[\mathcal{E}_1].
</math>
It supports our intuition that for two independent events, whether one of them occurs will not affect the chance of the other.


= Course info =
== Law of total probability ==
* '''Instructor ''': 尹一通，
The following fact is known as the law of total probability. It computes the probability by averaging over all possible cases.
:*email: yitong.yin@gmail.com, yinyt@nju.edu.cn
{{Theorem
:*office: 计算机系 804.
|Theorem (law of total probability)|
* '''Class meeting''': Tuesday 10am-12pm, 仙逸B-207.
:Let <math>\mathcal{E}_1,\mathcal{E}_2,\ldots,\mathcal{E}_n</math> be mutually disjoint events, and <math>\bigvee_{i=1}^n\mathcal{E}_i=\Omega</math> is the sample space.
* '''Office hour''': Wednesday 2-4pm, 计算机系 804.
:Then for any event <math>\mathcal{E}</math>,
::<math>
\Pr[\mathcal{E}]=\sum_{i=1}^n\Pr[\mathcal{E}\mid\mathcal{E}_i]\cdot\Pr[\mathcal{E}_i].
</math>
}}
{{Proof| Since <math>\mathcal{E}_1,\mathcal{E}_2,\ldots,\mathcal{E}_n</math> are mutually disjoint and <math>\bigvee_{i=1}^n\mathcal{E}_i=\Omega</math>, events <math>\mathcal{E}\wedge\mathcal{E}_1,\mathcal{E}\wedge\mathcal{E}_2,\ldots,\mathcal{E}\wedge\mathcal{E}_n</math> are also mutually disjoint, and <math>\mathcal{E}=\bigvee_{i=1}^n\left(\mathcal{E}\wedge\mathcal{E}_i\right)</math>. Then
:<math>
\Pr[\mathcal{E}]=\sum_{i=1}^n\Pr[\mathcal{E}\wedge\mathcal{E}_i],
</math>
which according to the definition of conditional probability, is <math>\sum_{i=1}^n\Pr[\mathcal{E}\mid\mathcal{E}_i]\cdot\Pr[\mathcal{E}_i]</math>.
}}


= Syllabus =
The law of total probability provides us a standard tool for breaking a probability into sub-cases. Sometimes, it helps the analysis.


=== 先修课程 Prerequisites ===
== A Chain of Conditioning ==
* 必须：离散数学，概率论，线性代数。
By the definition of conditional probability, <math>\Pr[A\mid B]=\frac{\Pr[A\wedge B]}{\Pr[B]}</math>. Thus, <math>\Pr[A\wedge B] =\Pr[B]\cdot\Pr[A\mid B]</math>. This hints us that we can compute the probability of the AND of events by conditional probabilities. Formally, we have the following theorem:
* 推荐：算法设计与分析。
{{Theorem|Theorem|
 
:Let <math>\mathcal{E}_1, \mathcal{E}_2, \ldots, \mathcal{E}_n</math>  be any <math>n</math> events. Then
=== Course materials ===
::<math>\begin{align}
* [[随机算法 (Spring 2013)/Course materials|<font size=3>教材和参考书</font>]]
\Pr\left[\bigwedge_{i=1}^n\mathcal{E}_i\right]
 
&=
=== 成绩 Grades ===
\prod_{k=1}^n\Pr\left[\mathcal{E}_k \mid \bigwedge_{i<k}\mathcal{E}_i\right].
* 课程成绩：本课程将会有六次作业和一次期末考试。最终成绩将由平时作业成绩和期末考试成绩综合得出。
\end{align}</math>
* 迟交：如果有特殊的理由，无法按时完成作业，请提前联系授课老师，给出正当理由。否则迟交的作业将不被接受。
}}
 
{{Proof|It holds that <math>\Pr[A\wedge B] =\Pr[B]\cdot\Pr[A\mid B]</math>. Thus, let <math>A=\mathcal{E}_n</math> and <math>B=\mathcal{E}_1\wedge\mathcal{E}_2\wedge\cdots\wedge\mathcal{E}_{n-1}</math>, then
=== <font color=red> 学术诚信 Academic Integrity </font>===
:<math>\begin{align}
学术诚信是所有从事学术活动的学生和学者最基本的职业道德底线，本课程将不遗余力的维护学术诚信规范，违反这一底线的行为将不会被容忍。
\Pr[\mathcal{E}_1\wedge\mathcal{E}_2\wedge\cdots\wedge\mathcal{E}_n]
 
&=
作业完成的原则：署你名字的工作必须由你完成。允许讨论，但作业必须独立完成，并在作业中列出所有参与讨论的人。不允许其他任何形式的合作——尤其是与已经完成作业的同学“讨论”。
\Pr[\mathcal{E}_1\wedge\mathcal{E}_2\wedge\cdots\wedge\mathcal{E}_{n-1}]\cdot\Pr\left[\mathcal{E}_n\mid \bigwedge_{i<n}\mathcal{E}_i\right].
 
\end{align}
本课程将对剽窃行为采取零容忍的态度。在完成作业过程中，对他人工作（出版物、互联网资料、其他人的作业等）直接的文本抄袭和对关键思想、关键元素的抄袭，按照 [http://www.acm.org/publications/policies/plagiarism_policy ACM Policy on Plagiarism]的解释，都将视为剽窃。剽窃者成绩将被取消。如果发现互相抄袭行为，<font color=red> 抄袭和被抄袭双方的成绩都将被取消</font>。因此请主动防止自己的作业被他人抄袭。
</math>
 
Recursively applying this equation to <math>\Pr[\mathcal{E}_1\wedge\mathcal{E}_2\wedge\cdots\wedge\mathcal{E}_{n-1}]</math> until there is only <math>\mathcal{E}_1</math> left, the theorem is proved.
学术诚信影响学生个人的品行，也关乎整个教育系统的正常运转。为了一点分数而做出学术不端的行为，不仅使自己沦为一个欺骗者，也使他人的诚实努力失去意义。让我们一起努力维护一个诚信的环境。
}}
 
= Assignments =
*[[随机算法 (Spring 2013)/Problem Set 1|Problem Set 1]], due on March 26, Tuesday, in class.
*[[随机算法 (Spring 2013)/Problem Set 2|Problem Set 2]], due on April 23, Tuesday, in class.
*[[随机算法 (Spring 2013)/Problem Set 3|Problem Set 3]], due on June 4, Tuesday, in class.
 
= Lecture Notes =
# [[随机算法 (Spring 2013)/Introduction and Probability Space|Introduction and Probability Space]]: checking matrix multiplication, polynomial identity testing
# [[随机算法 (Spring 2013)/Conditional Probability|Conditional Probability]]: polynomial identity testing, min-cut
# [[随机算法 (Spring 2013)/Random Variables and Expectations|Random Variables and Expectations]]: random quicksort, balls and bins 
# [[随机算法 (Spring 2013)/Moment and Deviation|Moment and Deviation]]: stable marriage, Markov's inequality, Chebyshev's inequality, median selection
# [[随机算法 (Spring 2013)/Threshold and Concentration|Threshold and Concentration]]:  random graphs, threshold phenomenon, Chernoff bound
# [[随机算法 (Spring 2013)/Applications of Chernoff Bound|Applications of Chernoff Bound]]: error reduction, set balancing, packet routing
# [[随机算法 (Spring 2013)/Concentration of Measure|Concentration of Measure]]: martingales, Azuma's inequality, Doob martingales, chromatic number of random graphs
# [[随机算法 (Spring 2013)/Random Projection|Random Projection]]: Johnson-Lindenstrauss Theorem
# [[随机算法 (Spring 2013)/Universal Hashing|Universal Hashing]]: <math>k</math>-wise independence, universal hash families, perfect hashing
# The Probabilistic Method
# Markov Chain and Random Walk
# Coupling and Mixing Time
# Expander Graphs
# Sampling and Counting
 
= The Probability Theory Toolkit =
* [http://en.wikipedia.org/wiki/Probability_space Probability space] and [http://en.wikipedia.org/wiki/Probability_axioms probability axioms]
* [http://en.wikipedia.org/wiki/Independence_(probability_theory)#Independent_events Independent events]
* [http://en.wikipedia.org/wiki/Conditional_probability Conditional probability]
* [http://en.wikipedia.org/wiki/Random_variable Random variable] and [http://en.wikipedia.org/wiki/Expected_value expectation]
* [http://en.wikipedia.org/wiki/Expected_value#Linearity Linearity of expectation]
* The [http://en.wikipedia.org/wiki/Law_of_total_probability law of total probability] and the [http://en.wikipedia.org/wiki/Law_of_total_expectation law of total expectation]
* The [http://en.wikipedia.org/wiki/Boole's_inequality union bound]
* [http://en.wikipedia.org/wiki/Bernoulli_trial Bernoulli trials]  
* [http://en.wikipedia.org/wiki/Geometric_distribution Geometric distribution]
* [http://en.wikipedia.org/wiki/Binomial_distribution Binomial distribution]
* [http://en.wikipedia.org/wiki/Markov's_inequality Markov's inequality]
* [http://en.wikipedia.org/wiki/Variance Variance] and [http://en.wikipedia.org/wiki/Covariance covariance]
* [http://en.wikipedia.org/wiki/Chebyshev's_inequality Chebyshev's inequality]
* [http://en.wikipedia.org/wiki/Chernoff_bound Chernoff bound]

Revision as of 10:48, 4 March 2013

Conditional Probability

In probability theory, the word "condition" is a verb. "Conditioning on the event ..." means that it is assumed that the event occurs.

Definition (conditional probability)
The conditional probability that event [math]\displaystyle{ \mathcal{E}_1 }[/math] occurs given that event [math]\displaystyle{ \mathcal{E}_2 }[/math] occurs is
[math]\displaystyle{ \Pr[\mathcal{E}_1\mid \mathcal{E}_2]=\frac{\Pr[\mathcal{E}_1\wedge \mathcal{E}_2]}{\Pr[\mathcal{E}_2]}. }[/math]

The conditional probability is well-defined only if [math]\displaystyle{ \Pr[\mathcal{E}_2]\neq0 }[/math].

For independent events [math]\displaystyle{ \mathcal{E}_1 }[/math] and [math]\displaystyle{ \mathcal{E}_2 }[/math], it holds that

[math]\displaystyle{ \Pr[\mathcal{E}_1\mid \mathcal{E}_2]=\frac{\Pr[\mathcal{E}_1\wedge \mathcal{E}_2]}{\Pr[\mathcal{E}_2]} =\frac{\Pr[\mathcal{E}_1]\cdot\Pr[\mathcal{E}_2]}{\Pr[\mathcal{E}_2]} =\Pr[\mathcal{E}_1]. }[/math]

It supports our intuition that for two independent events, whether one of them occurs will not affect the chance of the other.

Law of total probability

The following fact is known as the law of total probability. It computes the probability by averaging over all possible cases.

Theorem (law of total probability)
Let [math]\displaystyle{ \mathcal{E}_1,\mathcal{E}_2,\ldots,\mathcal{E}_n }[/math] be mutually disjoint events, and [math]\displaystyle{ \bigvee_{i=1}^n\mathcal{E}_i=\Omega }[/math] is the sample space.
Then for any event [math]\displaystyle{ \mathcal{E} }[/math],
[math]\displaystyle{ \Pr[\mathcal{E}]=\sum_{i=1}^n\Pr[\mathcal{E}\mid\mathcal{E}_i]\cdot\Pr[\mathcal{E}_i]. }[/math]
Proof.
Since [math]\displaystyle{ \mathcal{E}_1,\mathcal{E}_2,\ldots,\mathcal{E}_n }[/math] are mutually disjoint and [math]\displaystyle{ \bigvee_{i=1}^n\mathcal{E}_i=\Omega }[/math], events [math]\displaystyle{ \mathcal{E}\wedge\mathcal{E}_1,\mathcal{E}\wedge\mathcal{E}_2,\ldots,\mathcal{E}\wedge\mathcal{E}_n }[/math] are also mutually disjoint, and [math]\displaystyle{ \mathcal{E}=\bigvee_{i=1}^n\left(\mathcal{E}\wedge\mathcal{E}_i\right) }[/math]. Then
[math]\displaystyle{ \Pr[\mathcal{E}]=\sum_{i=1}^n\Pr[\mathcal{E}\wedge\mathcal{E}_i], }[/math]

which according to the definition of conditional probability, is [math]\displaystyle{ \sum_{i=1}^n\Pr[\mathcal{E}\mid\mathcal{E}_i]\cdot\Pr[\mathcal{E}_i] }[/math].

[math]\displaystyle{ \square }[/math]

The law of total probability provides us a standard tool for breaking a probability into sub-cases. Sometimes, it helps the analysis.

A Chain of Conditioning

By the definition of conditional probability, [math]\displaystyle{ \Pr[A\mid B]=\frac{\Pr[A\wedge B]}{\Pr[B]} }[/math]. Thus, [math]\displaystyle{ \Pr[A\wedge B] =\Pr[B]\cdot\Pr[A\mid B] }[/math]. This hints us that we can compute the probability of the AND of events by conditional probabilities. Formally, we have the following theorem:

Theorem
Let [math]\displaystyle{ \mathcal{E}_1, \mathcal{E}_2, \ldots, \mathcal{E}_n }[/math] be any [math]\displaystyle{ n }[/math] events. Then
[math]\displaystyle{ \begin{align} \Pr\left[\bigwedge_{i=1}^n\mathcal{E}_i\right] &= \prod_{k=1}^n\Pr\left[\mathcal{E}_k \mid \bigwedge_{i\lt k}\mathcal{E}_i\right]. \end{align} }[/math]
Proof.
It holds that [math]\displaystyle{ \Pr[A\wedge B] =\Pr[B]\cdot\Pr[A\mid B] }[/math]. Thus, let [math]\displaystyle{ A=\mathcal{E}_n }[/math] and [math]\displaystyle{ B=\mathcal{E}_1\wedge\mathcal{E}_2\wedge\cdots\wedge\mathcal{E}_{n-1} }[/math], then
[math]\displaystyle{ \begin{align} \Pr[\mathcal{E}_1\wedge\mathcal{E}_2\wedge\cdots\wedge\mathcal{E}_n] &= \Pr[\mathcal{E}_1\wedge\mathcal{E}_2\wedge\cdots\wedge\mathcal{E}_{n-1}]\cdot\Pr\left[\mathcal{E}_n\mid \bigwedge_{i\lt n}\mathcal{E}_i\right]. \end{align} }[/math]

Recursively applying this equation to [math]\displaystyle{ \Pr[\mathcal{E}_1\wedge\mathcal{E}_2\wedge\cdots\wedge\mathcal{E}_{n-1}] }[/math] until there is only [math]\displaystyle{ \mathcal{E}_1 }[/math] left, the theorem is proved.

[math]\displaystyle{ \square }[/math]
Retrieved from "https://tcs.nju.edu.cn/wiki/index.php?title=随机算法_(Spring_2013)&oldid=5455"