imported>Etone |
imported>Etone |
| Line 1: |
Line 1: |
| =Polynomial Identity Testing (PIT) = | | =Markov's Inequality= |
| The '''Polynomial Identity Testing (PIT)''' is such a problem: given as input two polynomials, determine whether they are identical. It plays a fundamental role in ''Identity Testing'' problems.
| |
|
| |
|
| First, let's consider the univariate ("one variable") case:
| | One of the most natural information about a random variable is its expectation, which is the first moment of the random variable. Markov's inequality draws a tail bound for a random variable from its expectation. |
| * '''Input:''' two polynomials <math>f, g\in\mathbb{F}[x]</math> of degree <math>d</math>.
| | {{Theorem |
| * Determine whether <math>f\equiv g</math> (<math>f</math> and <math>g</math> are identical).
| | |Theorem (Markov's Inequality)| |
| Here the <math>\mathbb{F}[x]</math> denotes the [http://en.wikipedia.org/wiki/Polynomial_ring ring of univariate polynomials] on a field <math>\mathbb{F}</math>. More precisely, a polynomial <math>f\in\mathbb{F}[x]</math> is
| | :Let <math>X</math> be a random variable assuming only nonnegative values. Then, for all <math>t>0</math>, |
| :<math>f(x)=\sum_{i=0}^\infty a_ix^i</math>,
| | ::<math>\begin{align} |
| where the coefficients <math>a_i</math> are taken from the field <math>\mathbb{F}</math>, and the addition and multiplication are also defined over the field <math>\mathbb{F}</math>.
| | \Pr[X\ge t]\le \frac{\mathbf{E}[X]}{t}. |
| And:
| | \end{align}</math> |
| * the '''degree''' of <math>f</math> is the highest <math>i</math> with non-zero <math>a_i</math>;
| |
| * a polynomial <math>f</math> is a '''zero-polynomial''', denoted as <math>f\equiv 0</math>, if all coefficients <math>a_i=0</math>.
| |
| | |
| Alternatively, we can consider the following equivalent problem by comparing the polynomial <math>f-g</math> (whose degree is at most <math>d</math>) with the zero-polynomial:
| |
| * '''Input:''' a polynomial <math>f\in\mathbb{F}[x]</math> of degree <math>d</math>.
| |
| * Determine whether <math>f\equiv 0</math> (<math>f</math> is the 0 polynomial).
| |
| | |
| The probalem is trivial if the input polynomial <math>f</math> is given explicitly: one can trivially solve the problem by checking whether all <math>d+1</math> coefficients are <math>0</math>. To make the problem nontrivial, we assume that the input polynomial is given implicitly as a ''black box'' (also called an ''oracle''): the only way the algorithm can access to <math>f</math> is to evaluate <math>f(x)</math> over some <math>x</math> from the field <math>\mathbb{F}</math>, <math>x</math> chosen by the algorithm.
| |
| | |
| A straightforward deterministic algorithm is to evaluate <math>f(x_1),f(x_2),\ldots,f(x_{d+1})</math> over <math>d+1</math> ''distinct'' elements <math>x_1,x_2,\ldots,x_{d+1}</math> from the field <math>\mathbb{F}</math> and check whether they are all zero. By the [https://en.wikipedia.org/wiki/Fundamental_theorem_of_algebra fundamental theorem of algebra], also known as polynomial interpolations, this guarantees to verify whether a degree-<math>d</math> univariate polynomial <math>f\equiv 0</math>.
| |
| {{Theorem|Fundamental Theorem of Algebra| | |
| :Any non-zero univariate polynomial of degree <math>d</math> has at most <math>d</math> roots.
| |
| }} | | }} |
| The reason for this fundamental theorem holding generally over any field <math>\mathbb{F}</math> is that any univariate polynomial of degree <math>d</math> factors uniquely into at most <math>d</math> irreducible polynomials, each of which has at most one root.
| | {{Proof| Let <math>Y</math> be the indicator such that |
| | :<math>\begin{align} |
| | Y &= |
| | \begin{cases} |
| | 1 & \mbox{if }X\ge t,\\ |
| | 0 & \mbox{otherwise.} |
| | \end{cases} |
| | \end{align}</math> |
|
| |
|
| The following simple randomized algorithm is natural:
| | It holds that <math>Y\le\frac{X}{t}</math>. Since <math>Y</math> is 0-1 valued, <math>\mathbf{E}[Y]=\Pr[Y=1]=\Pr[X\ge t]</math>. Therefore, |
| {{Theorem|Algorithm for PIT|
| | :<math> |
| *suppose we have a finite subset <math>S\subseteq\mathbb{F}</math> (to be specified later);
| | \Pr[X\ge t] |
| *pick <math>r\in S</math> ''uniformly'' at random;
| | = |
| *if <math>f(r) = 0</math> then return “yes” else return “no”;
| | \mathbf{E}[Y] |
| | \le |
| | \mathbf{E}\left[\frac{X}{t}\right] |
| | =\frac{\mathbf{E}[X]}{t}. |
| | </math> |
| }} | | }} |
|
| |
|
| This algorithm evaluates <math>f</math> at one point chosen uniformly at random from a finite subset <math>S\subseteq\mathbb{F}</math>. It is easy to see the followings:
| | == Generalization == |
| * If <math>f\equiv 0</math>, the algorithm always returns "yes", so it is always correct.
| | For any random variable <math>X</math>, for an arbitrary non-negative real function <math>h</math>, the <math>h(X)</math> is a non-negative random variable. Applying Markov's inequality, we directly have that |
| * If <math>f\not\equiv 0</math>, the algorithm may wrongly return "yes" (a '''false positive'''). But this happens only when the random <math>r</math> is a root of <math>f</math>. By the fundamental theorem of algebra, <math>f</math> has at most <math>d</math> roots, so the probability that the algorithm is wrong is bounded as
| |
| ::<math>\Pr[f(r)=0]\le\frac{d}{|S|}.</math>
| |
| | |
| By fixing <math>S\subseteq\mathbb{F}</math> to be an arbitrary subset of size <math>|S|=2d</math>, this probability of false positive is at most <math>1/2</math>. We can reduce it to an arbitrarily small constant <math>\delta</math> by repeat the above testing ''independently'' for <math>\log_2 \frac{1}{\delta}</math> times, since the error probability decays geometrically as we repeat the algorithm independently.
| |
| | |
| == Communication Complexity of Equality ==
| |
| The [http://en.wikipedia.org/wiki/Communication_complexity communication complexity] is introduced by Andrew Chi-Chih Yao as a model of computation with more than one entities, each with partial information about the input.
| |
| | |
| Assume that there are two entities, say Alice and Bob. Alice has a private input <math>a</math> and Bob has a private input <math>b</math>. Together they want to compute a function <math>f(a,b)</math> by communicating with each other. The communication follows a predefined '''communication protocol''' (the "algorithm" in this model). The complexity of a communication protocol is measured by the number of bits communicated between Alice and Bob in the worst case.
| |
| | |
| The problem of checking identity is formally defined by the function EQ as follows: <math>\mathrm{EQ}:\{0,1\}^n\times\{0,1\}^n\rightarrow\{0,1\}</math> and for any <math>a,b\in\{0,1\}^n</math>,
| |
| :<math> | | :<math> |
| \mathrm{EQ}(a,b)= | | \Pr[h(X)\ge t]\le\frac{\mathbf{E}[h(X)]}{t}. |
| \begin{cases} | | </math> |
| 1& \mbox{if } a=b,\\
| |
| 0& \mbox{otherwise.}
| |
| \end{cases}</math>
| |
|
| |
|
| A trivial way to solve EQ is to let Bob send his entire input string <math>b</math> to Alice and let Alice check whether <math>a=b</math>. This costs <math>n</math> bits of communications.
| | This trivial application of Markov's inequality gives us a powerful tool for proving tail inequalities. With the function <math>h</math> which extracts more information about the random variable, we can prove sharper tail inequalities. |
|
| |
|
| It is known that for deterministic communication protocols, this is the best we can get for computing EQ.
| | =Chebyshev's inequality= |
|
| |
|
| {{Theorem|Theorem (Yao 1979)| | | == Variance == |
| :Any deterministic communication protocol computing EQ on two <math>n</math>-bit strings costs <math>n</math> bits of communication in the worst-case. | | {{Theorem |
| | |Definition (variance)| |
| | :The '''variance''' of a random variable <math>X</math> is defined as |
| | ::<math>\begin{align} |
| | \mathbf{Var}[X]=\mathbf{E}\left[(X-\mathbf{E}[X])^2\right]=\mathbf{E}\left[X^2\right]-(\mathbf{E}[X])^2. |
| | \end{align}</math> |
| | :The '''standard deviation''' of random variable <math>X</math> is |
| | ::<math> |
| | \delta[X]=\sqrt{\mathbf{Var}[X]}. |
| | </math> |
| }} | | }} |
|
| |
|
| This theorem is much more nontrivial to prove than it looks, because Alice and Bob are allowed to interact with each other in arbitrary ways. The proof of this theorem is in Yao's [http://math.ucla.edu/~znorwood/290d.2.14s/papers/yao.pdf celebrated paper in 1979 with a humble title]. It pioneered the field of communication complexity.
| | The variance is the diagonal case for '''covariance'''. |
| | | {{Theorem |
| If we allow randomness in protocols, and also tolerate a small probabilistic error, the problem can be solved with significantly less communications. To present this randomized protocol, we need a few preparations:
| | |Definition (covariance)| |
| * We represent the inputs <math>a,b \in\{0,1\}^{n}</math> of Alice and Bob as two univariate polynomials of degree at most <math>n-1</math>, respectively
| | :The '''covariance''' of two random variables <math>X</math> and <math>Y</math> is |
| ::<math>f(x)=\sum_{i=0}^{n-1}a_ix^{i}</math> and <math>g(x)=\sum_{i=0}^{n-1}b_ix^{i}</math>. | | ::<math>\begin{align} |
| * The two polynomials <math>f</math> and <math>g</math> are defined over finite field <math>\mathbb{Z}_p=\{0,1,\ldots,p-1\}</math> for some suitable prime <math>p</math> (to be specified later), which means the additions and multiplications are modulo <math>p</math>.
| | \mathbf{Cov}(X,Y)=\mathbf{E}\left[(X-\mathbf{E}[X])(Y-\mathbf{E}[Y])\right]. |
| The randomized communication protocol is then as follows:
| | \end{align}</math> |
| {{Theorem|A randomized protocol for EQ|
| |
| '''Bob does''':
| |
| :* pick <math>r\in\mathbb{Z}_p</math> uniformly at random;
| |
| :* send <math>r</math> and <math>g(r)</math> to Alice;
| |
| '''Upon receiving''' <math>r</math> and <math>g(r)</math> '''Alice does''':
| |
| :* compute <math>f(r)</math>;
| |
| :* If <math>f(r)= g(r)</math> return "'''yes'''"; else return "'''no'''".
| |
| }} | | }} |
| The communication complexity of the protocol is given by the number of bits used to represent the values of <math>r</math> and <math>g(r)</math>.
| |
| Since the polynomials are defined over finite field <math>\mathbb{Z}_p</math> and the random number <math>f</math> is also chosen from <math>\mathbb{Z}_p</math>, this is bounded by <math>O(\log p)</math>.
| |
|
| |
| On the other hand the protocol makes mistakes only when <math>a\neq b</math> but wrongly answers "yes". This happens only when <math>f\not\equiv g</math> but <math>f(r)=g(r)</math>. The degrees of <math>f, g</math> are at most <math>n-1</math>, and <math>r</math> is chosen among <math>p</math> distinct values, we have
| |
| :<math>\Pr[f(r)=g(r)]\le \frac{n-1}{p}</math>.
| |
|
| |
|
| By choosing <math>p</math> to be a prime in the interval <math>[n^2, 2n^2]</math> (by [https://en.wikipedia.org/wiki/Bertrand%27s_postulate Chebyshev's theorem], such prime <math>p</math> always exists), the above randomized communication protocol solves the Equality function EQ with an error probability of false positive at most <math>O(1/n)</math>, with communication complexity <math>O(\log n)</math>, an EXPONENTIAL improvement to ANY deterministic communication protocol!
| | We have the following theorem for the variance of sum. |
|
| |
|
| == Schwartz-Zippel Theorem ==
| | {{Theorem |
| Now let's see the the true form of '''Polynomial Identity Testing (PIT)''', for multivariate polynomials:
| | |Theorem| |
| * '''Input:''' two <math>n</math>-variate polynomials <math>f, g\in\mathbb{F}[x_1,x_2,\ldots,x_n]</math> of degree <math>d</math>.
| | :For any two random variables <math>X</math> and <math>Y</math>, |
| * Determine whether <math>f\equiv g</math>.
| | ::<math>\begin{align} |
| The <math>\mathbb{F}[x_1,x_2,\ldots,x_n]</math> is the [http://en.wikipedia.org/wiki/Polynomial_ring#The_polynomial_ring_in_several_variables ring of multivariate polynomials] over field <math>\mathbb{F}</math>. An <math>n</math>-variate polynomial of degree <math>d</math>, written as a sum of monomials, is:
| | \mathbf{Var}[X+Y]=\mathbf{Var}[X]+\mathbf{Var}[Y]+2\mathbf{Cov}(X,Y). |
| :<math>f(x_1,x_2,\ldots,x_n)=\sum_{i_1,i_2,\ldots,i_n\ge 0\atop i_1+i_2+\cdots+i_n\le d}a_{i_1,i_2,\ldots,i_n}x_{1}^{i_1}x_2^{i_2}\cdots x_{n}^{i_n}</math>. | | \end{align}</math> |
| The '''degree''' or '''total degree''' of a monomial <math>a_{i_1,i_2,\ldots,i_n}x_{1}^{i_1}x_2^{i_2}\cdots x_{n}^{i_n}</math> is given by <math>i_1+i_2+\cdots+i_n</math> and the degree of a polynomial <math>f</math> is the maximum degree of monomials of nonzero coefficients.
| | :Generally, for any random variables <math>X_1,X_2,\ldots,X_n</math>, |
| | | ::<math>\begin{align} |
| As before, we also consider the following equivalent problem:
| | \mathbf{Var}\left[\sum_{i=1}^n X_i\right]=\sum_{i=1}^n\mathbf{Var}[X_i]+\sum_{i\neq j}\mathbf{Cov}(X_i,X_j). |
| * '''Input:''' a polynomial <math>f\in\mathbb{F}[x_1,x_2,\ldots,x_n]</math> of degree <math>d</math>.
| | \end{align}</math> |
| * Determine whether <math>f\equiv 0</math>.
| |
| | |
| If <math>f</math> is written explicitly as a sum of monomials, then the problem can be solved by checking whether all coefficients, and there at most <math>{n+d\choose d}\le (n+d)^{d}</math> coefficients in an <math>n</math>-variate polynomial of degree at most <math>d</math>.
| |
| | |
| A multivariate polynomial <math>f</math> can also be presented in its '''product form''', for example:
| |
| {{Theorem|Example|
| |
| The [http://en.wikipedia.org/wiki/Vandermonde_matrix Vandermonde matrix] <math>M=M(x_1,x_2,\ldots,x_n)</math> is defined as that <math>M_{ij}=x_i^{j-1}</math>, that is
| |
| :<math>M=\begin{bmatrix} | |
| 1 & x_1 & x_1^2 & \dots & x_1^{n-1}\\
| |
| 1 & x_2 & x_2^2 & \dots & x_2^{n-1}\\ | |
| 1 & x_3 & x_3^2 & \dots & x_3^{n-1}\\
| |
| \vdots & \vdots & \vdots & \ddots &\vdots \\
| |
| 1 & x_n & x_n^2 & \dots & x_n^{n-1}
| |
| \end{bmatrix}</math>. | |
| Let <math>f</math> be the polynomial defined as
| |
| :<math>
| |
| f(x_1,\ldots,x_n)=\det(M)=\prod_{j<i}(x_i-x_j).
| |
| </math> | |
| }} | | }} |
| For polynomials in product form, it is quite efficient to '''evaluate''' the polynomial at any specific point from the field over which the polynomial is defined, however, '''expanding''' the polynomial to a sum of monomials can be very expensive.
| | {{Proof| The equation for two variables is directly due to the definition of variance and covariance. The equation for <math>n</math> variables can be deduced from the equation for two variables. |
| | |
| The following is a simple randomized algorithm for testing identity of multivariate polynomials: | |
| {{Theorem|Randomized algorithm for multivariate PIT|
| |
| *suppose we have a finite subset <math>S\subseteq\mathbb{F}</math> (to be specified later);
| |
| * pick <math>r_1,r_2,\ldots,r_n\in S</math> ''uniformly'' and ''independently'' at random;
| |
| * if <math>f(\vec{r})=f(r_1,r_2,\ldots,r_n) = 0</math> then return “yes” else return “no”;
| |
| }} | | }} |
|
| |
|
| This algorithm evaluates <math>f</math> at one point chosen uniformly from an <math>n</math>-dimensional cube <math>S^n</math>, where <math>S\subseteq\mathbb{F}</math> is a finite subset. And:
| | For independent random variables, the expectation of a product equals the product of expectations. |
| * If <math>f\equiv 0</math>, the algorithm always returns "yes", so it is always correct.
| | {{Theorem |
| * If <math>f\not\equiv 0</math>, the algorithm may wrongly return "yes" (a '''false positive'''). But this happens only when the random <math>\vec{r}=(r_1,r_2,\ldots,r_n)</math> is a root of <math>f</math>. The probability of this bad event is upper bounded by the following famous result due to [https://pdfs.semanticscholar.org/b913/cf330852035f49b4ec5fe2db86c47d8a98fd.pdf Schwartz (1980)] and [http://www.cecm.sfu.ca/~monaganm/teaching/TopicsinCA15/zippel79.pdf Zippel (1979)].
| | |Theorem| |
| | | :For any two independent random variables <math>X</math> and <math>Y</math>, |
| {{Theorem|Schwartz-Zippel Theorem| | | ::<math>\begin{align} |
| : Let <math>f\in\mathbb{F}[x_1,x_2,\ldots,x_n]</math> be a multivariate polynomial of degree <math>d</math> over a field <math>\mathbb{F}</math>. If <math>f\not\equiv 0</math>, then for any finite set <math>S\subset\mathbb{F}</math>, and <math>r_1,r_2\ldots,r_n\in S</math> chosen uniformly and independently at random, | | \mathbf{E}[X\cdot Y]=\mathbf{E}[X]\cdot\mathbf{E}[Y]. |
| ::<math>\Pr[f(r_1,r_2,\ldots,r_n)=0]\le\frac{d}{|S|}.</math> | | \end{align}</math> |
| }} | | }} |
| The Schwartz-Zippel Theorem states that for any nonzero <math>n</math>-variate polynomial of degree at most <math>d</math>, the number of roots in any cube <math>S^n</math> is at most <math>d\cdot |S|^{n-1}</math>.
| | {{Proof| |
| | |
| Dana Moshkovitz gave a surprisingly simply and elegant [http://eccc.hpi-web.de/report/2010/096/ proof] of Schwartz-Zippel Theorem, using some advanced ideas. Now we introduce the standard proof by induction.
| |
| | |
| {{Proof| | |
| The theorem is proved by induction on <math>n</math>.
| |
| | |
| ;Induction basis''':'''
| |
| For <math>n=1</math>, this is the univariate case. Assume that <math>f\not\equiv 0</math>. Due to the fundamental theorem of algebra, any polynomial <math>f(x)</math> of degree at most <math>d</math> must have at most <math>d</math> roots, thus
| |
| :<math>\Pr[f(r)=0]\le\frac{d}{|S|}.
| |
| </math>
| |
| | |
| ;Induction hypothesis''':'''
| |
| Assume the theorem holds for any <math>m</math>-variate polynomials for all <math>m<n</math>.
| |
| | |
| ;Induction step''':'''
| |
| For any <math>n</math>-variate polynomial <math>f(x_1,x_2,\ldots,x_n)</math> of degree at most <math>d</math>, we write <math>f</math> as
| |
| :<math>f(x_1,x_2,\ldots,x_n)=\sum_{i=0}^kx_n^{i}f_i(x_1,x_2,\ldots,x_{n-1})</math>,
| |
| where <math>k</math> is the highest degree of <math>x_n</math>, which means the degree of <math>f_k</math> is at most <math>d-k</math> and <math>f_k\not\equiv 0</math>.
| |
| | |
| In particular, we write <math>f</math> as a sum of two parts:
| |
| :<math>f(x_1,x_2,\ldots,x_n)=x_n^k f_k(x_1,x_2,\ldots,x_{n-1})+\bar{f}(x_1,x_2,\ldots,x_n)</math>,
| |
| where both <math>f_k</math> and <math>\bar{f}</math> are polynomials, such that
| |
| * <math>f_k\not\equiv 0</math> is as above, whose degree is at most <math>d-k</math>;
| |
| * <math>\bar{f}(x_1,x_2,\ldots,x_n)=\sum_{i=0}^{k-1}x_n^i f_i(x_1,x_2,\ldots,x_{n-1})</math>, thus <math>\bar{f}(x_1,x_2,\ldots,x_n)</math> has no <math>x_n^{k}</math> factor in any term.
| |
| | |
| By the law of total probability, we have
| |
| :<math> | | :<math> |
| \begin{align} | | \begin{align} |
| &\Pr[f(r_1,r_2,\ldots,r_n)=0]\\
| | \mathbf{E}[X\cdot Y] |
| = | | &= |
| &\Pr[f(\vec{r})=0\mid f_k(r_1,r_2,\ldots,r_{n-1})=0]\cdot\Pr[f_k(r_1,r_2,\ldots,r_{n-1})=0]\\
| | \sum_{x,y}xy\Pr[X=x\wedge Y=y]\\ |
| &+\Pr[f(\vec{r})=0\mid f_k(r_1,r_2,\ldots,r_{n-1})\neq0]\cdot\Pr[f_k(r_1,r_2,\ldots,r_{n-1})\neq0]. | | &= |
| \end{align} | | \sum_{x,y}xy\Pr[X=x]\Pr[Y=y]\\ |
| </math>
| | &= |
| Note that <math>f_k(r_1,r_2,\ldots,r_{n-1})</math> is a polynomial on <math>n-1</math> variables of degree <math>d-k</math> such that <math>f_k\not\equiv 0</math>.
| | \sum_{x}x\Pr[X=x]\sum_{y}y\Pr[Y=y]\\ |
| By the induction hypothesis, we have
| | &= |
| :<math>
| | \mathbf{E}[X]\cdot\mathbf{E}[Y]. |
| \begin{align} | |
| (*)
| |
| &\qquad
| |
| &\Pr[f_k(r_1,r_2,\ldots,r_{n-1})=0]\le\frac{d-k}{|S|}.
| |
| \end{align} | |
| </math>
| |
| | |
| Now we look at the case conditioning on <math>f_k(r_1,r_2,\ldots,r_{n-1})\neq0</math>. Recall that <math>\bar{f}(x_1,\ldots,x_n)</math> has no <math>x_n^k</math> factor in any term, thus the condition <math>f_k(r_1,r_2,\ldots,r_{n-1})\neq0</math> guarantees that
| |
| :<math>f(r_1,\ldots,r_{n-1},x_n)=x_n^k f_k(r_1,r_2,\ldots,r_{n-1})+\bar{f}(r_1,r_2,\ldots,r_{n-1},x_n)=g_{r_1,\ldots,r_{n-1}}(x_n)</math>
| |
| is a nonzero univariate polynomial of <math>x_n</math> such that the degree of <math>g_{r_1,\ldots,r_{n-1}}(x_n)</math> is <math>k</math> and <math>g_{r_1,\ldots,r_{n-1}}\not\equiv 0</math>, for which we already known that the probability <math>g_{r_1,\ldots,r_{n-1}}(r_n)=0</math> is at most <math>\frac{k}{|S|}</math>.
| |
| Therefore,
| |
| :<math>
| |
| \begin{align} | |
| (**)
| |
| &\qquad
| |
| &\Pr[f(\vec{r})=0\mid f_k(r_1,r_2,\ldots,r_{n-1})\neq0]=\Pr[g_{r_1,\ldots,r_{n-1}}(r_n)=0\mid f_k(r_1,r_2,\ldots,r_{n-1})\neq0]\le\frac{k}{|S|}
| |
| \end{align} | | \end{align} |
| </math>.
| |
| Substituting both <math>(*)</math> and <math>(**)</math> back in the total probability, we have
| |
| :<math>
| |
| \Pr[f(r_1,r_2,\ldots,r_n)=0]
| |
| \le\frac{d-k}{|S|}+\frac{k}{|S|}=\frac{d}{|S|},
| |
| </math> | | </math> |
| which proves the theorem.
| |
| }} | | }} |
|
| |
|
| = Fingerprinting =
| | Consequently, covariance of independent random variables is always zero. |
| The polynomial identity testing algorithm in the Schwartz-Zippel theorem can be abstracted as the following framework:
| | {{Theorem |
| Suppose we want to compare two objects <math>X</math> and <math>Y</math>. Instead of comparing them directly, we compute random '''fingerprints''' <math>\mathrm{FING}(X)</math> and <math>\mathrm{FING}(Y)</math> of them and compare the fingerprints.
| | |Theorem| |
| | | :For any two independent random variables <math>X</math> and <math>Y</math>, |
| The fingerprints has the following properties:
| | ::<math>\begin{align} |
| * <math>\mathrm{FING}(\cdot)</math> is a function, meaning that if <math>X= Y</math> then <math>\mathrm{FING}(X)=\mathrm{FING}(Y)</math>.
| | \mathbf{Cov}(X,Y)=0. |
| * It is much easier to compute and compare the fingerprints.
| | \end{align}</math> |
| * Ideally, the domain of fingerprints is much smaller than the domain of original objects, so storing and comparing fingerprints are easy. This means the fingerprint function <math>\mathrm{FING}(\cdot)</math> cannot be an injection (one-to-one mapping), so it's possible that different <math>X</math> and <math>Y</math> are mapped to the same fingerprint. We resolve this by making fingerprint function ''randomized'', and for <math>X\neq Y</math>, we want the probability <math>\Pr[\mathrm{FING}(X)=\mathrm{FING}(Y)]</math> to be small.
| |
| | |
| In Schwartz-Zippel theorem, the objects to compare are polynomials from <math>\mathbb{F}[x_1,\ldots,x_n]</math>. Given a polynomial <math>f\in \mathbb{F}[x_1,\ldots,x_n]</math>, its fingerprint is computed as <math>\mathrm{FING}(f)=f(r_1,\ldots,r_n)</math> for <math>r_i</math> chosen independently and uniformly at random from some fixed set <math>S\subseteq\mathbb{F}</math>.
| |
| | |
| With this generic framework, for various identity testing problems, we may design different fingerprints <math>\mathrm{FING}(\cdot)</math>.
| |
| | |
| == Communication protocols for Equality by fingerprinting==
| |
| Now consider again the communication model where the two players Alice with a private input <math>x\in\{0,1\}^n</math> and Bob with a private input <math>y\in\{0,1\}^n</math> together compute a function <math>f(x,y)</math> by running a communication protocol.
| |
| | |
| We still consider the communication protocols for the equality function EQ
| |
| :<math>
| |
| \mathrm{EQ}(x,y)=
| |
| \begin{cases}
| |
| 1& \mbox{if } x=y,\\
| |
| 0& \mbox{otherwise.} | |
| \end{cases}</math> | |
| | |
| With the language of fingerprinting, this communication problem can be solved by the following generic scheme:
| |
| {{Theorem|Communication protocol for EQ by fingerprinting|
| |
| '''Bob does''':
| |
| :* choose a random fingerprint function <math>\mathrm{FING}(\cdot)</math> and compute the fingerprint of her input <math>\mathrm{FING}(y)</math>;
| |
| :* sends both the description of <math>\mathrm{FING}(\cdot)</math> and the value of <math>\mathrm{FING}(y)</math> to Alice;
| |
| '''Upon receiving''' the description of <math>\mathrm{FING}(\cdot)</math> and the value of <math>\mathrm{FING}(y)</math>, '''Alice does''':
| |
| :* computes <math>\mathrm{FING}(x)</math> and check whether <math>\mathrm{FING}(x)=\mathrm{FING}(y)</math>.
| |
| }} | | }} |
| In this way we have a randomized communication protocol for the equality function EQ with false positive. The communication cost as well as the error probability are reduced to the question of how to design this random fingerprint function <math>\mathrm{FING}(\cdot)</math> to guarantee:
| | {{Proof| |
| # A random fingerprint function <math>\mathrm{FING}(\cdot)</math> can be described succinctly.
| | :<math>\begin{align} |
| # The range of <math>\mathrm{FING}(\cdot)</math> is small, so the fingerprints are succinct.
| | \mathbf{Cov}(X,Y) |
| # If <math>x\neq y</math>, the probability <math>\Pr[\mathrm{FING}(x)=\mathrm{FING}(y)]</math> is small.
| | &=\mathbf{E}\left[(X-\mathbf{E}[X])(Y-\mathbf{E}[Y])\right]\\ |
| | | &= \mathbf{E}\left[X-\mathbf{E}[X]\right]\mathbf{E}\left[Y-\mathbf{E}[Y]\right] &\qquad(\mbox{Independence})\\ |
| === Fingerprinting by PIT===
| | &=0. |
| As before, we can define the fingerprint function as: for any bit-string <math>x\in\{0,1\}^n</math>, its random fingerprint is <math>\mathrm{FING}(x)=\sum_{i=1}^n x_i r^{i}</math>, where the additions and multiplications are defined over a finite field <math>\mathbb{Z}_p</math>, and <math>r</math> is chosen uniformly at random from <math>\mathbb{Z}_p</math>, where <math>p</math> is some suitable prime which can be represented in <math>\Theta(\log n)</math> bits. More specifically, we can choose <math>p</math> to be any prime from the interval <math>[n^2, 2n^2]</math>. Due to Chebyshev's theorem, such prime must exist.
| | \end{align}</math> |
| | |
| As we have shown before, it takes <math>O(\log p)=O(\log n)</math> bits to represent <math>\mathrm{FING}(y)</math> and to describe the random function <math>\mathrm{FING}(\cdot)</math> (since it a random function <math>\mathrm{FING}(\cdot)</math> from this family is uniquely identified by a random <math>r\in\mathbb{Z}_p</math>, which can be represented within <math>\log p=O(\log n)</math> bits). And it follows easily from the fundamental theorem of algebra that for any distinct <math>x, y\in\{0,1\}^n</math>,
| |
| :<math>
| |
| \Pr[\mathrm{FING}(x)=\mathrm{FING}(y)] \le \frac{n-1}{p}\le \frac{1}{n}. | |
| </math>
| |
| | |
| === Fingerprinting by randomized checksum===
| |
| Now we consider a new fingerprint function: We treat each input string <math>x\in\{0,1\}^n</math> as the binary representation of a number, and let <math>\mathrm{FING}(x)=x\bmod p</math> for some random prime <math>p</math> chosen from <math>[k]=\{0,1,\ldots,k-1\}</math>, for some <math>k</math> to be specified later.
| |
| | |
| Now a random fingerprint function <math>\mathrm{FING}(\cdot)</math> can be uniquely identified by this random prime <math>p</math>. The new communication protocol for EQ with this fingerprint is as follows:
| |
| {{Theorem|Communication protocol for EQ by random checksum|
| |
| '''Bob does''':
| |
| :for some parameter <math>k</math> (to be specified),
| |
| :* choose a prime <math>p\in[k]</math> uniformly at random;
| |
| :* send <math>p</math> and <math>x\bmod p</math> to Alice;
| |
| '''Upon receiving''' <math>p</math> and <math>x\bmod p</math>, '''Alice does''':
| |
| :* check whether <math>x\bmod p=y\bmod p</math>.
| |
| }} | | }} |
| The number of bits to be communicated is obviously <math>O(\log k)</math>. When <math>x\neq y</math>, we want to upper bound the error probability <math>\Pr[x\bmod p=y\bmod p]</math>.
| |
|
| |
| Suppose without loss of generality <math>x>y</math>. Let <math>z=x-y</math>. Then <math>z<2^n</math> since <math>x,y\in[2^n]</math>, and
| |
| <math>z\neq 0</math> for <math>x\neq y</math>. It holds that <math>x\equiv y\pmod p</math> if and only if <math>p\mid z</math>. Therefore, we only need to upper bound the probability
| |
| :<math>\Pr[z\bmod p=0]</math>
| |
| for an arbitrarily fixed <math>0<z<2^n</math>, and a uniform random prime <math>p\in[k]</math>.
| |
|
| |
| The probability <math>\Pr[z\bmod p=0]</math> is computed directly as
| |
| :<math>\Pr[z\bmod p=0]\le\frac{\mbox{the number of prime divisors of }z}{\mbox{the number of primes in }[k]}</math>.
| |
|
| |
| For the numerator, any positive <math>z<2^n</math> has at most <math>n</math> prime factors. To see this, by contradiction assume that <math>z</math> has more than <math>n</math> prime factors. Note that any prime number is at least 2. Then <math>z</math> must be greater than <math>2^n</math>, contradicting the fact that <math>z<2^n</math>.
| |
|
| |
| For the denominator, we need to lower bound the number of primes in <math>[k]</math>. This is given by the celebrated [http://en.wikipedia.org/wiki/Prime_number_theorem '''Prime Number Theorem (PNT)'''].
| |
|
| |
| {{Theorem|Prime Number Theorem|
| |
| :Let <math>\pi(k)</math> denote the number of primes less than <math>k</math>. Then <math>\pi(k)\sim\frac{k}{\ln k}</math> as <math>k\rightarrow\infty</math>.
| |
| }}
| |
|
| |
| Therefore, by choosing <math>k=2n^2\ln n</math>, we have that for a <math>0<z<2^n</math>, and a random prime <math>p\in[k]</math>,
| |
| :<math>\Pr[z\bmod p=0]\le\frac{n}{\pi(k)}\sim\frac{1}{n}</math>,
| |
| which means the for any inputs <math>x,y\in\{0,1\}^n</math>, if <math>x\neq y</math>, then the false positive is bounded as
| |
| :<math>\Pr[\mathrm{FING}(x)=\mathrm{FING}(y)]\le\Pr[|x-y|\bmod p=0]\le \frac{1}{n}</math>.
| |
| Moreover, by this choice of parameter <math>k=2n^2\ln n</math>, the communication complexity of the protocol is bounded by <math>O(\log k)=O(\log n)</math>.
| |
|
| |
| = Checking distinctness =
| |
| Consider the following problem of '''checking distinctness''':
| |
| *Given a sequence <math>x_1,x_2,\ldots,x_n\in\{1,2,\ldots,n\}</math>, check whether every element of <math>\{1,2,\ldots,n\}</math> appears '''exactly''' once.
| |
|
| |
| Obviously this problem can be solved in linear time and linear space (in addition to the space for storing the input) by maintaining a <math>n</math>-bit vector that indicates which numbers among <math>\{1,2,\ldots,n\}</math> have appeared.
| |
|
| |
| When this <math>n</math> is enormously large, <math>\Omega(n)</math> space cost is too expensive. We wonder whether we could solve this problem with a space cost (in addition to the space for storing the input) much less than <math>O(n)</math>. This can be done by fingerprinting if we tolerate a certain degree of inaccuracy.
| |
|
| |
| == Fingerprinting multisets ==
| |
| We consider the following more generalized problem, '''checking identity of multisets''':
| |
| * '''Input:''' two multisets <math>A=\{a_1,a_2,\ldots, a_n\}</math> and <math>B=\{b_1,b_2,\ldots, b_n\}</math> where <math>a_1,a_2,\ldots,b_1,b_2,\ldots,b_n\in \{1,2,\ldots,n\}</math>.
| |
| * Determine whether <math>A=B</math> (multiset equivalence).
| |
|
| |
| Here for a '''multiset''' <math>A=\{a_1,a_2,\ldots, a_n\}</math>, its elements <math>a_i</math> are not necessarily distinct. The '''multiplicity''' of an element <math>a_i</math> in a multiset <math>A</math> is the number of times <math>a_i</math> appears in <math>A</math>. Two multisets <math>A</math> and <math>B</math> are equivalent if they contain the same set of elements and the multiplicities of every element in <math>A</math> and <math>B</math> are equal.
| |
|
| |
| Obviously the above problem of checking distinctness can be treated as a special case of checking identity of multisets: by checking the identity of the multiset <math>A</math> and set <math>\{1,2,\ldots, n\}</math>.
| |
|
| |
| The following fingerprinting function for multisets was introduced by Lipton for solving multiset identity testing.
| |
|
| |
| {{Theorem|Fingerprint for multiset|
| |
| :Let <math>p</math> be a uniform random prime chosen from the interval <math>[(n\log n)^2,2(n\log n)^2]</math>. By Chebyshev's theorem, such prime must exist. And consider the the finite field <math>\mathbb{Z}_p=[p]</math>.
| |
|
| |
| :Given a multiset <math>A=\{a_1,a_2,\ldots,a_n\}</math>, we define a univariate polynomial <math>f_A\in\mathbb{Z}_p[x]</math> over the finite field <math>\mathbb{Z}_p</math> as follows:
| |
| ::<math>f_A(x)=\prod_{i=1}^n(x-a_i)</math>,
| |
| :where <math>+</math> and <math>\cdot</math> are defined over the finite field <math>\mathbb{Z}_p</math>.
| |
|
| |
|
| :We then define the random fingerprinting function as:
| | The variance of the sum of pairwise independent random variables is equal to the sum of variances. |
| ::<math>\mathrm{FING}(A)=f_A(r)=\prod_{i=1}^n(r-a_i)</math>, | | {{Theorem |
| :where <math>r</math> is chosen uniformly at random from <math>\mathbb{Z}_p</math>.
| | |Theorem| |
| | :For '''pairwise''' independent random variables <math>X_1,X_2,\ldots,X_n</math>, |
| | ::<math>\begin{align} |
| | \mathbf{Var}\left[\sum_{i=1}^n X_i\right]=\sum_{i=1}^n\mathbf{Var}[X_i]. |
| | \end{align}</math> |
| }} | | }} |
|
| |
|
| Since all computations of <math>\mathrm{FING}(A)=\prod_{i=1}^n(r-a_i)</math> are over the finite field <math>\mathbb{Z}_p</math>, the space cost for computing the fingerprint <math>\mathrm{FING}(A)</math> is only <math>O(\log p)=O(\log n)</math>.
| | ;Remark |
| | | :The theorem holds for '''pairwise''' independent random variables, a much weaker independence requirement than the '''mutual''' independence. This makes the second-moment methods very useful for pairwise independent random variables. |
| Moreover, the fingerprinting function <math>\mathrm{FING}(A)</math> is invariant under permutation of elements of the multiset <math>A=\{a_1,a_2,\ldots,a_n\}</math>, thus it is indeed a function of multisets (meaning every multiset has only one fingerprint). Therefore, if <math>A=B</math> then <math>\mathrm{FING}(A)=\mathrm{FING}(B)</math>.
| |
|
| |
|
| For two distinct multisets <math>A\neq B</math>, it is possible that <math>\mathrm{FING}(A)=\mathrm{FING}(B)</math>, but the following theorem due to Lipton bounds this error probability of false positive.
| | === Variance of binomial distribution === |
| | | For a Bernoulli trial with parameter <math>p</math>. |
| {{Theorem|Theorem (Lipton 1989)|
| |
| :Let <math>A=\{a_1,a_2,\ldots,a_n\}</math> and <math>B=\{b_1,b_2,\ldots,b_n\}</math> be two multisets whose elements are from <math>\{1,2,\ldots,n\}</math>. If <math>A\neq B</math>, then
| |
| ::<math>\Pr[\mathrm{FING}(A)= \mathrm{FING}(B)]=O\left(\frac{1}{n}\right)</math>.
| |
| }}
| |
| {{Proof|
| |
| Let <math>\tilde{f}_A(x)=\prod_{i=1}^n(x-a_i)</math> and <math>\tilde{f}_B(x)=\prod_{i=1}^n(x-b_i)</math> be two univariate polynomials defined over reals <math>\mathbb{R}</math>. Note that in contrast to <math>f_A(x)</math> and <math>f_B(x)</math>, the <math>+</math> and <math>\cdot</math> in <math>\tilde{f}_A(x), \tilde{f}_B(x)</math> do not modulo <math>p</math>. It is easy to verify that the polynomials <math>\tilde{f}_A(x), \tilde{f}_B(x)</math> have the following properties:
| |
| *<math>\tilde{f}_A\equiv \tilde{f}_B</math> if and only if <math>A=B</math>. Here <math>A=B</math> means the multiset equivalence.
| |
| *By the properties of finite field, for any value <math>r\in\mathbb{Z}_p</math>, it holds that <math>f_A(r)=\tilde{f}_A(r)\bmod p</math> and <math>f_B(r)=\tilde{f}_B(r)\bmod p</math>.
| |
| | |
| Therefore, assuming that <math>A\neq B</math>, we must have <math>\tilde{f}_A(x)\not\equiv \tilde{f}_B(x)</math>. Then by the law of total probability:
| |
| :<math> | | :<math> |
| \begin{align} | | X=\begin{cases} |
| \Pr[\mathrm{FING}(A)= \mathrm{FING}(B)] | | 1& \mbox{with probability }p\\ |
| &=
| | 0& \mbox{with probability }1-p |
| \Pr\left[f_A(r)=f_B(r)\mid f_A\not\equiv f_B\right]\Pr[f_A\not\equiv f_B]\\
| | \end{cases} |
| &\quad\,\,+\Pr\left[f_A(r)=f_B(r)\mid f_A\equiv f_B\right]\Pr[f_A\equiv f_B]\\ | |
| &\le \Pr\left[f_A(r)=f_B(r)\mid f_A\not\equiv f_B\right]+\Pr[f_A\equiv f_B].
| |
| \end{align} | |
| </math> | | </math> |
| Note that the degrees of <math>f_A,f_B</math> are at most <math>n</math> and <math>r</math> is chosen uniformly from <math>[p]</math>. By the Schwartz-Zippel theorem for univariate polynomials, the first probability
| | The variance is |
| :<math> | | :<math> |
| \Pr\left[f_A(r)=f_B(r)\mid f_A\not\equiv f_B\right]\le \frac{n}{p}=o\left(\frac{1}{n}\right), | | \mathbf{Var}[X]=\mathbf{E}[X^2]-(\mathbf{E}[X])^2=\mathbf{E}[X]-(\mathbf{E}[X])^2=p-p^2=p(1-p). |
| </math> | | </math> |
| since <math>p</math> is chosen from the interval <math>[(n\log n)^2,2(n\log n)^2]</math>.
| |
|
| |
|
| For the second probability <math>\Pr[f_A\equiv f_B]</math>, recall that <math>\tilde{f}_A\not\equiv \tilde{f}_B</math>, therefore there is at least a non-zero coefficient <math>c\le n^n</math> in <math>\tilde{f}_A-\tilde{f}_B</math>. The event <math>f_A\equiv f_B</math> occurs only if <math>c\bmod p=0</math>, which means
| | Let <math>Y</math> be a binomial random variable with parameter <math>n</math> and <math>p</math>, i.e. <math>Y=\sum_{i=1}^nY_i</math>, where <math>Y_i</math>'s are i.i.d. Bernoulli trials with parameter <math>p</math>. The variance is |
| :<math> | | :<math> |
| \begin{align} | | \begin{align} |
| \Pr[f_A\equiv f_B] | | \mathbf{Var}[Y] |
| &\le \Pr[c\bmod p=0]\\ | | &= |
| &=\frac{\text{number of prime factors of }c}{\text{number of primes in }[(n\log n)^2,2(n\log n)^2]}\\ | | \mathbf{Var}\left[\sum_{i=1}^nY_i\right]\\ |
| &\le \frac{n\log_2n}{\pi(2(n\log n)^2)-\pi((n\log n)^2)}. | | &= |
| | \sum_{i=1}^n\mathbf{Var}\left[Y_i\right] &\qquad (\mbox{Independence})\\ |
| | &= |
| | \sum_{i=1}^np(1-p) &\qquad (\mbox{Bernoulli})\\ |
| | &= |
| | p(1-p)n. |
| \end{align} | | \end{align} |
| </math> | | </math> |
| By the prime number theorem, <math>\pi(N)\rightarrow \frac{N}{\ln N}</math> as <math>N\to\infty</math>. Therefore,
| | |
| | == Chebyshev's inequality == |
| | {{Theorem |
| | |Theorem (Chebyshev's Inequality)| |
| | :For any <math>t>0</math>, |
| | ::<math>\begin{align} |
| | \Pr\left[|X-\mathbf{E}[X]| \ge t\right] \le \frac{\mathbf{Var}[X]}{t^2}. |
| | \end{align}</math> |
| | }} |
| | {{Proof| Observe that |
| | :<math>\Pr[|X-\mathbf{E}[X]| \ge t] = \Pr[(X-\mathbf{E}[X])^2 \ge t^2].</math> |
| | Since <math>(X-\mathbf{E}[X])^2</math> is a nonnegative random variable, we can apply Markov's inequality, such that |
| :<math> | | :<math> |
| \Pr[f_A\equiv f_B]=O\left(\frac{n\log n}{n^2\log n}\right)=O\left(\frac{1}{n}\right). | | \Pr[(X-\mathbf{E}[X])^2 \ge t^2] \le |
| | \frac{\mathbf{E}[(X-\mathbf{E}[X])^2]}{t^2} |
| | =\frac{\mathbf{Var}[X]}{t^2}. |
| </math> | | </math> |
| Combining everything together, we have
| |
| <math>\Pr[\mathrm{FING}(A)= \mathrm{FING}(B)]=O\left(\frac{1}{n}\right)</math>.
| |
| }} | | }} |