高级算法 (Fall 2023)/Fingerprinting

Checking Matrix Multiplication

The evolution of time complexity $O(n^{\omega })$ for matrix multiplication.

Let $\mathbb{F}$ be a feild (you may think of it as the filed $\mathbb{Q}$ of rational numbers, or the finite field ${\mathbb{Z}}_p$ of integers modulo prime $p$). We suppose that each field operation (addition, subtraction, multiplication, division) has unit cost. This model is called the unit-cost RAM model, which is an ideal abstraction of a computer.

Consider the following problem:

• Input: Three $n\times n$ matrices $A$, $B$, and $C$ over the field $\mathbb{F}$.
• Output: "yes" if $C=AB$ and "no" if otherwise.

A naive way to solve this is to multiply $A$ and $B$ and compare the result with $C$. The straightforward algorithm for matrix multiplication takes $O(n^3)$ time, assuming that each arithmetic operation takes unit time. The Strassen's algorithm discovered in 1969 now implemented by many numerical libraries runs in time $O(n^{{\log }_{2}7})\approx O(n^{2.81})$. Strassen's algorithm starts the search for fast matrix multiplication algorithms. The Coppersmith–Winograd algorithm discovered in 1987 runs in time $O(n^{2.376})$ but is only faster than Strassens' algorithm on extremely large matrices due to the very large constant coefficient. This has been the best known for decades, until recently Stothers got an $O(n^{2.374})$ algorithm in his PhD thesis in 2010, and independently Vassilevska Williams got an $O(n^{2.373})$ algorithm in 2012. Both these improvements are based on generalization of Coppersmith–Winograd algorithm. It is unknown whether the matrix multiplication can be done in time $O(n^{2+o(1)})$.

Freivalds Algorithm

The following is a very simple randomized algorithm due to Freivalds, running in $O(n^2)$ time:

Algorithm (Freivalds, 1979)

pick a vector $r\in \{0,1{\}}^n$ uniformly at random; if $A(Br)=Cr$ then return "yes" else return "no";

The product $A(Br)$ is computed by first multiplying $Br$ and then $A(Br)$. The running time of Freivalds algorithm is $O(n^2)$ because the algorithm computes 3 matrix-vector multiplications.

If $AB=C$ then $A(Br)=Cr$ for any $r\in \{0,1{\}}^n$, thus the algorithm will return a "yes" for any positive instance ($AB=C$). But if $AB\ne C$ then the algorithm will make a mistake if it chooses such an $r$ that $ABr=Cr$. However, the following lemma states that the probability of this event is bounded.

Lemma

If $AB\ne C$ then for a uniformly random $r\in \{0,1{\}}^n$, $Pr[ABr=Cr]\le \frac{1}{2}$.

Proof. Let $D=AB-C$. The event $ABr=Cr$ is equivalent to that $Dr=0$. It is then sufficient to show that for a $D\ne \mathbf{0}$, it holds that $Pr[Dr=\mathbf{0}]\le \frac{1}{2}$. Since $D\ne \mathbf{0}$, it must have at least one non-zero entry. Suppose that $D_{ij}\ne 0$. We assume the event that $Dr=\mathbf{0}$. In particular, the $i$-th entry of $Dr$ is $(Dr{)}_i=\sum _{k=1}^n{D}_{ik}{r}_k=0.$ The $r_j$ can be calculated by $r_j=-\frac{1}{D_{ij}}\sum _{k\ne j}^n{D}_{ik}{r}_k.$ Once all other entries $r_k$ with $k\ne j$ are fixed, there is a unique solution of $r_j$. Therefore, the number of $r\in \{0,1{\}}^n$ satisfying $Dr=\mathbf{0}$ is at most $2^{n-1}$. The probability that $ABr=Cr$ is bounded as $Pr[ABr=Cr]=Pr[Dr=\mathbf{0}]\le \frac{2^{n-1}}{2^n}=\frac{1}{2}$. $◻$

When $AB=C$, Freivalds algorithm always returns "yes"; and when $AB\ne C$, Freivalds algorithm returns "no" with probability at least 1/2.

To improve its accuracy, we can run Freivalds algorithm for $k$ times, each time with an independent $r\in \{0,1{\}}^n$, and return "yes" if and only if all running instances returns "yes".

Freivalds' Algorithm (multi-round)

pick $k$ vectors $r_1,r_2,\dots ,r_k\in \{0,1{\}}^n$ uniformly and independently at random; if $A(B{r}_i)=C{r}_i$ for all $i=1,\dots ,k$ then return "yes" else return "no";

If $AB=C$, then the algorithm returns a "yes" with probability 1. If $AB\ne C$, then due to the independence, the probability that all $r_i$ have $AB{r}_i=C_i$ is at most $2^{-k}$, so the algorithm returns "no" with probability at least $1-2^{-k}$. For any $0<ϵ<1$, choose $k={\log }_2\frac{1}{ϵ}$. The algorithm runs in time $O(n^2{\log }_2\frac{1}{ϵ})$ and has a one-sided error (false positive) bounded by $ϵ$.

Polynomial Identity Testing (PIT)

The Polynomial Identity Testing (PIT) is such a problem: given as input two polynomials, determine whether they are identical. It plays a fundamental role in Identity Testing problems.

First, let's consider the univariate ("one variable") case:

• Input: two polynomials $f,g\in \mathbb{F}[x]$ of degree $d$.
• Determine whether $f\equiv g$ ($f$ and $g$ are identical).

Here the $\mathbb{F}[x]$ denotes the ring of univariate polynomials on a field $\mathbb{F}$. More precisely, a polynomial $f\in \mathbb{F}[x]$ is

$f(x)=\sum _{i=0}^{\mathrm{\infty }}{a}_i{x}^i$,

where the coefficients $a_i$ are taken from the field $\mathbb{F}$, and the addition and multiplication are also defined over the field $\mathbb{F}$. And:

• the degree of $f$ is the highest $i$ with non-zero $a_i$;
• a polynomial $f$ is a zero-polynomial, denoted as $f\equiv 0$, if all coefficients $a_i=0$.

Alternatively, we can consider the following equivalent problem by comparing the polynomial $f-g$ (whose degree is at most $d$) with the zero-polynomial:

• Input: a polynomial $f\in \mathbb{F}[x]$ of degree $d$.
• Determine whether $f\equiv 0$ ($f$ is the 0 polynomial).

The problem is trivial if the input polynomial $f$ is given explicitly: one can trivially solve the problem by checking whether all $d+1$ coefficients are $0$. To make the problem nontrivial, we assume that the input polynomial is given implicitly as a black box (also called an oracle): the only way the algorithm can access to $f$ is to evaluate $f(x)$ over some $x$ from the field $\mathbb{F}$, $x$ chosen by the algorithm.

A straightforward deterministic algorithm is to evaluate $f(x_1),f(x_2),\dots ,f(x_{d+1})$ over $d+1$ distinct elements $x_1,x_2,\dots ,x_{d+1}$ from the field $\mathbb{F}$ and check whether they are all zero. By the fundamental theorem of algebra, also known as polynomial interpolations, this guarantees to verify whether a degree-$d$ univariate polynomial $f\equiv 0$.

Fundamental Theorem of Algebra

Any non-zero univariate polynomial of degree $d$ has at most $d$ roots.

The reason for this fundamental theorem holding generally over any field $\mathbb{F}$ is that any univariate polynomial of degree $d$ factors uniquely into at most $d$ irreducible polynomials, each of which has at most one root.

The following simple randomized algorithm is natural:

Algorithm for PIT

suppose we have a finite subset $S\subseteq \mathbb{F}$ (to be specified later); pick $r\in S$ uniformly at random; if $f(r)=0$ then return “yes” else return “no”;

This algorithm evaluates $f$ at one point chosen uniformly at random from a finite subset $S\subseteq \mathbb{F}$. It is easy to see the followings:

• If $f\equiv 0$, the algorithm always returns "yes", so it is always correct.
• If $f\not\equiv 0$, the algorithm may wrongly return "yes" (a false positive). But this happens only when the random $r$ is a root of $f$. By the fundamental theorem of algebra, $f$ has at most $d$ roots, so the probability that the algorithm is wrong is bounded as

$Pr[f(r)=0]\le \frac{d}{|S|}.$

By fixing $S\subseteq \mathbb{F}$ to be an arbitrary subset of size $|S|=2d$, this probability of false positive is at most $1/2$. We can reduce it to an arbitrarily small constant $\delta$ by repeat the above testing independently for ${\log }_2\frac{1}{\delta }$ times, since the error probability decays geometrically as we repeat the algorithm independently.

Communication Complexity of Equality

The communication complexity is introduced by Andrew Chi-Chih Yao as a model of computation with more than one entities, each with partial information about the input.

Assume that there are two entities, say Alice and Bob. Alice has a private input $a$ and Bob has a private input $b$. Together they want to compute a function $f(a,b)$ by communicating with each other. The communication follows a predefined communication protocol (the "algorithm" in this model). The complexity of a communication protocol is measured by the number of bits communicated between Alice and Bob in the worst case.

The problem of checking identity is formally defined by the function EQ as follows: $\mathrm{E}\mathrm{Q}:\{0,1{\}}^n\times \{0,1{\}}^n\to \{0,1\}$ and for any $a,b\in \{0,1{\}}^n$,

$\mathrm{E}\mathrm{Q}(a,b)=\{\begin{array}{ll}1& \text{if }a=b,\\ 0& \text{otherwise.}\end{array}$

A trivial way to solve EQ is to let Bob send his entire input string $b$ to Alice and let Alice check whether $a=b$. This costs $n$ bits of communications.

It is known that for deterministic communication protocols, this is the best we can get for computing EQ.

Theorem (Yao 1979)

Any deterministic communication protocol computing EQ on two $n$-bit strings costs $n$ bits of communication in the worst-case.

This theorem is much more nontrivial to prove than it looks, because Alice and Bob are allowed to interact with each other in arbitrary ways. The proof of this theorem is in Yao's celebrated paper in 1979 with a humble title. It pioneered the field of communication complexity.

If we allow randomness in protocols, and also tolerate a small probabilistic error, the problem can be solved with significantly less communications. To present this randomized protocol, we need a few preparations:

• We represent the inputs $a,b\in \{0,1{\}}^n$ of Alice and Bob as two univariate polynomials of degree at most $n-1$, respectively

$f(x)=\sum _{i=0}^{n-1}{a}_i{x}^i$ and $g(x)=\sum _{i=0}^{n-1}{b}_i{x}^i$.

• The two polynomials $f$ and $g$ are defined over finite field ${\mathbb{Z}}_p=\{0,1,\dots ,p-1\}$ for some suitable prime $p$ (to be specified later), which means the additions and multiplications are modulo $p$.

The randomized communication protocol is then as follows:

A randomized protocol for EQ

Bob does: pick $r\in {\mathbb{Z}}_p$ uniformly at random; send $r$ and $g(r)$ to Alice; Upon receiving $r$ and $g(r)$ Alice does: compute $f(r)$; If $f(r)=g(r)$ return "yes"; else return "no".

The communication complexity of the protocol is given by the number of bits used to represent the values of $r$ and $g(r)$. Since the polynomials are defined over finite field ${\mathbb{Z}}_p$ and the random number $f$ is also chosen from ${\mathbb{Z}}_p$, this is bounded by $O(\log p)$.

On the other hand the protocol makes mistakes only when $a\ne b$ but wrongly answers "yes". This happens only when $f\not\equiv g$ but $f(r)=g(r)$. The degrees of $f,g$ are at most $n-1$, and $r$ is chosen among $p$ distinct values, we have

$Pr[f(r)=g(r)]\le \frac{n-1}{p}$.

By choosing $p$ to be a prime in the interval $[n^2,2n^2]$ (by Chebyshev's theorem, such prime $p$ always exists), the above randomized communication protocol solves the Equality function EQ with an error probability of false positive at most $O(1/n)$, with communication complexity $O(\log n)$, an EXPONENTIAL improvement to ANY deterministic communication protocol!

Schwartz-Zippel Theorem

Now let's see the the true form of Polynomial Identity Testing (PIT), for multivariate polynomials:

• Input: two $n$-variate polynomials $f,g\in \mathbb{F}[x_1,x_2,\dots ,x_n]$ of degree $d$.
• Determine whether $f\equiv g$.

The $\mathbb{F}[x_1,x_2,\dots ,x_n]$ is the ring of multivariate polynomials over field $\mathbb{F}$. An $n$-variate polynomial of degree $d$, written as a sum of monomials, is:

$f(x_1,x_2,\dots ,x_n)=\sum _{\genfrac{}{}{0ex}{}{i_1,i_2,\dots ,i_n\ge 0}{i_1+i_2+\cdots +i_n\le d}}{a}_{i_1,i_2,\dots ,i_n}{x}_1^{i_1}{x}_2^{i_2}\cdots x_n^{i_n}$.

The degree or total degree of a monomial $a_{i_1,i_2,\dots ,i_n}{x}_1^{i_1}{x}_2^{i_2}\cdots x_n^{i_n}$ is given by $i_1+i_2+\cdots +i_n$ and the degree of a polynomial $f$ is the maximum degree of monomials of nonzero coefficients.

As before, we also consider the following equivalent problem:

• Input: a polynomial $f\in \mathbb{F}[x_1,x_2,\dots ,x_n]$ of degree $d$.
• Determine whether $f\equiv 0$.

If $f$ is written explicitly as a sum of monomials, then the problem can be solved by checking whether all coefficients, and there at most $(\genfrac{}{}{0ex}{}{n+d}{d})\le (n+d{)}^d$ coefficients in an $n$-variate polynomial of degree at most $d$.

A multivariate polynomial $f$ can also be presented in its product form, for example:

Example

The Vandermonde matrix $M=M(x_1,x_2,\dots ,x_n)$ is defined as that $M_{ij}=x_i^{j-1}$, that is $M=\left[\begin{array}{ccccc}1& x_1& x_1^2& \dots & x_1^{n-1}\\ 1& x_2& x_2^2& \dots & x_2^{n-1}\\ 1& x_3& x_3^2& \dots & x_3^{n-1}\\ ⋮& ⋮& ⋮& \ddots & ⋮\\ 1& x_n& x_n^2& \dots & x_n^{n-1}\end{array}\right]$. Let $f$ be the polynomial defined as $f(x_1,\dots ,x_n)=det(M)=\prod _{j<i}(x_i-x_j).$

For polynomials in product form, it is quite efficient to evaluate the polynomial at any specific point from the field over which the polynomial is defined, however, expanding the polynomial to a sum of monomials can be very expensive.

The following is a simple randomized algorithm for testing identity of multivariate polynomials:

Randomized algorithm for multivariate PIT

suppose we have a finite subset $S\subseteq \mathbb{F}$ (to be specified later); pick $r_1,r_2,\dots ,r_n\in S$ uniformly and independently at random; if $f(\overrightarrow{r})=f(r_1,r_2,\dots ,r_n)=0$ then return “yes” else return “no”;

This algorithm evaluates $f$ at one point chosen uniformly from an $n$-dimensional cube $S^n$, where $S\subseteq \mathbb{F}$ is a finite subset. And:

• If $f\equiv 0$, the algorithm always returns "yes", so it is always correct.
• If $f\not\equiv 0$, the algorithm may wrongly return "yes" (a false positive). But this happens only when the random $\overrightarrow{r}=(r_1,r_2,\dots ,r_n)$ is a root of $f$. The probability of this bad event is upper bounded by the following famous result due to Schwartz (1980) and Zippel (1979).

Schwartz-Zippel Theorem

Let $f\in \mathbb{F}[x_1,x_2,\dots ,x_n]$ be a multivariate polynomial of degree $d$ over a field $\mathbb{F}$. If $f\not\equiv 0$, then for any finite set $S\subset \mathbb{F}$, and $r_1,r_2\dots ,r_n\in S$ chosen uniformly and independently at random, $Pr[f(r_1,r_2,\dots ,r_n)=0]\le \frac{d}{|S|}.$

The Schwartz-Zippel Theorem states that for any nonzero $n$-variate polynomial of degree at most $d$, the number of roots in any cube $S^n$ is at most $d\cdot |S{|}^{n-1}$.

Dana Moshkovitz gave a surprisingly simply and elegant proof of Schwartz-Zippel Theorem, using some advanced ideas. Now we introduce the standard proof by induction.

Proof. The theorem is proved by induction on $n$. Induction basis: For $n=1$, this is the univariate case. Assume that $f\not\equiv 0$. Due to the fundamental theorem of algebra, any polynomial $f(x)$ of degree at most $d$ must have at most $d$ roots, thus $Pr[f(r)=0]\le \frac{d}{|S|}.$ Induction hypothesis: Assume the theorem holds for any $m$-variate polynomials for all $m<n$. Induction step: For any $n$-variate polynomial $f(x_1,x_2,\dots ,x_n)$ of degree at most $d$, we write $f$ as $f(x_1,x_2,\dots ,x_n)=\sum _{i=0}^k{x}_n^i{f}_i(x_1,x_2,\dots ,x_{n-1})$, where $k$ is the highest degree of $x_n$, which means the degree of $f_k$ is at most $d-k$ and $f_k\not\equiv 0$. In particular, we write $f$ as a sum of two parts: $f(x_1,x_2,\dots ,x_n)=x_n^k{f}_k(x_1,x_2,\dots ,x_{n-1})+\overline{f}(x_1,x_2,\dots ,x_n)$, where both $f_k$ and $\overline{f}$ are polynomials, such that $f_k\not\equiv 0$ is as above, whose degree is at most $d-k$; $\overline{f}(x_1,x_2,\dots ,x_n)=\sum _{i=0}^{k-1}{x}_n^i{f}_i(x_1,x_2,\dots ,x_{n-1})$, thus $\overline{f}(x_1,x_2,\dots ,x_n)$ has no $x_n^k$ factor in any term. By the law of total probability, we have $\begin{array}{rl}& Pr[f(r_1,r_2,\dots ,r_n)=0]\\ =& Pr[f(\overrightarrow{r})=0\mid f_k(r_1,r_2,\dots ,r_{n-1})=0]\cdot Pr[f_k(r_1,r_2,\dots ,r_{n-1})=0]\\ & +Pr[f(\overrightarrow{r})=0\mid f_k(r_1,r_2,\dots ,r_{n-1})\ne 0]\cdot Pr[f_k(r_1,r_2,\dots ,r_{n-1})\ne 0].\end{array}$ Note that $f_k(r_1,r_2,\dots ,r_{n-1})$ is a polynomial on $n-1$ variables of degree $d-k$ such that $f_k\not\equiv 0$. By the induction hypothesis, we have $\begin{array}{rlr}(\ast )& & Pr[f_k(r_1,r_2,\dots ,r_{n-1})=0]\le \frac{d-k}{|S|}.\end{array}$ Now we look at the case conditioning on $f_k(r_1,r_2,\dots ,r_{n-1})\ne 0$. Recall that $\overline{f}(x_1,\dots ,x_n)$ has no $x_n^k$ factor in any term, thus the condition $f_k(r_1,r_2,\dots ,r_{n-1})\ne 0$ guarantees that $f(r_1,\dots ,r_{n-1},x_n)=x_n^k{f}_k(r_1,r_2,\dots ,r_{n-1})+\overline{f}(r_1,r_2,\dots ,r_{n-1},x_n)=g_{r_1,\dots ,r_{n-1}}(x_n)$ is a nonzero univariate polynomial of $x_n$ such that the degree of $g_{r_1,\dots ,r_{n-1}}(x_n)$ is $k$ and $g_{r_1,\dots ,r_{n-1}}\not\equiv 0$, for which we already known that the probability $g_{r_1,\dots ,r_{n-1}}(r_n)=0$ is at most $\frac{k}{|S|}$. Therefore, $\begin{array}{rlr}(\ast \ast )& & Pr[f(\overrightarrow{r})=0\mid f_k(r_1,r_2,\dots ,r_{n-1})\ne 0]=Pr[g_{r_1,\dots ,r_{n-1}}(r_n)=0\mid f_k(r_1,r_2,\dots ,r_{n-1})\ne 0]\le \frac{k}{|S|}\end{array}$. Substituting both $(\ast )$ and $(\ast \ast )$ back in the total probability, we have $Pr[f(r_1,r_2,\dots ,r_n)=0]\le \frac{d-k}{|S|}+\frac{k}{|S|}=\frac{d}{|S|},$ which proves the theorem. $◻$

Detecting perfect matching

TBA

Edmonds matrix

Fingerprinting

The polynomial identity testing algorithm in the Schwartz-Zippel theorem can be abstracted as the following framework: Suppose we want to compare two objects $X$ and $Y$. Instead of comparing them directly, we compute random fingerprints $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(X)$ and $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(Y)$ of them and compare the fingerprints.

The fingerprints has the following properties:

• $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(\cdot )$ is a function, meaning that if $X=Y$ then $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(X)=\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(Y)$.
• It is much easier to compute and compare the fingerprints.
• Ideally, the domain of fingerprints is much smaller than the domain of original objects, so storing and comparing fingerprints are easy. This means the fingerprint function $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(\cdot )$ cannot be an injection (one-to-one mapping), so it's possible that different $X$ and $Y$ are mapped to the same fingerprint. We resolve this by making fingerprint function randomized, and for $X\ne Y$, we want the probability $Pr[\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(X)=\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(Y)]$ to be small.

In Schwartz-Zippel theorem, the objects to compare are polynomials from $\mathbb{F}[x_1,\dots ,x_n]$. Given a polynomial $f\in \mathbb{F}[x_1,\dots ,x_n]$, its fingerprint is computed as $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(f)=f(r_1,\dots ,r_n)$ for $r_i$ chosen independently and uniformly at random from some fixed set $S\subseteq \mathbb{F}$.

With this generic framework, for various identity testing problems, we may design different fingerprints $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(\cdot )$.

Communication protocols for Equality

Now consider again the communication model where the two players Alice with a private input $x\in \{0,1{\}}^n$ and Bob with a private input $y\in \{0,1{\}}^n$ together compute a function $f(x,y)$ by running a communication protocol.

We still consider the communication protocols for the equality function EQ

$\mathrm{E}\mathrm{Q}(x,y)=\{\begin{array}{ll}1& \text{if }x=y,\\ 0& \text{otherwise.}\end{array}$

With the language of fingerprinting, this communication problem can be solved by the following generic scheme:

Communication protocol for EQ by fingerprinting

Bob does: choose a random fingerprint function $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(\cdot )$ and compute the fingerprint of her input $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(y)$; sends both the description of $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(\cdot )$ and the value of $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(y)$ to Alice; Upon receiving the description of $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(\cdot )$ and the value of $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(y)$, Alice does: computes $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(x)$ and check whether $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(x)=\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(y)$.

In this way we have a randomized communication protocol for the equality function EQ with false positive. The communication cost as well as the error probability are reduced to the question of how to design this random fingerprint function $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(\cdot )$ to guarantee:

1. A random fingerprint function $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(\cdot )$ can be described succinctly.
2. The range of $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(\cdot )$ is small, so the fingerprints are succinct.
3. If $x\ne y$, the probability $Pr[\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(x)=\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(y)]$ is small.

Fingerprinting by PIT

As before, we can define the fingerprint function as: for any bit-string $x\in \{0,1{\}}^n$, its random fingerprint is $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(x)=\sum _{i=1}^n{x}_i{r}^i$, where the additions and multiplications are defined over a finite field ${\mathbb{Z}}_p$, and $r$ is chosen uniformly at random from ${\mathbb{Z}}_p$, where $p$ is some suitable prime which can be represented in $\mathrm{\Theta }(\log n)$ bits. More specifically, we can choose $p$ to be any prime from the interval $[n^2,2n^2]$. Due to Chebyshev's theorem, such prime must exist.

As we have shown before, it takes $O(\log p)=O(\log n)$ bits to represent $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(y)$ and to describe the random function $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(\cdot )$ (since it a random function $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(\cdot )$ from this family is uniquely identified by a random $r\in {\mathbb{Z}}_p$, which can be represented within $\log p=O(\log n)$ bits). And it follows easily from the fundamental theorem of algebra that for any distinct $x,y\in \{0,1{\}}^n$,

$Pr[\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(x)=\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(y)]\le \frac{n-1}{p}\le \frac{1}{n}.$

Fingerprinting by randomized checksum

Now we consider a new fingerprint function: We treat each input string $x\in \{0,1{\}}^n$ as the binary representation of a number, and let $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(x)=xmodp$ for some random prime $p$ chosen from $[k]=\{0,1,\dots ,k-1\}$, for some $k$ to be specified later.

Now a random fingerprint function $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(\cdot )$ can be uniquely identified by this random prime $p$. The new communication protocol for EQ with this fingerprint is as follows:

Communication protocol for EQ by random checksum

Bob does: for some parameter $k$ (to be specified), choose a prime $p\in [k]$ uniformly at random; send $p$ and $xmodp$ to Alice; Upon receiving $p$ and $xmodp$, Alice does: check whether $xmodp=ymodp$.

The number of bits to be communicated is obviously $O(\log k)$. When $x\ne y$, we want to upper bound the error probability $Pr[xmodp=ymodp]$.

Suppose without loss of generality $x>y$. Let $z=x-y$. Then $z<2^n$ since $x,y\in [2^n]$, and $z\ne 0$ for $x\ne y$. It holds that $x\equiv y(\mathrm{mod}p)$ if and only if $p\mid z$. Therefore, we only need to upper bound the probability

$Pr[zmodp=0]$

for an arbitrarily fixed $0<z<2^n$, and a uniform random prime $p\in [k]$.

The probability $Pr[zmodp=0]$ is computed directly as

$Pr[zmodp=0]\le \frac{\text{the number of prime divisors of }z}{\text{the number of primes in }[k]}$.

For the numerator, any positive $z<2^n$ has at most $n$ prime factors. To see this, by contradiction assume that $z$ has more than $n$ prime factors. Note that any prime number is at least 2. Then $z$ must be greater than $2^n$, contradicting the fact that $z<2^n$.

For the denominator, we need to lower bound the number of primes in $[k]$. This is given by the celebrated Prime Number Theorem (PNT).

Prime Number Theorem

Let $\pi (k)$ denote the number of primes less than $k$. Then $\pi (k)\sim \frac{k}{\ln k}$ as $k\to \mathrm{\infty }$.

Therefore, by choosing $k=2n^2\ln n$, we have that for a $0<z<2^n$, and a random prime $p\in [k]$,

$Pr[zmodp=0]\le \frac{n}{\pi (k)}\sim \frac{1}{n}$,

which means the for any inputs $x,y\in \{0,1{\}}^n$, if $x\ne y$, then the false positive is bounded as

$Pr[\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(x)=\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(y)]\le Pr[|x-y|modp=0]\le \frac{1}{n}$.

Moreover, by this choice of parameter $k=2n^2\ln n$, the communication complexity of the protocol is bounded by $O(\log k)=O(\log n)$.

Pattern matching

Consider the following problem of pattern matching, which has nothing to do with communication complexity.

• Input: a string $x\in \{0,1{\}}^n$ and a "pattern" $y\in \{0,1{\}}^m$.
• Determine whether the pattern $y$ is a contiguous substring of $x$. Usually, we are also asked to find the location of the substring.

A naive algorithm trying every possible match runs in $O(nm)$ time. The more sophisticated KMP algorithm inspired by automaton theory runs in $O(n+m)$ time.

A simple randomized algorithm, due to Karp and Rabin, uses the idea of fingerprinting and also runs in $O(n+m)$ time.

Let $X(j)=x_j{x}_{j+1}\cdots x_{j+m-1}$ denote the substring of $x$ of length $m$ starting at position $j$.

Algorithm (Karp-Rabin)

pick a random prime $p\in [k]$; for $j=1$ to $n-m+1$ do if $X(j)modp=ymodp$ then report a match; return "no match";

So the algorithm just compares the $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(X(j))$ and $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(y)$ for every $j$, with the same definition of fingerprint function $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(\cdot )$ as in the communication protocol for EQ.

By the same analysis, by choosing $k=n^{2}m\ln (n^{2}m)$, the probability of a single false match is

$Pr[X(j)modp=ymodp\mid X(j)\ne y]=O\left(\frac{1}{n^2}\right)$.

By the union bound, the probability that a false match occurs is $O\left(\frac{1}{n}\right)$.

The algorithm runs in linear time if we assume that we can compute $X(j)modp$ for each $j$ in constant time. This outrageous assumption can be made realistic by the following observation.

Lemma

Let $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(a)=amodp$. $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(X(j+1))\equiv 2(\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(X(j))-2^{m-1}{x}_j)+x_{j+m}(\mathrm{mod}p)$.

Proof. It holds that $X(j+1)=2(X(j)-2^{m-1}{x}_j)+x_{j+m}$. So the equation holds on the finite field modulo $p$. $◻$

Due to this lemma, each fingerprint $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(X(j))$ can be computed in an incremental way, each in constant time. The running time of the algorithm is $O(n+m)$.

Checking distinctness

Consider the following problem of checking distinctness:

• Given a sequence $x_1,x_2,\dots ,x_n\in \{1,2,\dots ,n\}$, check whether every element of $\{1,2,\dots ,n\}$ appears exactly once.

Obviously this problem can be solved in linear time and linear space (in addition to the space for storing the input) by maintaining a $n$-bit vector that indicates which numbers among $\{1,2,\dots ,n\}$ have appeared.

When this $n$ is enormously large, $\mathrm{\Omega }(n)$ space cost is too expensive. We wonder whether we could solve this problem with a space cost (in addition to the space for storing the input) much less than $O(n)$. This can be done by fingerprinting if we tolerate a certain degree of inaccuracy.

We consider the following more generalized problem, checking identity of multisets:

• Input: two multisets $A=\{a_1,a_2,\dots ,a_n\}$ and $B=\{b_1,b_2,\dots ,b_n\}$ where $a_1,a_2,\dots ,b_1,b_2,\dots ,b_n\in \{1,2,\dots ,n\}$.
• Determine whether $A=B$ (multiset equivalence).

Here for a multiset $A=\{a_1,a_2,\dots ,a_n\}$, its elements $a_i$ are not necessarily distinct. The multiplicity of an element $a_i$ in a multiset $A$ is the number of times $a_i$ appears in $A$. Two multisets $A$ and $B$ are equivalent if they contain the same set of elements and the multiplicities of every element in $A$ and $B$ are equal.

Obviously the above problem of checking distinctness can be treated as a special case of checking identity of multisets: by checking the identity of the multiset $A$ and set $\{1,2,\dots ,n\}$.

The following fingerprinting function for multisets was introduced by Lipton for solving multiset identity testing.

Fingerprint for multiset

Let $p$ be a uniform random prime chosen from the interval $[(n\log n{)}^2,2(n\log n{)}^2]$. By Chebyshev's theorem, such prime must exist. And consider the the finite field ${\mathbb{Z}}_p=[p]$. Given a multiset $A=\{a_1,a_2,\dots ,a_n\}$, we define a univariate polynomial $f_A\in {\mathbb{Z}}_p[x]$ over the finite field ${\mathbb{Z}}_p$ as follows: $f_A(x)=\prod _{i=1}^n(x-a_i)$, where $+$ and $\cdot$ are defined over the finite field ${\mathbb{Z}}_p$. We then define the random fingerprinting function as: $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(A)=f_A(r)=\prod _{i=1}^n(r-a_i)$, where $r$ is chosen uniformly at random from ${\mathbb{Z}}_p$.

Since all computations of $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(A)=\prod _{i=1}^n(r-a_i)$ are over the finite field ${\mathbb{Z}}_p$, the space cost for computing the fingerprint $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(A)$ is only $O(\log p)=O(\log n)$.

Moreover, the fingerprinting function $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(A)$ is invariant under permutation of elements of the multiset $A=\{a_1,a_2,\dots ,a_n\}$, thus it is indeed a function of multisets (meaning every multiset has only one fingerprint). Therefore, if $A=B$ then $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(A)=\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(B)$.

For two distinct multisets $A\ne B$, it is possible that $\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(A)=\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(B)$, but the following theorem due to Lipton bounds this error probability of false positive.

Theorem (Lipton 1989)

Let $A=\{a_1,a_2,\dots ,a_n\}$ and $B=\{b_1,b_2,\dots ,b_n\}$ be two multisets whose elements are from $\{1,2,\dots ,n\}$. If $A\ne B$, then $Pr[\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(A)=\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(B)]=O\left(\frac{1}{n}\right)$.

Proof. Let ${\tilde{f}}_A(x)=\prod _{i=1}^n(x-a_i)$ and ${\tilde{f}}_B(x)=\prod _{i=1}^n(x-b_i)$ be two univariate polynomials defined over reals $\mathbb{R}$. Note that in contrast to $f_A(x)$ and $f_B(x)$, the $+$ and $\cdot$ in ${\tilde{f}}_A(x),{\tilde{f}}_B(x)$ do not modulo $p$. It is easy to verify that the polynomials ${\tilde{f}}_A(x),{\tilde{f}}_B(x)$ have the following properties: ${\tilde{f}}_A\equiv {\tilde{f}}_B$ if and only if $A=B$. Here $A=B$ means the multiset equivalence. By the properties of finite field, for any value $r\in {\mathbb{Z}}_p$, it holds that $f_A(r)={\tilde{f}}_A(r)modp$ and $f_B(r)={\tilde{f}}_B(r)modp$. Therefore, assuming that $A\ne B$, we must have ${\tilde{f}}_A(x)\not\equiv {\tilde{f}}_B(x)$. Then by the law of total probability: $\begin{array}{rl}Pr[\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(A)=\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(B)]& =Pr[f_A(r)=f_B(r)\mid f_A\not\equiv f_B]Pr[f_A\not\equiv f_B]\\ & +Pr[f_A(r)=f_B(r)\mid f_A\equiv f_B]Pr[f_A\equiv f_B]\\ & \le Pr[f_A(r)=f_B(r)\mid f_A\not\equiv f_B]+Pr[f_A\equiv f_B].\end{array}$ Note that the degrees of $f_A,f_B$ are at most $n$ and $r$ is chosen uniformly from $[p]$. By the Schwartz-Zippel theorem for univariate polynomials, the first probability $Pr[f_A(r)=f_B(r)\mid f_A\not\equiv f_B]\le \frac{n}{p}=o\left(\frac{1}{n}\right),$ since $p$ is chosen from the interval $[(n\log n{)}^2,2(n\log n{)}^2]$. For the second probability $Pr[f_A\equiv f_B]$, recall that ${\tilde{f}}_A\not\equiv {\tilde{f}}_B$, therefore there is at least a non-zero coefficient $c\le n^n$ in ${\tilde{f}}_A-{\tilde{f}}_B$. The event $f_A\equiv f_B$ occurs only if $cmodp=0$, which means $\begin{array}{rl}Pr[f_A\equiv f_B]& \le Pr[cmodp=0]\\ & =\frac{\text{number of prime factors of }c}{\text{number of primes in }[(n\log n{)}^2,2(n\log n{)}^2]}\\ & \le \frac{n{\log }_{2}n}{\pi (2(n\log n{)}^2)-\pi ((n\log n{)}^2)}.\end{array}$ By the prime number theorem, $\pi (N)\to \frac{N}{\ln N}$ as $N\to \mathrm{\infty }$. Therefore, $Pr[f_A\equiv f_B]=O\left(\frac{n\log n}{n^2\log n}\right)=O\left(\frac{1}{n}\right).$ Combining everything together, we have $Pr[\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(A)=\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{G}(B)]=O\left(\frac{1}{n}\right)$. $◻$