随机算法 (Fall 2011)/Universal hash families

From EtoneWiki
Jump to: navigation, search

Hashing

Hashing is one of the oldest tools in Computer Science. Knuth's memorandum in 1963 on analysis of hash tables is now considered to be the birth of the area of analysis of algorithms.

  • Knuth. Notes on "open" addressing, July 22 1963. Unpublished memorandum.

The idea of hashing is simple: an unknown set of data items (or keys) are drawn from a large universe where ; in order to store in a table of entries (slots), we assume a consistent mapping (called a hash function) from the universe to a small range .

This idea seems clever: we use a consistent mapping to deal with an arbitrary unknown data set. However, there is a fundamental flaw for hashing.

  • For sufficiently large universe (), for any function, there exists a bad data set , such that all items in are mapped to the same entry in the table.

A simple use of pigeonhole principle can prove the above statement.

To overcome this situation, randomization is introduced into hashing. We assume that the hash function is a random mapping from to . In order to ease the analysis, the following ideal assumption is used:

Simple Uniform Hash Assumption (SUHA or UHA, a.k.a. the random oracle model):

A uniform random function is available and the computation of is efficient.

Families of universal hash functions

The assumption of completely random function simplifies the analysis. However, in practice, truly uniform random hash function is extremely expensive to compute and store. Thus, this simple assumption can hardly represent the reality.

There are two approaches for implementing practical hash functions. One is to use ad hoc implementations and wish they may work. The other approach is to construct class of hash functions which are efficient to compute and store but with weaker randomness guarantees, and then analyze the applications of hash functions based on this weaker assumption of randomness.

This route was took by Carter and Wegman in 1977 while they introduced universal families of hash functions.

Definition (universal hash families)
Let be a universe with . A family of hash functions from to is said to be -universal if, for any items and for a hash function chosen uniformly at random from , we have
A family of hash functions from to is said to be strongly -universal if, for any items , any values , and for a hash function chosen uniformly at random from , we have

In particular, for a 2-universal family , for any elements , a uniform random has

For a strongly 2-universal family , for any elements and any values , a uniform random has

This behavior is exactly the same as uniform random hash functions on any pair of inputs. For this reason, a strongly 2-universal hash family are also called pairwise independent hash functions.

Construction of 2-universal family of hash functions

The construction of pairwise independent random variables via modulo a prime introduced in Section 1 already provides a way of constructing a strongly 2-universal hash family.

Let be a prime. The function is defined by

and the family is

Lemma
is strongly 2-universal.
Proof.
In Section 1, we have proved the pairwise independence of the sequence of , for , which directly implies that is strongly 2-universal.
The original construction of Carter-Wegman

What if we want to have hash functions from to for non-prime and ? Carter and Wegman developed the following method.

Suppose that the universe is , and the functions map to , where . For some prime , let

and the family

Note that unlike the first construction, now .

Lemma (Carter-Wegman)
is 2-universal.
Proof.
Due to the definition of , there are many different hash functions in , because each hash function in corresponds to a pair of and . We only need to count for any particular pair of that , the number of hash functions that .

We first note that for any , . This is because would imply that , which can never happen since and (note that for an ). Therefore, we can assume that and for .

Due to the Chinese remainder theorem, for any that , for any that , there is exact one solution to satisfying:

After modulo , every has at most many that but . Therefore, for every pair of that , there exist at most pairs of and such that , which means there are at most many hash functions having for . For uniformly chosen from , for any ,

We prove that is 2-universal.

A construction used in practice

The main issue of Carter-Wegman construction is the efficiency. The mod operation is very slow, and has been so for more than 30 years.

The following construction is due to Dietzfelbinger et al. It was published in 1997 and has been practically used in various applications of universal hashing.

The family of hash functions is from to . With a binary representation, the functions map binary strings of length to binary strings of length . Let

and the family

This family of hash functions does not exactly meet the requirement of 2-universal family. However, Dietzfelbinger et al proved that is close to a 2-universal family. Specifically, for any input values , for a uniformly random ,

So is within an approximation ratio of 2 to being 2-universal. The proof uses the fact that odd numbers are relative prime to a power of 2.

The function is extremely simple to compute in c language. We exploit that C-multiplication (*) of unsigned u-bit numbers is done , and have a one-line C-code for computing the hash function:

h_a(x) = (a*x)>>(u-v)

The bit-wise shifting is a lot faster than modular. It explains the popularity of this scheme in practice than the original Carter-Wegman construction.

Collision number

Consider a 2-universal family of hash functions from to . Let be a hash function chosen uniformly from . For a fixed set of distinct elements from , say , the elements are mapped to the hash values . This can be seen as throwing balls to bins, with pairwise independent choices of bins.

As in the balls-into-bins with full independence, we are curious about the questions such as the birthday problem or the maximum load. These questions are interesting not only because they are natural to ask in a balls-into-bins setting, but in the context of hashing, they are closely related to the performance of hash functions.

The old techniques for analyzing balls-into-bins rely too much on the independence of the choice of the bin for each ball, therefore can hardly be extended to the setting of 2-universal hash families. However, it turns out several balls-into-bins questions can somehow be answered by analyzing a very natural quantity: the number of collision pairs.

A collision pair for hashing is a pair of elements which are mapped to the same hash value, i.e. . Formally, for a fixed set of elements , for any , let the random variable

The total number of collision pairs among the items is

Since is 2-universal, for any ,

The expected number of collision pairs is

In particular, for , i.e. items are mapped to hash values by a pairwise independent hash function, the expected collision number is .

Birthday problem

In the context of hash functions, the birthday problem ask for the probability that there is no collision at all. Since collision is something that we want to avoid in the applications of hash functions, we would like to lower bound the probability of zero-collision, i.e. to upper bound the probability that there exists a collision pair.

The above analysis gives us an estimation on the expected number of collision pairs, such that . Apply the Markov's inequality, for , we have

When , the number of collision pairs is with probability at most , therefore with probability at least , there is no collision at all. Therefore, we have the following theorem.

Theorem
If is chosen uniformly from a 2-universal family of hash functions mapping the universe to where , then for any set of items, where , the probability that there exits a collision pair is

Recall that for mutually independent choices of bins, for some , the probability that a collision occurs is about . For constant , this gives an essentially same bound as the pairwise independent setting. Therefore, the behavior of pairwise independent hash function is essentially the same as the uniform random hash function for the birthday problem. This is easy to understand, because birthday problem is about the behavior of collisions, and the definition of 2-universal hash function can be interpreted as "functions that the probability of collision is as low as a uniform random function".

Maximum load

Suppose that a fixed set of distinct items are mapped to random locations by a pairwise independent hash function from to . The load of a entry of the table is the number of items in mapped to . We want to bound the maximum load.

For uniform random hash function, this is exactly the maximum load in the balls-into-bins game. And we know that for , the maximum load is with high probability. This bound can be proved either by counting or by the Chernoff bound.

For pairwise independent hash functions, neither of previous techniques works any more. Nevertheless, we find that a bound on the maximum load can be directly implied by our analysis of collision number.

Let be a random variable which denotes the maximum load, i.e. the max number of balls in a bin by seeing elements as balls and has values as bins. Then the collision pairs contributed by this heaviest loaded bin is , so the total number of collision pairs is at least .

By our previous analysis, the expected number of collision pairs is . Therefore,

which implies that

In particular, when , i.e. when items are mapped to locations by a pairwise independent hash function, the maximum load is at most with probability at least 1/2. This bound is much weaker than the bound for uniform hash functions, but it is extremely general and holds for any 2-universal hash families. In fact, it was show by Alon et al that there exists 2-universal hash families which yields a maximum load that matches the above bound.

  • Alon, Dietzfelbinger, Miltersen, Petrank, and Tardos. Linear hash functions. Journal of the ACM (JACM), 1999.