Question: 2-3-4-5-6-7-8 trees. Describe algorithms for search and insertion in balanced 2-3-4-5-6-7-8 search trees.

Answer:
Search is the direct multiway-search generalization. In a node containing `k` sorted keys (`1 <= k <= 7`), compare the search key with the node keys, return if equal, otherwise follow one of the `k + 1` child links determined by the interval containing the key.

Insertion is the B-tree insertion algorithm for order 8. Maintain all leaves at the same depth. On the way down, if a child is full (7 keys / 8 links), split it before descending: promote its median key to the parent and replace the full child by two nodes containing the lower and upper halves. Then insert the new key in the appropriate leaf. If the root is full, split it first and create a new root.

The important correction is that a full 8-node is split into two roughly half-full nodes plus one promoted median key; it is not split into eight 2-nodes. Search and insertion both take logarithmic time because every internal node has between 2 and 8 children, except for the root.


Question: Memoryless. Show that red-black BSTs are not memoryless: for example, if you insert a key that is smaller than all the keys in the tree and then immediately delete the minimum, you may get a different tree.

Answer:
3.3.37 - Memoryless

It can be seen that red-black BSTs are not memoryless by observing the following example:

Inserting keys in the order N, G, D, F, E, O, L, K, M generates the red-black tree:

            (B)G
   (B)E              (B)N
(B)D  (B)F      (R)L      (B)O
            (B)K  (B)M

Tree after inserting minimum key (key A):

               (B)G
      (B)E              (B)N
   (B)D  (B)F      (R)L      (B)O
(R)A            (B)K  (B)M

Deleting minimum key (key A):

               (R)G                 (Flip colors)
      (R)E              (R)N
   (B)D  (B)F      (R)L      (B)O
(R)A            (B)K  (B)M

               (R)G                 (Rotate L right)
      (R)E              (R)L
   (B)D  (B)F      (B)K      (R)N
(R)A                      (B)M  (B)O

                   (R)L             (Rotate G left)
           (R)G            (R)N
      (R)E     (B)K     (B)M  (B)O
   (B)D  (B)F
(R)A

                   (B)L             (Flip colors)
           (B)G            (B)N
      (R)E     (B)K     (B)M  (B)O
   (B)D  (B)F
(R)A

                (B)L                (Delete min)
        (B)G            (B)N
   (R)E     (B)K     (B)M  (B)O
(B)D  (B)F

The final red-black tree is different from the original one.

Thanks to Hunter-Chen (https://github.com/Hunter-Chen) for mentioning that the previous answer was incorrect.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/186


Question: Delete the minimum. Implement the deleteMin() operation for red-black BSTs by maintaining the correspondence with the transformations given in the text for moving down the left spine of the tree while maintaining the invariant that the current node is not a 2-node.

Answer:
package chapter3.section3;

import edu.princeton.cs.algs4.StdOut;

/**
 * Created by Rene Argento on 19/06/17.
 */
public class Exercise39_DeleteTheMinimum {

    private class RedBlackBSTDeleteMin<Key extends Comparable<Key>, Value> extends RedBlackBST<Key, Value> {

        public void deleteMin() {
            if (isEmpty()) {
                return;
            }

            if (!isRed(root.left) && !isRed(root.right)) {
                root.color = RED;
            }

            root = deleteMin(root);

            if (!isEmpty()) {
                root.color = BLACK;
            }
        }

        protected Node deleteMin(Node node) {
            if (node.left == null) {
                return null;
            }

            if (!isRed(node.left) && !isRed(node.left.left)) {
                node = moveRedLeft(node);
            }

            node.left = deleteMin(node.left);
            return balance(node);
        }

        protected Node moveRedLeft(Node node) {
            //Assuming that node is red and both node.left and node.left.left are black,
            // make node.left or one of its children red
            flipColors(node);

            if (node.right != null && isRed(node.right.left)) {
                node.right = rotateRight(node.right);
                node = rotateLeft(node);
                flipColors(node);
            }

            return node;
        }

        protected Node balance(Node node) {
            if (node == null) {
                return null;
            }

            if (isRed(node.right) && !isRed(node.left)) {
                node = rotateLeft(node);
            }

            if (isRed(node.left) && node.left != null && isRed(node.left.left)) {
                node = rotateRight(node);
            }

            if (isRed(node.left) && isRed(node.right)) {
                flipColors(node);
            }

            node.size = size(node.left) + 1 + size(node.right);

            return node;
        }

        protected void flipColors(Node node) {
            if (node != null) {
                node.color = !node.color;

                if (node.left != null) {
                    node.left.color = !node.left.color;
                }
                if (node.right != null) {
                    node.right.color = !node.right.color;
                }
            }
        }
    }

    public static void main(String[] args) {
        Exercise39_DeleteTheMinimum deleteTheMinimum = new Exercise39_DeleteTheMinimum();
        RedBlackBSTDeleteMin<Integer, Integer> redBlackBST = deleteTheMinimum.new RedBlackBSTDeleteMin<>();

        redBlackBST.put(10, 10);
        redBlackBST.put(4, 4);
        redBlackBST.put(6, 6);
        redBlackBST.put(1, 1);
        redBlackBST.put(2, 2);
        redBlackBST.put(15, 15);
        redBlackBST.put(12, 12);

        while (!redBlackBST.isEmpty()) {
            for (Integer key : redBlackBST.keys()) {
                StdOut.println(key);
            }

            StdOut.println();

            StdOut.println("Delete min");
            redBlackBST.deleteMin();
        }
    }
}


Question: Delete the maximum. Implement the deleteMax() operation for red-black BSTs. Note that the transformations involved differ slightly from those in the previous exercise because red links are left-leaning.

Answer:
package chapter3.section3;

import edu.princeton.cs.algs4.StdOut;

/**
 * Created by Rene Argento on 21/06/17.
 */
public class Exercise40_DeleteTheMaximum {

    private class RedBlackBSTDeleteMax<Key extends Comparable<Key>, Value>  extends RedBlackBST<Key, Value> {

        public void deleteMax() {
            if (isEmpty()) {
                return;
            }

            if (!isRed(root.left) && !isRed(root.right)) {
                root.color = RED;
            }

            root = deleteMax(root);

            if (!isEmpty()) {
                root.color = BLACK;
            }
        }

        private Node deleteMax(Node node) {
            if (isRed(node.left)) {
                node = rotateRight(node);
            }

            if (node.right == null) {
                return null;
            }

            if (!isRed(node.right) && !isRed(node.right.left)) {
                node = moveRedRight(node);
            }

            node.right = deleteMax(node.right);
            return balance(node);
        }

        protected Node moveRedRight(Node node) {
            //Assuming that node is red and both node.right and node.right.left are black,
            // make node.right or one of its children red
            flipColors(node);

            if (node.left != null && isRed(node.left.left)) {
                node = rotateRight(node);
                flipColors(node);
            }

            return node;
        }

        protected Node balance(Node node) {
            if (node == null) {
                return null;
            }

            if (isRed(node.right) && !isRed(node.left)) {
                node = rotateLeft(node);
            }

            if (isRed(node.left) && node.left != null && isRed(node.left.left)) {
                node = rotateRight(node);
            }

            if (isRed(node.left) && isRed(node.right)) {
                flipColors(node);
            }

            node.size = size(node.left) + 1 + size(node.right);

            return node;
        }

        protected void flipColors(Node node) {
            if (node != null) {
                node.color = !node.color;

                if (node.left != null) {
                    node.left.color = !node.left.color;
                }
                if (node.right != null) {
                    node.right.color = !node.right.color;
                }
            }
        }

    }

    public static void main(String[] args) {
        Exercise40_DeleteTheMaximum deleteTheMaximum = new Exercise40_DeleteTheMaximum();
        RedBlackBSTDeleteMax<Integer, Integer> redBlackBST = deleteTheMaximum.new RedBlackBSTDeleteMax<>();

        redBlackBST.put(10, 10);
        redBlackBST.put(4, 4);
        redBlackBST.put(6, 6);
        redBlackBST.put(1, 1);
        redBlackBST.put(2, 2);
        redBlackBST.put(15, 15);
        redBlackBST.put(12, 12);

        while (!redBlackBST.isEmpty()) {

            for (Integer key : redBlackBST.keys()) {
                StdOut.println(key);
            }

            StdOut.println();

            StdOut.println("Delete max");
            redBlackBST.deleteMax();
        }
    }
}


Question: Delete. Implement the delete() operation for red-black BSTs, combining the methods of the previous two exercises with the delete() operation for BSTs.

Answer:
package chapter3.section3;

import edu.princeton.cs.algs4.StdOut;

/**
 * Created by Rene Argento on 22/06/17.
 */
public class Exercise41_Delete {

    private class RedBlackBSTDelete<Key extends Comparable<Key>, Value> extends RedBlackBST<Key, Value> {

        public void delete(Key key) {
            if (isEmpty()) {
                return;
            }

            if (!contains(key)) {
                return;
            }

            if (!isRed(root.left) && !isRed(root.right)) {
                root.color = RED;
            }

            root = delete(root, key);

            if (!isEmpty()) {
                root.color = BLACK;
            }
        }

        private Node delete(Node node, Key key) {
            if (node == null) {
                return null;
            }

            if (key.compareTo(node.key) < 0) {
                if (!isRed(node.left) && node.left != null && !isRed(node.left.left)) {
                    node = moveRedLeft(node);
                }

                node.left = delete(node.left, key);
            } else {
                if (isRed(node.left)) {
                    node = rotateRight(node);
                }

                if (key.compareTo(node.key) == 0 && node.right == null) {
                    return null;
                }

                if (!isRed(node.right) && node.right != null && !isRed(node.right.left)) {
                    node = moveRedRight(node);
                }

                if (key.compareTo(node.key) == 0) {
                    Node aux = min(node.right);
                    node.key = aux.key;
                    node.value = aux.value;
                    node.right = deleteMin(node.right);
                } else {
                    node.right = delete(node.right, key);
                }
            }

            return balance(node);
        }

        protected Node moveRedLeft(Node node) {
            flipColors(node);

            if (node.right != null && isRed(node.right.left)) {
                node.right = rotateRight(node.right);
                node = rotateLeft(node);
                flipColors(node);
            }

            return node;
        }

        protected Node moveRedRight(Node node) {
            flipColors(node);

            if (node.left != null && isRed(node.left.left)) {
                node = rotateRight(node);
                flipColors(node);
            }

            return node;
        }

        protected Node balance(Node node) {
            if (node == null) {
                return null;
            }

            if (isRed(node.right) && !isRed(node.left)) {
                node = rotateLeft(node);
            }

            if (isRed(node.left) && node.left != null && isRed(node.left.left)) {
                node = rotateRight(node);
            }

            if (isRed(node.left) && isRed(node.right)) {
                flipColors(node);
            }

            node.size = size(node.left) + 1 + size(node.right);

            return node;
        }

        protected void flipColors(Node node) {
            if (node != null) {
                node.color = !node.color;

                if (node.left != null) {
                    node.left.color = !node.left.color;
                }

                if (node.right != null) {
                    node.right.color = !node.right.color;
                }
            }
        }

    }

    public static void main(String[] args) {
        Exercise41_Delete delete = new Exercise41_Delete();
        RedBlackBSTDelete<Integer, Integer> redBlackBST = delete.new RedBlackBSTDelete<>();

        redBlackBST.put(10, 10);
        redBlackBST.put(4, 4);
        redBlackBST.put(6, 6);
        redBlackBST.put(1, 1);
        redBlackBST.put(2, 2);
        redBlackBST.put(15, 15);
        redBlackBST.put(12, 12);

        StdOut.println("Keys");
        delete.printKeys(redBlackBST);

        StdOut.println("Delete 1");
        redBlackBST.delete(1);
        delete.printKeys(redBlackBST);

        StdOut.println("Delete 15");
        redBlackBST.delete(15);
        delete.printKeys(redBlackBST);

        StdOut.println("Delete 10");
        redBlackBST.delete(10);
        delete.printKeys(redBlackBST);

        StdOut.println("Delete 6");
        redBlackBST.delete(6);
        delete.printKeys(redBlackBST);

        while (redBlackBST.size() > 0) {
            redBlackBST.delete(redBlackBST.select(0));
        }
        StdOut.println("Final size after deleting all keys: " + redBlackBST.size());
    }

    private void printKeys(RedBlackBSTDelete<Integer, Integer> redBlackBST) {
        for (Integer key : redBlackBST.keys()) {
            StdOut.println(key);
        }
        StdOut.println();
    }
}


Question: Suppose that keys are t-bit integers. For a modular hash function with prime M, prove that each key bit has the property that there exist two keys differing only in that bit that have different hash values.

Answer:
3.4.6

Proposition: With keys that are t-bit integers, for a hash function with prime M, each key bit has the property that there exist two keys differing only in that bit that have different hash values.

Proof: Consider the hash function k % M where k is an integer and M is a prime number.
Every time that we modify exactly one bit in an integer, we either add (when modifying from 0 to 1) or subtract (when modifying from 1 to 0) a value that is a power of 2 (0, 1, 2, 4, etc). Since we are never adding or subtracting a prime number (other than 2) or any prime number multiples, the modular hash function with a prime M (other than 2) will yield a different result for both numbers.

Example: 5-bit integer 16
16: 10000
17: 10001 (differing only in the 1st bit)
18: 10010 (differing only in the 2nd bit)
20: 10100 (differing only in the 3rd bit)
24: 11000 (differing only in the 4th bit)
8:  01000 (differing only in the 5th bit)

For M = 7

16 % 7 = 2
17 % 7 = 3
18 % 7 = 4
20 % 7 = 6
24 % 7 = 3
8 % 7 = 1

This definition does not hold for M = 2 though, as can be seen with the numbers 16 and 18, which differ by only 1 bit but have the same hash value in a modular hash function:
16 % 2 = 0
18 % 2 = 0


Question: Consider the idea of implementing modular hashing for integer keys with the code (a * k) % M, where a is an arbitrary fixed prime. Does this change mix up the bits sufficiently well that you can use nonprime M?

Answer:
3.4.7

If and only if a and M are coprime, the change will mix up the bits sufficiently well that a nonprime M can be used.

Proof:
1- (a * k) % M = hash
2- This means that a * k = M * q + hash, which is equivalent to hash = a * k - M * q, where q is a natural number.
3- Now let's consider that a and M are not coprime.
This means that there exists a non-zero natural number t where M = a * t.
hash = a * k - M * q expands to hash = a * k - a * t * q = a * (k - t * q)
This means that all hash values will be divisible by a, which greatly reduces the distribution of keys.
For example, if a = 37 and M = 37 * t, then all hash values will be only 0, 37, 74, 111, ...

Example where a is an arbitrary fixed prime:
a = 7
k = 1, 2, 3, 4
M = 6

(7 * 1) % 6 = 1
(7 * 2) % 6 = 2
(7 * 3) % 6 = 3
(7 * 4) % 6 = 4
(7 * 5) % 6 = 5
(7 * 6) % 6 = 0

Example where a is not an arbitrary fixed prime:
a = 8
k = 1, 2, 3, 4
M = 6

(8 * 1) % 6 = 2
(8 * 2) % 6 = 4
(8 * 3) % 6 = 0
(8 * 4) % 6 = 2
(8 * 5) % 6 = 4
(8 * 6) % 6 = 0

Thanks to dragon-dreamer (https://github.com/dragon-dreamer) for observing and providing the proof that a and M must be coprime.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/125


Question: How many empty lists do you expect to see when you insert N keys into a hash table with SeparateChainingHashST, for N=10, 10^2, 10^3, 10^4, 10^5, and 10^6? Hint: See Exercise 2.5.31.

Answer:
3.4.8

As we have seen on exercise 2.5.31, probability theory says that after generating N random values, the number of distinct values is about M(1 - e^(-a)) where M = the maximum value generated - 1 (in this case, the hash table length - 1), N = number of keys generated and a = N / M.

In this case M = N, so the hash table length is the same as the number of keys to be inserted:
a = 1
Number of distinct values = M(1 - e^(-1))

For N = 10
Number of distinct values = 10(1 - e^(-1))
Number of distinct values ~ 6

For N = 10^2
Number of distinct values = 10^2(1 - e^(-1))
Number of distinct values ~ 63

For N = 10^3
Number of distinct values = 10^3(1 - e^(-1))
Number of distinct values ~ 632

For N = 10^4
Number of distinct values = 10^4(1 - e^(-1))
Number of distinct values ~ 6321

For N = 10^5
Number of distinct values = 10^5(1 - e^(-1))
Number of distinct values ~ 63212

For N = 10^6
Number of distinct values = 10^6(1 - e^(-1))
Number of distinct values ~ 632120

Also, considering assumption J (uniform hashing assumption) the distinct keys will be distributed uniformly among the M buckets and the lists will have length > 1 only when duplicate keys are inserted. We know that in the real world other collisions may happen but it is a valid approximation. This means that the number of empty lists in the hash table will be equal to M - D where M is the hash table length and D is the number of distinct values inserted.

For N = 10
Number of empty lists expected = 10 - 6 = 4

For N = 10^2
Number of empty lists expected = 10^2 - 63 = 37

For N = 10^3
Number of empty lists expected = 10^3 - 632 = 368

For N = 10^4
Number of empty lists expected = 10^4 - 6321 = 3679

For N = 10^5
Number of empty lists expected = 10^5 - 63212 = 36788

For N = 10^6
Number of empty lists expected = 10^6 - 632120 = 367880


Question: Implement an eager delete() method for SeparateChainingHashST.

Answer:
package chapter3.section4;

import edu.princeton.cs.algs4.StdOut;

/**
 * Created by Rene Argento on 20/07/17.
 */
@SuppressWarnings("unchecked")
public class Exercise9 {

    private class SeparateChainingHashTableWithDelete<Key, Value> extends SeparateChainingHashTable<Key, Value> {

        public void delete(Key key) {
            if (key == null) {
                throw new IllegalArgumentException("Argument to delete() cannot be null");
            }

            if (isEmpty()) {
                return;
            }

            if (!contains(key)) {
                return;
            }

            symbolTable[hash(key)].delete(key);
            keysSize--;

            if (getLoadFactor() <= averageListSize / (double) 4) {
                resize(size / 2);
            }
        }

    }

    public static void main(String[] args) {
        Exercise9 exercise9 = new Exercise9();
        SeparateChainingHashTableWithDelete<Integer, Integer> separateChainingHashTableWithDelete =
                exercise9.new SeparateChainingHashTableWithDelete<>();

        separateChainingHashTableWithDelete.put(1, 1);
        separateChainingHashTableWithDelete.put(2, 2);
        separateChainingHashTableWithDelete.put(3, 3);
        separateChainingHashTableWithDelete.put(4, 4);
        separateChainingHashTableWithDelete.put(5, 5);
        separateChainingHashTableWithDelete.put(6, 6);
        separateChainingHashTableWithDelete.put(7, 7);

        StdOut.println("Keys");
        for (Integer key : separateChainingHashTableWithDelete.keys()) {
            StdOut.print(key + " ");
        }

        int[] keysToDelete = {-1, 1, 2, 7, 6, 4, 5, 3};
        for (int k : keysToDelete) {
            StdOut.println("\nDelete key " + k);
            separateChainingHashTableWithDelete.delete(k);

            for (Integer key : separateChainingHashTableWithDelete.keys()) {
                StdOut.print(key + " ");
            }
            StdOut.println("\nSize: " + separateChainingHashTableWithDelete.size());
        }
    }
}


Question: Insert the keys E A S Y Q U T I O N in that order into an initially empty table of size M =16 using linear probing. Use the hash function 11 k % M to transform the kth letter of the alphabet into a table index. Redo this exercise for M = 10.

Answer:
3.4.10

M = 16

key hash value
  E   7    0

0 null      8 null
1 null      9 null
2 null     10 null
3 null     11 null
4 null     12 null
5 null     13 null
6 null     14 null
7 E0       15 null

key hash value
  A   11   1

0 null      8 null
1 null      9 null
2 null     10 null
3 null     11 A1
4 null     12 null
5 null     13 null
6 null     14 null
7 E0       15 null

key hash value
  S    1   2

0 null      8 null
1 S2        9 null
2 null     10 null
3 null     11 A1
4 null     12 null
5 null     13 null
6 null     14 null
7 E0       15 null

key hash value
  Y    3   3

0 null      8 null
1 S2        9 null
2 null     10 null
3 Y3       11 A1
4 null     12 null
5 null     13 null
6 null     14 null
7 E0       15 null

key hash value
  Q   11   4

0 null      8 null
1 S2        9 null
2 null     10 null
3 Y3       11 A1
4 null     12 Q4
5 null     13 null
6 null     14 null
7 E0       15 null

key hash value
  U    7   5

0 null      8 U5
1 S2        9 null
2 null     10 null
3 Y3       11 A1
4 null     12 Q4
5 null     13 null
6 null     14 null
7 E0       15 null

key hash value
  T   12   6

0 null      8 U5
1 S2        9 null
2 null     10 null
3 Y3       11 A1
4 null     12 Q4
5 null     13 T6
6 null     14 null
7 E0       15 null

key hash value
  I    3   7

0 null      8 U5
1 S2        9 null
2 null     10 null
3 Y3       11 A1
4 I7       12 Q4
5 null     13 T6
6 null     14 null
7 E0       15 null

key hash value
  O   5    8

0 null      8 U5
1 S2        9 null
2 null     10 null
3 Y3       11 A1
4 I7       12 Q4
5 O8       13 T6
6 null     14 null
7 E0       15 null

key hash value
  N   10   9

0 null      8 U5
1 S2        9 null
2 null     10 N9
3 Y3       11 A1
4 I7       12 Q4
5 O8       13 T6
6 null     14 null
7 E0       15 null

M = 10

key hash value
  E   5    0

0 null      5 E0
1 null      6 null
2 null      7 null
3 null      8 null
4 null      9 null

key hash value
  A   1    1

0 null      5 E0
1 A1        6 null
2 null      7 null
3 null      8 null
4 null      9 null

key hash value
  S   9    2

0 null      5 E0
1 A1        6 null
2 null      7 null
3 null      8 null
4 null      9 S2

key hash value
  Y   5    3

0 null      5 E0
1 A1        6 Y3
2 null      7 null
3 null      8 null
4 null      9 S2

key hash value
  Q   7    4

0 null      5 E0
1 A1        6 Y3
2 null      7 Q4
3 null      8 null
4 null      9 S2

key hash value
  U   1    5

0 null      5 E0
1 A1        6 Y3
2 U5        7 Q4
3 null      8 null
4 null      9 S2

key hash value
  T   0    6

0 T6        5 E0
1 A1        6 Y3
2 U5        7 Q4
3 null      8 null
4 null      9 S2

key hash value
  I   9    7

0 T6        5 E0
1 A1        6 Y3
2 U5        7 Q4
3 I7        8 null
4 null      9 S2

key hash value
  O   5    8

0 T6        5 E0
1 A1        6 Y3
2 U5        7 Q4
3 I7        8 O8
4 null      9 S2

key hash value
  N   4    9

0 T6        5 E0
1 A1        6 Y3
2 U5        7 Q4
3 I7        8 O8
4 N9        9 S2

Thanks to Kongkille (https://github.com/Kongkille) for finding an error in some M = 16 insertions.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/37


Question: Give the contents of a linear-probing hash table that results when you insert the keys E A S Y Q U T I O N in that order into an initially empty table of initial size M = 4 that is expanded with doubling whenever half full. Use the hash function 11 k % M to transform the kth letter of the alphabet into a table index.

Answer:
3.4.11

key hash value
  E   3    0

0 null      2 null
1 null      3 E0

key hash value
  A   3    1

0 A1      2 null
1 null    3 E0

Doubles hash table size and M becomes 8

0 null      4 null
1 null      5 null
2 null      6 null
3 null      7 null

Reinsert A1
key hash value
  A    3   1

0 null      4 null
1 null      5 null
2 null      6 null
3 A1        7 null

Reinsert E0
key hash value
  E    7   0

0 null      4 null
1 null      5 null
2 null      6 null
3 A1        7 E0

key hash value
  S   1    2

0 null      4 null
1 S2        5 null
2 null      6 null
3 A1        7 E0

key hash value
  Y   3    3

0 null      4 Y3
1 S2        5 null
2 null      6 null
3 A1        7 E0

Doubles hash table size and M becomes 16

0 null      8 null
1 null      9 null
2 null     10 null
3 null     11 null
4 null     12 null
5 null     13 null
6 null     14 null
7 null     15 null

Reinsert A1
key hash value
  A   11   1

0 null      8 null
1 null      9 null
2 null     10 null
3 null     11 A1
4 null     12 null
5 null     13 null
6 null     14 null
7 null     15 null

Reinsert E0
key hash value
  E    7   0

0 null      8 null
1 null      9 null
2 null     10 null
3 null     11 A1
4 null     12 null
5 null     13 null
6 null     14 null
7 E0       15 null

Reinsert S2
key hash value
  S    1   2

0 null      8 null
1 S2        9 null
2 null     10 null
3 null     11 A1
4 null     12 null
5 null     13 null
6 null     14 null
7 E0       15 null

Reinsert Y3
key hash value
  Y    3   3

0 null      8 null
1 S2        9 null
2 null     10 null
3 Y3       11 A1
4 null     12 null
5 null     13 null
6 null     14 null
7 E0       15 null

key hash value
  Q   11   4

0 null      8 null
1 S2        9 null
2 null     10 null
3 Y3       11 A1
4 null     12 Q4
5 null     13 null
6 null     14 null
7 E0       15 null

key hash value
  U    7   5

0 null      8 U5
1 S2        9 null
2 null     10 null
3 Y3       11 A1
4 null     12 Q4
5 null     13 null
6 null     14 null
7 E0       15 null

key hash value
  T   12   6

0 null      8 U5
1 S2        9 null
2 null     10 null
3 Y3       11 A1
4 null     12 Q4
5 null     13 T6
6 null     14 null
7 E0       15 null

key hash value
  I    3   7

0 null      8 U5
1 S2        9 null
2 null     10 null
3 Y3       11 A1
4 I7       12 Q4
5 null     13 T6
6 null     14 null
7 E0       15 null

Doubles hash table size and M becomes 32

 0 null     16 null
 1 null     17 null
 2 null     18 null
 3 null     19 null
 4 null     20 null
 5 null     21 null
 6 null     22 null
 7 null     23 null
 8 null     24 null
 9 null     25 null
10 null     26 null
11 null     27 null
12 null     28 null
13 null     29 null
14 null     30 null
15 null     31 null

Reinsert A1
key hash value
  A   11   1

 0 null     16 null
 1 null     17 null
 2 null     18 null
 3 null     19 null
 4 null     20 null
 5 null     21 null
 6 null     22 null
 7 null     23 null
 8 null     24 null
 9 null     25 null
10 null     26 null
11 A1       27 null
12 null     28 null
13 null     29 null
14 null     30 null
15 null     31 null

Reinsert E0
key hash value
  E   23   0

 0 null     16 null
 1 null     17 null
 2 null     18 null
 3 null     19 null
 4 null     20 null
 5 null     21 null
 6 null     22 null
 7 null     23 E0
 8 null     24 null
 9 null     25 null
10 null     26 null
11 A1       27 null
12 null     28 null
13 null     29 null
14 null     30 null
15 null     31 null

Reinsert I7
key hash value
  I    3   7

 0 null     16 null
 1 null     17 null
 2 null     18 null
 3 I7       19 null
 4 null     20 null
 5 null     21 null
 6 null     22 null
 7 null     23 E0
 8 null     24 null
 9 null     25 null
10 null     26 null
11 A1       27 null
12 null     28 null
13 null     29 null
14 null     30 null
15 null     31 null

Reinsert Q4
key hash value
  Q   27   4

 0 null     16 null
 1 null     17 null
 2 null     18 null
 3 I7       19 null
 4 null     20 null
 5 null     21 null
 6 null     22 null
 7 null     23 E0
 8 null     24 null
 9 null     25 null
10 null     26 null
11 A1       27 Q4
12 null     28 null
13 null     29 null
14 null     30 null
15 null     31 null

Reinsert S2
key hash value
  S   17   2

 0 null     16 null
 1 null     17 S2
 2 null     18 null
 3 I7       19 null
 4 null     20 null
 5 null     21 null
 6 null     22 null
 7 null     23 E0
 8 null     24 null
 9 null     25 null
10 null     26 null
11 A1       27 Q4
12 null     28 null
13 null     29 null
14 null     30 null
15 null     31 null

Reinsert T6
key hash value
  T   28   6

 0 null     16 null
 1 null     17 S2
 2 null     18 null
 3 I7       19 null
 4 null     20 null
 5 null     21 null
 6 null     22 null
 7 null     23 E0
 8 null     24 null
 9 null     25 null
10 null     26 null
11 A1       27 Q4
12 null     28 T6
13 null     29 null
14 null     30 null
15 null     31 null

Reinsert U5
key hash value
  U    7   5

 0 null     16 null
 1 null     17 S2
 2 null     18 null
 3 I7       19 null
 4 null     20 null
 5 null     21 null
 6 null     22 null
 7 U5       23 E0
 8 null     24 null
 9 null     25 null
10 null     26 null
11 A1       27 Q4
12 null     28 T6
13 null     29 null
14 null     30 null
15 null     31 null

Reinsert Y3
key hash value
  Y   19   3

 0 null     16 null
 1 null     17 S2
 2 null     18 null
 3 I7       19 Y3
 4 null     20 null
 5 null     21 null
 6 null     22 null
 7 U5       23 E0
 8 null     24 null
 9 null     25 null
10 null     26 null
11 A1       27 Q4
12 null     28 T6
13 null     29 null
14 null     30 null
15 null     31 null

key hash value
  O    5   8

 0 null     16 null
 1 null     17 S2
 2 null     18 null
 3 I7       19 Y3
 4 null     20 null
 5 O8       21 null
 6 null     22 null
 7 U5       23 E0
 8 null     24 null
 9 null     25 null
10 null     26 null
11 A1       27 Q4
12 null     28 T6
13 null     29 null
14 null     30 null
15 null     31 null

key hash value
  N   26   9

 0 null     16 null
 1 null     17 S2
 2 null     18 null
 3 I7       19 Y3
 4 null     20 null
 5 O8       21 null
 6 null     22 null
 7 U5       23 E0
 8 null     24 null
 9 null     25 null
10 null     26 N9
11 A1       27 Q4
12 null     28 T6
13 null     29 null
14 null     30 null
15 null     31 null


Question:
Which of the following scenarios leads to expected linear running time for a random search hit in a linear-probing hash table?
a. All keys hash to the same index.
b. All keys hash to different indices.
c. All keys hash to an even-numbered index.
d. All keys hash to different even-numbered indices.

Answer:
Only scenario (a) necessarily leads to expected linear running time for a random search hit: if all keys hash to the same table index, they form one contiguous cluster, and a random hit probes a linear number of positions on average.

Scenario (b) is the best case. Scenario (d) is not necessarily linear, because different even positions may still keep probe sequences short when the load factor is moderate. Scenario (c) is not sufficient as stated: hashing only to even-numbered positions can create extra clustering, but it is not guaranteed to make every random successful search linear unless the keys also form a linear-size cluster.


Question:
Which of the following scenarios leads to expected linear running time for a search miss in a linear-probing hash table, assuming the search key is equally likely to hash to each table position?

a. All keys hash to the same index.
b. All keys hash to different indices.
c. All keys hash to an even-numbered index.
d. All keys hash to different even-numbered indices.

Answer:
Scenario (a) necessarily gives expected linear time for a search miss: all keys form one large cluster, and misses that hash into or before that cluster scan a linear number of entries.

Scenario (c) can also be linear when the even-index restriction creates a linear-size cluster, but it is not guaranteed solely from the statement. Scenarios (b) and (d) do not necessarily give linear miss cost; with well-spread keys and a bounded load factor, misses can remain constant expected time.


Question: How many compares could it take, in the worst case, to insert N keys into an initially empty table, using linear probing with array resizing?

Answer:
3.4.15

In the worst case all keys hash to the same index.
The exercise description does not mention the initial hash table size. This is important to compute the number of compares in a hash table with array resizing. This exercise answer assumes that the initial hash table size is N and its size is expanded with doubling whenever half full.

The number of compares per insert is:
1 for the first insert, 2 for the second insert, 3 for the third insert and so on, until the (N/2)th insert.
When the table is half full it is resized to 2N and the keys are reinserted, with 1, 2, 3, ..., N/2 compares.

Then, for the insert of the other keys there are N/2 + 1, N/2 + 2, ..., N compares per insert.

This is equal to:
Number of compares = (1 + 2 + 3 + ... + N/2) + 1 + 2 + 3 + ... + N/2 + (N/2 + 1) + (N/2 + 2) + ... + N
Number of compares = (N/2 + 1) * N / 2 / 2 + (N + 1) * N / 2
Number of compares = (N^2/2 + N) / 4 + (N^2 + N) / 2
Number of compares = (N^2/2 + N) / 4 + (2N^2 + 2N) / 4
Number of compares = (N^2/2 + (2N / 2)) / 4 + (2N^2 + 2N) / 4
Number of compares = (N^2 + 2N) / 8 + (4N^2 + 4N) / 8
Number of compares = (5N^2 + 6N) / 8

In the worst case, to insert N keys into an initially empty table, using linear probing with array resizing it would take (5N^2 + 6N) / 8 compares.


Question: Add a constructor to SeparateChainingHashST that gives the client the ability to specify the average number of probes to be tolerated for searches. Use array resizing to keep the average list size less than the specified value, and use the technique described on page 478 to ensure that the modulus for hash() is prime.

Answer:
package chapter3.section4;

import edu.princeton.cs.algs4.StdOut;

/**
 * Created by Rene Argento on 21/07/17.
 */
@SuppressWarnings("unchecked")
public class Exercise18 {

    private class SeparateChainingHashTableResize<Key, Value> extends SeparateChainingHashTable<Key, Value>{
        private int averageListSize;

        // The largest prime <= 2^i for i = 1 to 31
        // Used to distribute keys uniformly in the hash table after resizes
        // PRIMES[n] = 2^k - Ak where k is the power of 2 and Ak is the value to subtract to reach the previous prime number
        private final int[] PRIMES = {
                1, 1, 3, 7, 13, 31, 61, 127, 251, 509, 1021, 2039, 4093, 8191, 16381,
                32749, 65521, 131071, 262139, 524287, 1048573, 2097143, 4194301,
                8388593, 16777213, 33554393, 67108859, 134217689, 268435399,
                536870909, 1073741789, 2147483647
        };

        //The lg of the hash table size
        //Used in combination with PRIMES[] to distribute keys uniformly in the hash function after resizes
        private int lgM;

        public SeparateChainingHashTableResize(int initialSize, int averageListSize) {
            super(initialSize, averageListSize);

            this.size = initialSize;
            this.averageListSize = averageListSize;
            symbolTable = new SequentialSearchSymbolTable[size];

            for (int i = 0; i < size; i++) {
                symbolTable[i] = new SequentialSearchSymbolTable();
            }

            lgM = (int) (Math.log(size) / Math.log(2));
        }

        protected int hash(Key key) {
            int hash = key.hashCode() & 0x7fffffff;

            if (lgM < 26) {
                hash = hash % PRIMES[lgM + 5];
            }

            return hash % size;
        }

        public void resize(int newSize) {
            SeparateChainingHashTableResize<Key, Value> separateChainingHashTableTemp =
                    new SeparateChainingHashTableResize<>(newSize, averageListSize);

            for (Key key : keys()) {
                separateChainingHashTableTemp.put(key, get(key));
            }

            symbolTable = separateChainingHashTableTemp.symbolTable;
            size = separateChainingHashTableTemp.size;
            keysSize = separateChainingHashTableTemp.keysSize;
            lgM = separateChainingHashTableTemp.lgM;
        }

        public void put(Key key, Value value) {
            if (key == null) {
                throw new IllegalArgumentException("Key cannot be null");
            }

            if (value == null) {
                delete(key);
                return;
            }

            int hashIndex = hash(key);
            int currentSize = symbolTable[hashIndex].size;
            symbolTable[hashIndex].put(key, value);

            if (currentSize < symbolTable[hashIndex].size) {
                keysSize++;
            }

            if (getLoadFactor() >= averageListSize) {
                StdOut.println("Resize - doubling hash table size");

                resize(size * 2);
            }
        }

        public void delete(Key key) {
            if (key == null) {
                throw new IllegalArgumentException("Argument to delete() cannot be null");
            }

            if (isEmpty() || !contains(key)) {
                return;
            }

            symbolTable[hash(key)].delete(key);
            keysSize--;

            if (size > 1 && getLoadFactor() <= averageListSize / (double) 4) {
                StdOut.println("Resize - shrinking hash table size");

                resize(size / 2);
            }
        }
    }

    public static void main(String[] args) {
        Exercise18 exercise18 = new Exercise18();
        SeparateChainingHashTableResize<Integer, Integer> separateChainingHashTableResize =
                exercise18.new SeparateChainingHashTableResize<>(5, 2);

        for (int i = 0; i < 20; i++) {
            separateChainingHashTableResize.put(i, i);
        }
        StdOut.println("Expected: Resize - doubling hash table size 2x");

        for (int i = 0; i < 10; i++) {
            separateChainingHashTableResize.delete(i);
        }
        StdOut.println("Expected: Resize - shrinking hash table size");

        for (int i = 10; i < 15; i++) {
            separateChainingHashTableResize.delete(i);
        }
        StdOut.println("Expected: Resize - shrinking hash table size");
    }
}


Question: Add a method to LinearProbingHashST that computes the average cost of a search hit in the table, assuming that each key in the table is equally likely to be sought.

Answer:
package chapter3.section4;

import edu.princeton.cs.algs4.StdOut;

/**
 * Created by Rene Argento on 21/07/17.
 */
// Thanks to faame (https://github.com/faame) for fixing a bug in the compare count in this exercise.
// https://github.com/reneargento/algorithms-sedgewick-wayne/issues/236
public class Exercise20 {

    private class LinearProbingHashTableAvgSearchHitCost<Key, Value> extends LinearProbingHashTable<Key, Value> {

        LinearProbingHashTableAvgSearchHitCost(int size) {
            super(size);
        }

        public double getAverageCostOfSearchHit() {
            if (keysSize == 0) {
                return 0.0;
            }
            int totalCompares = 0;
            for (int i = 0; i < size; i++) {
                if (keys[i] == null) {
                    continue;
                }
                int compares = 1;
                for (int tableIndex = hash(keys[i]); !keys[tableIndex].equals(keys[i]);
                     tableIndex = (tableIndex + 1) % size) {
                    compares++;
                }
                totalCompares += compares;
            }
            return totalCompares / (double) keysSize;
        }

        private double totalNumberOfComparesForSearchHit;

        private void resize(int newSize) {
            LinearProbingHashTableAvgSearchHitCost<Key, Value> tempHashTable =
                    new LinearProbingHashTableAvgSearchHitCost<>(newSize);

            for (int i = 0; i < size; i++) {
                if (keys[i] != null) {
                    tempHashTable.put(keys[i], values[i]);
                }
            }

            keys = tempHashTable.keys;
            values = tempHashTable.values;
            size = tempHashTable.size;
            totalNumberOfComparesForSearchHit = tempHashTable.totalNumberOfComparesForSearchHit;
        }

        public Value get(Key key) {
            if (key == null) {
                throw new IllegalArgumentException("Argument to get() cannot be null");
            }
            int numberOfComparesBeforeFindingKey = 0;

            for (int tableIndex = hash(key); keys[tableIndex] != null; tableIndex = (tableIndex + 1) % size) {
                numberOfComparesBeforeFindingKey++;

                if (keys[tableIndex].equals(key)) {
                    totalNumberOfComparesForSearchHit += numberOfComparesBeforeFindingKey;
                    return values[tableIndex];
                }
            }
            return null;
        }

        public void put(Key key, Value value) {
            if (key == null) {
                throw new IllegalArgumentException("Key cannot be null");
            }

            if (value == null) {
                delete(key);
                return;
            }

            if (keysSize >= size / (double) 2) {
                resize(size * 2);
                lgM++;
            }

            int numberOfComparesBeforeFindingKey = 0;

            int tableIndex;
            for (tableIndex = hash(key); keys[tableIndex] != null; tableIndex = (tableIndex + 1) % size) {
                numberOfComparesBeforeFindingKey++;

                if (keys[tableIndex].equals(key)) {
                    totalNumberOfComparesForSearchHit += numberOfComparesBeforeFindingKey;
                    values[tableIndex] = value;
                    return;
                }
            }

            keys[tableIndex] = key;
            values[tableIndex] = value;
            keysSize++;
        }
    }

    public static void main(String[] args) {
        Exercise20 exercise20 = new Exercise20();
        LinearProbingHashTableAvgSearchHitCost<Integer, Integer> linearProbingHashTableAvgSearchHitCost =
                exercise20.new LinearProbingHashTableAvgSearchHitCost<>(20);

        linearProbingHashTableAvgSearchHitCost.put(5, 5);
        StdOut.println(linearProbingHashTableAvgSearchHitCost.getAverageCostOfSearchHit() + " Expected: 0.0");

        linearProbingHashTableAvgSearchHitCost.get(5);
        StdOut.println(linearProbingHashTableAvgSearchHitCost.getAverageCostOfSearchHit() + " Expected: 1.0");

        linearProbingHashTableAvgSearchHitCost.get(5);
        StdOut.println(linearProbingHashTableAvgSearchHitCost.getAverageCostOfSearchHit() + " Expected: 2.0");

        linearProbingHashTableAvgSearchHitCost.put(5, 5);
        StdOut.println(linearProbingHashTableAvgSearchHitCost.getAverageCostOfSearchHit() + " Expected: 3.0");

        linearProbingHashTableAvgSearchHitCost.put(7, 7);
        StdOut.println(linearProbingHashTableAvgSearchHitCost.getAverageCostOfSearchHit() + " Expected: 1.5");
    }
}


Question: Add a method to LinearProbingHashST that computes the average cost of a search miss in the table, assuming a random hash function. Note: You do not have to compute any hash functions to solve this problem.

Answer:
package chapter3.section4;

import edu.princeton.cs.algs4.StdOut;
import edu.princeton.cs.algs4.StdRandom;

/**
 * Created by Rene Argento on 21/07/17.
 */
public class Exercise21 {

    private class LinearProbingHashTableAvgSearchMissCost<Key, Value> extends LinearProbingHashTable<Key, Value> {

        LinearProbingHashTableAvgSearchMissCost(int size) {
            super(size);
        }

        // Average cost of search miss = ~1/2 * (1 + (1 / (1 - a)^2))
        public double getAverageCostOfSearchMissByLoadFactor() {
            double loadFactor = getLoadFactor();
            return 0.5 * (1 + (1 / Math.pow(1 - loadFactor, 2)));
        }

        // Average cost of search miss = 1 + N / 2M + (sum((cluster[i].size)^2)) / 2M for all clusters in the table
        public double getAverageCostOfSearchMissByClusterSizes() {
            int clusterSize = 0;
            int clusterSizeSquareSum = 0;

            for (int i = 0; i < size; i++) {
                if (keys[i] != null) {
                    clusterSize++;
                } else {
                    clusterSizeSquareSum += clusterSize * clusterSize;
                    clusterSize = 0;
                }
            }

            if (clusterSize != 0) {
                clusterSizeSquareSum += clusterSize * clusterSize;
            }
            return 1 + ((double) (keysSize + clusterSizeSquareSum)) / (size * 2);
        }
    }

    public static void main(String[] args) {
        Exercise21 exercise21 = new Exercise21();
        LinearProbingHashTableAvgSearchMissCost<Integer, Integer> linearProbingHashTableAvgSearchMissCost =
                exercise21.new LinearProbingHashTableAvgSearchMissCost<>(1000000);

        for (int i = 0; i < 500000; i++) {
            int randomKey = StdRandom.uniform(Integer.MAX_VALUE);
            linearProbingHashTableAvgSearchMissCost.put(randomKey, randomKey);
        }

        StdOut.printf("Average cost of search miss by load factor: %.2f\n",
                linearProbingHashTableAvgSearchMissCost.getAverageCostOfSearchMissByLoadFactor());
        StdOut.printf("Average cost of search miss by cluster sizes: %.2f\n",
                linearProbingHashTableAvgSearchMissCost.getAverageCostOfSearchMissByClusterSizes());
    }
}


Question: Consider modular hashing for string keys with R = 256 and M = 255. Show that this is a bad choice because any permutation of letters within a string hashes to the same value.

Answer:
3.4.23

Given R = 256 and M = 255, we shall prove that the hash of all strings with the same collection of chars is the same.
In fact, we can prove the proposition that hash(string) = (sum of chars) % 255 for all strings by induction on string
length.

For any string with only one char, the proposition holds trivially.
Let hash(str{k}) denote the hash of a string no longer than k, char(k) is the k-th char.
By induction assumption, we have:
hash(str{k}) = (sum of k chars) % 255

Then we have:
hash(str{k + 1}) = (hash(str{k}) * 256 + char(k + 1)) % 255
 = (hash(str{k}) * (255 + 1) + char(k + 1)) % 255
 = (hash(str{k}) * 255 + hash(str{k}) + char(k + 1)) % 255
 = (hash(str{k}) + char(k + 1)) % 255
 = ((sum of k chars) % 255 + char(k + 1)) % 255
 = ((sum of k chars + char(k + 1)) % 255) % 255
 = (sum of k + 1 chars) % 255
Note that (N * 255) % 255 = 0, and the char values are always less than 255, so the chars can be moved into the modular
expression.

Hence, by induction, the proposition holds for strings of any length.
All strings with the same collection of chars must have the same sum of chars, so by the proposition they must have the
same hash.

Thanks to luowyang (https://github.com/luowyang) for improving the answer and providing a proof for this exercise.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/95


Question: Double probing. Modify SeparateChainingHashST to use a second hash function and pick the shorter of the two lists. Give a trace of the process of inserting the keys E A S Y Q U T I O N in that order into an initially empty table of size M =3 using the function 11 k % M (for the kth letter) as the first hash function and the function 17 k % M (for the kth letter) as the second hash function. Give the average number of probes for random search hit and search miss in this table.

Answer:
Because 11 mod 3 and 17 mod 3 are both 2, the two hash functions are identical for this table size. Double probing therefore degenerates to ordinary separate chaining with one candidate chain.

Using A=1, B=2, ..., Z=26, the final chains are:

0: O, I, U
1: N, T, Q, E
2: Y, S, A

If new keys are inserted at the front of their chain, the average number of probes for a random successful search is

    (1 + 2 + 3 + 1 + 2 + 3 + 4 + 1 + 2 + 3) / 10 = 22 / 10 = 2.2

For an unsuccessful search, the chain searched is chosen by the hash value, so the average number of probes is

    (3 + 4 + 3) / 3 = 10 / 3

The trace should not average over a distinct second list, since the two hash functions choose the same list for every key when M = 3.


Question: Cuckoo hashing. Develop a symbol-table implementation that maintains two hash tables and two hash functions. Any given key is in one of the tables, but not both. When inserting a new key, hash to one of the tables; if the table position is occupied, replace that key with the new key and hash the old key into the other table (again kicking out a key that might reside there). If this process cycles, restart. Keep the tables less than half full. This method uses a constant number of equality tests in the worst case for search (trivial) and amortized constant time for insert.

Answer:
package chapter3.section4;

import edu.princeton.cs.algs4.Queue;
import edu.princeton.cs.algs4.StdOut;
import edu.princeton.cs.algs4.StdRandom;

import java.lang.reflect.Array;
import java.util.Arrays;

/**
 * Created by Rene Argento on 24/07/17.
 */
// Based on http://www.keithschwarz.com/interesting/code/cuckoo-hashmap/CuckooHashMap.java.html
@SuppressWarnings("unchecked")
public class Exercise31_CuckooHashing {

    private class CuckooHashing<Key, Value> {

        private class Entry {
            Key key;
            Value value;

            Entry(Key key, Value value) {
                this.key = key;
                this.value = value;
            }
        }

        private final class HashFunction<T> {
            private final int mA, mB;  // Coefficients for this hash function
            private final int mLgSize; // Log of the size of the hash tables.

            /**
             * Constructs a new hash function using the specified coefficients and
             * the log of the number of buckets in the hash table.
             */
            public HashFunction(int coefficientA, int coefficientB, int lgSize) {
                mA = coefficientA;
                mB = coefficientB;
                mLgSize = lgSize;
            }

            public int hash(Key key) {
            /* If the object is null, just evaluate to zero. */
                if (key == null) {
                    return 0;
                }

            /* Otherwise, split its hash code into upper and lower bits. */
                final int hashCode = key.hashCode();
                final int upper = hashCode >>> 16;
                final int lower = hashCode & (0xFFFF);

            /* Return the pairwise product of those bits, shifted down so that
             * only lgSize bits remain in the output.
             */
                return (upper * mA + lower * mB) >>> (32 - mLgSize);
            }
        }

        private int keysSize;
        private int size;

        private Entry keysAndValues[][];
        private HashFunction[] hashFunctions;

        CuckooHashing(int size) {
            this.size = size;

            keysAndValues = (Entry[][]) Array.newInstance(Entry.class,
                    2, size);

            // The lg of the hash table size
            // Used to distribute keys uniformly in the hash function
            int lgM = (int) (Math.log(size) / Math.log(2));

            hashFunctions = new HashFunction[2];
            for (int i = 0; i < 2; i++) {
                int randomCoefficientA = StdRandom.uniform(Integer.MAX_VALUE);
                int randomCoefficientB = StdRandom.uniform(Integer.MAX_VALUE);

                hashFunctions[i] = new HashFunction(randomCoefficientA, randomCoefficientB, lgM);
            }
        }

        public boolean isEmpty() {
            return keysSize == 0;
        }

        public int keysSize() {
            return keysSize;
        }

        private void updateHashFunctions() {
            int lgM = (int) (Math.log(size) / Math.log(2));

            for (int i = 0; i < 2; i++) {
                int randomCoefficientA = StdRandom.uniform(Integer.MAX_VALUE);
                int randomCoefficientB = StdRandom.uniform(Integer.MAX_VALUE);

                hashFunctions[i] = new HashFunction(randomCoefficientA, randomCoefficientB, lgM);
            }
        }

        private void resize(int newSize) {
            StdOut.println("New Size: " + newSize);

            size = newSize;

            Entry[][] oldEntries = keysAndValues;
            keysAndValues = (Entry[][]) Array.newInstance(Entry.class,
                    2, newSize);

            boolean tryToResize = true;

            while (tryToResize) {
                tryToResize = false;

                updateHashFunctions();

                for (Entry[] keysAndValues : keysAndValues) {
                    Arrays.fill(keysAndValues, null);
                }

                // Try to add all keys and values
                // Hash table 1
                for (Entry entry : oldEntries[0]) {
                    if (entry != null && tryToInsert(entry) != null) {
                        tryToResize = true;
                        break;
                    }
                }

                // Hash table 2
                if (!tryToResize) {
                    for (Entry entry : oldEntries[1]) {
                        if (entry != null && tryToInsert(entry) != null) {
                            tryToResize = true;
                            break;
                        }
                    }
                }
            }
        }

        private void rehash() {
            StdOut.println("Rehashing Keys: " + keysSize);

            Entry[] tempKeysAndValues = (Entry[]) Array.newInstance(Entry.class, keysSize);
            int tempKeysAndValuesIndex = 0;

            for (int i = 0; i < 2; i++) {
                for (Entry entry : keysAndValues[i]) {
                    if (entry != null) {
                        tempKeysAndValues[tempKeysAndValuesIndex++] = entry;
                    }
                }
            }

            boolean tryToRehash = true;

            while (tryToRehash) {
                tryToRehash = false;

                updateHashFunctions();

                for (Entry[] keysAndValues : keysAndValues) {
                    Arrays.fill(keysAndValues, null);
                }

                // Try to add all keys and values
                for (Entry entry : tempKeysAndValues) {
                    if (tryToInsert(entry) != null) {
                        tryToRehash = true;
                        break;
                    }
                }
            }
        }

        public boolean contains(Key key) {
            if (key == null) {
                throw new IllegalArgumentException("Argument to contains() cannot be null");
            }
            return get(key) != null;
        }

        public Value get(Key key) {
            if (key == null) {
                throw new IllegalArgumentException("Argument to get() cannot be null");
            }

            for (int hashTableIndex = 0; hashTableIndex < 2; hashTableIndex++) {
                int hash = hashFunctions[hashTableIndex].hash(key);

                if (keysAndValues[hashTableIndex][hash] != null && keysAndValues[hashTableIndex][hash].key.equals(key)) {
                    return keysAndValues[hashTableIndex][hash].value;
                }
            }
            return null;
        }

        public void put(Key key, Value value) {
            if (key == null) {
                throw new IllegalArgumentException("Key cannot be null");
            }

            if (value == null) {
                delete(key);
                return;
            }

            // Update key if it already exists
            for (int hashTableIndex = 0; hashTableIndex < 2; hashTableIndex++) {
                int hash = hashFunctions[hashTableIndex].hash(key);

                if (keysAndValues[hashTableIndex][hash] != null && keysAndValues[hashTableIndex][hash].key.equals(key)) {
                    keysAndValues[hashTableIndex][hash].value = value;
                    return;
                }
            }

            // Key does not exist, let's insert it
            // Check if the number of keys is equal or more than half of the hash table size
            if (keysSize >= size) {
                resize(size * 2);
            }

            Entry entry = new Entry(key, value);
            while (entry != null) {
                entry = tryToInsert(entry);

                if (entry != null) {
                    rehash();
                }
            }

            keysSize++;
        }

        /**
         * Given an Entry, tries to insert that entry into the hash table, taking
         * several iterations if necessary.
         *
         * @return The last displaced entry, or null if all collisions were resolved.
         */
        private Entry tryToInsert(Entry entry) {
            int maxTries = size + 1;
            int hashTableIndex = 0;

            for (int numberOfTries = 0; numberOfTries < maxTries; numberOfTries++) {
                int hash = hashFunctions[hashTableIndex].hash(entry.key);

                if (keysAndValues[hashTableIndex][hash] == null) {
                    keysAndValues[hashTableIndex][hash] = entry;
                    return null;
                }

                Entry entryToDisplace = keysAndValues[hashTableIndex][hash];
                keysAndValues[hashTableIndex][hash] = entry;

                entry = entryToDisplace;
                hashTableIndex = (hashTableIndex + 1) % 2;
            }
            return entry;
        }

        public void delete(Key key) {
            if (key == null) {
                throw new IllegalArgumentException("Argument to delete() cannot be null");
            }

            if (!contains(key)) {
                return;
            }

            for (int hashTableIndex = 0; hashTableIndex < 2; hashTableIndex++) {
                int hash = hashFunctions[hashTableIndex].hash(key);

                if (keysAndValues[hashTableIndex][hash] != null && keysAndValues[hashTableIndex][hash].key.equals(key)) {
                    keysAndValues[hashTableIndex][hash] = null;
                    break;
                }
            }

            keysSize--;

            if (keysSize > 1 && keysSize <= size / (double) 8) {
                resize(size / 2);
            }
        }

        public Iterable<Key> keys() {
            Queue<Key> keySet = new Queue<>();

            for (int hashTableIndex = 0; hashTableIndex < keysAndValues.length; hashTableIndex++) {
                for (Entry entry : keysAndValues[hashTableIndex]) {
                    if (entry != null) {
                        keySet.enqueue(entry.key);
                    }
                }
            }

            if (!keySet.isEmpty() && keySet.peek() instanceof Comparable) {
                Key[] keysToBeSorted = (Key[]) new Comparable[keySet.size()];
                for (int i = 0; i < keysToBeSorted.length; i++) {
                    keysToBeSorted[i] = keySet.dequeue();
                }

                Arrays.sort(keysToBeSorted);

                for (Key key : keysToBeSorted) {
                    keySet.enqueue(key);
                }
            }
            return keySet;
        }
    }

    public static void main(String[] args) {
        Exercise31_CuckooHashing exercise31_cuckooHashing = new Exercise31_CuckooHashing();
        CuckooHashing<Integer, Integer> cuckooHashing = exercise31_cuckooHashing.new CuckooHashing<>(16);

        for (int key = 1; key < 10; key++) {
            int randomKey = StdRandom.uniform(Integer.MAX_VALUE);
            cuckooHashing.put(randomKey, randomKey);
        }

        cuckooHashing.get(5);

        for (Integer key : cuckooHashing.keys()) {
            StdOut.print(key + " ");
        }

        StdOut.println();

        for (int key = 1; key < 1000000; key++) {
            cuckooHashing.put(key, key);
        }
        for (int key = 1; key < 1000000; key++) {
            cuckooHashing.delete(key);
        }

        for (int key = 1; key < 1500000; key++) {
            int randomKey = StdRandom.uniform(Integer.MAX_VALUE);
            cuckooHashing.put(randomKey, randomKey);
        }
    }
}


Question:
Hash attack. Find 2^N strings, each of length 2^N, that have the same hashCode() value, supposing that the hashCode() implementation for String is the following:
public int hashCode() {
 int hash = 0;
 for (int i = 0; i < length(); i ++)
  hash = (hash * 31) + charAt(i);
 return hash;
}
Strong hint: Aa and BB have the same value.

Answer:
The two strings `"Aa"` and `"BB"` have the same Java `String.hashCode()` value and the same length. Therefore any concatenation of equal-length blocks chosen from `{ "Aa", "BB" }` also has the same hash as any other concatenation with the same number of blocks.

To get at least `2^N` strings of length exactly `2^N`, use `m = 2^(N-1)` two-character blocks. This produces `2^m = 2^(2^(N-1))` colliding strings, each of length `2m = 2^N`; choose any `2^N` of them.

```java
private static void generate(int blocks, String prefix, List<String> result, int limit) {
    if (result.size() == limit) return;
    if (blocks == 0) {
        result.add(prefix);
        return;
    }
    generate(blocks - 1, prefix + "Aa", result, limit);
    generate(blocks - 1, prefix + "BB", result, limit);
}
```


Question:
Bad hash function. Consider the following hashCode() implementation for String, which was used in early versions of Java:
public int hashCode() {
 int hash = 0;
 int skip = Math.max(1, length()/8);
 for (int i = 0; i < length(); i += skip)
  hash = (hash * 37) + charAt(i);
 return hash;
}
Explain why you think the designers chose this implementation and then why you think it was abandoned in favor of the one in the previous exercise.

Answer:
3.4.33 - Bad hash function

I think the designers chose this implementation because it multiplies the hash value by a prime number several times. By skipping max(1, 1/8) chars in every iteration, it guarantees that the multiplication will only happen a constant number of time, improving the performance of the hash function.
I think it was abandoned because by skipping many characters on long strings, many similar strings (that differ by only a few characters) had the same hash code, causing several hash collisions.


Question: Develop classes HashSETint and HashSETdouble for maintaining sets of keys of primitive int and double types, respectively. (Eliminate code involving values in your solution to Exercise 3.5.4.)

Answer:
package chapter3.section5;

import edu.princeton.cs.algs4.Queue;
import edu.princeton.cs.algs4.StdOut;

import java.util.Arrays;

/**
 * Created by Rene Argento on 03/08/17.
 */
@SuppressWarnings("unchecked")
public class Exercise6 {

    private class HashSETint {

        private int keysSize;
        private int size;
        private int[] keys;

        public final static int EMPTY_KEY = Integer.MIN_VALUE;

        //The largest prime <= 2^i for i = 1 to 31
        //Used to distribute keys uniformly in the hash table after resizes
        //PRIMES[n] = 2^k - Ak where k is the power of 2 and Ak is the value to subtract to reach the previous prime number
        private final int[] PRIMES = {
                1, 1, 3, 7, 13, 31, 61, 127, 251, 509, 1021, 2039, 4093, 8191, 16381,
                32749, 65521, 131071, 262139, 524287, 1048573, 2097143, 4194301,
                8388593, 16777213, 33554393, 67108859, 134217689, 268435399,
                536870909, 1073741789, 2147483647
        };

        //The lg of the hash table size
        //Used in combination with PRIMES[] to distribute keys uniformly in the hash function after resizes
        private int lgM;

        private HashSETint(int size) {
            this.size = size;
            keys = new int[size];

            for (int i = 0; i < size; i++) {
                keys[i] = EMPTY_KEY;
            }

            lgM = (int) (Math.log(size) / Math.log(2));
        }

        public int size() {
            return keysSize;
        }

        public boolean isEmpty() {
            return keysSize == 0;
        }

        private int hash(int key) {
            int hash = Integer.valueOf(key).hashCode() & 0x7fffffff;

            if (lgM < 26) {
                hash = hash % PRIMES[lgM + 5];
            }

            return hash % size;
        }

        private double getLoadFactor() {
            return keysSize / (double) size;
        }

        private void resize(int newSize) {
            HashSETint tempSet = new HashSETint(newSize);

            for (int i = 0; i < size; i++) {
                if (keys[i] != EMPTY_KEY) {
                    tempSet.add(keys[i]);
                }
            }

            keys = tempSet.keys;
            size = tempSet.size;
        }

        public boolean contains(int key) {
            if (key == EMPTY_KEY) {
                throw new IllegalArgumentException("Invalid key");
            }

            for (int tableIndex = hash(key); keys[tableIndex] != EMPTY_KEY; tableIndex = (tableIndex + 1) % size) {
                if (keys[tableIndex] == key) {
                    return true;
                }
            }

            return false;
        }

        public void add(int key) {
            if (key == EMPTY_KEY) {
                throw new IllegalArgumentException("Invalid key");
            }

            if (keysSize >= size / (double) 2) {
                resize(size * 2);
                lgM++;
            }

            int tableIndex;
            for (tableIndex = hash(key); keys[tableIndex] != EMPTY_KEY; tableIndex = (tableIndex + 1) % size) {
                if (keys[tableIndex] == key) {
                    keys[tableIndex] = key;
                    return;
                }
            }

            keys[tableIndex] = key;
            keysSize++;
        }

        public void delete(int key) {
            if (key == EMPTY_KEY) {
                throw new IllegalArgumentException("Invalid key");
            }

            if (!contains(key)) {
                return;
            }

            int tableIndex = hash(key);
            while (keys[tableIndex] != key) {
                tableIndex = (tableIndex + 1) % size;
            }

            keys[tableIndex] = EMPTY_KEY;
            keysSize--;

            tableIndex = (tableIndex + 1) % size;

            while (keys[tableIndex] != EMPTY_KEY) {
                int keyToRedo = keys[tableIndex];

                keys[tableIndex] = EMPTY_KEY;
                keysSize--;

                add(keyToRedo);
                tableIndex = (tableIndex + 1) % size;
            }

            if (keysSize > 1 && keysSize <= size / (double) 8) {
                resize(size / 2);
                lgM--;
            }
        }

        public int[] keys() {
            Queue<Integer> keySet = new Queue<>();

            for (int key : keys) {
                if (key != EMPTY_KEY) {
                    keySet.enqueue(key);
                }
            }

            int[] keys = new int[keySet.size()];
            for (int i = 0; i < keys.length; i++) {
                keys[i] = keySet.dequeue();
            }

            Arrays.sort(keys);

            return keys;
        }

        @Override
        public String toString() {
            if (isEmpty()) {
                return "{ }";
            }

            StringBuilder stringBuilder = new StringBuilder("{");

            boolean isFirstKey = true;
            for (int key : keys()) {
                if (isFirstKey) {
                    isFirstKey = false;
                } else {
                    stringBuilder.append(",");
                }

                stringBuilder.append(" ").append(key);
            }

            stringBuilder.append(" }");
            return stringBuilder.toString();
        }
    }

    private class HashSETdouble {
        private int keysSize;
        private int size;
        private double[] keys;

        public final static double EMPTY_VALUE = Double.MIN_VALUE;

        //The largest prime <= 2^i for i = 1 to 31
        //Used to distribute keys uniformly in the hash table after resizes
        //PRIMES[n] = 2^k - Ak where k is the power of 2 and Ak is the value to subtract to reach the previous prime number
        private final int[] PRIMES = {
                1, 1, 3, 7, 13, 31, 61, 127, 251, 509, 1021, 2039, 4093, 8191, 16381,
                32749, 65521, 131071, 262139, 524287, 1048573, 2097143, 4194301,
                8388593, 16777213, 33554393, 67108859, 134217689, 268435399,
                536870909, 1073741789, 2147483647
        };

        //The lg of the hash table size
        //Used in combination with PRIMES[] to distribute keys uniformly in the hash function after resizes
        private int lgM;

        private HashSETdouble(int size) {
            this.size = size;
            keys = new double[size];

            for (int i = 0; i < size; i++) {
                keys[i] = EMPTY_VALUE;
            }

            lgM = (int) (Math.log(size) / Math.log(2));
        }

        public int size() {
            return keysSize;
        }

        public boolean isEmpty() {
            return keysSize == 0;
        }

        private int hash(double key) {
            int hash = Double.valueOf(key).hashCode() & 0x7fffffff;

            if (lgM < 26) {
                hash = hash % PRIMES[lgM + 5];
            }

            return hash % size;
        }

        private double getLoadFactor() {
            return keysSize / (double) size;
        }

        private void resize(int newSize) {
            HashSETdouble tempSet = new HashSETdouble(newSize);

            for (int i = 0; i < size; i++) {
                if (keys[i] != EMPTY_VALUE) {
                    tempSet.add(keys[i]);
                }
            }

            keys = tempSet.keys;
            size = tempSet.size;
        }

        public boolean contains(double key) {
            if (key == EMPTY_VALUE) {
                throw new IllegalArgumentException("Invalid key");
            }

            for (int tableIndex = hash(key); keys[tableIndex] != EMPTY_VALUE; tableIndex = (tableIndex + 1) % size) {
                if (keys[tableIndex] == key) {
                    return true;
                }
            }

            return false;
        }

        public void add(double key) {
            if (key == EMPTY_VALUE) {
                throw new IllegalArgumentException("Invalid key");
            }

            if (keysSize >= size / (double) 2) {
                resize(size * 2);
                lgM++;
            }

            int tableIndex;
            for (tableIndex = hash(key); keys[tableIndex] != EMPTY_VALUE; tableIndex = (tableIndex + 1) % size) {
                if (keys[tableIndex] == key) {
                    keys[tableIndex] = key;
                    return;
                }
            }

            keys[tableIndex] = key;
            keysSize++;
        }

        public void delete(double key) {
            if (key == EMPTY_VALUE) {
                throw new IllegalArgumentException("Invalid key");
            }

            if (!contains(key)) {
                return;
            }

            int tableIndex = hash(key);
            while (keys[tableIndex] != key) {
                tableIndex = (tableIndex + 1) % size;
            }

            keys[tableIndex] = EMPTY_VALUE;
            keysSize--;

            tableIndex = (tableIndex + 1) % size;

            while (keys[tableIndex] != EMPTY_VALUE) {
                double keyToRedo = keys[tableIndex];

                keys[tableIndex] = EMPTY_VALUE;
                keysSize--;

                add(keyToRedo);
                tableIndex = (tableIndex + 1) % size;
            }

            if (keysSize > 1 && keysSize <= size / (double) 8) {
                resize(size / 2);
                lgM--;
            }
        }

        public double[] keys() {
            Queue<Double> keySet = new Queue<>();

            for (double key : keys) {
                if (key != EMPTY_VALUE) {
                    keySet.enqueue(key);
                }
            }

            double[] keys = new double[keySet.size()];
            for (int i = 0; i < keys.length; i++) {
                keys[i] = keySet.dequeue();
            }

            Arrays.sort(keys);

            return keys;
        }

        @Override
        public String toString() {
            if (isEmpty()) {
                return "{ }";
            }

            StringBuilder stringBuilder = new StringBuilder("{");

            boolean isFirstKey = true;
            for (double key : keys()) {
                if (isFirstKey) {
                    isFirstKey = false;
                } else {
                    stringBuilder.append(",");
                }

                stringBuilder.append(" ").append(key);
            }

            stringBuilder.append(" }");
            return stringBuilder.toString();
        }
    }

    public static void main(String[] args) {
        Exercise6 exercise6 = new Exercise6();
        exercise6.testHashSTint();
        exercise6.testHashSTdouble();
    }

    private void testHashSTint() {
        StdOut.println("HashSTint test");
        HashSETint hashSTint = new HashSETint(5);

        hashSTint.add(5);
        hashSTint.add(1);
        hashSTint.add(9);
        hashSTint.add(2);
        hashSTint.add(0);
        hashSTint.add(99);
        hashSTint.add(-1);
        hashSTint.add(-2);
        hashSTint.add(3);
        hashSTint.add(-5);

        StdOut.println("Keys() test");

        for (Integer key : hashSTint.keys()) {
            StdOut.print(key + " ");
        }
        StdOut.println("\nExpected: -5 -2 -1 0 1 2 3 5 9 99");

        StdOut.println("\ntoString() test: " + hashSTint);

        StdOut.println("\nContains 0: " + hashSTint.contains(0) + " Expected: true");
        StdOut.println("Contains 100: " + hashSTint.contains(100) + " Expected: false");

        //Test delete()
        StdOut.println("\nDelete key 2");
        hashSTint.delete(2);
        StdOut.println(hashSTint);

        StdOut.println("\nDelete key 99");
        hashSTint.delete(99);
        StdOut.println(hashSTint);

        StdOut.println("\nDelete key -5");
        hashSTint.delete(-5);
        StdOut.println(hashSTint);
    }

    private void testHashSTdouble() {
        StdOut.println("\n\nHashSTdouble test");
        HashSETdouble hashSTdouble = new HashSETdouble(5);

        hashSTdouble.add(5.0);
        hashSTdouble.add(1.0);
        hashSTdouble.add(9.5);
        hashSTdouble.add(2.1);
        hashSTdouble.add(0);
        hashSTdouble.add(99.999);
        hashSTdouble.add(-1.05);
        hashSTdouble.add(-2.20);
        hashSTdouble.add(3.0);
        hashSTdouble.add(-5.9);

        StdOut.println("Keys() test");

        for (Double key : hashSTdouble.keys()) {
            StdOut.print(key + " ");
        }
        StdOut.println("\nExpected: -5.9 -2.2 -1.05 0.0 1.0 2.1 3.0 5.0 9.5 99.999");

        StdOut.println("\ntoString() test: " + hashSTdouble);

        StdOut.println("\nContains 0: " + hashSTdouble.contains(0) + " Expected: true");
        StdOut.println("Contains 100: " + hashSTdouble.contains(100) + " Expected: false");

        //Test delete()
        StdOut.println("\nDelete key 2.1");
        hashSTdouble.delete(2.1);
        StdOut.println(hashSTdouble);

        StdOut.println("\nDelete key 99.999");
        hashSTdouble.delete(99.999);
        StdOut.println(hashSTdouble);

        StdOut.println("\nDelete key -5.9");
        hashSTdouble.delete(-5.9);
        StdOut.println(hashSTdouble);
    }
}