Question: In the style of the figure in the text, show the Huffman coding tree construction process when you use Huffman for the string "it was the age of foolishness”. How many bits does the compressed bitstream require?

Answer:
5.5.10

text: it was the age of foolishness

Frequencies
2  2   5  1  2  4  2  3  1  3  2  1  1
i  t  SP  w  a  s  h  e  g  o  f  l  n

Ordered frequencies and trie construction
1  1  1  1  2  2  2  2  2  3  3  4   5
w  g  l  n  i  t  a  h  f  e  o  s  SP

1  1   2  2  2  2  2  2  3  3  4   5
l  n  1 1  i  t  a  h  f  e  o  s  SP
      w g

 2    2   2  2  2  2  2  3  3  4   5
1 1  1 1  i  t  a  h  f  e  o  s  SP
l n  w g

2  2  2  2  2  3  3      4     4   5
i  t  a  h  f  e  o   2    2   s  SP
                     1 1  1 1
                     l n  w g

2  2  2  3  3   4       4     4   5
a  h  f  e  o  2 2   2    2   s  SP
               i t  1 1  1 1
                    l n  w g

2  3  3   4    4       4     4   5
f  e  o  2 2  2 2   2    2   s  SP
         a h  i t  1 1  1 1
                   l n  w g

3   4    4       4     4   5    5
o  2 2  2 2   2    2   s  2 3  SP
   a h  i t  1 1  1 1     f e
             l n  w g

 4       4     4   5    5    7
2 2   2    2   s  2 3  SP  3   4
i t  1 1  1 1     f e      o  2 2
     l n  w g                 a h

4   5    5    7          8
s  2 3  SP  3   4    4       4
   f e      o  2 2  2 2   2    2
               a h  i t  1 1  1 1
                         l n  w g

 5    7          8           9
SP  3   4    4       4     4   5
    o  2 2  2 2   2    2   s  2 3
       a h  i t  1 1  1 1     f e
                 l n  w g

     8           9         12
 4       4     4   5    5      7
2 2   2    2   s  2 3  SP    3   4
i t  1 1  1 1     f e        o  2 2
     l n  w g                   a h

    12                   17
 5      7          8           9
SP    3   4    4       4     4   5
      o  2 2  2 2   2    2   s  2 3
         a h  i t  1 1  1 1     f e
                   l n  w g

                29
    12                    17
 5      7           8           9
SP    3   4     4       4     4   5
      o  2 2   2 2   2    2   s  2 3
         a h   i t  1 1  1 1     f e
                    l n  w g

Codeword table
key  value
SP     00
a      0110
e      1111
f      1110
g      10111
h      0111
i      1000
l      10100
n      10101
o      010
s      110
t      1001
w      10110

Compressed bitstream (characters only): 1000 1001 10110 0110 110 1001 0111 1111 0110 10111 1111 010 1110 1110 010 010 10100 1000 110 0111 10101 1111 110 110

Compressed bitstream (trie):
0001 00100000 001 01101111 001 01100001 01 01101000 00001 01101001 01 01110100 0001 01101100 01 01101110 001 01110111 01 01100111 001 01110011 001 01100110 01 01100101

Compressed bitstream (number of characters: 29):
00011101

Bits required: 103 + 142 + 8 = 253


Question: Suppose that all of the symbol frequencies are equal. Describe the Huffman code.

Answer:
If all symbol frequencies are equal, Huffman has no reason to prefer one symbol over another, so the code is a balanced prefix code. The code is not unique: ties in the priority queue may be broken in many ways.

If the alphabet size `R` is a power of two, every codeword has length `lg R`. If `R` is not a power of two, codeword lengths differ by at most one: each length is either `floor(lg R)` or `ceil(lg R)`.

It is not correct to say that Huffman always produces the same length as the original input. Compression depends on the original fixed-width representation and on the header/trie overhead.


Question: Characterize the tricky situation in LZW coding.

Answer:
5.5.17

The tricky situation in LZW coding happens during expansion when the codeword read to get the lookahead character has the same value as the next codeword to be added in the codeword table. This happens whenever the algorithm encounters cScSc, where c is a symbol and S is a string, cS is in the dictionary already but cSc is not.


Question: Prove the following fact about Huffman codes: If the frequency of symbol i is strictly larger than the frequency of symbol j, then the length of the codeword for symbol i is less than or equal to the length of the codeword for symbol j.

Answer:
5.5.22

If the frequency of symbol i is strictly larger than the frequency of symbol j, then the length of the codeword for symbol i is less than or equal to the length of the codeword for symbol j.

Proof:
Suppose that C is an optimal Huffman code, L(C) is the sum of all the codeword lengths in the code C, C(x) is the codeword associated to the symbol x in C, f(x) is the frequency of occurrence of symbol x and l(x) is the length of C(x).
Let C’ be the code interchanging C(i) and C(j), and, as mentioned on the exercise statement, f(i) > f(j).
Then
0 <= L(C’) - L(C)
   = E f(k) * l’(k) - E f(k) * l(k)
     k                k
   = f(i) * l(j) + f(j) * l(i) - f(i) * l(i) - f(j) * l(j)
   = (f(i) - f(j)) * (l(j) - l(i))

and hence l(j) - l(i) >= 0, or equivalently, l(i) <= l(j).

Reference: http://www.cs.cmu.edu/~aarti/Class/10704_Fall16/lec10.pdf


Question: What would be the result of breaking up a Huffman-encoded string into five-bit characters and Huffman-encoding that string?

Answer:
5.5.23

Breaking up a Huffman-encoded string into five-bit characters and Huffman-encoding that string is a process called Double Huffman Coding and it would lead to a string of final length between N / 5 and N.

Proof:

Consider that the string has N bits. The best case happens when all 5-bit strings are equal: the resulting compressed string length is N / 5.

Example:
01010010100101001010
is broken into
01010 01010 01010 01010

With the Huffman code 01010 -> 0, the string becomes:
0000

The original Huffman-encoded string length was 20 (N) and the final string length is 4 (N / 5).

With 5 bits there are 2^5 possible five-bit characters. The worst case happens when all five-bit characters are different and there is one occurrence of each possible five-bit character. In this case, the Huffman encoding tree will be a complete binary tree and all codewords will have length 5. All five-bit characters will be encoded to codewords of length 5 and the final string length will be N (the same length as the original Huffman-encoded string).

This is an idealized data-bit bound. A real Huffman file also stores a trie/header and may need padding, so for short inputs the second Huffman encoding can be larger than this bound suggests.


Question:
In the style of the figures in the text, show the encoding trie and the compression and expansion processes when LZW is used for the string
it was the best of times it was the worst of times

Answer:
5.5.24

text: it was the best of times it was the worst of times

Compression process

input   I     T     W     A     S     T     H     E     B     E     S     T     O     F     T     I     M     E     S     I     T     W     A     S     T     H     E     W     O     R     S     T     O     F     T     I     M     E     S
matches I     T     W     A     S     T     H     E     B     E     ST          O     F     T     I     M     ES          IT          WA          ST          HE          W     O     R     STO               FT          IM          ES
output  49    54    57    41    53    54    48    45    42    45    85          4F    46    54    49    4D    8A          81          83          85          87          57    4F    52    8B                8D          8F          8A         80

        IT 81 IT    IT    IT    IT    IT    IT    IT    IT    IT    IT          IT    IT    IT    IT    IT    IT          IT          IT          IT          IT          IT    IT    IT    IT                IT          IT          IT
              TW 82 TW    TW    TW    TW    TW    TW    TW    TW    TW          TW    TW    TW    TW    TW    TW          TW          TW          TW          TW          TW    TW    TW    TW                TW          TW          TW
                    WA 83 WA    WA    WA    WA    WA    WA    WA    WA          WA    WA    WA    WA    WA    WA          WA          WA          WA          WA          WA    WA    WA    WA                WA          WA          WA
                          AS 84 AS    AS    AS    AS    AS    AS    AS          AS    AS    AS    AS    AS    AS          AS          AS          AS          AS          AS    AS    AS    AS                AS          AS          AS
                                ST 85 ST    ST    ST    ST    ST    ST          ST    ST    ST    ST    ST    ST          ST          ST          ST          ST          ST    ST    ST    ST                ST          ST          ST
                                      TH 86 TH    TH    TH    TH    TH          TH    TH    TH    TH    TH    TH          TH          TH          TH          TH          TH    TH    TH    TH                TH          TH          TH
                                            HE 87 HE    HE    HE    HE          HE    HE    HE    HE    HE    HE          HE          HE          HE          HE          HE    HE    HE    HE                HE          HE          HE
                                                  EB 88 EB    EB    EB          EB    EB    EB    EB    EB    EB          EB          EB          EB          EB          EB    EB    EB    EB                EB          EB          EB
                                                        BE 89 BE    BE          BE    BE    BE    BE    BE    BE          BE          BE          BE          BE          BE    BE    BE    BE                BE          BE          BE
                                                              ES 8A ES          ES    ES    ES    ES    ES    ES          ES          ES          ES          ES          ES    ES    ES    ES                ES          ES          ES
                                                                    STO 8B      STO   STO   STO   STO   STO   STO         STO         STO         STO         STO         STO   STO   STO   STO               STO         STO         STO
                                                                                OF 8C OF    OF    OF    OF    OF          OF          OF          OF          OF          OF    OF    OF    OF                OF          OF          OF
                                                                                FT 8D FT    FT    FT    FT          FT          FT          FT          FT          FT    FT    FT    FT                FT          FT          FT
                                                                                TI 8E TI    TI    TI          TI          TI          TI          TI          TI    TI    TI    TI                TI          TI          TI
                                                                                IM 8F IM    IM          IM          IM          IM          IM          IM    IM    IM    IM                IM          IM          IM
                                                                                ME 90 ME          ME          ME          ME          ME          ME    ME    ME    ME                ME          ME          ME
                                                                                ESI 91      ESI         ESI         ESI         ESI         ESI   ESI   ESI   ESI               ESI         ESI         ESI
                                                                                ITW 92      ITW         ITW         ITW         ITW   ITW   ITW   ITW               ITW         ITW         ITW
                                                                                WAS 93      WAS         WAS         WAS   WAS   WAS   WAS               WAS         WAS         WAS
                                                                                STH 94      STH         STH   STH   STH   STH               STH         STH         STH
                                                                                HEW 95      HEW   HEW   HEW   HEW               HEW         HEW         HEW
                                                                                WO 96 WO    WO    WO                WO          WO          WO
                                                                                OR 97 OR    OR                OR          OR          OR
                                                                                RS 98 RS                RS          RS          RS
                                                                                STOF 99           STOF        STOF        STOF
                                                                                FTI 9A      FTI         FTI
                                                                                IME 9B      IME

Codeword table
key   value
IT     81
TW     82
WA     83
AS     84
ST     85
TH     86
HE     87
EB     88
BE     89
ES     8A
STO    8B
OF     8C
FT     8D
TI     8E
IM     8F
ME     90
ESI    91
ITW    92
WAS    93
STH    94
HEW    95
WO     96
OR     97
RS     98
STOF   99
FTI    9A
IME    9B

Encoding trie
                                                    Root
                                                     |
     ________________________________________________________________________________________________________
    /    |       |        |       |        |         |         |         |         |           |             \
   A 41  B 42    E 45     F 46    H 48     I 49      M 4D      O 4F      R 52      S 53        T 54           W 57
   |     |       |        |       |        |         |         |         |         |           |              |
   S 84  E 89  ____       T 8D    E 87    ____       E 90     ____       S 98      T 85     ________         ____
              /    \      |       |      /    \              /    \                |       /   |    \       /    \
             B 88   S 8A  I 9A    W 95  M 8F   T 81         F 8C   R 97           ____    H 86 I 8E  W 82  A 83   O 96
                    |                   |      |                                 /    \                    |
                    I 91                E 9B   W 92                             H 94   O 8B                S 93
                                                                                |
                                                                                F 99

Expansion process

input  49    54    57    41    53    54    48    45    42    45    85     4F    46    54    49    4D    8A     81     83     85     87     57    4F    52    8B      8D     8F     8A    80
output  I     T     W     A     S     T     H     E     B     E     ST     O     F     T     I     M     ES     IT     WA     ST     HE     W     O     R     STO     FT     IM     ES

     81 IT    IT    IT    IT    IT    IT    IT    IT    IT    IT    IT     IT    IT    IT    IT    IT    IT     IT     IT     IT     IT     IT    IT    IT    IT      IT     IT     IT
           82 TW    TW    TW    TW    TW    TW    TW    TW    TW    TW     TW    TW    TW    TW    TW    TW     TW     TW     TW     TW     TW    TW    TW    TW      TW     TW     TW
                 83 WA    WA    WA    WA    WA    WA    WA    WA    WA     WA    WA    WA    WA    WA    WA     WA     WA     WA     WA     WA    WA    WA    WA      WA     WA     WA
                       84 AS    AS    AS    AS    AS    AS    AS    AS     AS    AS    AS    AS    AS    AS     AS     AS     AS     AS     AS    AS    AS    AS      AS     AS     AS
                             85 ST    ST    ST    ST    ST    ST    ST     ST    ST    ST    ST    ST    ST     ST     ST     ST     ST     ST    ST    ST    ST      ST     ST     ST
                                   86 TH    TH    TH    TH    TH    TH     TH    TH    TH    TH    TH    TH     TH     TH     TH     TH     TH    TH    TH    TH      TH     TH     TH
                                         87 HE    HE    HE    HE    HE     HE    HE    HE    HE    HE    HE     HE     HE     HE     HE     HE    HE    HE    HE      HE     HE     HE
                                               88 EB    EB    EB    EB     EB    EB    EB    EB    EB    EB     EB     EB     EB     EB     EB    EB    EB    EB      EB     EB     EB
                                                     89 BE    BE    BE     BE    BE    BE    BE    BE    BE     BE     BE     BE     BE     BE    BE    BE    BE      BE     BE     BE
                                                           8A ES    ES     ES    ES    ES    ES    ES    ES     ES     ES     ES     ES     ES    ES    ES    ES      ES     ES     ES
                                                                 8B STO    STO   STO   STO   STO   STO   STO    STO    STO    STO    STO    STO   STO   STO   STO     STO    STO    STO
                                                                        8C OF    OF    OF    OF    OF    OF     OF     OF     OF     OF     OF    OF    OF    OF      OF     OF     OF
                                                                              8D FT    FT    FT    FT    FT     FT     FT     FT     FT     FT    FT    FT    FT      FT     FT     FT
                                                                                8E TI    TI    TI    TI     TI     TI     TI     TI     TI    TI    TI    TI      TI     TI     TI
                                                                                8F IM    IM    IM     IM     IM     IM     IM     IM    IM    IM    IM      IM     IM     IM
                                                                                90 ME    ME     ME     ME     ME     ME     ME    ME    ME    ME      ME     ME     ME
                                                                                91 ESI    ESI    ESI    ESI    ESI    ESI   ESI   ESI   ESI     ESI    ESI    ESI
                                                                                92 ITW    ITW    ITW    ITW    ITW   ITW   ITW   ITW     ITW    ITW    ITW
                                                                                93 WAS    WAS    WAS    WAS   WAS   WAS   WAS     WAS    WAS    WAS
                                                                                94 STH    STH    STH   STH   STH   STH     STH    STH    STH
                                                                                95 HEW    HEW   HEW   HEW   HEW     HEW    HEW    HEW
                                                                                96 WO    WO    WO    WO      WO     WO     WO
                                                                                97 OR    OR    OR      OR     OR     OR
                                                                                98 RS    RS      RS     RS     RS
                                                                                99 STOF    STOF   STOF   STOF
                                                                                9A FTI    FTI    FTI
                                                                                9B IME    IME

Inverse codeword table
key   value
81     IT
82     TW
83     WA
84     AS
85     ST
86     TH
87     HE
88     EB
89     BE
8A     ES
8B     STO
8C     OF
8D     FT
8E     TI
8F     IM
90     ME
91     ESI
92     ITW
93     WAS
94     STH
95     HEW
96     WO
97     OR
98     RS
99     STOF
9A     FTI
9B     IME


Question: Long repeats. Estimate the compression ratio achieved by run-length, Huffman, and LZW encoding for a string of length 2N formed by concatenating two copies of a random ASCII string of length N (see EXERCISE 5.5.9), under any assumptions that you think are reasonable.

Answer:
5.5.27 - Long repeats

Compressing 2 * 1000 random characters (for N = 1000), with 16000 bits (8 bits per character).

% java -cp algs4.jar:. RunLengthEncoding - < 5.5.27_random.txt | java -cp algs4.jar:. BinaryDump 0
53736 bits

Compression ratio: 53736 / 16000 = 335%

% java -cp algs4.jar:. edu.princeton.cs.algs4.Huffman - < 5.5.27_random.txt | java -cp algs4.jar:. BinaryDump 0
12624 bits

Compression ratio: 12624 / 16000 = 79%

% java -cp algs4.jar:. edu.princeton.cs.algs4.LZW - < 5.5.27_random.txt | java -cp algs4.jar:. BinaryDump 0
13416 bits

Compression ratio: 13416 / 16000 = 83%


Question: Molecules travel very quickly (faster than a speeding jet) but diffuse slowly because they collide with other molecules, thereby changing their direction. Extend the model to have a boundary shape where two vessels are connected by a pipe containing two different types of particles. Run a simulation and measure the fraction of particles of each type in each vessel as a function of time.

Answer:
6.9

Results

Time 0
LEFT VESSEL: 10 particles
Type 1: 10/10 Type 2: 0/10
RIGHT VESSEL: 10 particles
Type 1: 0/10 Type 2: 10/10

Time 1000
LEFT VESSEL: 14 particles
Type 1: 9/14 Type 2: 5/14
RIGHT VESSEL: 6 particles
Type 1: 1/6 Type 2: 5/6

Time 2000
LEFT VESSEL: 11 particles
Type 1: 6/11 Type 2: 5/11
RIGHT VESSEL: 9 particles
Type 1: 4/9 Type 2: 5/9

Time 3000
LEFT VESSEL: 13 particles
Type 1: 7/13 Type 2: 6/13
RIGHT VESSEL: 6 particles
Type 1: 2/6 Type 2: 4/6

Time 4000
LEFT VESSEL: 13 particles
Type 1: 7/13 Type 2: 6/13
RIGHT VESSEL: 7 particles
Type 1: 3/7 Type 2: 4/7

Time 5000
LEFT VESSEL: 10 particles
Type 1: 6/10 Type 2: 4/10
RIGHT VESSEL: 10 particles
Type 1: 4/10 Type 2: 6/10

Time 6000
LEFT VESSEL: 11 particles
Type 1: 6/11 Type 2: 5/11
RIGHT VESSEL: 9 particles
Type 1: 4/9 Type 2: 5/9

Time 7000
LEFT VESSEL: 10 particles
Type 1: 5/10 Type 2: 5/10
RIGHT VESSEL: 10 particles
Type 1: 5/10 Type 2: 5/10

Time 8000
LEFT VESSEL: 8 particles
Type 1: 3/8 Type 2: 5/8
RIGHT VESSEL: 12 particles
Type 1: 7/12 Type 2: 5/12

Time 9000
LEFT VESSEL: 10 particles
Type 1: 3/10 Type 2: 7/10
RIGHT VESSEL: 10 particles
Type 1: 7/10 Type 2: 3/10

Time 10000
LEFT VESSEL: 9 particles
Type 1: 5/9 Type 2: 4/9
RIGHT VESSEL: 11 particles
Type 1: 5/11 Type 2: 6/11

The system tends to achieve a balance between the number of particles of both types on both vessels.


Question: After running a simulation, negate all velocities and then run the system backward. It should return to its original state! Measure roundoff error by measuring the difference between the final and original states of the system.

Answer:
6.10

Roundoff error: 0.22176242098152166


Question: Add a method pressure() to Particle that measures pressure by accumulating the number and magnitude of collisions against walls. The pressure of the system is the sum of these quantities. Then add a method pressure() to CollisionSystem and write a client that validates the equation pv = nRT.

Answer:
The pressure measurement should use impulse delivered to the container walls, not particle volume.

For a vertical-wall collision, add impulse `2m|vx|`; for a horizontal-wall collision, add `2m|vy|`. Over elapsed time `t`, the 2D pressure analogue is force per boundary length:

`P = totalWallImpulse / (perimeter * t)`.

For the ideal-gas comparison in this 2D simulation, use the container area as `V` (the unit square has `V = 1`), and compute temperature from mean kinetic energy:

`T = average(m(vx^2 + vy^2)) / (2 kB)`.

Then compare the wall-collision measurement with

`P V = N kB T`,

or equivalently `P V = n R T` with `n = N / NA` and `R = NA kB`. The previous answer incorrectly used each particle's disk/sphere volume as the gas volume and squared the velocity magnitude again in the temperature calculation.


Question: Instrument the priority queue and test Pressure at various temperatures to identify the computational bottleneck. If warranted, try switching to a different priority-queue implementation for better performance at high temperatures.

Answer:
// Exercise13_PriorityQueuePerformance.java
package chapter6.eventdrivensimulation;

import chapter2.section4.IndexMinPriorityQueue;
import chapter2.section4.PriorityQueueResize;
import edu.princeton.cs.algs4.StdDraw;
import edu.princeton.cs.algs4.StdOut;
import edu.princeton.cs.algs4.StdRandom;
import edu.princeton.cs.algs4.Stopwatch;

/**
 * Created by Rene Argento on 12/07/18.
 */
public class Exercise13_PriorityQueuePerformance {

    private final double BOLTZMANN_CONSTANT = 1.3806488e-23;

    public class ParticleWithPressureAndId extends Particle {

        private static final int DIMENSION = 2;

        private int id;
        private double pressure;

        ParticleWithPressureAndId(int id, double positionX, double positionY, double velocityX, double velocityY,
                                  double radius, double mass) {
            super(positionX, positionY, velocityX, velocityY, radius, mass);
            this.id = id;
        }

        public double pressure() {
            return pressure;
        }

        private void addWallCollisionToPressure(boolean isHorizontal) {
            if (isHorizontal) {
                pressure += Math.abs(2 * mass * velocityY);
            } else {
                pressure += Math.abs(2 * mass * velocityX);
            }
        }

        // V = 4/3 (pi r^3)
        // V = volume
        // r = radius
        public double volume() {
            double volumeInCm3 = 4.0/3.0 * Math.PI * Math.pow(radius, 3);
            // Volume in dm^3
            return volumeInCm3 / 1000;
        }

        public double temperature() {
            double velocityMagnitude = Math.pow(velocityX, 2) + Math.pow(velocityY, 2);
            return mass * Math.pow(velocityMagnitude, 2) / (DIMENSION * BOLTZMANN_CONSTANT);
        }

        public void setVelocityX(double velocityX) {
            this.velocityX = velocityX;
        }

        public void setVelocityY(double velocityY) {
            this.velocityY = velocityY;
        }
    }

    public class CollisionSystemWithPressure {

        private class Event implements Comparable<Event> {

            private final double time;
            private final ParticleWithPressureAndId particleA;
            private final ParticleWithPressureAndId particleB;
            private final int collisionsCountA;
            private final int collisionsCountB;

            public Event(double time, ParticleWithPressureAndId particleA, ParticleWithPressureAndId particleB) {
                this.time = time;
                this.particleA = particleA;
                this.particleB = particleB;

                if (particleA != null) {
                    collisionsCountA = particleA.count();
                } else {
                    collisionsCountA = -1;
                }

                if (particleB != null) {
                    collisionsCountB = particleB.count();
                } else {
                    collisionsCountB = -1;
                }
            }

            public int compareTo(Event otherEvent) {
                if (this.time < otherEvent.time) {
                    return -1;
                } else if (this.time > otherEvent.time) {
                    return +1;
                } else {
                    return 0;
                }
            }

            public boolean isValid() {
                if (particleA != null && particleA.count() != collisionsCountA) {
                    return false;
                }

                if (particleB != null && particleB.count() != collisionsCountB) {
                    return false;
                }

                return true;
            }
        }

        private PriorityQueueResize<Event> priorityQueue;
        private IndexMinPriorityQueue<Event> indexMinPriorityQueue;
        private double time;

        private ParticleWithPressureAndId[] particles;
        private final int DRAW_EVENT_ID;

        private int numberOfParticles;
        private boolean useIndexPriorityQueue;

        private final double IDEAL_GAS_CONSTANT = 0.082057;
        private final double AVOGADROS_NUMBER = 6.022e23;

        private double totalTimeSpentOnInsert = 0;
        private double totalTimeSpentOnDeleteMin = 0;
        private double totalTimeSpentOnIsEmpty = 0;

        // Index min priority queue specific metrics
        private double totalTimeSpentOnContains = 0;
        private double totalTimeSpentOnMin = 0;
        private double totalTimeSpentOnDelete = 0;

        public CollisionSystemWithPressure(ParticleWithPressureAndId[] particles, boolean useIndexPriorityQueue) {
            StdDraw.enableDoubleBuffering();
            this.particles = particles;
            numberOfParticles = particles.length;

            DRAW_EVENT_ID = particles.length;
            this.useIndexPriorityQueue = useIndexPriorityQueue;
        }

        private void predictCollisions(ParticleWithPressureAndId particle, double limit) {
            if (particle == null) {
                return;
            }

            int particleId = particle.id;
            double currentSmallestEventTime = Double.POSITIVE_INFINITY;

            if (useIndexPriorityQueue) {
                Stopwatch timer = new Stopwatch();
                boolean containsParticleId = indexMinPriorityQueue.contains(particleId);
                totalTimeSpentOnContains += timer.elapsedTime();

                if (containsParticleId) {
                    timer = new Stopwatch();
                    indexMinPriorityQueue.delete(particleId);
                    totalTimeSpentOnDelete += timer.elapsedTime();
                }
            }

            for (int i = 0; i < particles.length; i++) {
                double deltaTime = particle.timeToHit(particles[i]);
                double eventTime = time + deltaTime;

                if (eventTime <= limit
                        && (!useIndexPriorityQueue || eventTime <= currentSmallestEventTime)) {
                    if (!useIndexPriorityQueue) {
                        Stopwatch timer = new Stopwatch();
                        priorityQueue.insert(new Event(eventTime, particle, particles[i]));
                        totalTimeSpentOnInsert += timer.elapsedTime();
                    } else {
                        Stopwatch timer = new Stopwatch();
                        boolean containsParticleId = indexMinPriorityQueue.contains(particleId);
                        totalTimeSpentOnContains += timer.elapsedTime();

                        if (containsParticleId) {
                            timer = new Stopwatch();
                            indexMinPriorityQueue.delete(particleId);
                            totalTimeSpentOnDelete += timer.elapsedTime();
                        }

                        timer = new Stopwatch();
                        indexMinPriorityQueue.insert(particleId, new Event(eventTime, particle, particles[i]));
                        totalTimeSpentOnInsert += timer.elapsedTime();

                        currentSmallestEventTime = eventTime;
                    }
                }
            }

            double deltaTimeVerticalWall = particle.timeToHitVerticalWall();
            double verticalWallEventTime = time + deltaTimeVerticalWall;

            if (verticalWallEventTime <= limit
                    && (!useIndexPriorityQueue || verticalWallEventTime <= currentSmallestEventTime)) {
                if (!useIndexPriorityQueue) {
                    Stopwatch timer = new Stopwatch();
                    priorityQueue.insert(new Event(verticalWallEventTime, particle, null));
                    totalTimeSpentOnInsert += timer.elapsedTime();
                } else {
                    Stopwatch timer = new Stopwatch();
                    boolean containsParticleId = indexMinPriorityQueue.contains(particleId);
                    totalTimeSpentOnContains += timer.elapsedTime();

                    if (containsParticleId) {
                        timer = new Stopwatch();
                        indexMinPriorityQueue.delete(particleId);
                        totalTimeSpentOnDelete += timer.elapsedTime();
                    }

                    timer = new Stopwatch();
                    indexMinPriorityQueue.insert(particleId, new Event(verticalWallEventTime, particle, null));
                    totalTimeSpentOnInsert += timer.elapsedTime();

                    currentSmallestEventTime = verticalWallEventTime;
                }
            }

            double deltaTimeHorizontalWall = particle.timeToHitHorizontalWall();
            double horizontalWallEventTime = time + deltaTimeHorizontalWall;

            if (horizontalWallEventTime <= limit
                    && (!useIndexPriorityQueue || horizontalWallEventTime <= currentSmallestEventTime)) {
                if (!useIndexPriorityQueue) {
                    Stopwatch timer = new Stopwatch();
                    priorityQueue.insert(new Event(horizontalWallEventTime, null, particle));
                    totalTimeSpentOnInsert += timer.elapsedTime();
                } else {
                    Stopwatch timer = new Stopwatch();
                    boolean containsParticleId = indexMinPriorityQueue.contains(particleId);
                    totalTimeSpentOnContains += timer.elapsedTime();

                    if (containsParticleId) {
                        timer = new Stopwatch();
                        indexMinPriorityQueue.delete(particleId);
                        totalTimeSpentOnDelete += timer.elapsedTime();
                    }

                    timer = new Stopwatch();
                    indexMinPriorityQueue.insert(particleId, new Event(horizontalWallEventTime, null, particle));
                    totalTimeSpentOnInsert += timer.elapsedTime();
                }
            }
        }

        public void redraw(double limit, double hertz) {
            StdDraw.clear();

            for (int i = 0; i < particles.length; i++) {
                particles[i].draw();
            }

            StdDraw.pause(20);
            StdDraw.show();

            if (time < limit) {
                if (!useIndexPriorityQueue) {
                    Stopwatch timer = new Stopwatch();
                    priorityQueue.insert(new Event(time + 1.0 / hertz, null, null));
                    totalTimeSpentOnInsert += timer.elapsedTime();
                } else {
                    Stopwatch timer = new Stopwatch();
                    indexMinPriorityQueue.insert(DRAW_EVENT_ID, new Event(time + 1.0 / hertz, null, null));
                    totalTimeSpentOnInsert += timer.elapsedTime();
                }
            }
        }

        public void simulate(double limit, double hertz) {
            if (!useIndexPriorityQueue) {
                priorityQueue = new PriorityQueueResize<>(PriorityQueueResize.Orientation.MIN);

                Stopwatch timer = new Stopwatch();
                priorityQueue.insert(new Event(0, null, null));
                totalTimeSpentOnInsert += timer.elapsedTime();
            } else {
                // Add 1 extra space for the draw event
                indexMinPriorityQueue = new IndexMinPriorityQueue<>(particles.length + 1);

                Stopwatch timer = new Stopwatch();
                indexMinPriorityQueue.insert(DRAW_EVENT_ID, new Event(0, null, null));
                totalTimeSpentOnInsert += timer.elapsedTime();
            }

            for (int i = 0; i < particles.length; i++) {
                predictCollisions(particles[i], limit);
            }

            StdOut.println("Testing with temperature: " + temperature());

            int nextCheckpoint = 1000;

            while (!isPriorityQueueEmpty()) {
                Event event;

                if (!useIndexPriorityQueue) {
                    Stopwatch timer = new Stopwatch();
                    event = priorityQueue.deleteTop();
                    totalTimeSpentOnDeleteMin += timer.elapsedTime();
                } else {
                    Stopwatch timer = new Stopwatch();
                    event = indexMinPriorityQueue.minKey();
                    totalTimeSpentOnMin += timer.elapsedTime();

                    timer = new Stopwatch();
                    indexMinPriorityQueue.deleteMin();
                    totalTimeSpentOnDeleteMin += timer.elapsedTime();
                }

                // ValidateEquation method is only used to simulate a Pressure test, but its results are not used
                if (event.time >= nextCheckpoint) {
                    validateEquation(time);
                    nextCheckpoint += 1000;
                }

                if (!event.isValid()) {
                    continue;
                }

                // Update particle positions
                for (int i = 0; i < particles.length; i++) {
                    particles[i].move(event.time - time);
                }
                // Update time
                time = event.time;

                ParticleWithPressureAndId particleA = event.particleA;
                ParticleWithPressureAndId particleB = event.particleB;

                if (particleA != null && particleB != null) {
                    particleA.bounceOff(particleB);
                } else if (particleA != null && particleB == null) {
                    particleA.bounceOffVerticalWall();
                    particleA.addWallCollisionToPressure(false);
                } else if (particleA == null && particleB != null) {
                    particleB.bounceOffHorizontalWall();
                    particleB.addWallCollisionToPressure(true);
                } else if (particleA == null && particleB == null) {
                    redraw(limit, hertz);
                }

                predictCollisions(particleA, limit);
                predictCollisions(particleB, limit);
            }

            identifyBottleneck();
        }

        private boolean isPriorityQueueEmpty() {
            boolean isEmpty;

            if (!useIndexPriorityQueue) {
                Stopwatch timer = new Stopwatch();
                isEmpty = priorityQueue.isEmpty();
                totalTimeSpentOnIsEmpty += timer.elapsedTime();
            } else {
                Stopwatch timer = new Stopwatch();
                isEmpty = indexMinPriorityQueue.isEmpty();
                totalTimeSpentOnIsEmpty += timer.elapsedTime();
            }

            return isEmpty;
        }

        private void validateEquation(double time) {
            double pressureMeasuredWithEquation = pressure();
            double pressureMeasuredWithWallCollisions = pressureWithWallCollisions(time);
            boolean unusedTest = pressureMeasuredWithEquation == pressureMeasuredWithWallCollisions;
        }

        // Ideal gas formula: P V = n R T
        // P = pressure
        // V = volume
        // n = number of moles
        // R = ideal gas constant
        // T = temperature
        //
        // P V = n R T
        // P = (n R T) / V
        public double pressure() {
            return (numberOfMoles() * IDEAL_GAS_CONSTANT * temperature()) / volume();
        }

        public double pressureWithWallCollisions(double time) {
            double systemPressure = 0;

            for (ParticleWithPressureAndId particle : particles) {
                systemPressure += particle.pressure();
            }

            return systemPressure / time;
        }

        public double volume() {
            double systemVolume = 0;

            for (ParticleWithPressureAndId particle : particles) {
                systemVolume += particle.volume();
            }

            return systemVolume;
        }

        // n = m / mw
        // n = number of moles
        // m = total mass
        // mw = molecular weight
        //
        // mw = pm * AC
        // pm = mass of one particle
        // AC = Avogadro's number
        public double numberOfMoles() {
            // All particle masses are the same
            double particleMass = particles[0].mass;
            double totalMass = particleMass * particles.length;

            double molecularWeight = particleMass * AVOGADROS_NUMBER;
            return totalMass / molecularWeight;
        }

        public double temperature() {
            double totalTemperature = 0;

            for (ParticleWithPressureAndId particle : particles) {
                totalTemperature += particle.temperature();
            }

            return totalTemperature / numberOfParticles;
        }

        // V = SQRT(2 kb T / M)
        // SQRT(V / 2) = (vx + vy) / 2
        private void setTemperature(double temperature) {
            for (ParticleWithPressureAndId particle : particles) {
                double newVelocityMagnitude =
                        Math.sqrt(2 * BOLTZMANN_CONSTANT * temperature / particle.getMass());
                double velocityComponent = Math.sqrt(newVelocityMagnitude / 2);
                particle.setVelocityX(velocityComponent);
                particle.setVelocityY(velocityComponent);
            }
        }

        private void identifyBottleneck() {
            double highestTimeSpent = 0;
            String computationalBottleneck = "";

            StdOut.printf("Total time spent on insert operations: %.5f\n", totalTimeSpentOnInsert);
            if (totalTimeSpentOnInsert > highestTimeSpent) {
                highestTimeSpent = totalTimeSpentOnInsert;
                computationalBottleneck = "insert";
            }

            StdOut.printf("Total time spent on deleteMin operations: %.5f\n", totalTimeSpentOnDeleteMin);
            if (totalTimeSpentOnDeleteMin > highestTimeSpent) {
                highestTimeSpent = totalTimeSpentOnDeleteMin;
                computationalBottleneck = "deleteMin";
            }

            StdOut.printf("Total time spent on isEmpty operations: %.5f\n", totalTimeSpentOnIsEmpty);
            if (totalTimeSpentOnIsEmpty > highestTimeSpent) {
                highestTimeSpent = totalTimeSpentOnIsEmpty;
                computationalBottleneck = "isEmpty";
            }

            if (useIndexPriorityQueue) {
                StdOut.printf("Total time spent on contains operations: %.5f\n", totalTimeSpentOnContains);
                if (totalTimeSpentOnContains > highestTimeSpent) {
                    highestTimeSpent = totalTimeSpentOnContains;
                    computationalBottleneck = "contains";
                }

                StdOut.printf("Total time spent on min operations: %.5f\n", totalTimeSpentOnMin);
                if (totalTimeSpentOnMin > highestTimeSpent) {
                    highestTimeSpent = totalTimeSpentOnMin;
                    computationalBottleneck = "min";
                }

                StdOut.printf("Total time spent on delete operations: %.5f\n", totalTimeSpentOnDelete);
                if (totalTimeSpentOnDelete > highestTimeSpent) {
                    computationalBottleneck = "delete";
                }
            }

            StdOut.println("Computational bottleneck: " + computationalBottleneck + " operations");
        }
    }

    private ParticleWithPressureAndId getRandomParticle(int id) {
        double positionX = StdRandom.uniform(0.0, 1.0);
        double positionY = StdRandom.uniform(0.0, 1.0);

        double velocityX = StdRandom.uniform(-0.005, 0.005);
        double velocityY = StdRandom.uniform(-0.005, 0.005);

        double radius = 0.0025;
        double mass = 0.5;

        return new ParticleWithPressureAndId(id, positionX, positionY, velocityX, velocityY, radius, mass);
    }

    private void doExperiment(int numberOfParticles, int simulationTime, double hertz) {
        StdOut.println("**** Standard priority queue tests ****");
        StdOut.println();

        double baseLineTemperature = doBaselineTest(numberOfParticles, simulationTime, hertz);

        double[] testTemperatures = {
                baseLineTemperature * 100,
                baseLineTemperature * 10000,
                baseLineTemperature * 1000000,
                baseLineTemperature * 100000000
        };

        for (int t = 0; t < testTemperatures.length; t++) {
            simulate(numberOfParticles, simulationTime, hertz, testTemperatures[t], false);
        }

        StdOut.println();
        StdOut.println("**** Index priority queue tests ****");

        // Also test the high temperatures with the index priority-queue
        for (int t = 2; t < testTemperatures.length; t++) {
            simulate(numberOfParticles, simulationTime, hertz, testTemperatures[t], true);
        }
    }

    private double doBaselineTest(int numberOfParticles, int simulationTime, double hertz) {
        ParticleWithPressureAndId[] particles = new ParticleWithPressureAndId[numberOfParticles];

        for (int i = 0; i < numberOfParticles; i++) {
            particles[i] = getRandomParticle(i);
        }

        CollisionSystemWithPressure collisionSystem = new CollisionSystemWithPressure(particles, false);
        collisionSystem.simulate(simulationTime, hertz);

        return collisionSystem.temperature();
    }

    private void simulate(int numberOfParticles, int simulationTime, double hertz, double testTemperature,
                          boolean useIndexPriorityQueue) {
        StdDraw.clear();
        StdOut.println();

        ParticleWithPressureAndId[] particles = new ParticleWithPressureAndId[numberOfParticles];
        for (int i = 0; i < numberOfParticles; i++) {
            particles[i] = getRandomParticle(i);
        }

        CollisionSystemWithPressure collisionSystemWithPressure =
                new CollisionSystemWithPressure(particles, useIndexPriorityQueue);
        collisionSystemWithPressure.setTemperature(testTemperature);

        collisionSystemWithPressure.simulate(simulationTime, hertz);
    }

    public static void main(String[] args) {
        int numberOfParticles = 30;
        int simulationTime = 10000;
        double hertz = 0.5;

        new Exercise13_PriorityQueuePerformance().doExperiment(numberOfParticles, simulationTime, hertz);
    }
}

Additional notes/results:
6.13

Tests done with a random baseline temperature T, 100 * T, 10000 * T, 1000000 * T and 100000000 * T.
The standard priority queue was used to test all the temperatures.
The index priority queue was also used to test the highest temperatures: 1000000 * T and 100000000 * T.

Results:

**** Standard priority queue tests ****

Testing with temperature: 5.817661439511093E12
Total time spent on insert operations: 0.02200
Total time spent on deleteMin operations: 0.01300
Total time spent on isEmpty operations: 0.00200
Computational bottleneck: insert operations

Testing with temperature: 8.270951314920601E14
Total time spent on insert operations: 0.00700
Total time spent on deleteMin operations: 0.01400
Total time spent on isEmpty operations: 0.00000
Computational bottleneck: delete min operations

Testing with temperature: 8.2709513149206064E16
Total time spent on insert operations: 0.00800
Total time spent on deleteMin operations: 0.01500
Total time spent on isEmpty operations: 0.00300
Computational bottleneck: delete min operations

Testing with temperature: 8.2709513149206047E18
Total time spent on insert operations: 0.01900
Total time spent on deleteMin operations: 0.06700
Total time spent on isEmpty operations: 0.00300
Computational bottleneck: delete min operations

Testing with temperature: 8.270951314920604E20
Total time spent on insert operations: 0.04700
Total time spent on deleteMin operations: 0.12100
Total time spent on isEmpty operations: 0.02500
Computational bottleneck: delete min operations

**** Index priority queue tests ****

Testing with temperature: 6.3274920297362524E18
Total time spent on insert operations: 0.00900
Total time spent on deleteMin operations: 0.00500
Total time spent on isEmpty operations: 0.00000
Total time spent on contains operations: 0.00300
Total time spent on min operations: 0.00100
Total time spent on delete operations: 0.00400
Computational bottleneck: insert operations

Testing with temperature: 6.327492029736252E20
Total time spent on insert operations: 0.00600
Total time spent on deleteMin operations: 0.00700
Total time spent on isEmpty operations: 0.00000
Total time spent on contains operations: 0.00300
Total time spent on min operations: 0.00200
Total time spent on delete operations: 0.00500
Computational bottleneck: deleteMin operations

For the standard priority queue on a lower temperature the computational bottleneck were the insert operations. However, for all higher temperatures, the computational bottleneck were the deleteMin operations.
For the index priority queue the computational bottleneck were also both the insert and deleteMin operations. In the first test the computational bottleneck were the insert operations. In the second test, with a higher temperature, the computational bottleneck were the deleteMin operations.


Question: Suppose that, in a three-level tree, we can afford to keep a links in internal memory, between b and 2b links in pages representing internal nodes, and between c and 2c items in pages representing external nodes. What is the maximum number of items that we can hold in such a tree, as a function of a, b, and c?

Answer:
6.14

The maximum number of items that we can hold in such a tree is achieved when there are "a" links in internal memory (the links between the first and second level of the tree), 2b links in pages representing internal nodes (the highest branching possible) and 2c items in pages representing external nodes (the highest number of items per external node).

Maximum number of items = a * 2b * 2c


Question: Estimate the average number of probes per search in a B-tree for S random searches, in a typical cache system, where the T most-recently-accessed pages are kept in memory (and therefore add 0 to the probe count). Assume that S is much larger than T.

Answer:
6.18

Let h be the number of pages examined by a search when nothing useful is cached; for a B-tree of order M with N keys, h is about log_M N to log_{M/2} N.

With random searches and S much larger than T, there is little locality among the leaf pages. The pages that reliably stay in the cache are the pages near the root, because every search touches them. If the cache can hold the top L levels of the tree, where

    1 + M + M^2 + ... + M^(L - 1) <= T,

then those L page probes usually cost 0, and the average number of disk probes per search is approximately

    max(0, h - L)

Lower-level pages may occasionally be among the T most recently accessed pages, but for random searches this contribution is small when S is much larger than T and the number of leaf/subtree pages is much larger than T.


Question: Consider the sibling split (or B*-tree) heuristic for B-trees: When it comes time to split a node because it contains M entries, we combine the node with its sibling. If the sibling has k entries with k < M - 1, we reallocate the items giving the sibling and the full node each about (M+k)/2 entries. Otherwise, we create a new node and give each of the three nodes about 2M/3 entries. Also, we allow the root to grow to hold about 4M/3 items, splitting it and creating a new root node with two entries when it reaches that bound. State bounds on the number of probes used for a search or an insertion in a B*-tree of order M with N items. Compare your bounds with the corresponding bounds for B-trees (see PROPOSITION B). Develop an insert implementation for B*-trees.

Answer:
6.20 - B* trees

Bounds on the number of probes used for a search or an insertion in a B*-tree of order M with N items:
Between log(M) N and log(2M/3) N probes.
This is because almost all internal nodes of the tree have between 2M / 3 and M - 1 links, since they are formed from a split of a full node with M keys and can only grow in size.
The only exception in which an internal node can have less than 2M / 3 links is when a node's child keys are reallocated with the creation of a new child node and its rightmost child does not get allocated enough entries. This happens when there are K keys to be reallocated to C child nodes and K / C < 2M / 3.

As seen in Proposition B, the bounds on the number of probes used for a search or an insertion in a B-tree of order M with N items is between log(M) N and log(M/2) N probes.
Since log(2M/3) N < log(M/2) N, B*-trees are more efficient for both search and insert operations. This efficiency comes with a tradeoff that the split() operation, which has a runtime complexity of O(M / 2) in B-trees, changes its runtime complexity to O(M^2).

B*-tree insertion outline:

```text
insert(key):
    if root is over the 4M/3 root limit:
        split root into two children and create a new root
    insert(root, key)

insert(page, key):
    if page is external:
        insert key into page in sorted order
        return

    child = page.childThatCanContain(key)
    insert(child, key)

    if child has M entries:
        sibling = an adjacent sibling of child under the same parent
        if sibling exists and sibling has fewer than M - 1 entries:
            redistribute child + separator key + sibling so each gets about (M + k) / 2 entries
            update the separator key in parent
        else:
            allocate a new page
            redistribute child + separator key + full sibling into three pages,
            each with about 2M / 3 entries
            replace the old separator in parent by two separator keys
```

The implementation differs from ordinary B-tree insertion at the overflow point: try redistribution with a sibling first; split two full siblings into three pages only when redistribution is impossible.


Question: Write a program to compute the average number of external pages for a B-tree of order M built from N random insertions into an initially empty tree. Run your program for reasonable values of M and N.

Answer:
6.21

Results:

 Order M |  Number of items |  AVG Number of External Pages
       4             100000                           42865
       4            1000000                          428592
       4           10000000                         4276006
      16             100000                            9424
      16            1000000                           94288
      16           10000000                          940648
      64             100000                            2277
      64            1000000                           22822
      64           10000000                          227472
     256             100000                             542
     256            1000000                            5650
     256           10000000                           57418

As expected, the higher the value of the order (M), the lower the number of external pages in the B-tree.
Also, when comparing experiments with the same order (M) but with different number of items (N), the higher the number of items, the higher the number of external pages in the B-tree.

Program structure:

```java
for (int m : new int[] {4, 16, 64, 256}) {
    for (int n : new int[] {100000, 1000000, 10000000}) {
        long total = 0;
        for (int t = 0; t < trials; t++) {
            BTreeSETWithExternalPageCounter<Integer> set =
                    new BTreeSETWithExternalPageCounter<>(0, m);
            for (int i = 0; i < n; i++) {
                set.add(StdRandom.uniformInt(Integer.MAX_VALUE));
            }
            total += set.externalPages();
        }
        StdOut.printf("%8d %12d %12.0f\n", m, n, total / (double) trials);
    }
}
```

`externalPages()` is maintained by incrementing the counter whenever an external page splits and a new external page is created.


Question: If your system supports virtual memory, design and conduct experiments to compare the performance of B-trees with that of binary search, for random searches in a huge symbol table.

Answer:
6.22

Results:

Number of searches |  B-tree time |  Binary search time
              1000          0.005                 0.001
            100000          0.082                 0.013
          10000000          6.352                 0.910

For random searches in a huge symbol table (with 10,000,000 entries) binary search has a better performance than B-trees.

Reference related to virtual memory in macs:
https://www.howtogeek.com/319151/why-you-shouldnt-turn-off-virtual-memory-on-your-mac/


Question: For your internal-memory implementation of Page in EXERCISE 6.15, run experiments to determine the value of M that leads to the fastest search times for a B-tree implementation supporting random search operations in a huge symbol table. Restrict your attention to values of M that are multiples of 100.

Answer:
6.23

Results:

 Order M |  Number of searches |  Total time
     100                  1000         0.005
     100                100000         0.198
     100              10000000        19.973
     200                  1000         0.002
     200                100000         0.195
     200              10000000        18.331
     400                  1000         0.003
     400                100000         0.188
     400              10000000        17.533
    1000                  1000         0.002
    1000                100000         0.176
    1000              10000000        16.818
    1500                  1000         0.002
    1500                100000         0.162
    1500              10000000        15.784
    2000                  1000         0.003
    2000                100000         0.205
    2000              10000000        17.821

The value of M (order) that leads to the fastest search times for a B-tree doing random search operations in a huge symbol table (with 10,000,000 entries) is 1500.


Question: Run experiments to compare search times for internal B-trees (using the value of M determined in the previous exercise), linear probing hashing, and red-black trees for random search operations in a huge symbol table.

Answer:
6.24

Results:

Number of searches |  B-tree time |  Linear probing hashing time |  Red-Black tree time
              1000          0.004                          0.001                  0.002
            100000          0.173                          0.019                  0.114
          10000000         16.659                          1.931                 11.297

On the experiments using B-trees with order 1500, linear probing hashing and red-black trees for random searches in a huge symbol table (with 10,000,000 entries), the best search performance by far was achieved by linear probing hashing, which took 1.931 seconds to search 10 million random keys.
It was followed by red-black trees, with 11.297 seconds to search 10 million random keys. B-trees had the worst search performance, taking 16.659 seconds to do the same searches.
Such results can be explained by the fact that linear probing hashing perform searches in O(1) while B-trees perform searches in O(log(M/2) N) - O(log(750) N) in this case - and red-black trees perform searches in O(lgN).


Question:
Give, in the style of the figure on page 882, the suffixes, sorted suffixes, index() and lcp() tables for the following strings:
a. abacadaba
b. mississippi
c. abcdefghij
d. aaaaaaaaaa

Answer:
6.25

a. abacadaba

suffixes      sorted suffix array
              i   index(i)   lcp(i)
0  abacadaba  0      8        0       a
1  bacadaba   1      6        1       aba
2  acadaba    2      0        3       abacadaba
3  cadaba     3      2        1       acadaba
4  adaba      4      4        1       adaba
5  daba       5      7        0       ba
6  aba        6      1        2       bacadaba
7  ba         7      3        0       cadaba
8  a          8      5        0       daba

b. mississippi

suffixes          sorted suffix array
                  i   index(i)    lcp(i)
 0  mississippi   0      10        0       i
 1  ississippi    1       7        1       ippi
 2  ssissippi     2       4        1       issippi
 3  sissippi      3       1        4       ississippi
 4  issippi       4       0        0       mississippi
 5  ssippi        5       9        0       pi
 6  sippi         6       8        1       ppi
 7  ippi          7       6        0       sippi
 8  ppi           8       3        2       sissippi
 9  pi            9       5        1       ssippi
10  i            10       2        3       ssissippi

c. abcdefghij

suffixes        sorted suffix array
                i   index(i)    lcp(i)
0  abcdefghij   0      0         0       abcdefghij
1  bcdefghij    1      1         0       bcdefghij
2  cdefghij     2      2         0       cdefghij
3  defghij      3      3         0       defghij
4  efghij       4      4         0       efghij
5  fghij        5      5         0       fghij
6  ghij         6      6         0       ghij
7  hij          7      7         0       hij
8  ij           8      8         0       ij
9  j            9      9         0       j

d. aaaaaaaaaa

suffixes        sorted suffix array
                i   index(i)    lcp(i)
0  aaaaaaaaaa   0      9         0       a
1  aaaaaaaaa    1      8         1       aa
2  aaaaaaaa     2      7         2       aaa
3  aaaaaaa      3      6         3       aaaa
4  aaaaaa       4      5         4       aaaaa
5  aaaaa        5      4         5       aaaaaa
6  aaaa         6      3         6       aaaaaaa
7  aaa          7      2         7       aaaaaaaa
8  aa           8      1         8       aaaaaaaaa
9  a            9      0         9       aaaaaaaaaa


Question:
Identify the problem with the following code fragment to compute all the suffixes for suffix sort:
suffix = "";
for (int i = s.length() - 1; i >= 0; i--)
{
  suffix = s.charAt(i) + suffix;
  suffixes[i] = suffix;
}

Answer:
The problem is that Java strings are immutable. Each statement

    suffix = s.charAt(i) + suffix;

creates a new String and copies all characters in the old suffix. The copied suffix lengths are 1, 2, 3, ..., N, so the total time and total character storage are quadratic, not linear.

A suffix-sort implementation should store integer starting positions, use a suffix object/view, or otherwise avoid materializing every suffix string by repeated concatenation.


Question: Under the assumptions described in SECTION 1.4. give the memory usage of a SuffixArray object with a string of length N.

Answer:
For the textbook `SuffixArray` representation with an inner `Suffix` object, each suffix object stores a reference to the shared text string and an `int` index.

Per `Suffix` object:

`16` object overhead + `8` text reference + `4` int index + `4` padding = `32` bytes.

`SuffixArray` object excluding the input `String` object itself:

`16` object overhead + `8` reference to `Suffix[]` + `24` array overhead + `8N` suffix references + `32N` suffix objects

`= 40N + 48` bytes.

If the `String` of length `N` is counted as part of the object graph under the Section 1.4 assumptions, add `56 + 2N` bytes for the string, giving

`42N + 104` bytes.

The previous `40N + 40` total was missing the outer object's reference to the suffix array.


Question: Write a SuffixArray client LCS that take two file-names as command-line arguments, reads the two text files, and finds the longest substring that appears in both in linear time. (In 1970, D. Knuth conjectured that this task was impossible.) Hint: Create a suffix array for s#t where s and t are the two text strings and # is a character that does not appear in either.

Answer:
```java
import edu.princeton.cs.algs4.In;
import edu.princeton.cs.algs4.StdOut;
import edu.princeton.cs.algs4.SuffixArray;

public class LCS {
    private static boolean inFirstText(int index, int firstLength) {
        return index < firstLength;
    }

    private static char delimiter(String a, String b) {
        for (char c = 1; c < Character.MAX_VALUE; c++) {
            if (a.indexOf(c) < 0 && b.indexOf(c) < 0) return c;
        }
        throw new IllegalArgumentException("no delimiter available");
    }

    public static void main(String[] args) {
        String s = new In(args[0]).readAll();
        String t = new In(args[1]).readAll();
        char sep = delimiter(s, t);
        String text = s + sep + t;
        int n1 = s.length();

        SuffixArray sa = new SuffixArray(text);
        String best = "";

        for (int i = 1; i < sa.length(); i++) {
            int a = sa.index(i);
            int b = sa.index(i - 1);
            if (a == n1 || b == n1) continue;
            if (inFirstText(a, n1) == inFirstText(b, n1)) continue;

            int length = sa.lcp(i);
            if (length > best.length()) {
                best = text.substring(a, a + length);
            }
        }
        StdOut.println(best);
    }
}
```

The scan after suffix-array construction is linear: the longest common substring must be the LCP of two adjacent suffixes that originate in different input files. To satisfy the exercise's linear-time requirement end to end, use a linear-time suffix-array construction for `s + sep + t`; the adjacent-LCP scan is already linear.


Question: If capacities are positive integers less than M, what is the maximum possible flow value for any st-network with V vertices and E edges? Give two answers, depending on whether or not parallel edges are allowed.

Answer:
Let C = M - 1 be the largest possible capacity.

If parallel edges are allowed, the maximum flow can be E * C: put all E edges directly from s to t, each with capacity C.

If parallel edges are not allowed, the source can have at most V - 1 outgoing edges, and using d independent capacity-C paths from s to t requires at least 2d - 1 edges if one path is the direct edge s->t and the other d - 1 paths go through distinct intermediate vertices. Therefore the maximum possible value is

    C * min(V - 1, floor((E + 1) / 2))

This bound is achievable by using the direct edge s->t plus as many two-edge paths s->v->t through distinct intermediate vertices as the vertex and edge budgets allow.


Question: Give an algorithm to solve the maxflow problem for the case that the network forms a tree if the sink is removed.

Answer:
Root the tree obtained by removing t at the source s. For each vertex v, compute bottom-up the maximum amount of flow that can be sent from v to t through the subtree rooted at v.

Let value(v) be:

    sum of capacities of edges v->t
  + sum over children w of min(capacity(v, w), value(w))

Compute value(v) by a postorder traversal of the tree. The answer is value(s).

This is linear time because each tree edge and each edge into the sink is inspected once. The formula is necessary because different branches of the tree can share upstream capacity; simply summing one source-to-sink path per source child is not sufficient for a general tree.


Question:
If true provide a short proof, if false give a counterexample:
a. In any max flow, there is no directed cycle on which every edge carries positive flow
b. There exists a max flow for which there is no directed cycle on which every edge carries positive flow
c. If all edge capacities are distinct, the max flow is unique
d. If all edge capacities are increased by an additive constant, the min cut remains unchanged
e. If all edge capacities are multiplied by a positive integer, the min cut remains unchanged

Answer:
6.38 - True of false

a. In any max flow, there is no directed cycle on which every edge carries positive flow.
False.
Counterexample:

    s
   | ^
2/2| |1/1
   v |
    1
    |
    |1/1
    v
    t

b. There exists a max flow for which there is no directed cycle on which every edge carries positive flow.
True.
Proof: Let f be a maximum flow and let C be a cycle on which every edge carries positive flow.
Let g = min(e E C) f(e). In other words, g is equal to the value of the minimum edge flow among the edges in cycle C.
Reducing the flow of each edge in C by g maintains the value of the max flow and sets the flow f(e) of at least one of the edges e E C to zero.

c. If all edge capacities are distinct, the max flow is unique.
False.
Counterexample:

      s
      | 1/1
      |
      v
      1
      |
     / \
2/0 /  |
   v   |
  2    | 3/1
   \   |
4/0 \  |
     v v
      t

      s
      | 1/1
      |
      v
      1
      |
     / \
2/1 /  |
   v   |
  2    | 3/0
   \   |
4/1 \  |
     v v
      t

d. If all edge capacities are increased by an additive constant, the min cut remains unchanged.
False.
Counterexample:

original network:

        s
        | 4/3
        v
        1
 1/1/   |1/1  \ 1/1    <- min cut
   v    v      v
   \    |     /
1/1 \   |1/1 / 1/1
     v  v   v
        t

After adding a constant value of 1 to all edge capacities:

        s
        | 5/3  <- min cut
        v
        1
 2/1/   |2/1  \ 2/1
   v    v      v
   \    |     /
2/1 \   |2/1 / 2/1
     v  v   v
        t

e. If all edge capacities are multiplied by a positive integer, the min cut remains unchanged.
True.
Proof: Let g be the positive integer for which every edge capacity is multiplied.
When all edge capacities are multiplied by g the value of every cut also gets multiplied by g.
Thus, the relative order of the cuts does not change and the min cut before the multiplication will remain the min cut after it.

References:
http://algo2.iti.kit.edu/sanders/courses/algdat03/sol3.pdf
https://stackoverflow.com/questions/40277603/is-minimum-cut-same-for-the-graph-after-increasing-edge-capacity-by-1-for-all-ed


Question: Complete the proof of PROPOSITION G: Show that each time an edge is a critical edge, the length of the augmenting path through it must increase by 2.

Answer:
Let `d_f(x)` be the BFS distance from `s` to vertex `x` in the residual network just before an augmentation, and suppose residual edge `u->v` is critical on a shortest augmenting path. Then

`d_f(v) = d_f(u) + 1`.

For `u->v` to become critical again later, the residual capacity of `u->v` must first be restored. The only way to restore it is to use the reverse edge `v->u` on some later augmenting path. At that later time, if the distance labels are `d'`, that path has

`d'(u) = d'(v) + 1`.

Edmonds-Karp BFS distances from `s` never decrease over the sequence of augmentations, so `d'(v) >= d_f(v)`. Therefore

`d'(u) = d'(v) + 1 >= d_f(v) + 1 = d_f(u) + 2`.

Thus, before edge `u->v` can be critical again, the shortest augmenting path distance to `u` has increased by at least 2, so the length of any augmenting path through that edge has increased by at least 2.


Question: Prove that the shortest-paths problem reduces to linear programming.

Answer:
6.50

Reduction from shortest paths problem to linear programming:

Method 1:
We consider a system of inequalities and equations that involve the following variables:
l(u,v) -> variable corresponding to the length of the directed edge u -> v.
x(u,v) -> indicator variable for whether edge u -> v is in the shortest path. It has value 1 when the edge is, and 0 when the edge is not.

Given a directed graph G with a source vertex s and a target vertex t.

Linear programming formulation:

   Minimize  E  l(u,v) * x(u,v)
            u->v

Subject to the constraints

   E x(s,v) - E x(v,s) = 1
   v          v

   E x(t,v) - E x(v,t) = -1
   v          v

for every vertex u other than s and t:
   E x(u,v) - E x(v,u) = 0
   v          v

   x(u,v) >= 0 for every edge u -> v

The constraints state that the path should start at s, end at t, and either pass through or avoid every other vertex v.
This selects the set of edges with minimal length, subject to the constraint that this set forms a path from s to t  - represented by the constraints that ensure that for all vertices except s and t the number of incoming and outcoming edges that are part of the path must be the same.

The solution is easily converted to a solution of the shortest paths problem: the edges u -> v for which x(u,v) is equal to 1 are part of the shortest path and the sum of all their lengths is the length of the shortest path from s to t.

Method 2:
We consider a system of inequalities and equations that involve the following variables:
d(v) -> variable corresponding to the shortest distance from s to vertex v.
l(u,v) -> variable corresponding to the length of the directed edge u -> v.

Given a directed graph G with a set of vertices V, a set of edges E and with a source vertex s and a target vertex t.

Linear programming formulation:

   Maximize d(t)

Subject to the constraints

   d(s) = 0
   d(v) - d(u) <= l(u,v) for every edge u -> v

The constraints state that in a shortest path from s to t, d(v) is at most the shortest path distance from s to v. And the value of d(t) will be the shortest path distance from s to t.
To find the edges in the shortest path from s to t, the following method can be used:
Pre-process the edges to map each vertex with its incoming edges.
Starting with vertex t, check which incoming neighbor vertex v has d(v) equal to d(t) - l(v,t).
Add the edge v -> t to the shortest path. Replace t with v and do the same check for its incoming neighbors, iterating this process until vertex s is reached.

To compute the shortest paths from s to every other vertex, the following linear programming formulation can be used:

   Maximize E d(v)
            v

Subject to the constraints

   d(s) = 0
   d(v) - d(u) <= l(u,v) for every edge u -> v

The resulting values of d(v) will be the shortest path distances from s to every vertex v.
To find the edges in the shortest path from s to any vertex v, the same process mentioned above can be used, replacing vertex t with vertex v.

To compute the shortest paths from all vertices to every other vertex (all-pairs shortest paths), the following linear programming formulation can be used:

d(u,v) -> variable corresponding to the shortest distance from vertex u to vertex v.

   Maximize E d(u,v)
            v

Subject to the constraints

   d(v,v) = 0 for every vertex v
   d(u,w) - d(u,v) <= l(v,w) for every vertex u in V and every edge v -> w in E

The resulting values of d(u,v) will be the shortest path distances from every vertex u to every vertex v.
To find the edges in the shortest paths from every vertex u to every vertex v, the same process mentioned above can be used, replacing vertex t with vertex v and vertex s with vertex u.

References:
https://en.wikipedia.org/wiki/Shortest_path_problem#Linear_programming_formulation
https://courses.engr.illinois.edu/cs498dl1/sp2015/notes/26-lp.pdf
http://www.cs.yale.edu/homes/aspnes/pinewiki/attachments/LinearProgramming/lp.pdf


Question: Could there be an algorithm that solves an NP-complete problem in an average time of NlogN, if P != NP? Explain your answer.

Answer:
Yes, this would not contradict P != NP as stated, because the question asks about average time rather than worst-case time. P is defined using worst-case polynomial time. An algorithm could have average running time O(N log N) under some input distribution while still taking super-polynomial time on worst-case instances.

If the guarantee were worst-case O(N log N) for an NP-complete problem, then P would equal NP. But an average-time statement alone does not imply that.


Question: Suppose that someone discovers an algorithm that is guaranteed to solve the boolean satisfiability problem in time proportional to 1.1^N. Does this imply that we can solve other NP-complete problems in time proportional to 1.1^N?

Answer:
Not necessarily with the same 1.1^N bound. Since SAT is NP-complete, every NP problem has a polynomial-time reduction to SAT. If an instance of another NP-complete problem of size N reduces to a SAT instance of size p(N), then the resulting running time would be roughly

    poly(N) + 1.1^p(N)

That is exponential if p is polynomial, but it is not necessarily proportional to 1.1^N. The base/exponent bound is not preserved by arbitrary polynomial reductions.


Question: What would be the significance of a program that could solve the integer linear programming problem in time proportional to 1.1^N?

Answer:
Integer linear programming is NP-complete, so a 1.1^N algorithm for ILP would be a major exponential-time improvement for that NP-complete problem and, via reductions, would give exponential-time algorithms for other NP problems.

It would not imply P = NP, because 1.1^N is still exponential. It also would not automatically imply that every NP problem can be solved in time 1.1^N in its own input size, because polynomial reductions can increase the instance size to p(N).


Question: Give a poly-time reduction from vertex cover to 0-1 integer linear inequality satisfiability.

Answer:
Given a vertex-cover instance (G = (V, E), k), create one 0-1 variable x_v for each vertex v. The intended meaning is x_v = 1 if v is included in the cover.

Add the constraints

    x_u + x_v >= 1        for every edge (u, v) in E
    sum_{v in V} x_v <= k
    x_v in {0, 1}         for every vertex v

The edge constraints require every edge to have at least one selected endpoint, and the sum constraint requires the selected set to have size at most k. Therefore the system is satisfiable if and only if G has a vertex cover of size at most k. The construction has one variable per vertex and one inequality per edge plus the size bound, so it is polynomial time.


Question: Prove that the problem of finding a Hamiltonian path in a directed graph is NP-complete, using the NP-completeness of the Hamiltonian-path problem for undirected graphs.

Answer:
6.55

Prove that the problem of finding a Hamiltonian path in a directed graph is NP-complete, using the NP-completeness of the Hamiltonian path problem for undirected graphs.

Let's call the Hamiltonian path problem in undirected graphs HPg and the Hamiltonian path problem in directed graphs HPdg.

Reduction from HPg to HPdg:
Replace every edge v - w in the undirected graph in HPg to two directed edges in the directed graph in HPdg:
A v -> w directed edge and a w -> v directed edge.
Solve HPdg.
The directed edges selected in the solution to HPdg can be mapped to the edges in the solution to HPg:
If either v -> w or w -> v are in HPdg solution, v - w is in HPg solution.

A problem is NP-complete if it is in NP and all problems in NP poly-time reduce to it.
HPdg is in NP because if given a Hamiltonian path as a solution, it is possible to check if all vertices are visited in polynomial time.
Since HPg is NP-complete, all problems in NP poly-time reduce to it.
As seen above, there is a poly-time reduction from HPg to HPdg. By transitive relation this shows that all problems in NP poly-time reduce to HPdg, and it is, therefore, NP-complete.

             All problems in NP

                     |
                     V

    Hamiltonian path problem for undirected graphs

                     |
                     V

    Hamiltonian path problem for directed graphs


Question: Suppose that two problems are known to be NP-complete. Does this imply that there is a poly-time reduction from one to the other?

Answer:
6.56

Yes, if two problems are known to be NP-complete this implies that there is a poly-time reduction from one to the other.
This is because all problems in NP poly-time reduce to any NP-complete problem.
All NP-complete problems are in NP, therefore, both problems poly-time reduce from one to the other.


Question: Suppose that X is NP-complete, X poly-time reduces to Y, and Y poly-time reduces to X. Is Y necessarily NP-complete?

Answer:
Yes, under the standard many-one polynomial-time reduction used for NP-completeness.

Since X is NP-complete, every problem in NP reduces to X. Because X <=p Y, every problem in NP also reduces to Y, so Y is NP-hard.

Also, Y <=p X and X is in NP. If y is an instance of Y, compute f(y), the corresponding instance of X. A certificate for f(y) can be verified in polynomial time, so Y is in NP.

Thus Y is both NP-hard and in NP, so Y is NP-complete.


Question: Suppose that we have an algorithm to solve the decision version of boolean satisfiability, which indicates that there exists an assignment of truth values to the variables that satisfies the boolean expression. Show how to find the assignment.

Answer:
Use the decision algorithm as a self-reduction oracle.

First ask whether the original formula is satisfiable. If not, there is no assignment. Otherwise process the variables one at a time while permanently keeping the choices already made.

For variable `x_i`, substitute all previous choices into the formula. Try setting `x_i = true` and call the SAT decision algorithm on the restricted formula. If it remains satisfiable, keep `x_i = true`; otherwise set `x_i = false`, which must preserve satisfiability because the current restricted formula was satisfiable before trying `true`.

After `n` oracle calls, all variables are fixed and the accumulated truth values form a satisfying assignment.


Question: Suppose that we have an algorithm to solve the decision version of the vertex cover problem, which indicates that there exists a vertex cover of a given size. Show how to solve the optimization version of finding the vertex cover of minimum cardinality.

Answer:
First find the optimum cardinality `k` by binary searching (or linearly searching) with the decision oracle: ask whether the graph has a vertex cover of size at most `k`.

Then recover an actual cover by self-reduction. Maintain the current graph `G`, target size `k`, and an initially empty cover `C`.

For each vertex `v`, ask whether `G - v` has a vertex cover of size `k - 1`. If yes, include `v` in `C`, delete `v` and all incident edges from `G`, and decrement `k`. If no, leave `v` out and continue. Stop when no edges remain or `k = 0`.

The selected vertices form a minimum vertex cover because each inclusion is certified by the decision oracle to still allow a cover of the remaining required size.


Question: Explain why the optimization version of the vertex cover problem is not necessarily a search problem.

Answer:
6.60

The optimization version of the vertex cover problem is not necessarily a search problem because its output (a vertex cover of minimum cardinality) cannot be certified to be correct in polynomial time. It is possible to certify in polynomial time that the output is a vertex cover, but not that it is a minimum cardinality vertex cover.


Question:
Suppose that X and Y are two search problems and that X poly-time reduces to Y. Which of the following can we infer?
a. If Y is NP-complete then so is X.
b. If X is NP-complete then so is Y.
c. If X is in P, then Y is in P.
d. If Y is in P, then X is in P.

Answer:
6.61

If X and Y are two search problems and X poly-time reduces to Y, we can infer that:

b. If X is NP-complete then so is Y.
This is because if both X and Y are search problems they are both in NP.
If X is NP-complete, then all problems in NP poly-time reduce to it. And if X poly-time reduces to Y, then all problems in NP also poly-time reduce to Y (through X).
Since Y is both in NP and all problems in NP poly-time reduce to it, Y is NP-complete.

d. If Y is in P, then X is in P.
If Y is in P it can be solved in polynomial time. If we can reduce X to Y then we can solve Y in polynomial time and consequently solve X in polynomial time, meaning that X is also in P.

The following alternatives are wrong:

a. If Y is NP-complete then so is X.
Not necessarily, there may not exist reductions from all problems in NP to X.

c. If X is in P, then Y is in P.
Not necessarily, Y may be NP-complete and currently it is unknown whether P = NP.

This assumes that "search problems" means NP search problems, so membership in NP is part of the premise.


Question:
Suppose that P != NP. Which of the following can we infer?
e. If X is NP-complete, then X cannot be solved in polynomial time.
f. If X is in NP, then X cannot be solved in polynomial time.
g. If X is in NP but not NP-complete, then X can be solved in polynomial time.
h. If X is in P, then X is not NP-complete.

Answer:
If P != NP, we can infer:

e. If X is NP-complete, then X cannot be solved in polynomial time.
If some NP-complete X were in P, then every problem in NP would reduce to X and also be in P, implying P = NP.

h. If X is in P, then X is not NP-complete.
If a problem in P were NP-complete, again all of NP would be in P.

The following do not follow:

f. If X is in NP, then X cannot be solved in polynomial time.
False: X might be in P, and P is a subset of NP.

g. If X is in NP but not NP-complete, then X can be solved in polynomial time.
Not necessarily. If P != NP, there may be problems in NP that are neither in P nor NP-complete.