<:> Calculating n-gram probabilities
One way to obtain the n-gram probabilities is to identify each
individual n-gram and then compute its relative frequency in the
sequence.
But this is exponentially costly.
Instead, we can use the chain rule to build up the probabilities
incrementally.
- p(w1|w2) = p(w1) p(w2|w1)
- p(w1|w2,w3) = p(w1) P(w2|w1) p(w3|w2,w1)
- ...
- p(w1|w2,w3 ... wN) = p(w1) p(w2|w1) p(w3|w2,w1) ... p(wN|wN-1 wN-2 ... W1)