Biostatistics 615/815 Lecture 23: The Baum-Welch Algorithm Advanced Hidden Markov Models

Transcription

1 Biostatistics 615/815 Lecture 23: The Algorithm Advanced Hidden Markov Models Hyun Min Kang April 12th, 2011 Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35

2 Annoucement Homework Final homework is announced Implementing two among E-M algorithm, Simulated Annealing, and Gibbs Sampler 815 Project Presentation : Tuesday April 19th Final report : Friday Apil 29th Final Exam Thursday April 21st, 10:30AM-12:30PM Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35

3 Key components in 815 Presentations Duration : 15 minutes Describe / illustrate what the problem is Key idea Results Challenges and lessons from implementations Comparisons with other alternatives (if possible) Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35

4 Recap - Gibbs Sampler Algorithm 1 Consider a particular choice of parameter values λ (t) 2 Define the next set of parameter values by Selecting a component to update, say i Sample value for λ (t+1) i, from p(λ i x, λ 1,, λ i 1, λ i+1,, λ k ) 3 Increment t and repeat the previous steps Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35

5 Recap - Gibbs Sampling for Gaussian Mixture Observed data : x = (x 1,, x n ) Parameters : z = (z 1,, z n ) where z i {1,, k} Sample each z i conditioned by all the other z Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35

6 Recap - Simulated Annealing and Gibbs Sampler Both Methods are Markov Chains The distribution of λ (t) only depends on λ (t 1) Update rule defines the transition probabilities between two states, requiring aperiodicity and irreducbility Both Methods are Metropolis-Hastings Algorithms Acceptance of proposed update is probabilistically determined by relative probabilities between the original and proposed states Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35

7 Today Algorithm An E-M algorithm for HMM parameter estimation Three main HMM algorithms The forward-backward algorithm The Viterbi algorithm The Algorithm Advanced HMM Expedited inference with uniform HMM Continuous-time Markov Process Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35

8 Revisiting Hidden Markov Model ()*# +,-,*+#!" " # $ # % # & #!" -! $"# - "#! %$# - $#! &/&0"1# %#! &# 3!" /' " 1 # 3!$ /' $ 1 # 3!% /' % 1 # 3!& /' & 1 # -,-#!" ' "# ' $# ' %# ' &# 2 # Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35

9 Statistical analysis with HMM HMM for a deterministic problem Given Given parameters λ = {π, A, B} and data o = (o 1,, o T ) Forward-backward algorithm Compute Pr(q t o, λ) Viterbi algorithm Compute arg max q Pr(q o, λ) HMM for a stochastic process / algorithm Generate random samples of o given λ Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35

10 Deterministic Inference using HMM If we know the exact set of parameters, the inference is deterministic given data No stochastic process involved in the inference procedure Inference is deterministic just as estimation of sample mean is deterministic The computational complexity of the inference procedure is exponential using naive algorithms Using dynamic programming, the complexity can be reduced to O(n 2 T) Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35

11 Using Stochastic Process for HMM Inference Using random process for the inference Randomly sampling o from Pr(o λ) Estimating arg max λ Pr(o λ) No deterministic algorithm available Simplex, E-M algorithm, or Simulated Annealing is possible apply Estimating the distribution Pr(λ o) Gibbs Sampling Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35

12 Recap : The E-M Algorithm Expectation step (E-step) Given the current estimates of parameters θ (t), calculate the conditional distribution of latent variable z Then the expected log-likelihood of data given the conditional distribution of z can be obtained Q(θ θ (t) ) = E z x,θ (t) [log p(x, z θ)] Maximization step (M-step) Find the parameter that maximize the expected log-likelihood θ (t+1) = arg max Q(θ θ t ) θ Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35

13 for estimating arg max λ Pr(o λ) Assumptions Transition matrix is identical between states a ij = Pr(q t+1 = i q t = j) = Pr(q t = i q t 1 = j) Emission matrix is identical between states b i (j) = Pr(o t = j q t = i) = Pr(o t=1 = j q t 1 = i) This is NOT the only possible assumption For example, a ij can be parameterized as a function of t Multiple sets of o independently drawn from the same distribution can be provided Other assumptions will result in different formulation of E-M algorithm Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35

14 E-step of the Algorithm 1 Run the forward-backward algorithm given λ (τ) α t (i) = Pr(o 1,, o t, q t = i λ (τ) ) β t (i) = Pr(o t+1,, o T q t = i, λ (τ) ) γ t (i) = Pr(q t = i o, λ (τ) ) = α t(i)β t (i) k α t(k)β t (k) 2 Compute ξ t (i, j) using α t (i) and β t (i) ξ t (i, j) = Pr(q t = i, q t+1 = j o, λ (τ) ) = α t(i)a ji b j (o t+1 )β t+1 (j) Pr(o λ (τ) ) α t (i)a ji b j (o t+1 )β t+1 (j) = (k,l) α t(k)a lk b l (o t+1 )β t+1 (l) Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35

15 M-step of the Algorithm Let λ (τ+1) = (π (τ+1), A (τ+1), B (τ+1) ) π (τ+1) (i) = a (τ+1) ij = b i (k) (τ+1) = T t=1 Pr(q t = i o, λ (τ) ) T t=1 = γ t(i) T T T 1 t=1 Pr(q t = j, q t+1 = i o, λ (τ) ) T 1 t=1 Pr(q = t = j o, λ (τ) ) T t=1 Pr(q t = i, o t = k o, λ (τ) ) T t=1 Pr(q t = i o, λ (τ) ) = T 1 t=1 ξ t(j, i) T 1 t=1 γ t(j) T t=1 γ t(i)i(o t = k) T t=1 γ t(i) A detailed derivation can be found at Welch, Hidden Markov Models and The Baum Welch Algorithm, IEEE Information Theory Society News Letter, Dec 2003 Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35

16 Implementing HMM Algorithms class HMM615 { public: int T; // number of instances int N; // number of states int O; // number of possible values std::vector<double> pis; // pis[i] : Pr(q_0=i) std::vector<double> trans; // trans[i*n+j] : Pr(q_t=j q_{t-1}=i) double symmtrans; std::vector<double> emis; // emis[i*n+j] : Pr(o_t=j q_t=i) std::vector<int> obs; // obs[t] : observed data from 0O std::vector<double> alphas; // alphas[t*n+i] : Pr(o_{1t},q_t=i lambda) std::vector<double> betas; // betas[t*n+i] : Pr(o_{t+1T} q_t=i,lambda) std::vector<double> gammas; // gammas[t*n+i] : Pr(q_t=i lambda,o_{1t}) }; double computeforwardbackward(); double computebaumwelch(double tol); Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35

17 Implementing Forward algorithm : α t (i) void HMM615::computeForwardBackward() { double sum = 0; // initialize alpha values for(int i=0; i < N; ++i) sum += (alphas[0*n + i] = pis[i] * emis[i*o+obs[0]]); for(int i=0; i < N; ++i) alphas[0*n + i] /= sum; // normalize sum(alphas) // iterate over alphas for each t for(int t=1; t < T; ++t) { for(int i=0; i < N; ++i) { alphas[t*n + i] = sum = 0; for(int j=0; j < N; ++j) alphas[t*n + i] += (alphas[(t-1)*n +j] * trans[j*n + i]); sum += (alphas[t*n + i] *= emis[i*o+obs[t]]); } for(int i=0; i < N; ++i) alphas[t*n + i] /= sum; // normalize sum(alphas) } // (continued to next slides) Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35

18 Implementing Backward algorithm : β t (i) // initialize the last element of betas sum = 0; for(int i=0; i < N; ++i) sum += emis[i*o+obs[t-1]]; for(int i=0; i < N; ++i) betas[(t-1)*n + i] = 1/sum; // main body of backward algorithm for(int t=t-2; t >=0; --t) { for(int i=0; i < N; ++i) { betas[t*n + i] = sum = 0; for(int j=0; j < N; ++j) betas[t*n + i] += (betas[(t+1)*n + j] * trans[i*n + j] * emis[j*o+obs[t+1]]); sum += (betas[t*n+i] * emis[i*o+obs[t]]); } sum = 0; // normalize sum (betas * emis ) for(int i=0; i < N; ++i) betas[t*n + i] /= sum; } // (continued to next slide) Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35

19 Combining forward-backward algorithm : γ t (i) // compute gammas = Pr(q_t o,lambda) for(int t=0; t < T; ++t) { sum = 0; for(int i=0; i < N; ++i) sum += (gammas[t*n + i] = alphas[t*n + i] * betas[t*n + i]); for(int i=0; i < N; ++i) } } gammas[t*n + i] /= sum; // normalize sum(gammas) Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35

20 Notes : Forward-backward algorithm Forward-backward algorithm for arbitrary numbers of states and observations Normalization of t α t(i) and i β t(i)b i (o t ) at each step Only relative values of α t (i) and β t (i) are important Avoids potential overflow / underflow due to numerical precision Why normalize i β t(i)b i (o t ) instead of i β t(i)? Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35

21 A small utility function : update // assign newval to dst, after computing the relative differences between them // note that dst is call-by-reference, and newval is call-by-value double HMM615::update(double& dst, double newval) { // calculate the relative differences double reldiff = fabs((dst-newval)/(newval+zeps)); // update the destination value dst = newval; // update the destination value return reldiff; } Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35

22 The Algorithm : Initialization void HMM615::computeBaumWelch(double tol) { double tmp, sum, reldiff = 1e9; std::vector<double> sumgammas(n,0), sumobsgammas(n*o,0); std::vector<double> xis(n*n,0), sumxis(n*n,0); // iterate until the difference is small enough for(int iter = 0; (iter < MAX_ITERATION) && ( reldiff > tol ); ++iter) { reldiff = 0; computeforwardbackward(); // run forward-backward algorithm for(int i=0; i < N; ++i) sumgammas[i] = 0; for(int i=0; i < N*O; ++i) sumobsgammas[i] = 0; for(int i=0; i < N*N; ++i) xis[i] = sumxis[i] = 0; Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35

23 The Algorithm : E-step // calculate \sum_t gamma_t(i) for(int t=0; t < T; ++t) { for(int i=0; i < N; ++i) { sumgammas[i] += gammas[t*n + i]; sumobsgammas[i*o + obs[t]] += gammas[t*n + i]; } } // calcupate \sum_t xi_t(i,j) for(int t=0; t < T-1; ++t) { sum = 0; for(int i=0; i < N; ++i) { for(int j=0; j < N; ++j) sum += ( xis[i*n+j] = ( alphas[t*n + i] * trans[i*n + j] * betas[(t+1)*n + j] *emis[j*o + obs[t+1] ] ) ); } for(int i=0; i < N*N; ++i) sumxis[i] += (xis[i] /= sum); } Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35

24 The Algorithm : M-step // update transition matrix for(int i=0; i < N; ++i) { for(int j=0; j < N; ++j) { tmp = sumxis[i*n+j] / (sumgammas[i]-gammas[(t-1)*n+i]+zeps); reldiff += update( trans[i*n+j], tmp ); } } } // update pis and emission matrix for(int i=0; i < N; ++i) { reldiff += update( pis[i], sumgammas[i]/t ); for(int j=0; j < O; ++j) reldiff += update (emis[i*o+j], sumobsgammas[i*o+j]/(sumgammas[i]+zeps)); } } // repeat until relative difference is small enough Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35

25 A working example : Biased coin example Model Observations : O = {1(Head), 2(Tail)} Hidden states : S = {1(Fair), 2(Biased)} Initial states : π = {08, 02} = A π 0 ( ) Transition probability : A(i, j) = a ij = ( ) Emission probability : B(i, j) = b j (i) = Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35

26 Biased coin example : main() function int main(int argc, char** argv) { // input/output routines omitted // assign an initial starting point double theta0 = 01; std::vector<double> trans(4, theta0); std::vector<double> emis(4, 05); std::vector<double> pis(2, 05); // initial pi = (05, 05) trans[0] = trans[3] = 1-theta0; // initial A = (09, 01; 01, 09) emis[2] = 07; emis[3] = 03; // initial B = (05, 07; 05; 03) HMM615 hmm(pis, trans, emis, observations); // constructor omitted in the slides hmmcomputebaumwelch(1e-3); // run algorithm } return 0; Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35

27 Biased coin example : Results user@host: / > baumwelchcoin biasedcoininput10000txt Iteration 216, normdiff = , pis = (080249, ), trans = ( , , , ), emis = ( , , , ) user@host: / > baumwelchcoin biasedcoininput1000txt Iteration 621, normdiff = , pis = ( , ), trans = ( , , , ), emis = ( , , , ) Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35

28 : Algorithm E-M algorithm for estimating HMM parameters Assumes identical transition and emission probabilities across t The framework can be accomodated for differently contrained HMM Requires many observations to reach a reliable estimates Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35

29 Rapid Inference with Definition π i = 1/n { θ i j a ij = 1 (n 1)θ i = j b i (k) has no restriction Independent transition between n states Useful model in genetics and speech recognition Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35

30 Rapid Inference with Definition π i = 1/n { θ i j a ij = 1 (n 1)θ i = j b i (k) has no restriction Independent transition between n states Useful model in genetics and speech recognition The Problem The time complexity of HMM inference is O(n 2 T) For large n, this still can be a substantial computational burden Can we reduce the time complexity by leveraging the simplicity? Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35

31 Forward Algorithm with Original Forward Algorithm α t (i) = Pr(o 1,, o t, q t = i λ) = Rapid Forward Algorithm for [ n ] α t (i) = j=1 α t 1(j)a ij b i (o t ) = [ n j=1 α t 1(j)a ij ] b i (o t ) [ (1 (n 1)θ)α t 1 (i) + j i α t 1(j)θ = [(1 nθ)α t 1 (i) + θ] b i (o t ) ] b i (o t ) Assuming normalizaed i α t(i) = 1 for every t The total time complexity is O(nT) Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35

32 Backward Algorithm with Original Forward Algorithm β t (i) = Pr(o t+1,, o T q t = i, λ) = Rapid Forward Algorithm for β t (i) = n β t+1 (j)a ji b j (o t+1 ) j=1 n β t+1 (j)a ji b j (o t+1 ) j=1 = (1 (n 1)θ)β t+1 (i)b i (o t+1 ) + θ j i β t+1(j)b j (o t+1 ) = (1 nθ)β t+1 (i)b t (o t+1 ) + θ Assuming i β t(i)b i (o t ) = 1 for every t (Now we know why!) Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35

33 : Rapid computation of forward-backward algorithm leveraging symmetric structure Rapid algorithm is also possible in a similar manner It is important to understand the computational details of exisitng methods to further tweak the method when necessary Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35

34 Continuous-time Markov Process () Example : Single dimensional Brownian motion A particle is moving along a line at a constant velocity v At a rate λ times per second, the particle changes the direction The position of particle is observed at arbitrary time points t 1, t 2,, t n How can we model the trajectory of the particles given observations? Other Applications Queueing theory Modeling recombination in diploid organisms Many other infinitesimal Models Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35

35 Key Idea of Transition Rate instead of Transition Probability In discret MP, a ij defines the transition probability between states In, transition rate ρ defines the transition probability within some time intervals Pr(q s = i, s (t, t + r) q t = i) = exp( ρ ii r) Difference from discret HMM The transition probability between time points are no longer identical However, the transition probability can be parameterized using ρ and the interval size Developing E-M algorithm for estimating ρ is more sophisticated than the algorithm Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35

36 Today The Algorithm Rapid inference with Continuous-time Markov Process () Next Lecture Overview for the Final Exam Hyun Min Kang Biostatistics 615/815 - Lecture 22 April 12th, / 35