Biostatistics 615/815 Lecture 10: Hidden Markov Models

Transcription

1 Biostatistics 615/815 Lecture 10: Hidden Markov Models Hyun Min Kang October 4th, 2012 Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33

2 Manhattan Tourist Problem Let C(r, c) be the optimal cost from (0, 0) to (r, c) Let h(r, c) be the weight from (r, c) to (r, c + 1) Let v(r, c) be the weight from (r, c) to (r + 1, c) We can recursively define the optimal cost as { C(r 1, c) + v(r 1, c) min r > 0, c > 0 C(r, c 1) + h(r, c 1) C(r, c) = C(r, c 1) + h(r, c 1) r = 0, c > 0 C(r 1, c) + v(r 1, c) r > 0, c = 0 0 r = 0, c = 0 Once C(r, c) is evaluated, it must be stored to avoid redundant computation Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33

3 Edit Distance Problem Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33

4 Dynamic Programming for Edit Distance Problem Input strings are x[1,, m] and y[1,, n] Let x i = x[1,, i] and y j = y[1,, j] be substrings of x and y Edit distance d(x, y) can be recursively defined as follows i j = 0 j i = 0 d(x i, y j ) = d(x i 1, y j ) + 1 min d(x i, y j 1 ) + 1 otherwise d(x i 1, y i 1 ) + I(x[i] y[j]) Similar to the Manhattan tourist problem, but with 3-way choice Time complexity is Θ(mn) Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33

5 Hidden Markov Models (s) A Markov model where actual state is unobserved Transition between states are probabilistically modeled just like the Markov process Typically there are observable outputs associated with hidden states The probability distribution of observable outputs given an hidden states can be obtained Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33

6 An example of 346$ 345$!"#!$ %&'$ 347$ 348$ 3466$ ()**+$ 3493$ 3493$,-)/+$ 3483$ 3435$ 012*+$ 34:3$ Direct Observation : (SUNNY, CLOUDY, RAINY) Hidden States : (HIGH, LOW) Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33

7 Mathematical representation of the example States S = {S 1, S 2 = (HIGH, LOW) Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33

8 Mathematical representation of the example States S = {S 1, S 2 = (HIGH, LOW) Outcomes O = {O 1, O 2, O 3 = (SUNNY, CLOUDY, RAINY) Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33

9 Mathematical representation of the example States S = {S 1, S 2 = (HIGH, LOW) Outcomes O = {O 1, O 2, O 3 = (SUNNY, CLOUDY, RAINY) Initial States π i = Pr(q 1 = S i ), π = {07, 03 Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33

10 Mathematical representation of the example States S = {S 1, S 2 = (HIGH, LOW) Outcomes O = {O 1, O 2, O 3 = (SUNNY, CLOUDY, RAINY) Initial States π i = Pr(q 1 = S i ), π = {07, 03 Transition A ij = Pr(q t+1 = S j q t = S i ) A = ( Emission B ij = b qt (o t ) = b Si (O j ) = Pr(o t = O j q t = S i ) B = ) ( ) Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33

11 Unconditional marginal probabilities What is the chance of rain in the day 4? ( ) ( ) Pr(q4 = S f(q 4 ) = 1 ) = (A T ) π = Pr(q 4 = S 2 ) 0331 Pr(o 4 = O 1 ) 0621 g(o 4 ) = Pr(o 4 = O 2 ) = B T f(q 4 ) = 0266 Pr(o 4 = O 3 ) 0233 The chance of rain in day 4 is 233% Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33

12 Marginal likelihood of data in Let λ = (A, B, π) For a sequence of observation o = {o 1,, o t, Pr(o λ) = q Pr(o q, λ) Pr(q λ) t t Pr(o q, λ) = Pr(o i q i, λ) = b qi (o i ) Pr(q λ) = π q1 t Pr(o λ) = q i=1 i=1 i=2 a qi 1 q i π q1 b q1 (o 1 ) t a qi 1 q i b qi (o i ) i=2 Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33

13 Naive computation of the likelihood Pr(o λ) = q t π q1 b q1 (o 1 ) a qi 1 q i b qi (o i ) i=2 Number of possible q = 2 t are exponentially growing with the number of observations Computational would be infeasible for large number of observations Algorithmic solution required for efficient computation Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33

14 More Markov Chain Question If the observation was (SUNNY,SUNNY,CLOUDY,RAINY,RAINY) from day 1 through day 5, what is the distribution of hidden states for each day? Need to know Pr(q t o, λ) Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33

15 Forward and backward probabilities q t = (q 1,, q t 1 ), q + t = (q t+1,, q T ) o t = (o 1,, o t 1 ), o + t = (o t+1,, o T ) Pr(q t = i o, λ) = Pr(q t = i, o λ) Pr(q = t = i, o λ) Pr(o λ) n j=1 Pr(q t = j, o λ) Pr(q t, o λ) = Pr(q t, o t, o t, o + t λ) = Pr(o + t q t, λ) Pr(o t q t, λ) Pr(o t q t, λ) Pr(q t λ) = Pr(o + t q t, λ) Pr(o t, o t, q t λ) = β t (q t )α t (q t ) If α t (q t ) and β t (q t ) is known, Pr(q t o, λ) can be computed in a linear time Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33

16 DP algorithm for calculating forward probability Key idea is to use (q t, o t ) o t q t 1 Each of q t 1, q t, and q t+1 is a Markov blanket α t (i) = Pr(o 1,, o t, q t = i λ) n = Pr(o t, o t, q t 1 = j, q t = i λ) = = j=1 n Pr(o t, q t 1 = j λ) Pr(q t = i q t 1 = j, λ) Pr(o t q t = i, λ) j=1 n α t 1 (j)a ji b i (o t ) j=1 α 1 (i) = π i b i (o 1 ) Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33

17 Conditional dependency in forward-backward algorithms Forward : (q t, o t ) o t q t 1 Backward : o t+1 o + t+1 q t+1!"!" "#$ % " % "&$ %!"! "#$%! "%! "&$%!"!" ' "#$% ' "% ' "&$%!" Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33

18 DP algorithm for calculating backward probability Key idea is to use o t+1 o + t+1 q t+1 β t (i) = Pr(o t+1,, o T q t = i, λ) n = Pr(o t+1, o + t+1, q t+1 = j q t = i, λ) = = β T (i) = 1 j=1 n Pr(o t+1 q t+1, λ) Pr(o + t+1 q t+1 = j, λ) Pr(q t+1 = j q t = i, λ) j=1 n β t+1 (j)a ij b j (o t+1 ) j=1 Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33

19 Putting forward and backward probabilities together Conditional probability of states given data Pr(q t = i o, λ) = = Pr(o, q t = S i λ) n j=1 Pr(o, q t = S j λ) α t (i)β t (i) n j=1 α t(j)β t (j) Time complexity is Θ(n 2 T) Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33

20 Finding the most likely trajectory of hidden states Given a series of observations, we want to compute Define δ t (i) as arg max Pr(q o, λ) q δ t (i) = max Pr(q, o λ) q Use dynamic programming algorithm to find the most likely path Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33

21 The algorithm Initialization δ 1 (i) = πb i (o 1 ) for 1 i n Maintenance δ t (i) = max j δ t 1 (j)a ji b i (o t ) ϕ t (i) = arg max j δ t 1 (j)a ji Termination Max likelihood is max i δ T (i) Optimal path can be backtracked using ϕ t (i) Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33

22 An example Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33

23 An example path When observations were (walk, shop, clean) Similar to Manhattan tourist problem Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33

24 A working example : Occasionally biased coin A generative Observations : O = {1(Head), 2(Tail) Hidden states : S = {1(Fair), 2(Biased) Initial states : π = {05, 05 ( ) Transition probability : A(i, j) = a ij = ( ) Emission probability : B(i, j) = b i (j) = Questions Given coin toss observations, estimate the probability of each state Given coin toss observations, what is the most likely series of states? Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33

25 Implementing - Matrix615h #ifndef MATRIX_615_H #define MATRIX_615_H #include <vector> // to avoid multiple inclusion of same headers template <class T> class Matrix615 { public: std::vector< std::vector<t> > data; Matrix615(int nrow, int ncol, T val = 0) { dataresize(nrow); // make n rows for(int i=0; i < nrow; ++i) { data[i]resize(ncol,val); // make n cols with default value val int rownums() { return (int)datasize(); int colnums() { return ( datasize() == 0 )? 0 : (int)data[0]size(); ; #endif // MATRIX_615_H Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33

26 Implementations - 615h #ifndef 615_H #define 615_H #include "Matrix615h" class 615 { public: // parameters int nstates; // n : number of possible states int nobs; // m : number of possible output values int ntimes; // t : number of time slots with observations std::vector<double> pis; // initial states std::vector<int> outs; // observed outcomes Matrix615<double> trans; // trans[i][j] corresponds to A_{ij Matrix615<double> emis; // storages for dynamic programming Matrix615<double> alphas, betas, gammas, deltas; Matrix615<int> phis; std::vector<int> path; Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33

27 Implementations - 615h 615(int states, int obs, int times) : nstates(states), nobs(obs), ntimes(times), trans(states, states, 0), emis(states, obs, 0), alphas(times, states, 0), betas(times, states, 0), gammas(times, states, 0), deltas(times, states, 0), phis(times, states, 0) { pisresize(nstates); pathresize(ntimes); void forward(); // given below void backward(); // void forwardbackward(); // given below void viterbi(); // ; #endif // 615_H Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33

28 Implementations - 615::forward() void 615::forward() { for(int i=0; i < nstates; ++i) { alphasdata[0][i] = pis[i] * emisdata[i][outs[0]]; for(int t=1; t < ntimes; ++t) { for(int i=0; i < nstates; ++i) { alphasdata[t][i] = 0; for(int j=0; j < nstates; ++j) { alphasdata[t][i] += (alphasdata[t-1][j] * transdata[j][i] * emisdata[i][outs[t]]); Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33

29 Implementations - 615::backward() void 615::backward() { for(int i=0; i < nstates; ++i) { betasdata[ntimes-1][i] = 1; for(int t=ntimes-2; t >=0; --t) { for(int i=0; i < nstates; ++i) { betasdata[t][i] = 0; for(int j=0; j < nstates; ++j) { betasdata[t][i] += (betasdata[t+1][j] * transdata[i][j] * emisdata[j][outs[t+1]]); Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33

30 Implementations - 615::forwardBackward() void 615::forwardBackward() { forward(); backward(); for(int t=0; t < ntimes; ++t) { double sum = 0; for(int i=0; i < nstates; ++i) { sum += (alphasdata[t][i] * betasdata[t][i]); for(int i=0; i < nstates; ++i) { gammasdata[t][i] = (alphasdata[t][i] * betasdata[t][i])/sum; Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33

31 Implementations - 615::viterbi() void 615::viterbi() { for(int i=0; i < nstates; ++i) { deltasdata[0][i] = pis[i] * emisdata[i][ outs[0] ]; for(int t=1; t < ntimes; ++t) { for(int i=0; i < nstates; ++i) { int maxidx = 0; double maxval = deltasdata[t-1][0] * transdata[0][i] * emisdata[i][ outs[t] ]; for(int j=1; j < nstates; ++j) { double val = deltasdata[t-1][j] * transdata[j][i] * emisdata[i][ outs[t] ]; if ( val > maxval ) { maxidx = j; maxval = val; deltasdata[t][i] = maxval; phisdata[t][i] = maxidx; Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33

32 Implementations - 615::viterbi() (cont d) // backtrack viterbi path double maxdelta = deltasdata[ntimes-1][0]; path[ntimes-1] = 0; for(int i=1; i < nstates; ++i) { if ( maxdelta < deltasdata[ntimes-1][i] ) { maxdelta = deltasdata[ntimes-1][i]; path[ntimes-i] = i; for(int t=ntimes-2; t >= 0; --t) { path[t] = phisdata[t+1][ path[t+1] ]; Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33

33 Implementations - biasedcoincpp #include <iostream> #include <iomanip> int main(int argc, char** argv) { std::vector<int> toss; std::string tok; while( std::cin >> tok ) { if ( tok == "H" ) tosspush_back(0); else if ( tok == "T" ) tosspush_back(1); else { std::cerr << "Cannot recognize input " << tok << std::endl; return -1; int T = tosssize(); 615 hmm(2, 2, T); hmmtransdata[0][0] = 095; hmmtransdata[0][1] = 005; hmmtransdata[1][0] = 02; hmmtransdata[1][1] = 08; Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33

34 Implementations - biasedcoincpp hmmemisdata[0][0] = 05; hmmemisdata[0][1] = 05; hmmemisdata[1][0] = 09; hmmemisdata[1][1] = 01; hmmpis[0] = 05; hmmpis[1] = 05; hmmouts = toss; hmmforwardbackward(); hmmviterbi(); std::cout << "TIME\tTOSS\tP(FAIR)\tP(BIAS)\tMLSTATE" << std::endl; std::cout << std::setiosflags(std::ios::fixed) << std::setprecision(4); for(int t=0; t < T; ++t) { std::cout << t+1 << "\t" << (toss[t] == 0? "H" : "T") << "\t" << hmmgammasdata[t][0] << "\t" << hmmgammasdata[t][1] << "\t" << (hmmpath[t] == 0? "FAIR" : "BIASED" ) << std::endl; return 0; Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33

35 Example runs $ cat ~hmkang/public/615/data/toss20txt ~hmkang/public/615/bin/biasedcoin TIME TOSS P(FAIR) P(BIAS) MLSTATE 1 H FAIR 2 T FAIR 3 H FAIR 4 T FAIR 5 H FAIR 6 H FAIR 7 T FAIR 8 H BIASED 9 H BIASED 10 H BIASED 11 H BIASED 12 H BIASED 13 H BIASED 14 T BIASED 15 H BIASED 16 H BIASED 17 H BIASED 18 H BIASED 19 H BIASED 20 H BIASED Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33

36 Today - Hidden Markov Models algorithm algorithm Example Next Week Basic Usage of STL containers Using Eigen Library Interfacing between R and C++ Hyun Min Kang Biostatistics 615/815 - Lecture 10 October 4th, / 33