Application of Maximum Entropy Markov Models on the Protein Secondary Structure Predictions

Size: px

Start display at page:

Download "Application of Maximum Entropy Markov Models on the Protein Secondary Structure Predictions"

Ellen Blake
5 years ago
Views:

1 Applcaton of Maxmum Entropy Markov Models on the Proten Secondary Structure Predctons Yohan Km Department of Chemstry and Bochemstry Unversty of Calforna, San Dego La Jolla, CA Abstract An applcaton of a new probablstc modelng framework, Maxmum Entropy Markov Model, on the proten secondary structure predcton problem s descrbed [6]. As n prevous domans of problem, such as the task of segmentng a body of text, wthn whch MEMM was appled [6], the secondary structure predcton problem requres labelng an observaton sequence of alphabets. Ths paper s an exploratory effort that attempts to establsh the feasblty of usng the new framework on ths longstandng problem. Our ntal results produced wth a rather smple MEMM model show promse ( 58% accuracy) and suggest drectons for further mprovements. 1 Introducton The problem of predctng the secondary structure of protens has been around even before the structure of frst proten was solved by the x-ray crystallography method [1]. The database collectng those structures solved snce s a testament to the fact that there exst recurrng shapes representng varous parts of the proten whose geometres are guded by the composton of the amno acd sequence [7]. Intally these recurrng shapes were gven secondary structure labels by the human experts n the area. However ths method of labelng ntroduced subjectvty. In 1983, Kabsch et al ntroduced the DSSP program that consstently assgned secondary structure labels to the solved structures. The program bases ts method of labelng on the hydrogen bondng patterns found n the solved structure. Snce the growth of database holdng only proten sequences outpaced that of solved proten structures, much effort has been made n the area of predctng structures from the amno sequences. One form of predctng the proten structure from the amno acd sequence s the secondary structure predcton. Instead of predctng the full 3-D coordnates of the structure, the task s to predct a sequence of secondary structure labels based on the amno acd sequence alone. Present work uses the set of secondary structure labels whose sze s 3 (.e. Helx, Col, and Sheet).

2 Prevous works on predctng secondary structures of protens have yelded the best percent accuracy rangng from 63% to 71% [8]. These numbers, however, should be taken wth cauton snce performance of a method based on a tranng set may vary when traned on a dfferent tranng set. MEMM was recently ntroduced to model sequental data [6]. Some of the examples that ths new framework can be appled nclude taggng parts-of-speech on to a body of a text, labelng segments of questons and answer on Frequently-Asked- Questons lst (FAQs), and others. These types of problem are so called Input- Output problem. The essental task s to take an nput sequence and produce an output sequence that contans labels of the correspondng parts of the nput sequence. The proten secondary structure predcton s another Input-Output problem. In ths case, the task s to label a gven sequence of amno acds, X, wth a sequence of secondary structure labels, S. Here, the alphabet szes of X and S are 20 and 3, respectvely. MEMM shows better performance than the Hdden Markov Models (HMM) at least n the problem domans that t was ntally mplemented. Although the work that followed [5] showed that MEMM suffers from the label-bas problem and t ntroduced yet another model, mplementng MEMM was drven by followng motvatons: 1. offers a new testng ground to tackle the well establshed problem (the proten secondary structure predcton) and shed lght on MEMM s performance 2. allows overlappng features (features wll be precsely defned later n the paper) 3. t s a condtonal model No prevous applcaton of MEMM on the secondary structure predcton problem has been found n the lterature. Wth the potental advantage that MEMM offers overlappng features and that t s a condtonal model dependent only on the observaton sequence, present work ntroduces the new modelng framework to a long-standng problem of predctng the proten secondary structure from an amno acd sequence. The paper focuses on how MEMM was used n the secondary structure predcton problem. Present mplementaton of MEMM dffers n two aspects from the prevous ones. Notably there are no state transton probablty functons but a sngle functon that assgns probablty value to any possble combnaton of state and fragment. Ths mples that Vterb algorthm s not requred to nfer the state sequence. In addton, types of feature functons that were chosen are dfferent from the ones chosen for, say for nstance, segmentng FAQ problems. Ths s due to the basc dfference n the nature of the problems. In the later part of the paper, a general overvew of how the model s traned s gven and the results of the applcaton of MEMM on Salzberg s data set are shown [9]. 2 Maxmum Entropy Markov Model A summary of the new modelng framework ntroduced s gven [6]. The essental prncple behnd Maxmum Entropy approach s to create a model that satsfes all known constrants but otherwse treat the unknowns unformly [2]. The fnal model after tranng s a collecton of separately traned transton functons, P s (s x) and a vector of parameters λ. The traned functon P s (s x) outputs a probablty value of seeng a state transton from s to s gven observaton x. When P s (s x) s summed over s, the sum should be 1 by the defnton of the probablty functon.

3 1 P s ( s x) = exp a f a ( s, x) Z( s', x) a ' λ (1) Z(s,x) n equaton (1) s a normalzng factor. Features f a (s,x) are bnary functons that capture the mportant relatonshp among state and observaton sequences. Present work does not explctly ncorporate the state transtons. In other words, no functons of P s (s x) are constructed. Instead, only one probablty functon P(s x) s used, and the observaton x s characterzed by a set of 300 feature functons. The decson to use only one probablty functon was forced by the choce and the total number of feature functons. However, t should be noted that the feature functons employed n ths work mplctly capture the dependence on prevous states. Detaled dscussons on feature functons wll be gven later. The absence of state transton functons makes the use of Vterb algorthm to nfer the state sequence unnecessary. One can nfer the state sequence just by pckng the state label at each postons of the sequence that has the hghest probablty based on the P(s x). 3 Feature Selecton One last mportant aspect of the current mplementaton s the task of selectng the set of features that capture the mportant relatonshps among state and observaton sequences. No systematc effort was made to select the best set of feature functons. Selectng a set of features was made dffcult by the uncertanty of how many features and what knds are needed to suffcently capture the relatonshp among the amno acd sequence and the correspondng secondary structure labels. In addton, due to long tranng tme needed, only a small number of combnatons of feature functons were explored. Another dffculty n choosng features was due to nherent dffculty n understandng what the language of amno acds s. To gve an analogy usng textsegmentng FAQs as an example, some features such as ndent-present and contans-queston-mark are mmedately understood by the reader to have mportant roles n segmentng FAQs nto queston and answer pars. The reader has an ntutve noton that the two features mentoned have some relevance n dentfyng dfferent segments of FAQs. Ths ntuton s made possble snce the reader understands what each of the observaton sequence means (n ths case, a body of Englsh text). To extend the analogy of the reader and the observaton sequence, the reader n the case of secondary structure predcton s not human but a dense collecton of cellular machneres that understand what the amno acd sequence means and that carry out myrads of operatons dctated by the amno acd compostons. In the end, the set of features that are descrbed by [3] are used. Ths set of features s smple, and one can mmedately see relatonshps among state and observaton sequence that t s tryng to capture. The present work uses 300 feature functons that are succnctly represented by the followng equaton: s, ) = 1f matches x table(k) s 0 f otherwse and x ( k f (2)

4 Table 1: Frst 6 entres of table(k). k s aa p Table1: State labels and amno acds are represented wth ntegers. k = ndex of the feature functon = poston n the observaton sequence s = possble state assgnment at poston. s = 3. x = possble amno acd sequence fragment of length 5 at poston. aa = amno acd ndex rangng from 1 to 20. p = poston of the amno acd wthn an amno acd sequence fragment. Two examples usng ths set of feature functons f 1 (1, [ ]) = 1 and f 1 (1, [ ]) = 0 are provded to clarfy the way n whch these feature functons are used. The feature functon wth an ndex k = 1 means that ths functon returns a value of 1 only f the arguments (.e. s and an amno acd at poston p of the fragment x ) of the functon match label and amno acd stored n the table at k-th row (Fgure 1). In the frst example, s = 1 and x = [ ]. Havng k = 1 means that ths feature functon returns 1 only when t sees a state that has a value of 1 and an amno acd at poston 1 wthn fragment x wth value of 1. In the second example, the feature functon sees a state value of 1; however, t sees an amno acd value of 2 at poston 1 of fragment x = [ ]. Consequently, the latter feature functon yelds 0. Ths set of feature functons attempts to capture, borrowng the term used n [3], the amount of nformaton that an amno acd at poston p has on the state assgnment of the amno acd at the mddle of the wndow (Fgure 2). There are 3 possble secondary structure labels, 20 possble amno acds, and 5 possble postons. Ths yelds the total number of 300 unque permutatons possble for the trplet (label, amno acd, poston) where each permutaton s captured by one feature functon.

5 Fgure 2: Indvdual amno acds at varous postons of a fragment contrbutng nformaton to yeld the fnal state assgnment for the mddle amno acd. An amno acd sequence fragment of length 5 s presented wth ts mddle resdue labeled as H or Helx. Each one of I1, I2, I3, I4, and I5 s represented by a feature functon. Snce there are 3 states, 20 amno acds, and 5 postons, the total number of feature functons s Parameter Estmaton wth Generalzed Iteratve Scalng The MEMM model s traned by maxmzng the log probablty of the state sequence gven the observaton sequence. Ths mples that the emprcal expectatons of the features have to equal to those calculated usng the traned model. Ths condton s shown by the equaton (3). N N 1 1 f ( s, x ) = P( s x ) fk ( s, x ) N 1 1 k = N = s S N = total length of the tranng sequence of state and observaton pars. The term on the left s the emprcal expectaton of the feature functon f k ( ). The term on the rght s the expectaton calculated usng the model. Ths work follows the same outlne of the algorthm provded n earler work to estmate the parameters [6]. One mnor dfference to note s that the arbtrary constant C was set to 1. The traned MEMM model s a set of converged parameters λ and the sngle probablty functon P(s x). It should be noted that the fnal values of λ ndcate how much each feature functon nfluence n determnng the probablty of s gven x. Roughly speakng, greater the value of a parameter, the more t allows the correspondng feature functon to contrbute to the fnal probablty value. 5 Expermental Results A data set used n earler works [8] provded both the tranng and testng data sets. Brefly, ths data set contaned amno acd sequences of expermentally solved structures and correspondng sequences of secondary structure labels as determned by the DSSP program [4]. The orgnal data set was processed to ft the needs of the model. Pars of amno acd sequences and secondary structure label sequences wthn the orgnal data set (3)

6 were combned nto one fle. Then, another fle was generated that had two columns: 1. frst column was allotted for the fragments of amno acd sequence derved by sldng a wndow of length 5 along the amno acd sequence 2. second column was for the state labels assgned to the mddle resdues of the correspondng fragments. The processed fle contaned total of 18,790 lnes, each contanng a fragment-label par. The tranng set conssted of roughly half of the processed fle. The test set was the latter half. MEMM was mplemented wthn Matlab envronment. Tranng was stopped after 4 teratons (parameters converged to wthn second decmals), and the entre tranng took approxmately 16 hours on Pentum II 400 Mhz. Table 2: Comparsons of % correct labels among varous methods. Methods % correct PEBLS (1992) 71 Zhang et al (1992) 66.4 Qan & Sejnowsk (1988) 64.3 Holley & Karplus (1989) 63.2 MEMM random 39.9 Table 2 s shown to llustrate the range n whch prevously developed methods performed relatve to MEMM. These numbers are not drectly comparable snce dfferent tranng sets were used. In addton relatvely poor performance of MEMM may be attrbuted to the fact that t only took nto account 2 neghborng amno acds whereas other methods such as PEBLS and Qan & Sejnowsk used 8 neghborng amno acds or more. Salzberg et al reported that ther method, PEBLS, consdered 9 neghborng amno acds because they found ths number to be optmal. Fgure 3: One-to-one parng of the correct sequence label at top and MEMM predcted sequence at the bottom for the frst 100 fragments n the test set. MEMM predcts a rough outlne of the patches of contnuous secondary structure labels. In ths partcular example, for two long stretches of contnuous secondary structures (frst 1/3 and last 1/3 of the sequences shown above), MEMM gets majorty of the labels correct. In the mddle part of the sequences, where correct labels fluctuate more often than the other regons, MEMM performs poorly.

7 6 Conclusons and Future Drectons A new probablstc modelng framework was appled to the proten secondary structure predcton problem. The set of smple features ntroduced by [3] was used. The results of the experment on Salzberg s data set show some promses ( 58% correct labels). Ths s over 20% better than randomly assgnng the labels wth emprcal dstrbuton of secondary structure labels n the data set (H = 26.1 %; C = 54.4%; E = 19.5 %). The results of the experment should be vewed wth the knowledge that only fragments of length 5 were consdered n the model. As was noted n [8], a better fragment length than the one used here can probably be found for the current choce of feature set. More over, many varatons on the selecton of features are possble. Present work dd not take the advantage of overlappng features offered by MEMM. Although MEMM offers nterestng aspect to modelng the sequental data, the experence wth mplementng the model suggests that the archtecture of MEMM puts much constrant on the number of features that can be used durng tranng. Greater the number of features, the calculaton of expectaton value of each feature functon ncreases correspondngly (see the outlne of algorthm n [6]). Wth tranng set of sze N and feature set of sze M, the tme complexty of the tranng s O(j*M*N), where j s the number of teratons. The number of feature functons can be sgnfcantly decreased by ntroducng overlappng feature functons. Much room for mprovements exsts n the area of feature selecton. 7 References [1] Bald, P. Bonformatcs. (2001). The Machne Learnng Approach. 2 nd ed. [2] Berger, A. L., Della Petra, S. A., & Della Petra, V. J. (1996). A Maxmum Entropy Approach to Natural Language Processng. Computatonal Lngustcs. 22, 1. [3] Garner, J., Osguthorpe, D. J., & Robson, B. (1978). Analyss of the Accuracy and Implcatons of Smple Methods for Predctng the Secondary Structure of Globular Protens. Journal of Molecular Bology, 120, [4] Kabsch, W. & Sander, C. (1983). Dctonary of Proten Secondary Structure: Pattern Recognton of Hydrogen-Bonded and Geometrcal Features. Bopolymers, 22, [5] Lafferty, J., McCallum, A., & Perera, F. (2001). Condtonal Random Felds: Probablstc Models for Segmentng and Labelng Sequence Data. Proc. ICML. [6] McCallum, A., Fretag, D. & Perera, F. (2000). Maxmum Entropy Markov Models for Informaton Extracton and Segmentaton. Proc. ICML. [7] Proten Data Bank. [8] Salzberg, S. & Cost, S. (1992). Predctng Proten Secondary Structure wth a Nearest-neghbor Algorthm. Journal of Molecular Bology. 227, [9] Salzberg s Data Set.

Content Based Image Retrieval Using 2-D Discrete Wavelet with Texture Feature with Different Classifiers

Content Based Image Retrieval Using 2-D Discrete Wavelet with Texture Feature with Different Classifiers IOSR Journal of Electroncs and Communcaton Engneerng (IOSR-JECE) e-issn: 78-834,p- ISSN: 78-8735.Volume 9, Issue, Ver. IV (Mar - Apr. 04), PP 0-07 Content Based Image Retreval Usng -D Dscrete Wavelet wth