Lecture 5: Markov models
|
|
- Jeremy Kelly
- 5 years ago
- Views:
Transcription
1 Master s course Bioinformatics Data Analysis and Tools Lecture 5: Markov models Centre for Integrative Bioinformatics
2 Problem in biology Data and patterns are often not clear cut When we want to make a method to recognise a pattern (e.g. a sequence motif), we have to learn from the data (e.g. maybe there are other differences between sequences that have the pattern and those that do not) This leads to Data mining and Machine learning
3 Contents: A widely used machine learning approach: Markov models Markov chain models (1st order, higher order and inhomogeneous models; parameter estimation; classification) Interpolated Markov models (and back-off models) Hidden Markov models (forward, backward and Baum- Welch algorithms; model topologies; applications to gene finding and protein family modeling
4
5 Markov Chain Models a Markov chain model is defined by: a set of states some states emit symbols other states (e.g. the begin state) are silent a set of transitions with associated probabilities the transitions emanating from a given state define a distribution over the possible next states
6 Markov Chain Models given some sequence x of length L, we can ask how probable the sequence is given our model for any probabilistic model of sequences, we can write this probability as key property of a (1st order) Markov chain: the probability of each X i depends only on X i-1
7 Markov Chain Models Pr(cggt) = Pr(c)Pr(g c)pr(g g)pr(t g)
8 Markov Chain Models Can also have an end state, allowing the model to represent: Sequences of different lengths Preferences for sequences ending with particular symbols
9 Markov Chain Models The transition parameters can be denoted by where a x i = 1xi Pr( x x ) i i 1 Similarly we can denote the probability of a sequence x as ax i 1 x i Where a Bxi represents the transition from the begin state
10 Example Application CpG islands CGdinucleotides are rarer in eukaryotic genomes than expected given the independent probabilities of C, G but the regions upstream of genes are richer in CG dinucleotides than elsewhere CpG islands useful evidence for finding genes Could predict CpG islands with Markov chains one to represent CpG islands one to represent the rest of the genome Example includes using Maximum likelihood and Bayes statistical data and feeding it to a HM model
11 Estimating the Model Parameters Given some data (e.g. a set of sequences from CpG islands), how can we determine the probability parameters of our model? One approach: maximum likelihood estimation given a set of data D set the parameters θ to maximize Pr(D θ) i.e. make the data D look likely under the model
12 Maximum Likelihood Estimation Suppose we want to estimate the parameters Pr(a), Pr(c), Pr(g), Pr(t) And we re given the sequences: accgcgctta gcttagtgac tagccgttac Then the maximum likelihood estimates are: Pr(a) = 6/30 = 0.2 Pr(g) = 7/30 = Pr(c) = 9/30 = 0.3 Pr(t) = 8/30 = 0.267
13
14
15
16
17 These data are derived from genome sequences
18
19
20
21 Higher Order Markov Chains An nth order Markov chain over some alphabet is equivalent to a first order Markov chain over the alphabet of n-tuples Example: a 2nd order Markov model for DNA can be treated as a 1st order Markov model over alphabet: AA, AC, AG, AT, CA, CC, CG, CT, GA, GC, GG, GT, TA, TC, TG, and TT (i.e. all possible dipeptides)
22 A Fifth Order Markov Chain
23 Inhomogenous Markov Chains In the Markov chain models we have considered so far, the probabilities do not depend on where we are in a given sequence In an inhomogeneous Markov model, we can have different distributions at different positions in the sequence Consider modeling codons in protein coding regions
24 Inhomogenous Markov Chains
25 A Fifth Order Inhomogenous Markov Chain
26 Selecting the Order of a Markov Chain Model Higher order models remember more history Additional history can have predictive value Example: predict the next word in this sentence fragment finish (up, it, first, last,?) now predict it given more history Fast guys finish
27 Selecting the Order of a Markov Chain Model However, the number of parameters we need to estimate grows exponentially with the order for modeling DNA we need parameters for an nth order model, with n 5 normally The higher the order, the less reliable we can expect our parameter estimates to be estimating the parameters of a 2nd order homogenous Markov chain from the complete genome of E. Coli, we would see each word > 72,000 times on average estimating the parameters of an 8th order chain, we would see each word ~ 5 times on average
28 Interpolated Markov Models The IMM idea: manage this trade-off by interpolating among models of various orders Simple linear interpolation:
29 Interpolated Markov Models We can make the weights depend on the history for a given order, we may have significantly more data to estimate some words than others General linear interpolation
30 Gene Finding: Search by Content Encoding a protein affects the statistical properties of a DNA sequence some amino acids are used more frequently than others (Leu more popular than Trp) different numbers of codons for different amino acids (Leu has 6, Trp has 1) for a given amino acid, usually one codon is used more frequently than others This is termed codon preference Codon preferences vary by species
31 Codon Preference in E. Coli AA codon / Gly GGG 1.89 Gly GGA 0.44 Gly GGU Gly GGC Glu GAG Glu GAA Asp GAU Asp GAC 43.26
32 Search by Content Common way to search by content build Markov models of coding & noncoding regions apply models to ORFs (Open Reading Frames) or fixedsized windows of sequence GeneMark [Borodovsky et al.] popular system for identifying genes in bacterial genomes uses 5th order inhomogenous Markov chain models
33 The GLIMMER System Salzberg et al., 1998 System for identifying genes in bacterial genomes Uses 8th order, inhomogeneous, interpolated Markov chain models
34 IMMs in GLIMMER How does GLIMMER determine the values? First, let us express the IMM probability calculation recursively:
35 IMMs in GLIMMER If we haven t seen x i-1 x i-n more than 400 times, then compare the counts for the following: Use a statistical test ( χ 2 ) to get a value d indicating our confidence that the distributions represented by the two sets of counts are different
36 IMMs in GLIMMER χ 2 score when comparing n th -order with n-1 th -order Markov model (preceding slide)
37 The GLIMMER method 8th order IMM vs. 5th order Markov model Trained on 1168 genes (ORFs really) Tested on 1717 annotated (more or less known) genes
38
39
40 Plot sensitivity over 1-specificity
41
42
43 Hidden Markov models (HMMs) Given say a T in our input sequence, which state emitted it?
44 Hidden Markov models (HMMs) Hidden State We will distinguish between the observed parts of a problem and the hidden parts In the Markov models we have considered previously, it is clear which state accounts for each part of the observed sequence In the model above (preceding slide), there are multiple states that could account for each part of the observed sequence this is the hidden part of the problem states are decoupled from sequence symbols
45 HMM-based homology searching HMM for ungapped alignment Transition probabilities and Emission probabilities Gapped HMMs also have insertion and deletion states (next slide)
46 Profile HMM: m=match state, I-insert state, d=delete state; go from left to right. I and m states output amino acids; d states are silent. d 1 d 2 d 3 d 4 I 0 I 1 I 2 I 3 I 4 m 0 m 1 m 2 m 3 m 4 m 5 Start End Model for alignment with insertions and deletions
47 HMM-based homology searching Most widely used HMM-based profile searching tools currently are SAM-T99 (Karplus et al., 1998) and HMMER2 (Eddy, 1998) formal probabilistic basis and consistent theory behind gap and insertion scores HMMs good for profile searches, bad for alignment (due to parametrisation of the models) HMMs are slow
48 Homology-derived Secondary Structure of Proteins (HSSP) Sander & Schneider, 1991 It s all about trying to push don t know region down
49 The Parameters of an HMM
50 HMM for Eukaryotic Gene Finding Figure from A. Krogh, An Introduction to Hidden Markov Models for Biological Sequences
51 A Simple HMM
52 Three Important Questions How likely is a given sequence? the Forward algorithm What is the most probable path for generating a given sequence? the Viterbi algorithm How can we learn the HMM parameters given a set of sequences? the Forward-Backward (Baum-Welch) algorithm
53 Three basic problems of HMMs Once we have an HMM, there are three problems of interest. (1) The Evaluation Problem Given an HMM and a sequence of observations, what is the probability that the observations are generated by the model? (2) The Decoding Problem Given a model and a sequence of observations, what is the most likely state sequence in the model that produced the observations? (3) The Learning Problem Given a model and a sequence of observations, how should we adjust the model parameters in order to maximize Evaluation problem can be used for isolated (word) recognition. Decoding problem is related to the continuous recognition as well as to the segmentation. Learning problem must be solved, if we want to train an HMM for the subsequent use of recognition tasks.
54 Three Important Questions How likely is a given sequence? Forward algorithm What is the most probable path for generating a given sequence? How can we learn the HMM parameters given a set of sequences?
55 How Likely is a Given Sequence? The probability that the path is taken and the sequence is generated: (assuming begin/end are the only silent states on path)
56 How Likely is a Given Sequence?
57 How Likely is a Given Sequence? The probability over all paths is: but the number of paths can be exponential in the length of the sequence... the Forward algorithm enables us to compute this efficiently
58 How Likely is a Given Sequence: The Forward Algorithm Define f k (i) to be the probability of being in state k Having observed the first i characters of x we want to compute f N (L), the probability of being in the end state having observed all of x We can define this recursively
59 How Likely is a Given Sequence:
60 Initialisation: The forward algorithm f 0 (0) = 1 (start), f k (0) = 0 (other silent states k) probability that we re in start state and have observed 0 characters from the sequence Recursion: f l (i) = e l (i)σ k f k (i-1)a kl (emitting states), f l (i) = Σ k f k (i)a kl (silent states) Termination: Pr(x) = Pr(x 1 x L ) = f N (L) = Σ k f k (L)a kn probability that we are in the end state and have observed the entire sequence
61 Forward algorithm example
62 Three Important Questions How likely is a given sequence? What is the most probable path for generating a given sequence? Viterbi algorithm How can we learn the HMM parameters given a set of sequences?
63 Finding the Most Probable Path: The Viterbi Algorithm Define v k (i) to be the probability of the most probable path accounting for the first i characters of x and ending in state k We want to compute v N (L), the probability of the most probable path accounting for all of the sequence and ending in the end state Can be defined recursively Can use DP to find v N (L) efficiently
64 Finding the Most Probable Path: Initialisation: The Viterbi Algorithm v 0 (0) = 1 (start), v k (0) = 0 (non-silent states) Recursion for emitting states (i =1 L): Recursion for silent states:
65 Finding the Most Probable Path: The Viterbi Algorithm
66 Three Important Questions How likely is a given sequence? (clustering) What is the most probable path for generating a given sequence? (alignment) How can we learn the HMM parameters given a set of sequences? The Baum-Welch Algorithm
67 The Learning Problem Generally, the learning problem is how to adjust the HMM parameters, so that the given set of observations (called the training set) is represented by the model in the best way for the intended application. Thus it would be clear that the ``quantity'' we wish to optimize during the learning process can be different from application to application. In other words there may be several optimization criteria for learning, out of which a suitable one is selected depending on the application. There are two main optimization criteria found in the literature; Maximum Likelihood (ML) and Maximum Mutual Information (MMI).
68 The Learning Task Given: a model a set of sequences (the training set) Do: find the most likely parameters to explain the training sequences The goal is find a model that generalizes well to sequences we haven t seen before
69 Learning Parameters If we know the state path for each training sequence, learning the model parameters is simple no hidden state during training count how often each parameter is used normalize/smooth to get probabilities process just like it was for Markov chain models If we don t know the path for each training sequence, how can we determine the counts? key insight: estimate the counts by considering every path weighted by its probability
70 Learning Parameters: The Baum-Welch Algorithm An EM (expectation maximization) approach, a forward-backward algorithm Algorithm sketch: initialize parameters of model iterate until convergence Calculate the expected number of times each transition or emission is used Adjust the parameters to maximize the likelihood of these expected values Baum-Welch has as important feature that it always converges
71 The Expectation step
72 The Expectation step
73 The Expectation step
74 The Expectation step
75 The Expectation step First, we need to know the probability of the i th symbol being produced by state q, given sequence x: Pr( π i =k x) Given this we can compute our expected counts for state transitions, character emissions
76 The Expectation step
77 The Backward Algorithm
78 The Expectation step
79 The Expectation step
80 The Expectation step
81 The Maximization step
82 The Maximization step
83 The Baum-Welch Algorithm Initialize parameters of model Iterate until convergence calculate the expected number of times each transition or emission is used adjust the parameters to maximize the likelihood of these expected values This algorithm will converge to a local maximum (in the likelihood of the data given the model) Usually in a fairly small number of iterations
Biology 644: Bioinformatics
A statistical Markov model in which the system being modeled is assumed to be a Markov process with unobserved (hidden) states in the training data. First used in speech and handwriting recognition In
More informationHMMConverter A tool-box for hidden Markov models with two novel, memory efficient parameter training algorithms
HMMConverter A tool-box for hidden Markov models with two novel, memory efficient parameter training algorithms by TIN YIN LAM B.Sc., The Chinese University of Hong Kong, 2006 A THESIS SUBMITTED IN PARTIAL
More informationHIDDEN MARKOV MODELS AND SEQUENCE ALIGNMENT
HIDDEN MARKOV MODELS AND SEQUENCE ALIGNMENT - Swarbhanu Chatterjee. Hidden Markov models are a sophisticated and flexible statistical tool for the study of protein models. Using HMMs to analyze proteins
More informationCS6220: DATA MINING TECHNIQUES
CS6220: DATA MINING TECHNIQUES Sequence Data Instructor: Yizhou Sun yzsun@ccs.neu.edu November 22, 2015 Announcement TRACE faculty survey myneu->self service tab Homeworks HW5 will be the last homework
More informationGenome 559. Hidden Markov Models
Genome 559 Hidden Markov Models A simple HMM Eddy, Nat. Biotech, 2004 Notes Probability of a given a state path and output sequence is just product of emission/transition probabilities If state path is
More informationBMI/CS Lecture #22 - Stochastic Context Free Grammars for RNA Structure Modeling. Colin Dewey (adapted from slides by Mark Craven)
BMI/CS Lecture #22 - Stochastic Context Free Grammars for RNA Structure Modeling Colin Dewey (adapted from slides by Mark Craven) 2007.04.12 1 Modeling RNA with Stochastic Context Free Grammars consider
More informationEukaryotic Gene Finding: The GENSCAN System
Eukaryotic Gene Finding: The GENSCAN System BMI/CS 776 www.biostat.wisc.edu/bmi776/ Spring 2016 Anthony Gitter gitter@biostat.wisc.edu These slides, excluding third-party material, are licensed under CC
More informationChapter 6. Multiple sequence alignment (week 10)
Course organization Introduction ( Week 1,2) Part I: Algorithms for Sequence Analysis (Week 1-11) Chapter 1-3, Models and theories» Probability theory and Statistics (Week 3)» Algorithm complexity analysis
More informationECE521: Week 11, Lecture March 2017: HMM learning/inference. With thanks to Russ Salakhutdinov
ECE521: Week 11, Lecture 20 27 March 2017: HMM learning/inference With thanks to Russ Salakhutdinov Examples of other perspectives Murphy 17.4 End of Russell & Norvig 15.2 (Artificial Intelligence: A Modern
More informationGLIMMER. Dennis Flottmann
GLIMMER Dennis Flottmann 1 Agenda Who invented GLIMMER? What is GLIMMER? How GLIMMER works IMMs ICMs GLIMMER live demonstration GLIMMER today and in comparison to other tools 2 Who invented GLIMMER? Steven
More informationGribskov Profile. Hidden Markov Models. Building a Hidden Markov Model #$ %&
Gribskov Profile #$ %& Hidden Markov Models Building a Hidden Markov Model "! Proteins, DNA and other genomic features can be classified into families of related sequences and structures How to detect
More informationMachine Learning. Computational biology: Sequence alignment and profile HMMs
10-601 Machine Learning Computational biology: Sequence alignment and profile HMMs Central dogma DNA CCTGAGCCAACTATTGATGAA transcription mrna CCUGAGCCAACUAUUGAUGAA translation Protein PEPTIDE 2 Growth
More informationHidden Markov Models. Slides adapted from Joyce Ho, David Sontag, Geoffrey Hinton, Eric Xing, and Nicholas Ruozzi
Hidden Markov Models Slides adapted from Joyce Ho, David Sontag, Geoffrey Hinton, Eric Xing, and Nicholas Ruozzi Sequential Data Time-series: Stock market, weather, speech, video Ordered: Text, genes Sequential
More information15-780: Graduate Artificial Intelligence. Computational biology: Sequence alignment and profile HMMs
5-78: Graduate rtificial Intelligence omputational biology: Sequence alignment and profile HMMs entral dogma DN GGGG transcription mrn UGGUUUGUG translation Protein PEPIDE 2 omparison of Different Organisms
More informationFaster Gradient Descent Training of Hidden Markov Models, Using Individual Learning Rate Adaptation
Faster Gradient Descent Training of Hidden Markov Models, Using Individual Learning Rate Adaptation Pantelis G. Bagos, Theodore D. Liakopoulos, and Stavros J. Hamodrakas Department of Cell Biology and
More information3.4 Multiple sequence alignment
3.4 Multiple sequence alignment Why produce a multiple sequence alignment? Using more than two sequences results in a more convincing alignment by revealing conserved regions in ALL of the sequences Aligned
More informationHidden Markov Models Review and Applications. hidden Markov model. what we see model M = (,Q,T) states Q transition probabilities e Ax
Hidden Markov Models Review and Applications 1 hidden Markov model what we see x y model M = (,Q,T) states Q transition probabilities e Ax t AA e Ay observation observe states indirectly emission probabilities
More informationComputational Molecular Biology
Computational Molecular Biology Erwin M. Bakker Lecture 3, mainly from material by R. Shamir [2] and H.J. Hoogeboom [4]. 1 Pairwise Sequence Alignment Biological Motivation Algorithmic Aspect Recursive
More informationMultiple Sequence Alignment Based on Profile Alignment of Intermediate Sequences
Multiple Sequence Alignment Based on Profile Alignment of Intermediate Sequences Yue Lu and Sing-Hoi Sze RECOMB 2007 Presented by: Wanxing Xu March 6, 2008 Content Biology Motivation Computation Problem
More informationChapter 8 Multiple sequence alignment. Chaochun Wei Spring 2018
1896 1920 1987 2006 Chapter 8 Multiple sequence alignment Chaochun Wei Spring 2018 Contents 1. Reading materials 2. Multiple sequence alignment basic algorithms and tools how to improve multiple alignment
More informationCS313 Exercise 4 Cover Page Fall 2017
CS313 Exercise 4 Cover Page Fall 2017 Due by the start of class on Thursday, October 12, 2017. Name(s): In the TIME column, please estimate the time you spent on the parts of this exercise. Please try
More informationPROTEIN MULTIPLE ALIGNMENT MOTIVATION: BACKGROUND: Marina Sirota
Marina Sirota MOTIVATION: PROTEIN MULTIPLE ALIGNMENT To study evolution on the genetic level across a wide range of organisms, biologists need accurate tools for multiple sequence alignment of protein
More informationC E N T R. Introduction to bioinformatics 2007 E B I O I N F O R M A T I C S V U F O R I N T. Lecture 13 G R A T I V. Iterative homology searching,
C E N T R E F O R I N T E G R A T I V E B I O I N F O R M A T I C S V U Introduction to bioinformatics 2007 Lecture 13 Iterative homology searching, PSI (Position Specific Iterated) BLAST basic idea use
More informationBLAST, Profile, and PSI-BLAST
BLAST, Profile, and PSI-BLAST Jianlin Cheng, PhD School of Electrical Engineering and Computer Science University of Central Florida 26 Free for academic use Copyright @ Jianlin Cheng & original sources
More informationHidden Markov Models. Mark Voorhies 4/2/2012
4/2/2012 Searching with PSI-BLAST 0 th order Markov Model 1 st order Markov Model 1 st order Markov Model 1 st order Markov Model What are Markov Models good for? Background sequence composition Spam Hidden
More informationProfiles and Multiple Alignments. COMP 571 Luay Nakhleh, Rice University
Profiles and Multiple Alignments COMP 571 Luay Nakhleh, Rice University Outline Profiles and sequence logos Profile hidden Markov models Aligning profiles Multiple sequence alignment by gradual sequence
More informationUsing Hidden Markov Models for Multiple Sequence Alignments Lab #3 Chem 389 Kelly M. Thayer
Página 1 de 10 Using Hidden Markov Models for Multiple Sequence Alignments Lab #3 Chem 389 Kelly M. Thayer Resources: Bioinformatics, David Mount Ch. 4 Multiple Sequence Alignments http://www.netid.com/index.html
More informationEECS730: Introduction to Bioinformatics
EECS730: Introduction to Bioinformatics Lecture 04: Variations of sequence alignments http://www.pitt.edu/~mcs2/teaching/biocomp/tutorials/global.html Slides adapted from Dr. Shaojie Zhang (University
More information!"#$ Gribskov Profile. Hidden Markov Models. Building an Hidden Markov Model. Proteins, DNA and other genomic features can be
Gribskov Profile $ Hidden Markov Models Building an Hidden Markov Model $ Proteins, DN and other genomic features can be classified into families of related sequences and structures $ Related sequences
More informationWeighted Finite-State Transducers in Computational Biology
Weighted Finite-State Transducers in Computational Biology Mehryar Mohri Courant Institute of Mathematical Sciences mohri@cims.nyu.edu Joint work with Corinna Cortes (Google Research). 1 This Tutorial
More informationAssignment 2. Unsupervised & Probabilistic Learning. Maneesh Sahani Due: Monday Nov 5, 2018
Assignment 2 Unsupervised & Probabilistic Learning Maneesh Sahani Due: Monday Nov 5, 2018 Note: Assignments are due at 11:00 AM (the start of lecture) on the date above. he usual College late assignments
More informationGLOBEX Bioinformatics (Summer 2015) Multiple Sequence Alignment
GLOBEX Bioinformatics (Summer 2015) Multiple Sequence Alignment Scoring Dynamic Programming algorithms Heuristic algorithms CLUSTAL W Courtesy of jalview Motivations Collective (or aggregate) statistic
More informationCS839: Probabilistic Graphical Models. Lecture 10: Learning with Partially Observed Data. Theo Rekatsinas
CS839: Probabilistic Graphical Models Lecture 10: Learning with Partially Observed Data Theo Rekatsinas 1 Partially Observed GMs Speech recognition 2 Partially Observed GMs Evolution 3 Partially Observed
More informationStructured Learning. Jun Zhu
Structured Learning Jun Zhu Supervised learning Given a set of I.I.D. training samples Learn a prediction function b r a c e Supervised learning (cont d) Many different choices Logistic Regression Maximum
More informationHidden Markov Models in the context of genetic analysis
Hidden Markov Models in the context of genetic analysis Vincent Plagnol UCL Genetics Institute November 22, 2012 Outline 1 Introduction 2 Two basic problems Forward/backward Baum-Welch algorithm Viterbi
More informationINTRODUCTION TO BIOINFORMATICS
Molecular Biology-2017 1 INTRODUCTION TO BIOINFORMATICS In this section, we want to provide a simple introduction to using the web site of the National Center for Biotechnology Information NCBI) to obtain
More informationSpeech Recognition Lecture 8: Acoustic Models. Eugene Weinstein Google, NYU Courant Institute Slide Credit: Mehryar Mohri
Speech Recognition Lecture 8: Acoustic Models. Eugene Weinstein Google, NYU Courant Institute eugenew@cs.nyu.edu Slide Credit: Mehryar Mohri Speech Recognition Components Acoustic and pronunciation model:
More informationMultiple Sequence Alignment Gene Finding, Conserved Elements
Multiple Sequence Alignment Gene Finding, Conserved Elements Definition Given N sequences x 1, x 2,, x N : Insert gaps (-) in each sequence x i, such that All sequences have the same length L Score of
More informationFinding homologous sequences in databases
Finding homologous sequences in databases There are multiple algorithms to search sequences databases BLAST (EMBL, NCBI, DDBJ, local) FASTA (EMBL, local) For protein only databases scan via Smith-Waterman
More informationComputational Genomics and Molecular Biology, Fall
Computational Genomics and Molecular Biology, Fall 2015 1 Sequence Alignment Dannie Durand Pairwise Sequence Alignment The goal of pairwise sequence alignment is to establish a correspondence between the
More informationUsing Hidden Markov Models to Detect DNA Motifs
San Jose State University SJSU ScholarWorks Master's Projects Master's Theses and Graduate Research Spring 5-13-2015 Using Hidden Markov Models to Detect DNA Motifs Santrupti Nerli San Jose State University
More informationStephen Scott.
1 / 33 sscott@cse.unl.edu 2 / 33 Start with a set of sequences In each column, residues are homolgous Residues occupy similar positions in 3D structure Residues diverge from a common ancestral residue
More informationINTRODUCTION TO BIOINFORMATICS
Molecular Biology-2019 1 INTRODUCTION TO BIOINFORMATICS In this section, we want to provide a simple introduction to using the web site of the National Center for Biotechnology Information NCBI) to obtain
More informationε-machine Estimation and Forecasting
ε-machine Estimation and Forecasting Comparative Study of Inference Methods D. Shemetov 1 1 Department of Mathematics University of California, Davis Natural Computation, 2014 Outline 1 Motivation ε-machines
More informationA reevaluation and benchmark of hidden Markov Models
04-09-2014 1 A reevaluation and benchmark of hidden Markov Models Jean-Paul van Oosten Prof. Lambert Schomaker 04-09-2014 2 Hidden Markov model fields & variants Automatic speech recognition Gene sequence
More informationIntroduction to Hidden Markov models
1/38 Introduction to Hidden Markov models Mark Johnson Macquarie University September 17, 2014 2/38 Outline Sequence labelling Hidden Markov Models Finding the most probable label sequence Higher-order
More informationof Manchester The University COMP14112 Markov Chains, HMMs and Speech Revision
COMP14112 Lecture 11 Markov Chains, HMMs and Speech Revision 1 What have we covered in the speech lectures? Extracting features from raw speech data Classification and the naive Bayes classifier Training
More informationMSCBIO 2070/02-710: Computational Genomics, Spring A4: spline, HMM, clustering, time-series data analysis, RNA-folding
MSCBIO 2070/02-710:, Spring 2015 A4: spline, HMM, clustering, time-series data analysis, RNA-folding Due: April 13, 2015 by email to Silvia Liu (silvia.shuchang.liu@gmail.com) TA in charge: Silvia Liu
More informationCOMP90051 Statistical Machine Learning
COMP90051 Statistical Machine Learning Semester 2, 2016 Lecturer: Trevor Cohn 20. PGM Representation Next Lectures Representation of joint distributions Conditional/marginal independence * Directed vs
More informationResearch Article An Improved Scoring Matrix for Multiple Sequence Alignment
Hindawi Publishing Corporation Mathematical Problems in Engineering Volume 2012, Article ID 490649, 9 pages doi:10.1155/2012/490649 Research Article An Improved Scoring Matrix for Multiple Sequence Alignment
More informationBrief review from last class
Sequence Alignment Brief review from last class DNA is has direction, we will use only one (5 -> 3 ) and generate the opposite strand as needed. DNA is a 3D object (see lecture 1) but we will model it
More informationPattern Recognition. Kjell Elenius. Speech, Music and Hearing KTH. March 29, 2007 Speech recognition
Pattern Recognition Kjell Elenius Speech, Music and Hearing KTH March 29, 2007 Speech recognition 2007 1 Ch 4. Pattern Recognition 1(3) Bayes Decision Theory Minimum-Error-Rate Decision Rules Discriminant
More informationBLAST: Basic Local Alignment Search Tool Altschul et al. J. Mol Bio CS 466 Saurabh Sinha
BLAST: Basic Local Alignment Search Tool Altschul et al. J. Mol Bio. 1990. CS 466 Saurabh Sinha Motivation Sequence homology to a known protein suggest function of newly sequenced protein Bioinformatics
More informationEfficient Implementation of a Generalized Pair HMM for Comparative Gene Finding. B. Majoros M. Pertea S.L. Salzberg
Efficient Implementation of a Generalized Pair HMM for Comparative Gene Finding B. Majoros M. Pertea S.L. Salzberg ab initio gene finder genome 1 MUMmer Whole-genome alignment (optional) ROSE Region-Of-Synteny
More informationCISC 636 Computational Biology & Bioinformatics (Fall 2016)
CISC 636 Computational Biology & Bioinformatics (Fall 2016) Sequence pairwise alignment Score statistics: E-value and p-value Heuristic algorithms: BLAST and FASTA Database search: gene finding and annotations
More informationData Mining Technologies for Bioinformatics Sequences
Data Mining Technologies for Bioinformatics Sequences Deepak Garg Computer Science and Engineering Department Thapar Institute of Engineering & Tecnology, Patiala Abstract Main tool used for sequence alignment
More informationk-means demo Administrative Machine learning: Unsupervised learning" Assignment 5 out
Machine learning: Unsupervised learning" David Kauchak cs Spring 0 adapted from: http://www.stanford.edu/class/cs76/handouts/lecture7-clustering.ppt http://www.youtube.com/watch?v=or_-y-eilqo Administrative
More informationDynamic Programming User Manual v1.0 Anton E. Weisstein, Truman State University Aug. 19, 2014
Dynamic Programming User Manual v1.0 Anton E. Weisstein, Truman State University Aug. 19, 2014 Dynamic programming is a group of mathematical methods used to sequentially split a complicated problem into
More informationSAPLING: Suffix Array Piecewise Linear INdex for Genomics Michael Kirsche
SAPLING: Suffix Array Piecewise Linear INdex for Genomics Michael Kirsche mkirsche@jhu.edu StringBio 2018 Outline Substring Search Problem Caching and Learned Data Structures Methods Results Ongoing work
More informationAn Introduction to Hidden Markov Models
An Introduction to Hidden Markov Models Max Heimel Fachgebiet Datenbanksysteme und Informationsmanagement Technische Universität Berlin http://www.dima.tu-berlin.de/ 07.10.2010 DIMA TU Berlin 1 Agenda
More informationStatistical Methods for NLP
Statistical Methods for NLP Information Extraction, Hidden Markov Models Sameer Maskey * Most of the slides provided by Bhuvana Ramabhadran, Stanley Chen, Michael Picheny Speech Recognition Lecture 4:
More informationLecture 3: February Local Alignment: The Smith-Waterman Algorithm
CSCI1820: Sequence Alignment Spring 2017 Lecture 3: February 7 Lecturer: Sorin Istrail Scribe: Pranavan Chanthrakumar Note: LaTeX template courtesy of UC Berkeley EECS dept. Notes are also adapted from
More informationToPS User Guide. André Yoshiaki Kashiwabara Ígor Bonadio Vitor Onuchic Alan Mitchell Durham
ToPS User Guide André Yoshiaki Kashiwabara Ígor Bonadio Vitor Onuchic Alan Mitchell Durham January de 2013 ii Contents 1 Introduction 1 1.1 Supported Features................................ 1 2 Build
More informationDynamic Programming. Ellen Feldman and Avishek Dutta. February 27, CS155 Machine Learning and Data Mining
CS155 Machine Learning and Data Mining February 27, 2018 Motivation Much of machine learning is heavily dependent on computational power Many libraries exist that aim to reduce computational time TensorFlow
More informationGraphical Models & HMMs
Graphical Models & HMMs Henrik I. Christensen Robotics & Intelligent Machines @ GT Georgia Institute of Technology, Atlanta, GA 30332-0280 hic@cc.gatech.edu Henrik I. Christensen (RIM@GT) Graphical Models
More informationExpectation Maximization. Machine Learning 10701/15781 Carlos Guestrin Carnegie Mellon University
Expectation Maximization Machine Learning 10701/15781 Carlos Guestrin Carnegie Mellon University April 10 th, 2006 1 Announcements Reminder: Project milestone due Wednesday beginning of class 2 Coordinate
More informationDocumentation of HMMEditor 1.0
Documentation of HMMEditor 1.0 HMMEditor 1.0 stands for profile Hidden Markov Model (phmm) Visual Editor. It is a tool to visualize and edit phmm in HMMer format. HMMer format is also used by Pfam protein
More informationCS273: Algorithms for Structure Handout # 4 and Motion in Biology Stanford University Thursday, 8 April 2004
CS273: Algorithms for Structure Handout # 4 and Motion in Biology Stanford University Thursday, 8 April 2004 Lecture #4: 8 April 2004 Topics: Sequence Similarity Scribe: Sonil Mukherjee 1 Introduction
More informationQuiz Section Week 8 May 17, Machine learning and Support Vector Machines
Quiz Section Week 8 May 17, 2016 Machine learning and Support Vector Machines Another definition of supervised machine learning Given N training examples (objects) {(x 1,y 1 ), (x 2,y 2 ),, (x N,y N )}
More informationMetaPhyler Usage Manual
MetaPhyler Usage Manual Bo Liu boliu@umiacs.umd.edu March 13, 2012 Contents 1 What is MetaPhyler 1 2 Installation 1 3 Quick Start 2 3.1 Taxonomic profiling for metagenomic sequences.............. 2 3.2
More informationAlignment of Long Sequences
Alignment of Long Sequences BMI/CS 776 www.biostat.wisc.edu/bmi776/ Spring 2009 Mark Craven craven@biostat.wisc.edu Pairwise Whole Genome Alignment: Task Definition Given a pair of genomes (or other large-scale
More information8/19/13. Computational problems. Introduction to Algorithm
I519, Introduction to Introduction to Algorithm Yuzhen Ye (yye@indiana.edu) School of Informatics and Computing, IUB Computational problems A computational problem specifies an input-output relationship
More informationGlobal Alignment Scoring Matrices Local Alignment Alignment with Affine Gap Penalties
Global Alignment Scoring Matrices Local Alignment Alignment with Affine Gap Penalties From LCS to Alignment: Change the Scoring The Longest Common Subsequence (LCS) problem the simplest form of sequence
More informationHidden Markov Model II
Hidden Markov Model II A brief review of HMM 1/20 HMM is used to model sequential data. Observed data are assumed to be emitted from hidden states, where the hidden states is a Markov chain. A HMM is characterized
More informationHARDWARE ACCELERATION OF HIDDEN MARKOV MODELS FOR BIOINFORMATICS APPLICATIONS. by Shakha Gupta. A project. submitted in partial fulfillment
HARDWARE ACCELERATION OF HIDDEN MARKOV MODELS FOR BIOINFORMATICS APPLICATIONS by Shakha Gupta A project submitted in partial fulfillment of the requirements for the degree of Master of Science in Computer
More informationCSCI 599 Class Presenta/on. Zach Levine. Markov Chain Monte Carlo (MCMC) HMM Parameter Es/mates
CSCI 599 Class Presenta/on Zach Levine Markov Chain Monte Carlo (MCMC) HMM Parameter Es/mates April 26 th, 2012 Topics Covered in this Presenta2on A (Brief) Review of HMMs HMM Parameter Learning Expecta2on-
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 5 Inference
More informationConditional Random Fields and beyond D A N I E L K H A S H A B I C S U I U C,
Conditional Random Fields and beyond D A N I E L K H A S H A B I C S 5 4 6 U I U C, 2 0 1 3 Outline Modeling Inference Training Applications Outline Modeling Problem definition Discriminative vs. Generative
More informationProtein Sequence Classification Using Probabilistic Motifs and Neural Networks
Protein Sequence Classification Using Probabilistic Motifs and Neural Networks Konstantinos Blekas, Dimitrios I. Fotiadis, and Aristidis Likas Department of Computer Science, University of Ioannina, 45110
More informationEECS730: Introduction to Bioinformatics
EECS730: Introduction to Bioinformatics Lecture 06: Multiple Sequence Alignment https://upload.wikimedia.org/wikipedia/commons/thumb/7/79/rplp0_90_clustalw_aln.gif/575px-rplp0_90_clustalw_aln.gif Slides
More informationNew String Kernels for Biosequence Data
Workshop on Kernel Methods in Bioinformatics New String Kernels for Biosequence Data Christina Leslie Department of Computer Science Columbia University Biological Sequence Classification Problems Protein
More informationA Visualization Tool to Improve the Performance of a Classifier Based on Hidden Markov Models
A Visualization Tool to Improve the Performance of a Classifier Based on Hidden Markov Models Gleidson Pegoretti da Silva, Masaki Nakagawa Department of Computer and Information Sciences Tokyo University
More information10-701/15-781, Fall 2006, Final
-7/-78, Fall 6, Final Dec, :pm-8:pm There are 9 questions in this exam ( pages including this cover sheet). If you need more room to work out your answer to a question, use the back of the page and clearly
More informationCS 6784 Paper Presentation
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data John La erty, Andrew McCallum, Fernando C. N. Pereira February 20, 2014 Main Contributions Main Contribution Summary
More informationMultiple Sequence Alignment II
Multiple Sequence Alignment II Lectures 20 Dec 5, 2011 CSE 527 Computational Biology, Fall 2011 Instructor: Su-In Lee TA: Christopher Miles Monday & Wednesday 12:00-1:20 Johnson Hall (JHN) 022 1 Outline
More informationAs of August 15, 2008, GenBank contained bases from reported sequences. The search procedure should be
48 Bioinformatics I, WS 09-10, S. Henz (script by D. Huson) November 26, 2009 4 BLAST and BLAT Outline of the chapter: 1. Heuristics for the pairwise local alignment of two sequences 2. BLAST: search and
More informationUser Guide Written By Yasser EL-Manzalawy
User Guide Written By Yasser EL-Manzalawy 1 Copyright Gennotate development team Introduction As large amounts of genome sequence data are becoming available nowadays, the development of reliable and efficient
More informationSequence Alignment (chapter 6) p The biological problem p Global alignment p Local alignment p Multiple alignment
Sequence lignment (chapter 6) p The biological problem p lobal alignment p Local alignment p Multiple alignment Local alignment: rationale p Otherwise dissimilar proteins may have local regions of similarity
More informationFastA & the chaining problem
FastA & the chaining problem We will discuss: Heuristics used by the FastA program for sequence alignment Chaining problem 1 Sources for this lecture: Lectures by Volker Heun, Daniel Huson and Knut Reinert,
More informationFastA and the chaining problem, Gunnar Klau, December 1, 2005, 10:
FastA and the chaining problem, Gunnar Klau, December 1, 2005, 10:56 4001 4 FastA and the chaining problem We will discuss: Heuristics used by the FastA program for sequence alignment Chaining problem
More informationGreedy Algorithms Huffman Coding
Greedy Algorithms Huffman Coding Huffman Coding Problem Example: Release 29.1 of 15-Feb-2005 of TrEMBL Protein Database contains 1,614,107 sequence entries, comprising 505,947,503 amino acids. There are
More informationHidden Markov Model for Sequential Data
Hidden Markov Model for Sequential Data Dr.-Ing. Michelle Karg mekarg@uwaterloo.ca Electrical and Computer Engineering Cheriton School of Computer Science Sequential Data Measurement of time series: Example:
More informationModeling time series with hidden Markov models
Modeling time series with hidden Markov models Advanced Machine learning 2017 Nadia Figueroa, Jose Medina and Aude Billard Time series data Barometric pressure Temperature Data Humidity Time What s going
More informationLearning Hidden Markov Models for Regression using Path Aggregation
Learning Hidden Markov Models for Regression using Path Aggregation Keith Noto Dept of Computer Science and Engineering University of California San Diego La Jolla, CA 92093 knoto@csucsdedu Abstract We
More informationToday. Lecture 4: Last time. The EM algorithm. We examine clustering in a little more detail; we went over it a somewhat quickly last time
Today Lecture 4: We examine clustering in a little more detail; we went over it a somewhat quickly last time The CAD data will return and give us an opportunity to work with curves (!) We then examine
More informationSequence Alignment. Ulf Leser
Sequence Alignment Ulf Leser his Lecture Approximate String Matching Edit distance and alignment Computing global alignments Local alignment Ulf Leser: Bioinformatics, Summer Semester 2016 2 ene Function
More informationHPC methods for hidden Markov models (HMMs) in population genetics
HPC methods for hidden Markov models (HMMs) in population genetics Peter Kecskemethy supervised by: Chris Holmes Department of Statistics and, University of Oxford February 20, 2013 Outline Background
More informationBiostrings. Martin Morgan Bioconductor / Fred Hutchinson Cancer Research Center Seattle, WA, USA June 2009
Biostrings Martin Morgan Bioconductor / Fred Hutchinson Cancer Research Center Seattle, WA, USA 15-19 June 2009 Biostrings Representation DNA, RNA, amino acid, and general biological strings Manipulation
More informationA NEW GENERATION OF HOMOLOGY SEARCH TOOLS BASED ON PROBABILISTIC INFERENCE
205 A NEW GENERATION OF HOMOLOGY SEARCH TOOLS BASED ON PROBABILISTIC INFERENCE SEAN R. EDDY 1 eddys@janelia.hhmi.org 1 Janelia Farm Research Campus, Howard Hughes Medical Institute, 19700 Helix Drive,
More informationLectures by Volker Heun, Daniel Huson and Knut Reinert, in particular last years lectures
4 FastA and the chaining problem We will discuss: Heuristics used by the FastA program for sequence alignment Chaining problem 4.1 Sources for this lecture Lectures by Volker Heun, Daniel Huson and Knut
More information