Multiple sequence alignment by Genetic Algorithms

Transcription

1 Bi04b_1 Unit 04b: Multiple sequence alignment by Genetic Algorithms...What Biology can do for Computer Science... Bi04b_2 9 Biological ideas used in Genetic Algorithms (GA) and Genetic Programming (GP) The Darwinian principle of survival of the fittest asexual mutation operation sexual recombination (crossover) operation inversion operation gene regulation gene duplication gene deletion embryos development of embryo into organism from Koza (1993) 1

2 Genetic Algorithm (GA) Bi04b_3 Definition The genetic algorithm is a mathematical algorithm that transforms a set (population) of mathematical objects (typically fixed-length binary character strings), each with an associated fitness value, into a new set (new generation of the population) of offspriing objects, using operations patterned after naturally-occurring genetic operations and the Darwinian principle of reproduction and survival of the fittest. The GA is a very general computational approach that can be tailored to solve optimization - or search tasks for very different problems and settings. from Koza (1993) Genetic Algorithm, Concepts Bi04b_4 Parent population Fitness score 3 1 individuals GENETIC OPERATIONS offspring population Fitness score loop 2

3 Types of individuals for GA Bi04b_ AZKLMN binary strings character strings tree structure technical constructions, differing in detail representations directly usable by GA begin loop add end begin add met end computer programs GAs on programs: genetic programming Types of individuals for GA, ctd. Bi04b_6 particular (special) fold of a protein a particular alignment of N msa via GAs sequences C B F E a particular journey for the travelling salesman A D 3

4 Types of individuals for GA, ctd. Bi04b_7 vascular tree supplying N sites of tissue Preparatory Step 1 to implement a GA: Recast the representation of individuals to strings or trees Fitness scores for types of individuals Bi04b_8 Parent population Fitness score 3 1 individuals GENETIC OPERATIONS offspring population Fitness score loop 4

5 Fitness scores for types of individuals, ctd. Bi04b_9 Problem to optimize Fitness Score, example maximum load - weight of material begin loop add end begin add met end rate of successful perceptions, TP, FP, TN, FN - Energy of protein Fitness scores for types of individuals, ctd. Bi04b_10 alignment score C B F E - Length of journey A D - Blood volume Preparatory Step 2 for implementing a GA: Define algorithm to compute fitness score 5

6 Genetic operations for GA Bi04b_11 Parent population Fitness score 3 offspring population Fitness score 4 individuals 1 4 GENETIC OPERATIONS loop Genetic operations Bi04b_12 Darwinian reproduction (copy operation): ACKBDF ACKBDF Individuals with higher fitness (F) are stochastically chosen more likely, e.g. via p 1-e -kf best individuals are not necessarily chosen worst individual is not necessarily excluded A certain fraction of population undergoes reproduction (either exact or randomly selected) sexual crossover asexual Mutation 6

7 Genetic operations, ctd. Bi04b_13 Darwinian reproduction (asexual copy operation) sexual crossover from Brown (1999) asexual mutation high resolution Genetic operations, ctd. Bi04b_14 Darwinian reproduction (asexual copy operation) sexual crossover high resolution asexual mutation 7

8 Genetic operations, ctd. Bi04b_15 Darwinian reproduction (asexual copy operation) sexual crossover The predominant operation with GAs A certain fraction of individuals goes into mating pool based on fitness. Or: tournament selection: mate best bull with best cow. Two parental individuals (strings, trees) are chosen based on fitness Pick a point in the genome (the same for both parents) to become the recombinant joint asexual mutation Branch migration shifts recombinant joint Bi04b_16 Darwinian reproduction (asexual copy operation) sexual crossover high resolution asexual mutation 8

9 sexual crossover, details Bi04b_17 for string representation: pick position of recombinant joint stochastically between 1 and L-1 (L = length of genome representation string) father L-1 L mother L-1 L parential string chromosomes join recombinants... L-1 L... L-1 L offspring string chromosomes sexual crossover with strings Bi04b_18 gene 1 gene 2 gene 3 gene 4 gene 5 gene 6 A K B L Y R T S A B L M M L L K A S R R A A A L L B M N N L red color long hair 3 legs B L L L B A R T S A L M M M A A K S A A L A B B L M M N A L blue color short hair 4 legs B L L L B A R T S A L M M L L K A S R R A A A L L B M N N L blue color short hair 3 legs A K B L Y R T S A B L M M M A A K S A A L A B B L M M N A L red color long hair 4 legs color, hair length color, # of legs in this example: inherited together (linked features) in this example: inherited separately (independent) 9

10 sexual crossover with strings, ctd. Bi04b_19 gene 1 gene 2 gene 3 gene 4 gene 5 gene 6 A K B L Y R T S A B L M M L L K A S R R A A A L L B M N N L red color long hair 3 legs B L L L B A R T S A L M M M A A K S A A L A B B L M M N A L blue color short hair 4 legs B L L L B A R T S A L M M L L K A S R R A A A L L B M N N L blue color short hair 3 legs A K B L Y R T S A B L M M M A A K S A A L A B B L M M N A L red color long hair 4 legs features on genes close to each other: features on genes lying far apart: likely to be transmitted linked to each other more likely to be disrupted and transmitted independently from each other sexual crossover with strings, ctd. Bi04b_20 From model we can see: features on genes close to each other: features on genes lying far apart: likely to be transmitted linked to each other more likely to be disrupted and transmitted independently from each other For real genetics: reverse the argument to define gene-distance: observe frequency for linked VS: independent transmittence of features derive a measure of gene-distance (CM: centi Morgan) within genome-maps (physical maps) 10

11 Gene Distance and Gene Maps Bi04b_21 from Brown (1999) high resolution Zwei Gene, die relativ eng benachbart auf einem Chromosom liegen, werden durch ein Crossingover mit geringerer Wahrscheinlichkeit entkoppelt als solche, die weiter voneinander entfernt sind. Weiße Augen (w) und gelbe Körper (y) rekombinieren deshalb seltener als weiße Augen und kleine Flügel (m). Sturtevants Karte für fünf Gene des X-Chromosoms von Drosophila. Abkürzungen: y, gelber Körper; w, weiße Augen; v, zinnoberrote Augen m, kleine Flügel; r, rudimentäre Flügel. GAs & Co-adapted Sets of genes Bi04b_22 Genes close to each other on the chromosome are less likely to be saparated by a crossover. Therefore place things adjacent if they are a good combination (e.g. long legs and long neck). ability of the GA to solve the problem depends on this kind of choices. In nature: if cooperative beneficial features get together close (due to crossover) they are from then on inherited more effectively together (called inversion ). General idea how GAs work: they generate coadapted pairs that tend to get commoted in the population from Koza (1993) 11

12 Caveats regarding analogy to natural crossover Bi04b_23 real genes usually stand for a feature (color) which is expressed/ not expressed (but they don t toggle between features, e.g. colors). GA-genomes normally toggle. In addition to crossover - modeled after naturein GAs many other artificial genetic operators may be freely designed to fit the representation and perform specifically suitable jobs in optimization (will be shown in examples) Mutation Operation Bi04b_24 VERY occasional - maybe 1 bit/character per generation Choose one parental string (asexual) based on fitness. Pick point from 1 to L (using a uniform random distribution) A L L M A A K parent from Koza (1993) A L K M A A K offspring Mutation is a localized search, changing one factor only! Similar to Monte Carlo Move in Gibbs sampling mode! Point 3 chosen and mutated 12

13 GA example run (4 individuals, 3-dimensional optimization) Select individuals of population for mating mating pool by chance, according to fitness Bi04b_25 Selected NOT selected individual Generation 0 Mating pool for sexual crossover Generation 1 genome fitness prob genome fitness genome fitness prob Total Worst Average Best selected selected twice from Koza (1993) GA example run (4 individuals, 3-dimensional optimization) Perform genomic operations on members of mating pool by chance, according to fitness Bi04b_26 crossover copy individual Generation 0 Mating pool for sexual crossover Generation 1 genome fitness prob genome fitness genome fitness prob Total Worst Average Best mutation from Koza (1993) 13

14 Genetic Algorithms are probabilistic Bi04b_27 Creation of the initial random population (generation 0) (uniform distribution) Probabilistic selection of participant(s) for the genetic operation (unequal probabilities, based on fitness) Probabilistic selection of the type of operation (unequal probabilities) Probabilistic selection of crossover or mutation point (equal or unequal probabilities) (Often) probabilistic selection of fitness cases (uniform distribution) In each run you get a different answer: A GA is a multi-run-algo from Koza (1993) Promising GA Application Areas Bi04b_28 Problem areas involving many variables that are interrelated in highly non-linear ways Problem areas involving many variables whose interrelationship is not well understood Problem areas where a good approximate solution is satisfactory (and no one is expecting a perfect solution) design control classification, pattern recognition, image processing forecasting model building and data mining from Koza (1993) 14

15 Promising GA/GP Areas, ctd. Bi04b_29 Problem areas where discovery of the size and shape of the solution is a major part of the problem Problem areas where large computerized databases are accumulating and computerized techniques are needed to analyze the data genome and protein sequences satellite data astronomy petroleum financial databases marketing databases World Wide Web from Koza (1993) Promising GA/GP Application Areas, ctd. Bi04b_30 Problem areas for which human find it very difficult to write good programs... from Koza (1993) 15

16 Multiple sequence alignment by GA Program SAGA (after Notredame C, Higgins DG: SAGA: sequence alignment by genetic algorithm. Nucleic Acids Res. 1996;24: ) 1524) Bi04b_31 Initialisation Evaluation Breeding 1. create G 0 2. evaluate the population of generation n (G n ) 3. if the population is stabilised then End 4. select the individuals to replace 5. evaluate the expected number of offspring (EO) for each individual (fitness based) 6. select the parent(s) from G n 7. select the operator 8. generate the new child 9. keep or discard the new child in G n+1 End 10. goto 6 until all the children have been successfully put into G n n = n goto Evaluation 13. end 1. Initialisation in SAGA Bi04b_32 Initialisation: generate population of 100 random alignments each individual (alignment) contains only terminal gaps chose random offset (e.g. between 0 and 50 for each sequence) pad with leading and trailing gaps to make all sequences equally long. 16

17 Initialisation, example Bi04b_33 A 1 A 2 offset =1 A m WGKVNVDEVGGEAL- --WDKVNEEEVGGEAL-- -WGKVGAHAGEYGAEAL- --WSKVGGHAGEYGAEAL --WGKVNVDEVGGEAL-- WDKVNEEEVGGEAL---- -WGKVGAHAGEYGAEAL- --WSKVGGHAGEYGAEAL offset =4 offset =2 WGKVNVDEVGGEAL WDKVNEEEVGGEAL- WGKVGAHAGEYGAEAL-- -WSKVGGHAGEYGAEAL- possible realization: find length L max of longest sequence choose unique length of all alignments L A > L max for each sequence with length L i in alignment: toss for offset from equal probabilities between 1 offset (L A -L i ) Evaluation in SAGA Bi04b_34 Given a multiple alignment A of N sequences: Fitness = 1 Alignment cost ( A) = N i 1 Wij cost Aij i= 2 j= 1 = 1. ( ) weight cost of pairwise alignment cost(a ij ) is computed from substitution matrices (PAMs, BLOSUMs) with affine gap penalities (variants exist for vis à vis gaps) 17

18 2.-5. Evaluation in SAGA, ctd. Bi04b_ choose the best 50% of alignments of G n for direct copy operation (i.e. will make up half of the next generation this is a method of overlapping generations individuals not selected for copy will be replaced) evaluate for each alignment in G n the expected number of offspring (EO) based on fitness. Usually 0 EO Breeding in SAGA (genetic OP other than copy) Bi04b_ stochastically (proportional to EO, unequal probabilities) select parents from G n for the mating pool stochastically select an operator (operators are specifically designed for a chosen representation scheme, see below). apply operator to (1 or 2) parent(s) and generate children check children for duplicats within generation G n+1 : if duplicates occur then discard parents & children and repeat from item 6. repeat until enough children (e.g. 50% of population) are generated 18

19 3. Termination condition in SAGA Bi04b_37 there is no theoretically proven criterium for convergence heuristic stop, if no improvement found over the last 100 generations Closeups on Operators in SAGA Bi04b_38 [ traditional operators: sexual crossover ] asexual mutation GA-operators specifically designed for multiple sequence alignments crossover (2 different types) gap insertion block shuffling block searching local optimal or sub-optimal rearrangement 2 modes of usage for each operator: stochastic semi hill-climbing 19

20 One point crossover in SAGA Bi04b_39 acts on 2 parent alignments (sexual) 1 st parent is cut straight at random position 2 nd parent is tailored to let pieces fit together 2 different children may be produced spaces at junction are filled with gaps at random (stochastic mode) keep child with better score (semi hill climbing mode) (this) operator is both: crossover + mutation operator may disrupt coherent parts of a sequence One point crossover in SAGA, ctd. Bi04b_40 Parent Alignment 1 Parent Alignment 2 WGKVN---VDEVGGEAL- WDKVNEEE---VGGEAL- WGKVG--AHAGEYGAEAL WSKVGGHA--GEYGAEAL --WGKVNVDEVG-GEAL WD--KVNEEEVG-GEAL WGKVGA-HAGEYGAEAL WSKVGGHAGE-YGAEAL WGKV--NVDEVG-GEAL WDKV--NEEEVG-GEAL WGKVGA-HAGEYGAEAL WSKVGGHAGE-YGAEAL + --WGKVN---VDEVGGEAL- WD--KVNEEE---VGGEAL- WGKV--G--AHAGEYGAEAL WSKV--GGHA--GEYGAEAL Child Alignment 1 Child Alignment 2 WGKV--NVDEVG-GEAL WDKV--NEEEVG-GEAL WGKVGA-HAGEYGAEAL WSKVGGHAGE-YGAEAL from Notredame (1996) Chosen Child Alignment 20

21 Uniform crossover in SAGA Bi04b_41 acts on 2 parent alignments (sexual) designed after natural crossover (multiple) exchanges between parents are promoted between zones of homology implemented modes: stochastic or semi-hillclimbing Uniform crossover in SAGA, ctd. Bi04b_42 Parent Alignment 1 * * WG K VNVDEV-- G GEAL WD K VNEEEV-- G GEAL WG K VGAHAGEY G AEAL WS K VGGHAGEY G AEAL Parent Alignment 2 * * WG- K V--NVDEV G GE-AL W-D K V--NEEEV G G-EAL W-G K VGAHAGEY G AEA-L -WS K VGGHAGEY G AEAL- * * WG K V--NVDEV G GEAL WD K V--NEEEV G GEAL WG K VGAHAGEY G AEAL WS K VGGHAGEY G AEAL + * Position consistent K between the two parents * * WG- K VNVDEV-- G GE-AL W-D K VNEEEV-- G G-EAL W-G K VGAHAGEY G AEA-L -WS K VGGHAGEY G AEAL- Child Alignment 1 Child Alignment 2 from Notredame (1996) WGKV--NVDEVG-GEAL WDKV--NEEEVG-GEAL WGKVGA-HAGEYGAEAL WSKVGGHAGE-YGAEAL Chosen Child Alignment 21

22 Gap insertion operator in SAGA Bi04b_43 acts on 1 parent alignment (asexual) split sequences into 2 groups (G 1, G 2 ) (ideally derived from phylogenetic tree of sequences) choose insertion point P1 randomly insert random number of gaps at P1 into all sequences G 1 chooe insertion point P2 randomly insert same # of gaps at P2 into all sequences G2 all sequences increase in length by chosen # of gaps Above stochastic mode of operator can be made semi hill climbing by selecting everything randomly as above, except for P1. Try all possible P1 and take best. Gap insertion operator in SAGA, ctd. Bi04b_44 Insertion of the gaps in the parent alignment (stochastic mode) P1 gaps in G1 seq1 seq2 seq3 seq4 seq5 WGKVNVDEVGGEA-GL WDKVNEEEVGGEA-GL WGKVGAHAGEYGAEAL WSKVGGHAGEYGAEAL WAKVEADVAGHGQDIL G1 G2 WGKV--NVDEVGGEA-GL WDKV--NEEEVGGEA-GL WGKVGAHAGEYGAEAL-- WSKVGGHAGEYGAEAL-- WAKVEADVAGHGQDIL-- gaps in G2 P2 from Notredame (1996) 22

23 Gap insertion operator in SAGA, ctd. Bi04b_45 Insertion of the gaps in the parent alignment (semi hill climbing mode) seq1 seq2 seq3 seq4 seq5 from Notredame (1996) sliding P1 P1 WGKVNVDEVGGEA-GL WDKVNEEEVGGEA-GL WGKVGAHAGEYGAEAL WSKVGGHAGEYGAEAL WAKVEADVAGHGQDIL P2 G1 G2 For each possible position of P1 generate 1 child! --WGKVNVDEVGGEA-GL --WDKVNEEEVGGEA-GL WGKVGAHAGEYGAEAL-- WSKVGGHAGEYGAEAL-- WAKVEADVAGHGQDIL-- W--GKVNVDEVGGEA-GL W--DKVNEEEVGGEA-GL WGKVGAHAGEYGAEAL-- WSKVGGHAGEYGAEAL-- WAKVEADVAGHGQDIL--... WGKVNVDEVGGEA-GL-- WDKVNEEEVGGEA-GL-- WGKVGAHAGEYGAEAL-- WSKVGGHAGEYGAEAL-- WAKVEADVAGHGQDIL-- select optimum child Block shuffling in SAGA Bi04b_46 block definition, modified for shuffling operators: block = set of overlapping stretches of residues, each being delimited by a gap or by an end of sequence WGKVN--VDEVGGEAL WGKVGAHAGEYGAEAL WDKV--NEEEVGGEAL WSKVGGHAGEYGAEAL WAKVEADVAGHGQDIL WGKVN--VDEVGGEAL WGKVGAHAGEYGAEAL WDKV--NEEEVGGEAL WSKVGGHAGEYGAEAL WAKVEADVAGHGQDIL from Notredame (1996) 23

24 Block shuffling in SAGA, ctd. Bi04b_47 Shuffling type 1: Move a full block of gaps (or a full block of residues). WGKVN--VDEVGGEAL WGKVGAHAGEYGAEAL WDKV--NEEEVGGEAL WSKVGGHAGEYGAEAL WAKVEADVAGHGQDIL WGKV--NVDEVGGEAL WGKVGAHAGEYGAEAL WDK--VNEEEVGGEAL WSKVGGHAGEYGAEAL WAKVEADVAGHGQDIL Example shown: Move block of gaps to left by 1. from Notredame (1996) Block shuffling in SAGA, ctd. Bi04b_48 Shuffling type 2: Split the block horizontally and move one of the sub blocks to the left or to the right. The subdivision of a block is made according to the tree (cf. gap insertion operator). WGKVN--VDEVGGEAL WGKVGAHAGEYGAEAL WDKV--NEEEVGGEAL WSKVGGHAGEYGAEAL WAKVEADVAGHGQDIL Example: Shift left by 1 for block in group G1 Shift nothing for block in group G2 WGKV--NVDEVGGEAL WGKVGAHAGEYGAEAL WDKV--NEEEVGGEAL WSKVGGHAGEYGAEAL WAKVEADVAGHGQDIL seq1 seq2 seq3 seq4 seq5 G1 G2 from Notredame (1996) 24

25 Block shuffling in SAGA, ctd. Bi04b_49 Shuffling type 3: Split the block vertically and move one half to the left or to the right. The move can be made stochastic or in a semi-hill climbing way, looking for the best position. WGKVN--VDEVGGEAL WGKVGAHAGEYGAEAL WDKV--NEEEVGGEAL WSKVGGHAGEYGAEAL WAKVEADVAGHGQDIL WGKVNV--DEVGGEAL WGKVGAHAGEYGAEAL WDKV-N-EEEVGGEAL WSKVGGHAGEYGAEAL WAKVEADVAGHGQDIL from Notredame (1996) Block searching & moving operator, ctd. Bi04b_50...Therefore we designed a crude method that... [ refer to conventional definition of block] select substring of (random length) in 1 sequence search all other sequences for best match to substring add substring found to the old one to generate a profile in each of the other sequences find string best matching to profile and add it to profile. Search extends only over a window around profile move strings inside sequences to reconstruct block 25

26 Block searching & moving operator, ctd....therefore we designed a crude method that... Bi04b_51 No further description given in original paper...inspecting the above derivation one can easily see that... This block searching mutation generates more dramatic changes than any of the other operators. Local optimal & suboptimal rearrangement Bi04b_52 Authors suggest additional and very heuristic manipulations, such as: optimize gaps inside a given block exhaustive (all possibilities) examination local alignment via genetic algorithm (LAGA) 26

27 Dynamic scheduling of operators in SAGA Bi04b_53 22 operators in total initially each operator has probability 1/22 computes running averages of efficiency for each operator based on improvement achieved include last operator, second last, etc. with decreasing weights operator usage probability: total improvement number of children created p = max (p, p min ), i.e. apply each operator at least with minimum usage probability Self tuning of operator selection in SAGA Bi04b_54 control chart to monitor usage from Notredame (1996) 27

28 Performance of SAGA, Summary Bi04b_55 very satisfactory on test cases compared to other programs (e.g. CLUSTALW) satisfying and even superior to others when checked with alignments based on 3D-structure (golden Standard) the more sophisticated operators are essential, SAGA does not work with simple mutation & crossover! 28