Computational biology course IST 2015/2016

Transcription

1 Computational biology course IST 2015/2016

2 Introduc)on to Algorithms! Algorithms: problem- solving methods suitable for implementation as a computer program! Data structures: objects created to organize data for processing Algotithms and Data structures go hand in hand 2

3 Introduc)on to Algorithms! Strategy:! Specification (define properties)! Arquitecture (algorithm and data structures)! Algorithm analysis (time and memory)! Implementation (programming language)! Test (define inputs to verify defined properties)! The choice of the best algorithm for a particular task can be a complicated process, perhaps involving sophisticated mathematical analysis 3

4 Algorithm Analysis! We should not use an algorithm without having an idea of what resources it might be expected to perform! One of the first steps is to do empirical analysis:! Given two algorithms to solve the same problem, run them both to see which one takes longer! First challenge:! Develop a correct and complete implementation! Second challenge:! Determine the nature of the input data and other factors that have direct influence on the experiments to be performed! What input data?! Actual data; Random data; Perverse data 4

5 Algorithm Analysis! When empirical studies start to consume a significant amount of time, mathematical analysis is called for! Why perform mathematical analysis of algorithms:! To compare different algorithms for the same task! To predict performance in a new environment! To set values of algorithms parameters 5

6 Algorithm Analysis! The first step is to separate the analysis from the implementation, that is, to identify the abstract operations on which the algorithm is based:! How many times one execute the code fragment i = a[x]! Property of the algorithm! How many nanoseconds might be required to execute that particular code fragment on our computer! Property of the computer! Although the number of abstract operations involved can be large, in principle, the performance of an algorithm typically depends on only a few quantities 6

7 Growth of Func)ons! Most algorithms have a primary parameter N that affects the running time most significantly:! The degree of a polynomial! The size of a file to be sorted and searched! The number of characters in a text string! Or some other abstract measure of the size of the problem being considered! The analysis is often reduced to just one parameter by expressing one of the parameters as a function of the other or by considering one parameter at a time, holding the other constant 7

8 Growth of Func)ons! The algorithms typically have running times proportional to one of the following functions: 1 - Most instructions of most programs are executed once or at most only a few times - If all the instructions of the program have this property, the program s running time is constant log N - The running time of the program is logarithmic - The program gets slightly slower as N grows - Whenever N doubles, logn increases by a constante, but N logn does not double until N increases to N 2 - The running time of the program is linear - This situation is optimal for an algorithm that must process N inputs (or produce N outputs) 8

9 Growth of Func)ons N log N This running time arises when algorithms solve a problem by breaking it up into smaller subproblems, solving them independently, and them combining the solutions - If N is 1 million N log N is perhaps 20 millions N 2 - The running time is quadratic - Arise in algorithms that process all pairs of data items - Pratical for use on only relatively small problems N 3 - The program running time is cubic - If N = 100, N 3 = 1 million (ex: matrices product) 2 N - Few algorithms with exponential running time are likely to be appropriate for pratical use - Typical in brute- force solutions - If N = 20, 2 N = 1 million 9

10 Growth of Func)ons Other examples: Regard lg lg N as a constant lg lg = 8 Values of commonly encountered functions lg N N N N lg N N (lg N) 2 N 3/2 N

11 Growth of Func)ons Seconds conversions seconds minutes hours days weeks months years decades centuries never 11

12 Big- Oh Nota)on! The mathematical artifact that allows us to suppress detail when we are analyzing algorithms is called the O- notation! Definition: A function g(n) is said to be O(f(N)) if there exist constants c 0 and N 0 such that g(n) < c 0 f(n) for all N > N 0! The O- notation is used for three distinct purposes:! To bound the error that we make when we ignore small terms in mathematical formulas! To bound the error that we make when we ignore parts of a program that contribute a small amount to the total being analyzed! To allow us to classify algorithms according to upper bounds on their total running times 12

13 Bounding a func)on with an O- approxima)on! The oscillating curve represents a function, g(n), which we are trying to approximate N 0 c 0 f(n)! g(n) = O(f(N)) - the value of g(n) falls below some curve the shape f() to the right of some vertical line g(n) f(n) 13

14 Big- Oh Nota)on! Transformations in an expression that uses the O- notation, some examples:! f(n) O(f(N))! c. O(f(N)) = O(c. f(n)) = O(f(N))! O(f) + O(g) = O(f+g) = O(max(f,g))! O(f). O(g) = O(f. g)! O(f) + O(g) = O(f) if g(n) f(n) for N > N 0 A formula with one O- term is named an asymptotic expression 14

15 Examples of Algorithm Analysis! Considere the following C statement: for (i = 0; i < N; i++) { instructions; }! The running time is O(N)! Considere the following C statement: for (i = 0; i < N; i++) { for (j = 0; j < N; j++) { instructions; } }! The running time is O(N 2 ) 15

16 Examples of Algorithm Analysis! Considere the following code: for (i = 0; i < N; i++) { for (j = i; j < N; j++) { instructions; } } The internal loop does N + (N-1) + (N-2) = N(N+1)/2 O(N 2 ) 16

17 Basic Recurrences! Many algorithms are based on the principle of recursively decomposing a large problem into one or more smaller ones, using solutions to the subproblems to solve original problem int factorial (int N) { if (N == 0) return 1; return N*factorial(N-1); } 17

18 Basic Recurrences! Considere the following program: int func (int a[], int N){ several instructions ; a = func(a, N-1); }! This formula arises for a program that loops through the input to eliminate one item: C N = C N- 1 + f(n) recurrence relation! 18

19 Basic Recurrences C N = C N-1 + N for N 2 with C 1 = 1 Solution: C N is about N 2 /2 Proof: We telescope the equation by applying it to itself Important: The recursive decomposition in an algorithm is directly reflected in its analysis C N = C N 1 + N = C N 2 + (N 1) + N = C N 3 + (N 2) + (N 1) + N! = C (N 2) + (N 1) + N = (N 2) + (N 1) + N N N +1 = ( ) 2 19

20 Examples Example 1: N = 10 3 M = 10 3 Algorithm O(NM) Machine: 1GHz and 1Gb RAM (10 9 steps/second)! Example 2:! N = 10 9! M = 10 9! Algorithm O(NM) O(N logm)! Machine: 1GHz and 1Gb RAM (10 9 steps/second) Time: 1ms! Time: 10 9 s => 30 years 9s 20

21 Algorithm Analysis! Considere the analysis of sequential search and binary search algorithms! Compare these algorithms! Imagine a credit- card company that has N credit risks or stolen credit cards, and wants to check whether any of M given transactions involves any one of the N bad numbers! N on the order of 10 3 to 10 6! M on the order of 10 6 to

22 Sequen)al search This function checks whether the number v is among a previously stored set of numbers in a[l],...,a[r] int search (int a[], int v, int l, int r) { int i; for (i = l; i <= r; i++) if (v == a[i]) return i ; return 1; } 22

23 Sequen)al search The running time depends on the data Depends on whether or not the object is in the array Depends on the object position, first or last position in the array To make a prediction we need to make an assumption about the data All the numbers are randomly chosen Separate successful and unsuccessful cases 23

24 Sequen)al search Property: Sequential search examines N numbers for each unsuccessful search and about N/2 for each successful search on average! The running time is proportional to N! For each input element! The total cost is O(MN)! How to improve the algorithm?! Put the numbers in the table in order (sorting)! N numbers for each search in the worst case! N/2 numbers for each search in average (sucess or not) 24

25 Binary Search Classical solution to the search problem that is more efficient than sequential search Idea: If the numbers in the table are in order, we can eliminate half of them from consideration by comparing the one that we seek with the one at the middle position in the table If it is equal, it is a successful search If it is less, apply the method to the left half If it is greater, apply the method to the right half 25

26 Binary Search int search (int a[], int v, int l, int r) { while (r >= l) { int m = (l+r) / 2 ; } if (v == a[m]) return m ; if (v < a[m]) r = m-1; else l = m+1 ; } return 1 ; 26

27 Binary Search Property: Binary search never examines more than lg N + 1 numbers Proof: (illustrates the use of recurrence relations) T N T N/2 + 1 for N 2 with T 1 = 1! Solve a huge search problem with up to 1 million numbers with at most 20 comparisons per transaction! The search problem is so important that several methods have been developed that are even faster than this one 27

28 Data Structures! Basic types! Integers (int Index);! Floating- point (float Stock, double Ind);! Characteres (char Answer)! Grouping data (static or dinamic allocation)! Arrays ( elements of the same type)! int Array1[N]! Two- dimensional (Array2[N 1 ]...[N k ] ou Array2[N 1,...,N k ])! Heaps (Heap[N])! Structures (mix different types of data) 28

29 Data Structures! Dynamic: grow and shrink during their lifetime! Lists! Linked lists! Stack, Queue, Priority queue! Doubly- linked, circular list! Trees! Graphs! Compound types! Arrays of arrays; arrays of structs; Lists of structs; Multilist,... 29

30 Array Definition: An array is a fixed collection of same-type data that are stored contiguously and that are accessible by an index ex: int Tab1[4]; /* 4 integers array */ char Tab2[3][2]; /* 3 by 2 array of characters */ char Tab3[5][14][2]; /* 5 by 14 by 2 array of characters */ Tab1[0] Tab1[1] Tab1[2] Tab1[3] 30

31 Linked List Definition: A linked list is a set of items where each item is part of a node that also contains a link to a node head item item item typedef struct _list { Item item; struct _list * next; } list; null link 31

32 Data Structures! Organizing the data for processing is an essential step in the development of a computer program! For the same data, some data structures require more or less space than others! For the same operations on the data, some data structures lead to more or less efficient algorithms than others! Main operations:! Access and Modify - > Arrays! Insert and Remove - > Linked Lists 32

33 Access and Modify Access/Modify Array List First O(1) O(1) Last O(1) O(k 1 ) Just mantain a pointer to the head node Next O(1) O(1) Specified O(1) O(k 2 ) k 1 total number of elements k 2 specific element position (k1 k2) 33

34 Inser)on Insertion Array List First O(k 1 ) O(1) Last O(1) O(k 1 ) Specified O(k 1 -k 2 ) O(k 2 ) Just mantain a pointer to the head node Conditional O(k 1 ) O(k 2 ) k1 total number of elements before insertion k2 inserted element position (k1 k2) 34

35 Algorithms Design Techniques! Most common design techniques:! Exhaustive Search (Brute Force)! Branch- and- Bound Algorithms (Pruning)! Greedy Algorithms! Dynamic Programming! Divide- and- Conquer Algorithms! Randomized Algorithms 35

36 Exhaus)ve Search! Examines every possible alternative to find one particular solution! These are the easiest algorithms to design and understand! Often the first step in designing more efficient algorithms! In general are too slow to be pratical for anything but the smallest instances 36

37 E. S. Biological Problems! DNA restriction mapping! Regulatory motif finding 37

38 Branch- and- Bound Algorithms! Omit a large number of alternatives 38

39 Greedy Algorithms! Iterative procedures that choose among a number of alternatives at each iteration! Some alternatives may lead to correct solutions while others may not! Greedy algorithms choose the most attractive alternative at each iteration, for example, the largest denomination possible 39

40 Biological Problems! Genome rearrangements! Motif finding! Other example:! Considere the following numbers : ! Add 10 using a greedy algorithm! The result is not the best solution 40

41 Dynamic Programming! Break the problem into smaller subproblems and use the solutions of the subproblems to construct the solution of the larger one! Dynamic programming organizes computations to avoid recomputing values that you already know, which can often save a great deal of time 41

42 Biological Problems! DNA sequence comparison! Gene prediction Algorithms! Needleman- Wunch algorithm! Smith- Waterman algorithm Tools! BLAST! FAST- A! CLUSTAL 42

43 Divide- and- Conquer! Split the problem into smaller subproblems! Solve the subproblems independently! Combine the solutions of subproblems into a solution of the original problem! The combination step is critical 43

44 Problems! Local alignment problem! Sorting problems (Mergsort) 44

45 Randomized Algorithms! Roll a die to decide where to go in the next step! These algorithms help solve pratical problems and some of them have a competitive advantage over deterministic algorithms 45

46 Biological Problems! Motif finding (Gibbs Sampling) 46

47 Machine Learning! Decision trees! Clustering techniques! Hidden Markov Models!... 47

48 Biological Problems! Gene finding! Protein classification 48

49 Graph Algorithms! Graph theory was born in the eighteenth century when Leonhard Euler solved the famous Königsberg Bridge problem! Find a tour crossing every bridge just once 49

50 Biological Problems! DNA Sequencing! Fragment Assembly in DNA Sequencing! Protein Sequencing and Identification 50

51 Combinatorial PaOern Matching! Looks for exact or approximate occurrences of given patterns in a long text! Exact Matching : Boyer- Moore algorithm! Data structures : Suffix trees 51

52 Biological Problems! Repeat Finding! DNA databases search! Protein databases search! Motif finding! Motif extraction 52