QB LECTURE #1: Algorithms and Dynamic Programming
|
|
- Ashlynn Howard
- 5 years ago
- Views:
Transcription
1 QB LECTURE #1: Algorithms and Dynamic Programming Adam Siepel Nov. 16, 2015
2 2 Plan for Today Introduction to algorithms Simple algorithms and running time Dynamic programming Soon: sequence alignment
3 3 Intro to Algorithms An algorithm is a sequence of instructions to solve a well-defined class of problems Usually defined in terms of inputs and outputs Two key questions: Does it work correctly? How fast? Elegance is prized, but not at the expense of correctness or speed
4 4 Examples of Algorithms Trivial: linear search, sum elements of list, search for largest element More subtle: binary search, quick sort, graph search Tricky: dynamic programming, branchand-bound search, minimum spanning tree Deep: nonlinear optimization, expectation maximization for MLE, fast Fourier transform, sorting by reversals
5 5 Running Time We study running time as a function of the size of the input, ignoring constants (depend on implementation) Can consider worst-case, best-case, average-case, amortized running time Worst-case most widely used: big-o notation Examples: linear-search = O(n), binary search = O(log n), sorting = O(n log n), sequence alignment = O(nm) Roughly speaking, algorithms are tractable if polynomial (e.g., O(n k )), otherwise intractable
6 Searching for GATTACA Where is GATTACA in the human genome? Strategy 1: Brute Force T G A T T A C A G A T T A C C G A T T A C A No match at offset 1
7 Searching for GATTACA Where is GATTACA in the human genome? Strategy 1: Brute Force T G A T T A C A G A T T A C C G A T T A C A Match at offset 2
8 Searching for GATTACA Where is GATTACA in the human genome? Strategy 1: Brute Force T G A T T A C A G A T T A C C G A T T A C A No match at offset 3
9 Searching for GATTACA Where is GATTACA in the human genome? Strategy 1: Brute Force T G A T T A C A G A T T A C C G A T T A C A No match at offset 9 <- Checking each possible position takes time
10 Brute Force Analysis Brute Force: At every possible offset in the genome: Do all of the characters of the query match? Analysis Simple, easy to understand Genome length = n [3B] Query length = m [7] Comparisons: (n-m+1) * m [21B] Overall runtime: O(nm) [How long would it take if we double the genome size, read length?] [How long would it take if we double both?]
11 Brute Force Reflections Why check every position? GATTACA can't possibly start at position 15 [WHY?] T G A T T A C A G A T T A C C G A T T A C A Improve runtime to O(n + m) [3B + 7] If we double both, it just takes twice as long Knuth-Morris-Pratt, 1977 Boyer-Moyer, 1977, 1991 For one-off scans, this is the best we can do (optimal performance) We have to read every character of the genome, and every character of the query For short queries, runtime is dominated by the length of the genome
12 Suffix Arrays: Searching the Phone Book What if we need to check many queries? We don't need to check every page of the phone book to find 'Schatz' Sorting alphabetically lets us immediately skip 96% (25/26) of the book without any loss in accuracy Sorting the genome: Suffix Array (Manber & Myers, 1991) Sort every suffix of the genome Split into n suffixes Sort suffixes alphabetically [Challenge Question: How else could we split the genome?]
13 Searching the Index Strategy 2: Binary search Compare to the middle, refine as higher or lower Searching for GATTACA Lo = 1; Hi = 15; Lo Hi # Sequence Pos 1 ACAGATTACC 6 2 ACC 13 3 AGATTACC 8 4 ATTACAGATTACC 3 5 ATTACC 10 6 C 15 7 CAGATTACC 7 8 CC 14 9 GATTACAGATTACC 2 10 GATTACC 9 11 TACAGATTACC 5 12 TACC TGATTACAGATTACC 1 14 TTACAGATTACC 4 15 TTACC 11
14 Searching the Index Strategy 2: Binary search Compare to the middle, refine as higher or lower Searching for GATTACA Lo = 1; Hi = 15; Mid = (1+15)/2 = 8 Middle = Suffix[8] = CC Lo Hi # Sequence Pos 1 ACAGATTACC 6 2 ACC 13 3 AGATTACC 8 4 ATTACAGATTACC 3 5 ATTACC 10 6 C 15 7 CAGATTACC 7 8 CC 14 9 GATTACAGATTACC 2 10 GATTACC 9 11 TACAGATTACC 5 12 TACC TGATTACAGATTACC 1 14 TTACAGATTACC 4 15 TTACC 11
15 Searching the Index Strategy 2: Binary search Compare to the middle, refine as higher or lower Searching for GATTACA Lo = 1; Hi = 15; Mid = (1+15)/2 = 8 Middle = Suffix[8] = CC => Higher: Lo = Mid + 1 Lo Hi # Sequence Pos 1 ACAGATTACC 6 2 ACC 13 3 AGATTACC 8 4 ATTACAGATTACC 3 5 ATTACC 10 6 C 15 7 CAGATTACC 7 8 CC 14 9 GATTACAGATTACC 2 10 GATTACC 9 11 TACAGATTACC 5 12 TACC TGATTACAGATTACC 1 14 TTACAGATTACC 4 15 TTACC 11
16 Searching the Index Strategy 2: Binary search Compare to the middle, refine as higher or lower Searching for GATTACA Lo = 1; Hi = 15; Mid = (1+15)/2 = 8 Middle = Suffix[8] = CC => Higher: Lo = Mid + 1 Lo = 9; Hi = 15; Lo Hi # Sequence Pos 1 ACAGATTACC 6 2 ACC 13 3 AGATTACC 8 4 ATTACAGATTACC 3 5 ATTACC 10 6 C 15 7 CAGATTACC 7 8 CC 14 9 GATTACAGATTACC 2 10 GATTACC 9 11 TACAGATTACC 5 12 TACC TGATTACAGATTACC 1 14 TTACAGATTACC 4 15 TTACC 11
17 Searching the Index Strategy 2: Binary search Compare to the middle, refine as higher or lower Searching for GATTACA Lo = 1; Hi = 15; Mid = (1+15)/2 = 8 Middle = Suffix[8] = CC => Higher: Lo = Mid + 1 Lo = 9; Hi = 15; Mid = (9+15)/2 = 12 Middle = Suffix[12] = TACC Lo Hi # Sequence Pos 1 ACAGATTACC 6 2 ACC 13 3 AGATTACC 8 4 ATTACAGATTACC 3 5 ATTACC 10 6 C 15 7 CAGATTACC 7 8 CC 14 9 GATTACAGATTACC 2 10 GATTACC 9 11 TACAGATTACC 5 12 TACC TGATTACAGATTACC 1 14 TTACAGATTACC 4 15 TTACC 11
18 Searching the Index Strategy 2: Binary search Compare to the middle, refine as higher or lower Searching for GATTACA Lo = 1; Hi = 15; Mid = (1+15)/2 = 8 Middle = Suffix[8] = CC => Higher: Lo = Mid + 1 Lo = 9; Hi = 15; Mid = (9+15)/2 = 12 Middle = Suffix[12] = TACC => Lower: Hi = Mid - 1 Lo = 9; Hi = 11; Lo Hi # Sequence Pos 1 ACAGATTACC 6 2 ACC 13 3 AGATTACC 8 4 ATTACAGATTACC 3 5 ATTACC 10 6 C 15 7 CAGATTACC 7 8 CC 14 9 GATTACAGATTACC 2 10 GATTACC 9 11 TACAGATTACC 5 12 TACC TGATTACAGATTACC 1 14 TTACAGATTACC 4 15 TTACC 11
19 Searching the Index Strategy 2: Binary search Compare to the middle, refine as higher or lower Searching for GATTACA Lo = 1; Hi = 15; Mid = (1+15)/2 = 8 Middle = Suffix[8] = CC => Higher: Lo = Mid + 1 Lo = 9; Hi = 15; Mid = (9+15)/2 = 12 Middle = Suffix[12] = TACC => Lower: Hi = Mid - 1 Lo = 9; Hi = 11; Mid = (9+11)/2 = 10 Middle = Suffix[10] = GATTACC Lo Hi # Sequence Pos 1 ACAGATTACC 6 2 ACC 13 3 AGATTACC 8 4 ATTACAGATTACC 3 5 ATTACC 10 6 C 15 7 CAGATTACC 7 8 CC 14 9 GATTACAGATTACC 2 10 GATTACC 9 11 TACAGATTACC 5 12 TACC TGATTACAGATTACC 1 14 TTACAGATTACC 4 15 TTACC 11
20 Searching the Index Strategy 2: Binary search Compare to the middle, refine as higher or lower Searching for GATTACA Lo = 1; Hi = 15; Mid = (1+15)/2 = 8 Middle = Suffix[8] = CC => Higher: Lo = Mid + 1 Lo = 9; Hi = 15; Mid = (9+15)/2 = 12 Middle = Suffix[12] = TACC => Lower: Hi = Mid - 1 Lo = 9; Hi = 11; Mid = (9+11)/2 = 10 Middle = Suffix[10] = GATTACC => Lower: Hi = Mid - 1 Lo = 9; Hi = 9; Lo Hi # Sequence Pos 1 ACAGATTACC 6 2 ACC 13 3 AGATTACC 8 4 ATTACAGATTACC 3 5 ATTACC 10 6 C 15 7 CAGATTACC 7 8 CC 14 9 GATTACAGATTACC 2 10 GATTACC 9 11 TACAGATTACC 5 12 TACC TGATTACAGATTACC 1 14 TTACAGATTACC 4 15 TTACC 11
21 Searching the Index Strategy 2: Binary search Compare to the middle, refine as higher or lower Searching for GATTACA Lo = 1; Hi = 15; Mid = (1+15)/2 = 8 Middle = Suffix[8] = CC => Higher: Lo = Mid + 1 Lo = 9; Hi = 15; Mid = (9+15)/2 = 12 Middle = Suffix[12] = TACC => Lower: Hi = Mid - 1 Lo = 9; Hi = 11; Mid = (9+11)/2 = 10 Middle = Suffix[10] = GATTACC => Lower: Hi = Mid - 1 Lo = 9; Hi = 9; Mid = (9+9)/2 = 9 Middle = Suffix[9] = GATTACA => Match at position 2! Lo Hi # Sequence Pos 1 ACAGATTACC 6 2 ACC 13 3 AGATTACC 8 4 ATTACAGATTACC 3 5 ATTACC 10 6 C 15 7 CAGATTACC 7 8 CC 14 9 GATTACAGATTACC 2 10 GATTACC 9 11 TACAGATTACC 5 12 TACC TGATTACAGATTACC 1 14 TTACAGATTACC 4 15 TTACC 11
22 Binary Search Analysis Binary Search Initialize search range to entire list mid = (hi+lo)/2; middle = suffix[mid] if query matches middle: done else if query < middle: pick low range else if query > middle: pick hi range Repeat until done or empty range [WHEN?] Analysis More complicated method How many times do we repeat? How many times can it cut the range in half? Find smallest x such that: n/(2 x ) 1; x = lg 2 (n) [32] Total Runtime: O(m lg n) More complicated, but much faster! Looking up a query loops 32 times instead of 3B [How long does it take to search 6B or 24B nucleotides?]
23 Sorting Quickly sort these numbers into ascending order: 14, 29, 6, 31, 39, 64, 78, 50, 13, 63, 61, 19 [How do you do it?] 6, 14, 29, 31, 39, 64, 78, 50, 13, 63, 61, 19 6, 13, 14, 29, 31, 39, 64, 78, 50, 63, 61, 19 6, 13, 14, 19, 29, 31, 39, 64, 78, 50, 63, 61 6, 13, 14, 19, 29, 31, 39, 64, 78, 50, 63, 61 6, 13, 14, 19, 29, 31, 39, 64, 78, 50, 63, 61 6, 13, 14, 19, 29, 31, 39, 50, 64, 78, 63, 61 6, 13, 14, 19, 29, 31, 39, 50, 61, 64, 78, 63 6, 13, 14, 19, 29, 31, 39, 50, 61, 63, 64, 78 6, 13, 14, 19, 29, 31, 39, 50, 61, 63, 64, 78 6, 13, 14, 19, 29, 31, 39, 50, 61, 63, 64, 78 6, 13, 14, 19, 29, 31, 39, 50, 61, 63, 64, 78 6, 13, 14, 19, 29, 31, 39, 50, 61, 63, 64, 78
24 Selection Sort Analysis Selection Sort (Input: list of n numbers) for pos = 1 to n // find the smallest element in [pos, n] smallest = pos for check = pos+1 to n if (list[check] < list[smallest]): smallest = check Analysis // move the smallest element to the front tmp = list[smallest] list[pos] = list[smallest] list[smallest] = tmp Outer loop: pos = 1 to n Inner loop: check = pos to n Running time: Outer * Inner = O(n 2 ) [Challenge Questions: Why is this slow? / Can we sort any faster?]
25 Divide and Conquer Selection sort is slow because it rescans the entire list for each element How can we split up the unsorted list into independent ranges? Hint 1: Binary search splits up the problem into 2 independent ranges (hi/lo) Hint 2: Assume we know the median value of a list n < = > 2 x n/2 < = > = < = > 4 x n/4 < = > = < = > = < = > = < = > 8 x n/8 16 x n/16 [How many times can we split a list in half?] 2 i x n/2 i
26 Algorithmic Complexity What is the runtime as a function of the input size?
27 27 Recursion Algorithms that call themselves are recursive Example: partialsum(n) Input: positive integer n Output: sum of integers from 1 to n if (n = 1) return 1; else return n + partialsum(n 1); Recursive algorithms can always be redefined as iterative algorithms (sometimes desirable)
28 Fibonacci Sequence def fib(n): if n == 0 or n == 1: return n else: f(5) F(6) f(4) return fib(n 1) + fib(n 2) f(4) f(3) f(3) f(2) f(3) f(2) f(2) f(1) f(2) f(1) f(1) f(0) f(2) f(1) f(1) f(0) f(1) f(0) f(1) f(0) f(1) f(0)
29 Fibonacci Sequence def fib(n): if n == 0 or n == 1: return n else: return fib(n 1) + fib(n 2) [How long would it take for F(7)?] [What is the running time?]
30 Bottom-up Fibonacci Sequence def fib(n): table = [0] * (n+1) table[0] = 0 table[1] = 1 for i in range(2,n+1): table[i] = table[i 2] + table[i 1] return table[n] [How long will it take for F(7)?] [What is the running time?]
31 31 Dynamic Programming Recursion with reuse of intermediate computations Widely used in computational biology: alignment, phylogenetics, HMMs Also graphical models, operations research, signal processing, timeseries analysis, etc.
32 Dynamic Programming General approach for solving (some) complex problems When applicable, the method takes far less time than naive methods. Polynomial time (O(n) or O(n 2 ) instead of exponential time (O(2 n ) or O(3 n )) Requirements: Overlapping subproblems Optimal substructure Applications: Fibonacci Longest Increasing Subsequence Sequence alignment, Dynamic Time Warp, Viterbi F(5) F(3) F(6) F(4) F(2) Not applicable: Traveling salesman problem, Clique finding, Subgraph isomorphism, F(1) F(0) The cheapest flight from airport A to airport B involves a single connection through airport C, but the cheapest flight from airport A to airport C involves a connection through some other airport D.
33 33 Introduction to Graphs 5 2 A graph consists of nodes and edges. The edges may be directed or undirected, and may be weighted or unweighted. A path from node u to node v is a sequence of connected edges leading from u to v The length of a path is its total number of edges. The weight of a path is the sum of the weights of all edges. Many optimization problems can be cast as problems of finding paths in graphs
34 34 Manhattan Tourist Problem Problem: move from source (59th St. & 8th Ave.) to sink (42nd & Lexington), maximizing number of tourist attractions City can be described as a graph with vertices for corners and edges for streets. This graph is a grid (lattice) The numbers of attractions are weights. The tourist can only move S&E, which directs the edges The problem is to find maximum weight from source to sink (not actual path)
35 35
36 36 Base Cases The best weight at the source is s0,0 = 0 What is the best weight along the northern boundary: s0,1, s0,2,...? Answer: s 0,j = s 0,j 1 + w (0,j 1) (0,j), 1 j m Similarly, for the western boundary: s i,0 = s i 1,0 + w (i 1,0) (i,0), 1 i n
37 37 Key Recurrence How do we express si,j in terms of a subproblem? Answer: s i,j = max { s i 1,j + w (i 1,j) (i,j) s i,j 1 + w (i,j 1) (i,j)
38 38 The Algorithm Running time: O(nm)
39 39 Key Points Identify sub-problem, find recurrence Recursive algorithm is not enough: no savings unless computations are reused Can be useful to convert to problem of finding path in graph
40 40 That s All Homework is coming (today or tomorrow). Will be due Nov 30.
41 Suffix Array Construction How can we store the suffix array? [How many characters are in all suffixes combined?] Pos Hopeless to explicitly store 4.5 billion billion characters Instead use implicit representation Keep 1 copy of the genome, and a list of sorted offsets Storing 3 billion offsets fits on a server (12GB) Searching the array is very fast, but it takes time to construct This time will be amortized over many, many searches Run it once "overnight" and save it away for all future queries TGATTACAGATTACC
42 QuickSort Analysis
43 QuickSort Analysis
44 QuickSort in Python list.sort() The goal of software engineering is to build libraries of correct reusable functions that implement higher level ideas Build complex software out of simple components Software tends to be 90% plumbing, 10% research You still need to know how they work Python requires an explicit representation of the strings
45 Longest Increasing Subsequence Given a sequence of N numbers A 1, A 2, A 3, A N, find the longest monotonically increasing subsequence 29, 6, 14, 31, 39, 78, 63, 50, 13, 64, 61, 19 Greedy approach (always extend the subsequence if you can): 29, 6, 14, 31, 39, 78, 63, 50, 13, 64, 61, 19 => 4 Brute force: Try all possible O(2 n ) subsequences 29, 6, 14, 31, 39, 78, 63, 50, 13, 64, 61, 19 => 1 29, 6, 14, 31, 39, 78, 63, 50, 13, 64, 61, 19 => invalid 29, 6, 14, 31, 39, 78, 63, 50, 13, 64, 61, 19 => invalid 29, 6, 14, 31, 39, 78, 63, 50, 13, 64, 61, 19 => 2
46 Longest Increasing Subsequence Idea: The solution for all N numbers depends on the solution for the first N-1 Look through the previous values to find the longest subsequence ending at X such that A X < A N Dynamic Programming: Def: L[j] is the longest increasing subsequence ending at position j Base case: L[0] = 0 Recurrence: LIS=max{L[i]} L[ j] = max{ L[i] }+1 i< j A[i ]<A[ j ] Index Value LIS Prev [What s the LIS of 0,8,4,12,2,10,6,14,1,9,5,13,3,11,7,15?]
47 Longest Increasing Subsequence // Initialize L[0] = 0; P[0] = i j n // Iteratively apply recurrence for i = 1 to N // find the best LIS to extend bestlis = 0; bestidx = -1 for j = 1 to i if ((A[j] <= A[i])) && (L[j] > bestlis)) bestlis = L[j]; bestidx = j L[i] = bestlis + 1; P[i] = bestidx A[1] A[2] A[3] A[i] A[j] L[1] L[2] L[3] L[i] L[i]+1 P[1] P[2] P[3] P[j] i A[n] // Scan the L array to find the overall LIS LIS = 0 for j = 1 to N if (L[j] > LIS) LIS = L[j] print The LIS is $LIS [What s the running time?]
CS : Data Structures
CS 600.226: Data Structures Michael Schatz Nov 16, 2016 Lecture 32: Mike Week pt 2: BWT Assignment 9: Due Friday Nov 18 @ 10pm Remember: javac Xlint:all & checkstyle *.java & JUnit Solutions should be
More informationCS : Data Structures Michael Schatz. Nov 14, 2016 Lecture 32: BWT
CS 600.226: Data Structures Michael Schatz Nov 14, 2016 Lecture 32: BWT HW8 Assignment 8: Competitive Spelling Bee Out on: November 2, 2018 Due by: November 9, 2018 before 10:00 pm Collaboration: None
More informationCS : Data Structures
CS 600.226: Data Structures Michael Schatz Nov 30, 2016 Lecture 35: Topological Sorting Assignment 10: Due Monday Dec 5 @ 10pm Remember: javac Xlint:all & checkstyle *.java & JUnit Solutions should be
More informationLecture 2. Dynammic Programming Notes.txt Page 1
Lecture 2. Dynammic Programming Notes.txt Page 1 Dynamic Programming Michael Schatz (mschatz@cshl.edu) =============================================================================== Motivation: Many optimization
More informationDivide and Conquer Algorithms. Problem Set #3 is graded Problem Set #4 due on Thursday
Divide and Conquer Algorithms Problem Set #3 is graded Problem Set #4 due on Thursday 1 The Essence of Divide and Conquer Divide problem into sub-problems Conquer by solving sub-problems recursively. If
More informationCS : Data Structures Michael Schatz. Nov 26, 2016 Lecture 34: Advanced Sorting
CS 600.226: Data Structures Michael Schatz Nov 26, 2016 Lecture 34: Advanced Sorting Assignment 9: StringOmics Out on: November 16, 2018 Due by: November 30, 2018 before 10:00 pm Collaboration: None Grading:
More informationAlgorithm classification
Types of Algorithms Algorithm classification Algorithms that use a similar problem-solving approach can be grouped together We ll talk about a classification scheme for algorithms This classification scheme
More informationAlgorithm Design and Recursion. Search and Sort Algorithms
Algorithm Design and Recursion Search and Sort Algorithms Objectives To understand the basic techniques for analyzing the efficiency of algorithms. To know what searching is and understand the algorithms
More informationLecturers: Sanjam Garg and Prasad Raghavendra March 20, Midterm 2 Solutions
U.C. Berkeley CS70 : Algorithms Midterm 2 Solutions Lecturers: Sanjam Garg and Prasad aghavra March 20, 207 Midterm 2 Solutions. (0 points) True/False Clearly put your answers in the answer box in front
More informationLecture 12: Divide and Conquer Algorithms
Lecture 12: Divide and Conquer Algorithms Study Chapter 7.1 7.4 1 Divide and Conquer Algorithms Divide problem into sub-problems Conquer by solving sub-problems recursively. If the sub-problems are small
More informationChapter 6. Dynamic Programming. Modified from slides by Kevin Wayne. Copyright 2005 Pearson-Addison Wesley. All rights reserved.
Chapter 6 Dynamic Programming Modified from slides by Kevin Wayne. Copyright 2005 Pearson-Addison Wesley. All rights reserved. 1 Think recursively (this week)!!! Divide & conquer and Dynamic programming
More informationLecture 19 Sorting Goodrich, Tamassia
Lecture 19 Sorting 7 2 9 4 2 4 7 9 7 2 2 7 9 4 4 9 7 7 2 2 9 9 4 4 2004 Goodrich, Tamassia Outline Review 3 simple sorting algorithms: 1. selection Sort (in previous course) 2. insertion Sort (in previous
More informationUNIT 5B Binary Search
205/09/30 UNIT 5B Binary Search Course Announcements Written exam next week (Wed. Oct 7 ) Practice exam available on the Resources page Exam reviews: Sunday afternoon; watch Piazza for times and places
More informationIntroduction to Algorithms / Algorithms I Lecturer: Michael Dinitz Topic: Dynamic Programming I Date: 10/6/16
600.463 Introduction to Algorithms / Algorithms I Lecturer: Michael Dinitz Topic: Dynamic Programming I Date: 10/6/16 11.1 Introduction Dynamic programming can be very confusing until you ve used it a
More informationDivide and Conquer. Bioinformatics: Issues and Algorithms. CSE Fall 2007 Lecture 12
Divide and Conquer Bioinformatics: Issues and Algorithms CSE 308-408 Fall 007 Lecture 1 Lopresti Fall 007 Lecture 1-1 - Outline MergeSort Finding mid-point in alignment matrix in linear space Linear space
More informationCS 137 Part 8. Merge Sort, Quick Sort, Binary Search. November 20th, 2017
CS 137 Part 8 Merge Sort, Quick Sort, Binary Search November 20th, 2017 This Week We re going to see two more complicated sorting algorithms that will be our first introduction to O(n log n) sorting algorithms.
More informationSankalchand Patel College of Engineering - Visnagar Department of Computer Engineering and Information Technology. Assignment
Class: V - CE Sankalchand Patel College of Engineering - Visnagar Department of Computer Engineering and Information Technology Sub: Design and Analysis of Algorithms Analysis of Algorithm: Assignment
More information1 Dynamic Programming
Recitation 13 Dynamic Programming Parallel and Sequential Data Structures and Algorithms, 15-210 (Fall 2013) November 20, 2013 1 Dynamic Programming Dynamic programming is a technique to avoid needless
More informationAdmin. How's the project coming? After these slides, read chapter 13 in your book. Quizzes will return
Recursion CS 1 Admin How's the project coming? After these slides, read chapter 13 in your book Yes that is out of order, but we can read it stand alone Quizzes will return Tuesday Nov 29 th see calendar
More informationCSE 431/531: Analysis of Algorithms. Greedy Algorithms. Lecturer: Shi Li. Department of Computer Science and Engineering University at Buffalo
CSE 431/531: Analysis of Algorithms Greedy Algorithms Lecturer: Shi Li Department of Computer Science and Engineering University at Buffalo Main Goal of Algorithm Design Design fast algorithms to solve
More informationSolutions to Problem 1 of Homework 3 (10 (+6) Points)
Solutions to Problem 1 of Homework 3 (10 (+6) Points) Sometimes, computing extra information can lead to more efficient divide-and-conquer algorithms. As an example, we will improve on the solution to
More informationTotal Points: 60. Duration: 1hr
CS800 : Algorithms Fall 201 Nov 22, 201 Quiz 2 Practice Total Points: 0. Duration: 1hr 1. (,10) points Binary Heap. (a) The following is a sequence of elements presented to you (in order from left to right):
More informationSorting Pearson Education, Inc. All rights reserved.
1 19 Sorting 2 19.1 Introduction (Cont.) Sorting data Place data in order Typically ascending or descending Based on one or more sort keys Algorithms Insertion sort Selection sort Merge sort More efficient,
More informationString Matching. Pedro Ribeiro 2016/2017 DCC/FCUP. Pedro Ribeiro (DCC/FCUP) String Matching 2016/ / 42
String Matching Pedro Ribeiro DCC/FCUP 2016/2017 Pedro Ribeiro (DCC/FCUP) String Matching 2016/2017 1 / 42 On this lecture The String Matching Problem Naive Algorithm Deterministic Finite Automata Knuth-Morris-Pratt
More informationSorting is a problem for which we can prove a non-trivial lower bound.
Sorting The sorting problem is defined as follows: Sorting: Given a list a with n elements possessing a total order, return a list with the same elements in non-decreasing order. Remember that total order
More informationSubsequence Definition. CS 461, Lecture 8. Today s Outline. Example. Assume given sequence X = x 1, x 2,..., x m. Jared Saia University of New Mexico
Subsequence Definition CS 461, Lecture 8 Jared Saia University of New Mexico Assume given sequence X = x 1, x 2,..., x m Let Z = z 1, z 2,..., z l Then Z is a subsequence of X if there exists a strictly
More informationAlgorithm Design and Analysis
Algorithm Design and Analysis LECTURE 16 Dynamic Programming (plus FFT Recap) Adam Smith 9/24/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Discrete Fourier Transform
More informationDesign and Analysis of Algorithms 演算法設計與分析. Lecture 7 April 15, 2015 洪國寶
Design and Analysis of Algorithms 演算法設計與分析 Lecture 7 April 15, 2015 洪國寶 1 Course information (5/5) Grading (Tentative) Homework 25% (You may collaborate when solving the homework, however when writing
More informationRecall from Last Time: Big-Oh Notation
CSE 326 Lecture 3: Analysis of Algorithms Today, we will review: Big-Oh, Little-Oh, Omega (Ω), and Theta (Θ): (Fraternities of functions ) Examples of time and space efficiency analysis Covered in Chapter
More informationCSE373: Data Structure & Algorithms Lecture 18: Comparison Sorting. Dan Grossman Fall 2013
CSE373: Data Structure & Algorithms Lecture 18: Comparison Sorting Dan Grossman Fall 2013 Introduction to Sorting Stacks, queues, priority queues, and dictionaries all focused on providing one element
More informationCMPSCI 311: Introduction to Algorithms Practice Final Exam
CMPSCI 311: Introduction to Algorithms Practice Final Exam Name: ID: Instructions: Answer the questions directly on the exam pages. Show all your work for each question. Providing more detail including
More informationComputer Algorithms. Introduction to Algorithm
Computer Algorithms Introduction to Algorithm CISC 4080 Yanjun Li 1 What is Algorithm? An Algorithm is a sequence of well-defined computational steps that transform the input into the output. These steps
More informationCSE 373 MAY 24 TH ANALYSIS AND NON- COMPARISON SORTING
CSE 373 MAY 24 TH ANALYSIS AND NON- COMPARISON SORTING ASSORTED MINUTIAE HW6 Out Due next Wednesday ASSORTED MINUTIAE HW6 Out Due next Wednesday Only two late days allowed ASSORTED MINUTIAE HW6 Out Due
More informationDynamic Programming. Lecture Overview Introduction
Lecture 12 Dynamic Programming 12.1 Overview Dynamic Programming is a powerful technique that allows one to solve many different types of problems in time O(n 2 ) or O(n 3 ) for which a naive approach
More informationDivide & Conquer Algorithms
Divide & Conquer Algorithms Outline 1. MergeSort 2. Finding the middle vertex 3. Linear space sequence alignment 4. Block alignment 5. Four-Russians speedup 6. LCS in sub-quadratic time Section 1: MergeSort
More informationDynamic Programming. Introduction, Weighted Interval Scheduling, Knapsack. Tyler Moore. Lecture 15/16
Dynamic Programming Introduction, Weighted Interval Scheduling, Knapsack Tyler Moore CSE, SMU, Dallas, TX Lecture /6 Greedy. Build up a solution incrementally, myopically optimizing some local criterion.
More informationAlgorithms and Data Structures Lesson 3
Algorithms and Data Structures Lesson 3 Michael Schwarzkopf https://www.uni weimar.de/de/medien/professuren/medieninformatik/grafische datenverarbeitung Bauhaus University Weimar May 30, 2018 Overview...of
More informationIntroduction to Computer Science
Introduction to Computer Science CSCI 109 Readings St. Amant, Ch. 4, Ch. 8 China Tianhe-2 Andrew Goodney Spring 2018 An algorithm (pronounced AL-go-rithum) is a procedure or formula for solving a problem.
More informationPseudo code of algorithms are to be read by.
Cs502 Quiz No1 Complete Solved File Pseudo code of algorithms are to be read by. People RAM Computer Compiler Approach of solving geometric problems by sweeping a line across the plane is called sweep.
More informationCSE 332: Data Structures & Parallelism Lecture 12: Comparison Sorting. Ruth Anderson Winter 2019
CSE 332: Data Structures & Parallelism Lecture 12: Comparison Sorting Ruth Anderson Winter 2019 Today Sorting Comparison sorting 2/08/2019 2 Introduction to sorting Stacks, queues, priority queues, and
More informationPresentation for use with the textbook, Algorithm Design and Applications, by M. T. Goodrich and R. Tamassia, Wiley, Merge Sort & Quick Sort
Presentation for use with the textbook, Algorithm Design and Applications, by M. T. Goodrich and R. Tamassia, Wiley, 2015 Merge Sort & Quick Sort 1 Divide-and-Conquer Divide-and conquer is a general algorithm
More informationCSE 143. Complexity Analysis. Program Efficiency. Constant Time Statements. Big Oh notation. Analyzing Loops. Constant Time Statements (2) CSE 143 1
CSE 1 Complexity Analysis Program Efficiency [Sections 12.1-12., 12., 12.9] Count number of instructions executed by program on inputs of a given size Express run time as a function of the input size Assume
More informationString Matching Algorithms
String Matching Algorithms 1. Naïve String Matching The naïve approach simply test all the possible placement of Pattern P[1.. m] relative to text T[1.. n]. Specifically, we try shift s = 0, 1,..., n -
More informationCSE 373: Data Structures and Algorithms
CSE 373: Data Structures and Algorithms Lecture 19: Comparison Sorting Algorithms Instructor: Lilian de Greef Quarter: Summer 2017 Today Intro to sorting Comparison sorting Insertion Sort Selection Sort
More informationModule 08: Searching and Sorting Algorithms
Module 08: Searching and Sorting Algorithms Topics: Searching algorithms Sorting algorithms 1 Application: Searching a list Suppose you have a list L. How could you determine if a particular value is in
More informationTest 1 - Closed Book Last 4 Digits of Student ID # Multiple Choice. Write your answer to the LEFT of each problem. 3 points each
CSE 5311 Test 1 - Closed Book Summer 2009 Name Last 4 Digits of Student ID # Multiple Choice. Write your answer to the LEFT of each problem. 3 points each 1. Which of the following statements is true?
More information8/19/13. Computational problems. Introduction to Algorithm
I519, Introduction to Introduction to Algorithm Yuzhen Ye (yye@indiana.edu) School of Informatics and Computing, IUB Computational problems A computational problem specifies an input-output relationship
More informationMeasuring algorithm efficiency
CMPT 225 Measuring algorithm efficiency Timing Counting Cost functions Cases Best case Average case Worst case Searching Sorting O Notation O notation's mathematical basis O notation classes and notations
More informationWhat is sorting? Lecture 36: How can computation sort data in order for you? Why is sorting important? What is sorting? 11/30/10
// CS Introduction to Computation " UNIVERSITY of WISCONSIN-MADISON Computer Sciences Department Professor Andrea Arpaci-Dusseau Fall Lecture : How can computation sort data in order for you? What is sorting?
More informationAdvanced Algorithms and Data Structures
Advanced Algorithms and Data Structures Prof. Tapio Elomaa Course Basics A new 7 credit unit course Replaces OHJ-2156 Analysis of Algorithms We take things a bit further than OHJ-2156 We will assume familiarity
More informationCSci 231 Final Review
CSci 231 Final Review Here is a list of topics for the final. Generally you are responsible for anything discussed in class (except topics that appear italicized), and anything appearing on the homeworks.
More informationSorting and Searching
Sorting and Searching Sorting o Simple: Selection Sort and Insertion Sort o Efficient: Quick Sort and Merge Sort Searching o Linear o Binary Reading for this lecture: http://introcs.cs.princeton.edu/python/42sort/
More informationAlgorithms for Data Science
Algorithms for Data Science CSOR W4246 Eleni Drinea Computer Science Department Columbia University Thursday, October 1, 2015 Outline 1 Recap 2 Shortest paths in graphs with non-negative edge weights (Dijkstra
More informationIntroduction to Computers and Programming. Today
Introduction to Computers and Programming Prof. I. K. Lundqvist Lecture 10 April 8 2004 Today How to determine Big-O Compare data structures and algorithms Sorting algorithms 2 How to determine Big-O Partition
More informationUnit-2 Divide and conquer 2016
2 Divide and conquer Overview, Structure of divide-and-conquer algorithms, binary search, quick sort, Strassen multiplication. 13% 05 Divide-and- conquer The Divide and Conquer Paradigm, is a method of
More informationCAD Algorithms. Categorizing Algorithms
CAD Algorithms Categorizing Algorithms Mohammad Tehranipoor ECE Department 2 September 2008 1 Categorizing Algorithms Greedy Algorithms Prim s Algorithm (Minimum Spanning Tree) A subgraph that is a tree
More informationData Structures and Algorithms CMPSC 465
Data Structures and Algorithms CMPSC 465 LECTURES 7-8 More Divide and Conquer Multiplication Adam Smith S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson John McCarthy (1927
More informationSorting & Growth of Functions
Sorting & Growth of Functions CSci 588: Data Structures, Algorithms and Software Design Introduction to Algorithms, Cormen et al., Chapter 3 All material not from online sources or text copyright Travis
More informationAnalysis of Algorithms. Unit 4 - Analysis of well known Algorithms
Analysis of Algorithms Unit 4 - Analysis of well known Algorithms 1 Analysis of well known Algorithms Brute Force Algorithms Greedy Algorithms Divide and Conquer Algorithms Decrease and Conquer Algorithms
More informationIN101: Algorithmic techniques Vladimir-Alexandru Paun ENSTA ParisTech
IN101: Algorithmic techniques Vladimir-Alexandru Paun ENSTA ParisTech License CC BY-NC-SA 2.0 http://creativecommons.org/licenses/by-nc-sa/2.0/fr/ Outline Previously on IN101 Python s anatomy Functions,
More informationDivide and Conquer Sorting Algorithms and Noncomparison-based
Divide and Conquer Sorting Algorithms and Noncomparison-based Sorting Algorithms COMP1927 16x1 Sedgewick Chapters 7 and 8 Sedgewick Chapter 6.10, Chapter 10 DIVIDE AND CONQUER SORTING ALGORITHMS Step 1
More informationDivide & Conquer Algorithms
Divide & Conquer Algorithms Outline MergeSort Finding the middle point in the alignment matrix in linear space Linear space sequence alignment Block Alignment Four-Russians speedup Constructing LCS in
More informationCSC 505, Spring 2005 Week 6 Lectures page 1 of 9
CSC 505, Spring 2005 Week 6 Lectures page 1 of 9 Objectives: learn general strategies for problems about order statistics learn how to find the median (or k-th largest) in linear average-case number of
More informationAlgorithm Analysis. Spring Semester 2007 Programming and Data Structure 1
Algorithm Analysis Spring Semester 2007 Programming and Data Structure 1 What is an algorithm? A clearly specifiable set of instructions to solve a problem Given a problem decide that the algorithm is
More informationDIVIDE & CONQUER. Problem of size n. Solution to sub problem 1
DIVIDE & CONQUER Definition: Divide & conquer is a general algorithm design strategy with a general plan as follows: 1. DIVIDE: A problem s instance is divided into several smaller instances of the same
More informationLecture #2. 1 Overview. 2 Worst-Case Analysis vs. Average Case Analysis. 3 Divide-and-Conquer Design Paradigm. 4 Quicksort. 4.
COMPSCI 330: Design and Analysis of Algorithms 8/28/2014 Lecturer: Debmalya Panigrahi Lecture #2 Scribe: Yilun Zhou 1 Overview This lecture presents two sorting algorithms, quicksort and mergesort, that
More information15110 Principles of Computing, Carnegie Mellon University - CORTINA. Binary Search. Required: List L of n unique elements.
UNIT 5B Binary Search 1 Binary Search Required: List L of n unique elements. The elements must be sorted in increasing order. Result: The index of a specific element (called the key) or None if the key
More informationCS/ENGRD 2110 Object-Oriented Programming and Data Structures Spring 2012 Thorsten Joachims. Lecture 10: Asymptotic Complexity and
CS/ENGRD 2110 Object-Oriented Programming and Data Structures Spring 2012 Thorsten Joachims Lecture 10: Asymptotic Complexity and What Makes a Good Algorithm? Suppose you have two possible algorithms or
More informationExtended Introduction to Computer Science CS1001.py Lecture 7: Basic Algorithms: Sorting, Merge; Time Complexity and the O( ) notation
Extended Introduction to Computer Science CS1001.py Lecture 7: Basic Algorithms: Sorting, Merge; Time Complexity and the O( ) notation Instructors: Daniel Deutch, Amir Rubinstein Teaching Assistants: Michal
More informationAlgorithms and Data Structures CS-CO-412
Algorithms and Data Structures CS-CO-412 David Vernon Professor of Informatics University of Skövde Sweden david@vernon.eu www.vernon.eu Algorithms and Data Structures 1 Copyright D. Vernon 2014 Algorithmic
More informationRecursion: The Beginning
Department of Computer Science and Engineering Chinese University of Hong Kong This lecture will introduce a useful technique called recursion. If used judiciously, this technique often leads to elegant
More informationLecture Notes 14 More sorting CSS Data Structures and Object-Oriented Programming Professor Clark F. Olson
Lecture Notes 14 More sorting CSS 501 - Data Structures and Object-Oriented Programming Professor Clark F. Olson Reading for this lecture: Carrano, Chapter 11 Merge sort Next, we will examine two recursive
More informationCS 350 Final Algorithms and Complexity. It is recommended that you read through the exam before you begin. Answer all questions in the space provided.
It is recommended that you read through the exam before you begin. Answer all questions in the space provided. Name: Answer whether the following statements are true or false and briefly explain your answer
More informationWhat is a minimal spanning tree (MST) and how to find one
What is a minimal spanning tree (MST) and how to find one A tree contains a root, the top node. Each node has: One parent Any number of children A spanning tree of a graph is a subgraph that contains all
More informationCSE 431/531: Algorithm Analysis and Design (Spring 2018) Greedy Algorithms. Lecturer: Shi Li
CSE 431/531: Algorithm Analysis and Design (Spring 2018) Greedy Algorithms Lecturer: Shi Li Department of Computer Science and Engineering University at Buffalo Main Goal of Algorithm Design Design fast
More information10/11/2013. Chapter 3. Objectives. Objectives (continued) Introduction. Attributes of Algorithms. Introduction. The Efficiency of Algorithms
Chapter 3 The Efficiency of Algorithms Objectives INVITATION TO Computer Science 1 After studying this chapter, students will be able to: Describe algorithm attributes and why they are important Explain
More informationOverview of Sorting Algorithms
Unit 7 Sorting s Simple Sorting algorithms Quicksort Improving Quicksort Overview of Sorting s Given a collection of items we want to arrange them in an increasing or decreasing order. You probably have
More informationCSE 417 Branch & Bound (pt 4) Branch & Bound
CSE 417 Branch & Bound (pt 4) Branch & Bound Reminders > HW8 due today > HW9 will be posted tomorrow start early program will be slow, so debugging will be slow... Review of previous lectures > Complexity
More informationObject-oriented programming. and data-structures CS/ENGRD 2110 SUMMER 2018
Object-oriented programming 1 and data-structures CS/ENGRD 2110 SUMMER 2018 Lecture 8: Sorting http://courses.cs.cornell.edu/cs2110/2018su Lecture 7 Recap 2 Introduced a formal notation for analysing the
More informationCHENNAI MATHEMATICAL INSTITUTE M.Sc. / Ph.D. Programme in Computer Science
CHENNAI MATHEMATICAL INSTITUTE M.Sc. / Ph.D. Programme in Computer Science Entrance Examination, 5 May 23 This question paper has 4 printed sides. Part A has questions of 3 marks each. Part B has 7 questions
More informationLecture 7 Quicksort : Principles of Imperative Computation (Spring 2018) Frank Pfenning
Lecture 7 Quicksort 15-122: Principles of Imperative Computation (Spring 2018) Frank Pfenning In this lecture we consider two related algorithms for sorting that achieve a much better running time than
More informationLecture 2: Getting Started
Lecture 2: Getting Started Insertion Sort Our first algorithm is Insertion Sort Solves the sorting problem Input: A sequence of n numbers a 1, a 2,..., a n. Output: A permutation (reordering) a 1, a 2,...,
More informationCSED233: Data Structures (2017F) Lecture12: Strings and Dynamic Programming
(2017F) Lecture12: Strings and Dynamic Programming Daijin Kim CSE, POSTECH dkim@postech.ac.kr Strings A string is a sequence of characters Examples of strings: Python program HTML document DNA sequence
More information1 Format. 2 Topics Covered. 2.1 Minimal Spanning Trees. 2.2 Union Find. 2.3 Greedy. CS 124 Quiz 2 Review 3/25/18
CS 124 Quiz 2 Review 3/25/18 1 Format You will have 83 minutes to complete the exam. The exam may have true/false questions, multiple choice, example/counterexample problems, run-this-algorithm problems,
More informationQuickSort
QuickSort 7 4 9 6 2 2 4 6 7 9 4 2 2 4 7 9 7 9 2 2 9 9 1 QuickSort QuickSort on an input sequence S with n elements consists of three steps: n n n Divide: partition S into two sequences S 1 and S 2 of about
More informationIntroduction to Computer Science and Programming for Astronomers
Introduction to Computer Science and Programming for Astronomers Lecture 7. István Szapudi Institute for Astronomy University of Hawaii February 21, 2018 Outline 1 Reminder 2 Reminder We have seen that
More informationAlgorithmic Paradigms
Algorithmic Paradigms Greedy. Build up a solution incrementally, myopically optimizing some local criterion. Divide-and-conquer. Break up a problem into two or more sub -problems, solve each sub-problem
More informationProblem solving paradigms
Problem solving paradigms Bjarki Ágúst Guðmundsson Tómas Ken Magnússon Árangursrík forritun og lausn verkefna School of Computer Science Reykjavík University Today we re going to cover Problem solving
More informationSorting. Sorting in Arrays. SelectionSort. SelectionSort. Binary search works great, but how do we create a sorted array in the first place?
Sorting Binary search works great, but how do we create a sorted array in the first place? Sorting in Arrays Sorting algorithms: Selection sort: O(n 2 ) time Merge sort: O(nlog 2 (n)) time Quicksort: O(n
More informationComputational biology course IST 2015/2016
Computational biology course IST 2015/2016 Introduc)on to Algorithms! Algorithms: problem- solving methods suitable for implementation as a computer program! Data structures: objects created to organize
More informationCSE 146. Asymptotic Analysis Interview Question of the Day Homework 1 & Project 1 Work Session
CSE 146 Asymptotic Analysis Interview Question of the Day Homework 1 & Project 1 Work Session Comparing Algorithms Rough Estimate Ignores Details Or really: independent of details What are some details
More informationSAMPLE OF THE STUDY MATERIAL PART OF CHAPTER 6. Sorting Algorithms
SAMPLE OF THE STUDY MATERIAL PART OF CHAPTER 6 6.0 Introduction Sorting algorithms used in computer science are often classified by: Computational complexity (worst, average and best behavior) of element
More informationDynamic programming. Trivial problems are solved first More complex solutions are composed from the simpler solutions already computed
Dynamic programming Solves a complex problem by breaking it down into subproblems Each subproblem is broken down recursively until a trivial problem is reached Computation itself is not recursive: problems
More informationAlgorithms. Algorithms 1.4 ANALYSIS OF ALGORITHMS
ROBERT SEDGEWICK KEVIN WAYNE Algorithms ROBERT SEDGEWICK KEVIN WAYNE 1.4 ANALYSIS OF ALGORITHMS Algorithms F O U R T H E D I T I O N http://algs4.cs.princeton.edu introduction observations mathematical
More informationAlgorithms. Lecture Notes 5
Algorithms. Lecture Notes 5 Dynamic Programming for Sequence Comparison The linear structure of the Sequence Comparison problem immediately suggests a dynamic programming approach. Naturally, our sub-instances
More informationCSE 373: Data Structures and Algorithms
CSE 373: Data Structures and Algorithms Lecture 19: Comparison Sorting Algorithms Instructor: Lilian de Greef Quarter: Summer 2017 Today Intro to sorting Comparison sorting Insertion Sort Selection Sort
More informationTopics Applications Most Common Methods Serial Search Binary Search Search by Hashing (next lecture) Run-Time Analysis Average-time analysis Time anal
CSC212 Data Structure t Lecture 18 Searching Instructor: George Wolberg Department of Computer Science City College of New York @ George Wolberg, 2016 1 Topics Applications Most Common Methods Serial Search
More informationCMSC 451: Dynamic Programming
CMSC 41: Dynamic Programming Slides By: Carl Kingsford Department of Computer Science University of Maryland, College Park Based on Sections 6.1&6.2 of Algorithm Design by Kleinberg & Tardos. Dynamic Programming
More informationCSI33 Data Structures
Outline Department of Mathematics and Computer Science Bronx Community College October 11, 2017 Outline Outline 1 Chapter 6: Recursion Outline Chapter 6: Recursion 1 Chapter 6: Recursion Measuring Complexity
More informationCSE373: Data Structures and Algorithms Lecture 4: Asymptotic Analysis. Aaron Bauer Winter 2014
CSE373: Data Structures and Algorithms Lecture 4: Asymptotic Analysis Aaron Bauer Winter 2014 Previously, on CSE 373 We want to analyze algorithms for efficiency (in time and space) And do so generally
More information