CS 440 Theory of Algorithms /

Transcription

1 CS 440 Theory of Algorithms / CS 468 Algorithms in Bioinformaticsi Introduction Copyright 2007 Pearson Addison-Wesley. All rights reserved. Copyright 2007 Pearson Addison-Wesley. All rights reserved. The most valuable acquisitions in a scientific or technical education are the general-purpose mental tools which remain serviceable for a lifetime. George Forsythe, Whatto t Do Till the Computer Scientist Comes, 1967 "A person well trained in computer science knows how to deal with algorithms; how to construct them, manipulate them, understand them, analyze them. This knowledge prepares him for much more than writinggood computer programs; it is a general-purpose mental tool which will be a definite aid to his understanding of other subjects, whether they be chemistry, linguistics, i or music, etc." DonaldKnuth, Sl SelectedPapers tdp on Computer Science, 1996 Two ideas lie gleaming on the jeweler's velvet. The first is the calculus, the second, the algorithm. The calculus and the rich body of mathematical analysis to which it gave rise made modern science possible; but tithas been the algorithm thathas made possible the modern world. David Berlinski, The Advent of the Algorithm, 2000 Copyright 2007 Pearson Addison-Wesley. All rights reserved. Design and Analysis of Algorithms - Chapter 1 1-1

2 What is an Algorithm? An algorithm is a sequence of unambiguous instructions for solving a problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time. Copyright 2007 Pearson Addison-Wesley. All rights reserved. Design and Analysis of Algorithms - Chapter Are algorithms just of theoretical intesrets? Application areas of algorithms Copyright 2007 Pearson Addison-Wesley. All rights reserved. Design and Analysis of Algorithms - Chapter 1 1-3

3 What is an algorithm? Recipe, process, method, technique, procedure, routine, with following requirements: 1. Finiteness terminates after a finite number of steps 2. Definiteness rigorously and unambiguously specified 3. Input valid inputs are clearly specified 4. Output can be proved to produce the correct output given a valid input 5. Effectiveness steps are sufficiently simple and basic Copyright 2007 Pearson Addison-Wesley. All rights reserved. Design and Analysis of Algorithms - Chapter Why study algorithms? Theoretical importance The core of computer science Practical importance A practitioner s toolkit of known algorithms Framework for designing and analyzing algorithms for new problems Copyright 2007 Pearson Addison-Wesley. All rights reserved. Design and Analysis of Algorithms - Chapter 1 1-5

4 Historical Perspective Euclid s algorithm for finding the greatest common divisor Muhammad ibn Musa al-khwarizmi 9 th century mathematician Copyright 2007 Pearson Addison-Wesley. All rights reserved. Design and Analysis of Algorithms - Chapter Notion of Algorithm problem algorithm input computer output t Algorithmic solution Copyright 2007 Pearson Addison-Wesley. All rights reserved. Design and Analysis of Algorithms - Chapter 1 1-7

5 Example of Computational Problem: Sorting Statement of problem: Input: A sequence of n numbers <a 1, a 2,, a n > Output: A reordering of the input sequence <a 1, a 2,, a n> n so that a i a j whenever i < j Instance: The sequence <5, 3, 2, 8, 3> Algorithms: Selection sort Insertion sort Merge sort (many others) Copyright 2007 Pearson Addison-Wesley. All rights reserved. Design and Analysis of Algorithms - Chapter Selection Sort Input: array Output: array sorted in non-decreasing order Algorithm: for =1 to swap with smallest of see also pseudocode, section 3.1 Copyright 2007 Pearson Addison-Wesley. All rights reserved. Design and Analysis of Algorithms - Chapter 1 1-9

6 Euclid s Algorithm Problem: Find gcd(m,n), the greatest common divisor of two nonnegative, not both zero integers m and n Examples: gcd(60,24) = 12, gcd(60,0) = 60, gcd(0,0) =? Euclid s algorithm is based on repeated application of equality gcd(m,n) = gcd(n, m mod n) until the second number becomes 0, which makes the problem trivial. Example: gcd(60,24) = gcd(24,12) = gcd(12,0) = 12 Copyright 2007 Pearson Addison-Wesley. All rights reserved Two descriptions of Euclid s algorithm Step 1 If n = 0, return m and stop; otherwise go to Step 2 Step 2 Divide m by n and assign the value fo the remainder to r Step 3 Assign the value of n to m and the value of r to n. Go to Step 1. while n 0do r m mod n m n n r return m Copyright 2007 Pearson Addison-Wesley. All rights reserved. 1-11

7 Other methods for computing gcd(m,n) Consecutive integer checking algorithm Step 1 Assign the value of min{m,n} to t Step 2 Divide m by t. If the remainder is 0,go to Step 3; otherwise, go to Step 4 Step 3 Divide n by t. If the remainder is 0, return t and stop; otherwise, go to Step 4 Step 4 Decrease t by 1 and go to Step 2 Copyright 2007 Pearson Addison-Wesley. All rights reserved Other methods for gcd(m,n) [cont.] Middle-school procedure Step 1 Find the prime factorization of m Step 2 Find the prime factorization of n Step 3 Find all the common prime factors Step 4 Compute the product of all the common prime factors and return it as gcd(m,n) Is this an algorithm? Copyright 2007 Pearson Addison-Wesley. All rights reserved. 1-13

8 Sieve of Eratosthenes Input: Integer n 2 Output: List of primes less than or equal to n for p 2to n do A[p] p for p 2to n do if A[p] 0 //p hasn t been previously eliminated from the list j p* p while j n do A[j] 0 //mark element as eliminated j j + p Example: Copyright 2007 Pearson Addison-Wesley. All rights reserved More Problems Multiplication of two integers (e.g., 1526 x 77) American way British way Chinese way ( rch_type=&aq=6&oq=multi ) Egyptian way Coin change problem Greedy Brute force (exhaustive search) Other techniques? Copyright 2007 Pearson Addison-Wesley. All rights reserved. Design and Analysis of Algorithms - Chapter

9 Problem Solving Problem 1: Old world puzzle A peasant finds himself on a riverbank with a wolf, a goat, and a head of cabbage. He needs to transport all three to the other side of the river in his boat. However, the boat has room for only the peasant himself and one other item. In his absence, the wolf would eat the goat, and the goat would eatthe cabbage. Solve this problem, or prove it has no solution. * The peasant is vegetarian who doesn t like to eat cabbage. Copyright 2007 Pearson Addison-Wesley. All rights reserved Problem Solving Problem 2: New world puzzle There are four people, A, B, C, and D, who want to cross a bridge; they all begin on the same side. You have 17 minutes to get them all across to the other side. It is night, and they have one flashlight. A maximum of two people can cross the bridge at one time. Any party that crosses, either one or two people, must have the flashlight with them. The flashlight must be walked edb back and forth; it cannot be thrown, for example. A takes 1 minute to cross the bridge, B takes 2 minutes, C takes 5 minutes, and D takes 10 minutes. A pair must walk together at the rate of the slower person s pace. For example, if A and D walk across first, 10 minutes have elapsed when they get to the other side of the bridge. If D returns the flashlight, a total of 20 minutes have passed and you have failed the mission. Solve this problem, or prove it has no solution. Copyright 2007 Pearson Addison-Wesley. All rights reserved. 1-17

10 Fundamentals of Algorithmic Problem Solving Copyright 2007 Pearson Addison-Wesley. All rights reserved Some Well-known Computational Problems Problems Sorting Searching Shortest paths in a graph Minimum spanning tree Primality testing Traveling salesman problem Knapsack problem Chess Towers of Hanoi Program termination Problem Categories Graph Problem String Processing Combinatorial Problems Geometric Problems Numerical Problems Copyright 2007 Pearson Addison-Wesley. All rights reserved. Design and Analysis of Algorithms - Chapter

11 Basic Issues Related to Algorithms How to design algorithms How to express algorithms Proving correctness Efficiency Theoretical analysis Empirical analysis Optimality Copyright 2007 Pearson Addison-Wesley. All rights reserved. Design and Analysis of Algorithms - Chapter Algorithm design strategies Brute force Greedy approach Divide and conquer Dynamic programming Decrease and conquer Backtracking and Branch and bound Transform and conquer Space and time tradeoffs There are more, for example, Machine learning, Randomized algorithms, Genetic algorithms, etc. Copyright 2007 Pearson Addison-Wesley. All rights reserved. Design and Analysis of Algorithms - Chapter

12 Analysis of Algorithms How good is the algorithm? Correctness Time efficiency Space efficiency i Does there exist a better algorithm? Lower bounds Optimality Copyright 2007 Pearson Addison-Wesley. All rights reserved. Design and Analysis of Algorithms - Chapter