Data Structure Lecture#5: Algorithm Analysis (Chapter 3) U Kang Seoul National University

Transcription

1 Data Structure Lecture#5: Algorithm Analysis (Chapter 3) U Kang Seoul National University U Kang 1

2 In This Lecture Learn how to evaluate algorithm efficiency Learn the concept of average case, best case, and worst case running time Learn the important concept of Asymptotic Analysis Big-Oh Omega Theta U Kang 2

3 Algorithm Efficiency There are often many approaches (algorithms) to solve a problem. How do we choose between them? At the heart of computer program design are two (sometimes conflicting) goals. 1. To design an algorithm that is easy to understand, code, and debug. 2. To design an algorithm that makes efficient use of the computer s resources. U Kang 3

4 Algorithm Efficiency (cont) Goal (1) is the concern of Software Engineering. Goal (2) is the concern of data structures and algorithm analysis. When goal (2) is important, how do we measure an algorithm s cost? U Kang 4

5 How to Measure Efficiency? Measuring efficiency Empirical comparison (run programs) Asymptotic Algorithm Analysis Critical resources: time, space, programmers effort, ease of use For most algorithms, running time depends on size of the input. Running time is expressed as T(n) for some function T on input size n. We count the number of basic operations U Kang 5

6 How to Measure Efficiency? We count the number of basic operations Basic operation: its time to complete does not depend on the values of its operands or n. Corresponds to one statement in a programming language Exception: for/while loop Addition, subtraction, division, multiplication Assignment Comparison U Kang 6

7 Examples of Growth Rate Example 1. Position of largest value in "A */ static int largest(int[] A) { int currlarge = 0; // Position of largest for (int i=1; i<a.length; i++) if (A[currlarge] < A[i]) currlarge = i; // Remember pos return currlarge; // Return largest pos } U Kang 7

8 Examples (cont) Example 2: Assignment statement. x = 1; Example 3: sum = 0; for (i=1; i<=n; i++) for (j=1; j<=n; j++) sum++; } U Kang 8

9 Growth Rate Graph Stirling s Approximation nn! ~ 2ππnn( nn ee )nn U Kang 9

10 Best, Worst, Average Cases Not all inputs of a given size take the same time to run. Sequential search for K in an array of n integers: begin at first element in the array and look at each element in turn until K is found Best case cost: Worst case cost: Average case cost: U Kang 10

11 Which Analysis to Use? Average time? Best case time? Worst case time? U Kang 11

12 Which Analysis to Use? While average time appears to be the fairest measure, it may be difficult to determine. When is the best case time important? When is the worst case time important? U Kang 12

13 Faster Computer or Algorithm? Suppose we buy a computer 10 times faster. i.e., process 10 times larger number of operations at the same interval n: size of input that can be processed in one second on old computer n : size of input that can be processed in one second on new computer f(x) = # of operations for input of size x Find n such that f(n ) = 10 f(n) U Kang 13

14 Asymptotic Analysis: Big-oh (Informal) Definition: T(n) is in the set O(f(n)) if T(n) cf(n) for large n (c is a constant you can choose) Examples T(n) = 5n is in the set O(n) T(n) = 5n + 6 is in the set O(n) T(n) = 4n 2 + 3n + 7 is in the set O(n 2 ) T(n) = 4n 2 + 3n + 7 is not in set O(n) Why? U Kang 14

15 Asymptotic Analysis: Big-oh (Formal) Definition: For T(n) a non-negatively valued function, T(n) is in the set O(f(n)) if there exist two positive constants c and n 0 such that T(n) cf(n) for all n > n 0. Use: The algorithm is in O(n 2 ) in [best, average, worst] case. Meaning: For all data sets big enough (i.e., n>n 0 ), the algorithm always executes in less than cf(n) steps in [best, average, worst] case. U Kang 15

16 Big-oh Notation (cont) Big-oh notation indicates an upper bound. Example: If T(n) = 3n 2 then T(n) is in O(n 2 ). Look for the tightest upper bound: While T(n) = 3n 2 is in O(n 3 ), we prefer O(n 2 ). U Kang 16

17 Big-Oh Examples Example 1: Finding value X in an array (average cost). Then T(n) = c s n/2. For all values of n > 1, c s n/2 c s n. Therefore, the definition is satisfied for f(n)=n, n 0 = 1, and c = c s. Hence, T(n) is in O(n). U Kang 17

18 Big-Oh Examples (2) Example 2: Suppose T(n) = c 1 n 2 + c 2 n, where c 1 and c 2 are positive. c 1 n 2 + c 2 n c 1 n 2 + c 2 n 2 (c 1 + c 2 )n 2 for all n > 1. Then T(n) cn 2 whenever n > n 0, for c = c 1 + c 2 and n 0 = 1. Therefore, T(n) is in O(n 2 ) by definition. Example 3: T(n) = d. Then T(n) is in O(1). What is c? What is n 0? U Kang 18

19 A Common Misunderstanding The best case for my algorithm is n=1 because that is the fastest. WRONG! Big-oh refers to a growth rate as n grows to. Best case is defined for the input of size n that is cheapest among all inputs of size n. U Kang 19

20 Big-Omega Definition: For T(n) a non-negatively valued function, T(n) is in the set Ω(g(n)) if there exist two positive constants c and n 0 such that T(n) cg(n) for all n > n 0. Meaning: For all data sets big enough (i.e., n > n 0 ), the algorithm always requires more than cg(n) steps. Lower bound. U Kang 20

21 Big-Omega Example T(n) = c 1 n 2 + c 2 n. c 1 n 2 + c 2 n c 1 n 2 for all n > 1. T(n) cn 2 for c = c 1 and all n > n 0 = 1. Therefore, T(n) is in Ω(n 2 ) by the definition. We want the greatest lower bound. U Kang 21

22 Theta Notation When big-oh and Ω coincide, we indicate this by using Θ (big-theta) notation. Definition: An algorithm is said to be in Θ(h(n)) if it is in O(h(n)) and it is in Ω(h(n)). U Kang 22

23 What You Need to Know The concept of average case, best case, and worst case running time When they are important Evaluate the [average, best case, worst case] running time of codes Intuition and definition of Asymptotic Analysis Big-Oh, Omega, Theta Evaluate the complexity of codes using asymptotic analysis U Kang 23

24 Questions? U Kang 24

25 Big O in Real World Built for Yeosu EXPO in 2012 U Kang 25

26 Big O in Real World Big-O show U Kang 26