Choice of C++ as Language

Transcription

1 EECS 281: Data Structures and Algorithms Principles of Algorithm Analysis Choice of C++ as Language All algorithms implemented in this book are in C++, but principles are language independent That is, an algorithm that is n 3 in C++, is also n 3 in Java, Pascal, Fortran Today There are many ways to solve an algorithmic problem Goal of 281 is to solve problems using most efficient method where efficient is defined in terms of time and space Key to 281 is ability to successfully analyze different methods/algorithms

2 Objective Use known mathematical principles to determine efficiency, and thus guide choice of algorithms Determine complexity by Method 1: Experimentation: run tests on data sets Method 2: Solution of Recurrence: mathematical solution to representation of recursive algorithm Method 3: Analysis: divide algorithm into constituent parts Nature of Input Data Determine nature of input data best case: data that promotes best possible behavior of algorithm average case: data that promotes expected behavior of algorithm worst case: pathological data that promotes worst possible behavior of algorithm Desirable Requirements for a General Methodology for Analyzing Algorithms Takes into account all possible input Allows evaluating relative efficiency of any two algorithms independent of hardware and software environment Allows studying a high-level description without actually implementing or running the code

3 Method 1: Empirical Analysis Given two algorithms to solve the same problem, run both codes on varying, typical data sets to determine which algorithm takes longer Can be used to validate other analyses Note that when empirical analyses take a long time, one should use other methods Method 2: Recurrence Relations Based upon mathematical solution of representation of recursive algorithms Factorial Program T(n) = t 1, when n = 0 T(n - 1) + t 2, when n > 0 Will be discussed later in context of recursive algorithms Method 3: Analysis (Decomposition/Observation) Based upon analysis, decomposition, and observation of algorithms Decomposition (Operation Count) analysis in the small e.g., what does currentmax <- A[0] cost? Observation analysis in the large i.e., all primitive operations cost the same

4 Detailed Model of Computer (Goodrich) Primitive Operations (each costs 1) a) Variable assignment b) Arithmetic operation c) Comparison d) Array indexing or Pointer reference e) Function call f) Function return Counting Primitive Operations Define a set of high-level primitive operations that are independent of the programming language used: Assignment statement Calling a function Compare operation Arithmetic operation Count how many primitive operations are executed as an estimate of the running time for the algorithm Decomposition (Operation Count) Algorithm arraymax(a, n) Input array A of n ints Output max element of A currentmax A[0] for i 1 to n 1 do if A[i]>currentMax then currentmax A[i] return currentmax Best case (if never true) Worst case (if always true)

5 Observation Algorithm arraymax(a, n) Input array A of n ints Output max element of A currentmax A[0] for i 1 to n 1 do if A[i]>currentMax then currentmax A[i] return currentmax O( ) O( ) O( ) O( ) O( ) Best case (if never true) O( ) Worst case (if always true) O( ) An Aside (or is it?) What is the sum of all integers from 0 to 4, inclusive? What is the sum of all integers from 3 to 12, inclusive? What is the sum of all integers from 1 to 25, inclusive? Asymptotic Algorithm Analysis In algorithm analysis, focus on the growth rate of the running time as a function of the input size N. e.g. big-picture observation: arraymax grows proportionally to n Asymptote: In mathematics, a line or curve that acts as the limit of another line or curve. [Britannica Concise Encyclopedia ]

6 Growth of Functions Most algorithms defined in terms of primary parameter N size of file number of input data characters in text string N is typically directly proportional to size of data set being processed Growth of Functions Typically described in terms of 1 (constant) log N (logarithmic) N (linear) N log N (N log N) N 2 (quadratic) N 3 (cubic) 2 N (exponential) O-Notation (Upper Bound) Def n: A function f(n) is said to be O(g(N)) if there exist constants c 0 and N 0, such that f(n) < c 0 g(n) for all N > N 0 Notes Nothing is implied for N N 0, except N 0 1 Nothing is implied about c 0, except c 0 > 0

7 O-Notation Purpose Quantify upper bound (worst case performance) on total running time Ignore small terms in formulas Ignore trivial parts of program Ignore constants in notation Focus on largest (leading) terms of a mathematical expression Defined in terms of asymptotic tightness Omega Notation (Lower Bound) Def n: A function f(n) is said to be Ω(g(N)) if there exist constants c 0 and N 0, such that f(n) > c 0 g(n) for all N > N 0 Notes Nothing is implied for N N 0, except N 0 1 Nothing is implied about c 0, except c 0 > 0 Omega Notation Purpose Quantify lower bound (best case performance) on total running time Define the asymptotic lower bound of an algorithm s performance

8 Theta Notation (Upper and Lower Bound) Def n: A function f(n) is said to be θ(g(n)) if there exist constants c 0, c 1, and N 0, such that 0 c 0 g(n) f(n) c 1 g(n) for all N > N 0 Notes Nothing is implied for N N 0, except N 0 1 Nothing is implied about c 0 or c 1, except c 0 > 0 Rules for Analysis: Sequential Composition S 1 ; S 2 ; S m ; t s1 + t s2 + t s3 = max(o(s 1 ), O(s 2 ), O(s 3 )) Rules for Analysis: Iteration for (S 1 ;S 2 ;S 3 ) S 4 ; t s1 + t s2 (n+1) + t s3 (n) + t s4 (n) = max(o(s 1 ), O(s 2 ) x (n+1), O(s 3 ) x (n), O(s 4 ) x (n)) = max(o(1), O(n+1), O(n), O(s 4 ) x (n)) = O((s 4 ) x (n))

9 Rules for Analysis: Conditional Execution if (S 1 ) S 2 ; else S 3 ; t s1 + t s2 + t s3 = max(o(s 1 ), O(s 2 ), O(s 3 )) Other Mathematical Functions Ceiling Fxn x : smallest integer >= x Floor Fxn x : largest integer <= x lg N: binary logarithm (log 2 N) N!: N factorial Summary: Algorithm Complexity Three methods to determine algorithm performance empirical solution of recurrence relations analysis To understand performance, need language to describe Big- Oh: asymptotic upper bound Omega: asymptotic lower bound Theta: asymptotic upper and lower bound

10 Loose Ends Section 3.2 provides a a quick mathematical review Summation Logarithms and exponents Proposition 3.16 on p.126 Big-Oh rules Lots of examples in Section 3.5