Algorithm. Algorithm Analysis. Algorithm. Algorithm. Analyzing Sorting Algorithms (Insertion Sort) Analyzing Algorithms 8/31/2017

Transcription

1 8/3/07 Analysis Introduction to Analysis Model of Analysis Mathematical Preliminaries for Analysis Set Notation Asymptotic Analysis What is an algorithm? An algorithm is any well-defined computational procedure that takes some value, or set of values, as input and produces some value, or set of values, as output. Input Output An algorithm is a sequence of computational steps that transform the input into the output. An algorithm is a tool for solving a well-specified computational problem. Ex) sorting problem A sequence of n numbers <a, a, a 3, a n > Sorting Permutation <a, a, a 3, a n > of input sequence such that a a a 3 a n What is a correct algorithm for solving a problem? An algorithm is said to be correct if it halts with the correct output for every possible input. What criteria are used to determine which algorithm is better? Measure of efficiency is speed How long an algorithm takes to produce output. How do we evaluate which algorithm is better? By analyzing algorithms mathematically 3 4 Analyzing s (Insertion Sort) Model of Implementation Single processor random access machine instructions are executed one after another (no concurrent execution!) The Data (input or output) are integer and floating point. Since the time taken by an algorithm grows with the size of the input, the running time of an algorithm can be described as a function of the size of the input. The running time of an algorithm on a particular input is the number of primitive operations or steps executed. Input: A sequence of n numbers <a, a, a 3, a n > Output: Permutation <a, a, a 3, a n > of input sequence such that a a a 3 a n Insert 3 Insert Insert 4 Insert Insert

2 8/3/07 (Insertion Sort) For each iteration, an element is placed into the correct position in the sorted sub list. { 3 for j = to length of array A do 0 i = i A[i + ] = key 3 4 (Insertion Sort) For each iteration, an element is placed into the correct position in the sorted sub list. { 3 for j = to length of array A do 0 i = i A[i + ] = key 3 4 C C C 3 C 4 C 5 C 6 C (Insertion Sort: Best Case) (Insertion Sort: Best Case) Insert Insert 3 Insert 4 Insert 5 Insert { 3 for j = to length of array A do 0 i = i A[i + ] = key 3 4 C (n ) C (n ) C 3 (n ) C 4 (n ) C 5 (n ) C 6 (n ) C 7 (n ) T(n) = c n + c (n ) + c 3(n ) + c 4(n ) + c 7(n ) = (c + c + c 3 + c 4 + c 7)n - (c + c 3 + c 4 + c 7) 9 0 (Insertion Sort: Worst Case) (Insertion Sort: Worst Case) Insert 5 Insert 4 Insert 3 Insert Insert { 3 for j = to length of array A do 0 i = i A[i + ] = key 3 4 C (n ) C (n ) C 3(n ) n(n + ) C 4( ) n(n ) C 5( ) n(n ) C 6( ) C 7(n )

3 8/3/07 (Insertion Sort: Worst Case) Insertion Sort: Worst Case T n = C n + C n + C 3 n + C 4 n n + + C 5 n n + C 6 n n + C 7 n Importance of Worst Case Analysis The worst-case running time gives an upper bound on the running time for any input. For some algorithms, the worst case occurs fairly often. The average case is usually as bad as the worst case. = C4 + C5 + C6 n + C + C + C 3 + C4 C5 C6 + C 7 n (C + C + C 3 + C 4 + C 7 ) 3 4 Sets & Set Notations Instead of using an exact running time, we can use simplifying abstractions to ease our analysis of algorithms. We can use asymptotic notation to describe the speed of an algorithm as an order of growth based on the input size n. Ex) Insertion Sort best case: order of growth = n worst case: order of growth = n Sets A set is a collection of objects (members of the set) without repetition. Set notations : x A: x A: A B: A = B A B: empty set (a set with no element) x is a member of set A x is not a member of set A set A is contained in set B if and only if A B and B A if A B but not B A, then set A is a proper subset of set B 5 6 Sets & Set Notations Sets can be specified using roster form (aka tabular form). E.g.: A = {, 4, 6, 8, 0, - strict enumeration B = {, 4, 6, - pattern enumeration Or using a set builder (aka set former). E.g.: A = {x x > 0, x is an even integer Operations: A B: (Union) {x: x A or x B A B: (Intersection) {x: x A and x B A B: (Difference) {x: x A and x B A B: (Cartesian Product) {(x, y): x A and y B A : (Power Set), is the set of all subsets of A

4 8/3/07 (Examples) E.g.) Let A ={, and B = {, 3. Then A B = A B = A B = A B = A = {,, 3 { { {(, ), (, 3), (, ), (, 3) {, {, {, {, Ā: The complement of set A. To explain complementation, we need to define a Universal set U If U is specified, then Ā ={x: x U and x A 9 0 E.g.) Let U = {x: x > 0, x N and A = {x: x > 0, x N, x is even where N is natural number Set A and set B are disjoint if A B = DeMorgan s Laws Ā ={x x > 0, x is odd A B A B A B A B Functions A function is a rule that assigns to elements of one set a unique element of another set. f: A B The domain of f is a subset of A. The range of f is a subset of B. f is a total function if the domain of function f is all elements of A. f is a partial function if the domain of function f is a proper subset of A. Big O-Notation For a given function g(n), we denote by O(g(n)) the set of functions such that, O(g(n))= {f(n) there exist positive constants c and n 0 such that 0 f(n) cg(n) for all n n 0 f(n) = O(g(n)) indicates that a function f(n) is a member of O(g(n)). We say that g(n) is an asymptotic upper bound for f(n)

5 8/3/07 f(n) = (g(n)) f(n) = O(g(n)), since -notation is stronger notation than O- notation. Written set-theoretically (g(n)) O(g(n)). 5 6 Example: Show that 7n 5 = O(n ) Solution: Based on big-o notation, we need to find two integers c and n 0 such that 7n 5 cn for all n n 0 With c = 7 and n 0 = 7n 5 7 n for all n 7n 5 = O(n ) Example: Show that 35n = O(n 3 ) Solution: Based on the big-o notation, we need find two integers c and n 0 such that 35n c n 3 for all n n 0 With c = 36 and n 0 = 5 35n n 3 for all n 5 35n = O(n 3 ) 7 8 Example: Show that 6 n + n = O( n ) Solution: Based on the big-o notation, we need find two integers c and n 0 such that 6 n + n c n for all n n 0 With c = 7 and n 0 = 5 6 n + n 7 n for all n 5 6 n + n = O( n ) Big -Notation For a given function g(n), we denote by (g(n)) the set of functions such that (g(n)) = {f(n) there exists positive constants c, c and n 0 such that 0 c g(n) f(n) c g(n) for all n n 0 f(n) = (g(n)) indicates that a function f(n) is a member of (g(n)). We say that g(n) is an asymptotically tight bound for f(n)

6 8/3/07 Ex) Show n 3n ( n ) We need to find out positive c, c and n 0 such that cn n 3n cn for all n n0 3 c c n 3 Left hand side : c is hold with c and n 7 n 4 3 Right hand side : c is hold with c and n n n n 3n n is hold with n Big -Notation For a given function g(n), we denote by (g(n)) the set of functions such that (g(n)) = {f(n) there exist positive constants c and n 0 such that 0 cg(n) f(n) for all n n 0 We write f(n) = (g(n)) to indicate that a function f(n) is a member of (g(n)), i.e., that g(n) is an asymptotic lower bound for f(n) Example: Show 3n + = (n) By Big- notation we need to find out two integer c and n 0 such that 3n + cn for all n n 0 With c = and n, 3n + n for all n 3n + = (n) Example: Show n + n 3 = (n 00 ) By Big- notation we need to find out two integer c and n 0 such that n + n 3 cn 00 for all n n 0 With c = and large enough n 0, n + n 3 n 00 for all n n 0 is true, since left hand side grows exponentially and right hand side grows by polynomial

7 8/3/07 Asymptotic Notations (Little o) Little o-notation For a given function g(n), we denote by o(g(n)) the set of functions such that o(g(n)) = {f(n) for any positive constant c > 0, there exists a constant n 0 such that 0 f(n) < cg(n) for all n n 0 The asymptotic upper bound provided by big O- notation may or may not be asymptotically tight. Small o-notation is used to denote an upper bound that is not asymptotically tight. Asymptotic Notations (Little o) The relation f(n) = o(g(n)) implies that lim 0 n g( n) Ex) n = o(n ) but n o(n ) Asymptotic Notations (Little ω) Little ω-notation For a given function g(n), we denote by ω(g(n)) the set of functions such that ω(g(n)) = {f(n) for any positive constant c > 0, there exists a constant n 0 such that 0 cg(n) < f(n) The asymptotic lower bound provided by big - notation may or may not be asymptotically tight. Small ω-notation denotes a lower bound that is not asymptotically tight. Asymptotic Notations (Little ω) The relation f(n) = ω(g(n)) implies that lim n g ( n ) Ex) ( n n ) but n ( n ) Properties of Order Properties of Order Transitivity if f(n) O(g(n)) and g(n) O(h(n)) then f(n) O(h(n)) If f(n) (g(n)) and g(n) (h(n)) then f(n) (h(n)) If f(n) (g(n)) and g(n) (h(n)) then f(n) (h(n)) If f(n) o(g(n)) and g(n) o(h(n)) then f(n) o(h(n)) If f(n) (g(n)) and g(n) (h(n)) then f(n) (h(n)) Reflexivity O( ) ( ) ( ) Symmetry f(n) (g(n)) if and only if g(n) (f(n)) Transpose Symmetry f(n) O(g(n)) if and only if g(n) (f(n)) f(n) o(g(n)) if and only if g(n) (f(n)) 4 4 7