A graphical view of big-o notation. c*g(n) f(n) f(n) = O(g(n))

Transcription

1 ca see that time required to search/sort grows with size of We How do space/time eeds of program grow with iput size? iput. time: cout umber of operatios as fuctio of iput Executio size operatio Assigmet: couts as 1 operatio (operatio cout of side expressio is determied separately) righthad Loop: umber of operatios/iteratio * umber of loop iteratios Method ivocatio: umber of operatios executed betwee executio of iermost assigmet statemet does 2 oatig-poit Each 3 loads, 1 store, ad some iteger operatios to idex ito operatios, total umber of operatios = a 3 + b 2 + c + d (for some So a,b,c,d) most cases, we are oly iterested i the most sigicat I term i the expressio for executio time as a (fastest-growig) size drop all terms except leadig term, ad igore costat Example: matrix multiplicatio for (i = 0 i < i++) Let us focus o executio time. Space aalysis is similar. for (j = 0 j < j++) for (k = 0 k < k++) C[i][j] = C[i][j] + A[i][k] + B[k][j] Problem size: Basic operatio: arithmetic/logical operatio couts as 1 the arrays. Statemet is executed 3 times. whe a method is ivoked ad whe ivocatio returs Asymptotic complexity: O( 3 ) 4 2 Asymptotic complexity: fuctio of iput size. Asymptotic complexity: Asymptotic Complexity express required umber of operatios as a fuctio of iput multiplier Example: f(x) = 13* + 8 f(x) = O() 1 3

2 f() ad g() be fuctios. We say that f() is of order g(), Let O(g()) if there is a costat c > 0 such that for all but a writte f() = for all > 0: f() = for all > 0: operatio cout might deped ot oly o size of iput Subtlety: also o the value of the iput (look at liear search or biary but complexity gives a idea of how rapidly space/time Asymptotic grow as problem size grows. requiremets we have a computig device that ca execute 1000 Suppose per secod. Here is the size of the problem that ca be operatios i a secod, a miute ad a hour by algorithms of dieret solved complexity. asymptotic 1 secod 1 miute 1hour Complexity ,000 3,600,000 log , prove that f() = O(g()), d a 0 ad c such that To c g() for all > 0. f() Formal deitio of O() otatio: ite umber of positive values of, f() c g() I other words, g() sooer or later overtakes f() as gets large. Example: f() = + 5 g() =. We show that f() = O(g()). Choose c = 6: Example: f() = 17 g() = 3 2. We show that f() = O(g()). Choose c = 6: A graphical view of big-o otatio c*g() f() f() = O(g()) search). For big-o determiatio, use worst-case sceario

3 searchig ad sortig algorithms, you ca usually determie For complexity by coutig comparisos. big-o you usually ed up doig some xed umber of Reaso: operatios per compariso. arithmetic/logical of times while loop executes depeds ot oly o size of iput Number a but o the values i it ad the value of v. array liear search, worst-case sceario is that all array elemets are For examied. Aalysis of biary search is a little more dicult. public static boolea biarysearch(comparable[] a, Object v) { Example: selectio sort... public static void selectiosort(comparable[] a) { <-- array of size while (left <= right){ <-- how may times does this loop execute? <-- iteratios for (it i = 0 i < a.legth i++) { middle = (left + right)/2 it MiPos = i it test = a[middle].compareto(v) <-- compariso for (it j = i+1 j < a.legth j++) { <-- -i-1 iteratios if (test < 0) left = middle+1 <-- compariso if (a[j].compareto(a[mipos]) < 0) else MiPos = j if (test == 0) { Comparable temp = a[i] retur true a[i] = a[mipos] a[mipos] = temp else right = middle-1 Total umber of comparisos = (-1) + (-2) = (-1)/2 //if we reach here, we didt fid the object Complexity: O( 2 ) retur false Example: liear search public static boolea liearsearch(comparable[] a, Object v) { it i = 0 while (i < a.legth) <-- How may times does this loop execute??? if (a[i].compareto(v) == 0) retur true <-- compariso else i++ retur false Big-O complexity: worst-case sceario So complexity of liear search = O() where is size of iput array. 9 11

4 of iteratios of while loop depeds o values i array ad Number of v. value lets make worst case estimates: if array is of size, what is OK, worst case umber of iteratios you make? the to see that if size of array is, rst iteratio cuts the size of Easy iterval you eed to look at to at most ceilig((-1)/2). So if the ay time you have a procedure whose complexity is less tha O(), Note: meas ituitively that procedure does ot examie all of the iput. it Aalysis of merge-sort: public static Comparable[] mergesort(comparable[] A, it low, it high) { if (low < high - 1) //at least three elemets {it mid = (low + high)/2 Comparable[] A1 = mergesort(a, low, mid) <-- comparisos i method Comparable[] A2 = mergesort(a, mid +1, high) <-- comparisos i method Why is this wrog? retur merge(a1,a2) <-- comparisos i method... c(1) = 0 c(2) = 1 c() = 2c(/2) + It ca be show that c() = O(log 2 ()) Aalysis of quicksort: tricky public static void quicksort(comparable[] A, it l, it h) { if (l < h) {it p = partitio(a,l+1,h,a[l]) <--- comparisos //move pivot ito its fial restig place //swap A[p-1] ad A[l] Comparable temp = A[p-1] A[p-1] = A[l] A[l] = temp c() is worst-case umber of comparisos, quicksort(a,l,p-1) <-- comparisos c(1) = 1 c(2) = 2 quicksort(a,p,h) <-- comparisos c() = 1 + c(ceilig((-1)/2)) Icorrect attempt: It ca be show that c() = O(log 2 ()) c(1) = 1 c(2) = 1 c() = (-1) + 2c(/2) partitio sortig the two partitioed arrays 13 15

5 for the same problem ca vary eormously i asymptotic Programs eciecy. for quicksort: oe of the partitioed array is empty, ad Worst-case other has (-1) elemets! the the average (ot worst-case), quick-sort rus i log 2 () time, O is why it is usually preferred i practice. which approach to avoidig worst-case behavior: pick pivot carefully so it Oe array i half. May heuristics for doig this, but oe of them partitios ca be show that c() = O(2 ). Cost of computig value is It i the size of the iput! expoetial Iterative Fiboacci Code fib () = fib(-1) + fib(-2) > 2 = fib(-1) + fib(-2) fib() = 1 fib(1) = 1 fib(2) is a recursive program: Here it fib(it ) { static ( <= 2) retur 1 if dad = 1 graddad = 1 curret = 1; for (i = 3; i <= ; i++) { graddad = dad; dad = curret; curret = dad +graddad; pritf("aswer is " + curret); else retur fib(-1) + fib(-2) Number of times loop is executed is bouded by. Each iteratio does some costat amout of work. => Time complexity of algorithm = O() Remember: big-o is worst-case complexity. fib(5) fib(4) fib(3) fib(3) fib(2) fib(2) fib(1) So actual recurrece relatio is c(1) = 1 c(2) = 1 c() = (-1) + c(-1) fib(2) fib(1) partitio sortig the two partitioed arrays c() = c(-1) + c(-2) + 2 It ca be show that c() = O( 2 ) c(2) = 1 c(1) = 1 ca guaratee that worst-case behavior will ot show up

6 CS 211, you are expected to kow the complexity of the I we discuss i class. algorithms are also expected to kow how to determie iformally the You complexity (i closed-form) of toy recursive programs asymptotic Sheet for closed-form expressios Cheat relatio Closed-form Example Recurrece similar to merge-sort or biary search c(1) = 1 c() = O() Liear search c() = 1 + c(-1) c(1) = 1 c() = + c(-1) c() = O(^2) Quicksort c(1) = 1 c() = 1 + c(/2) c() = O(log()) Biary search (1) = 1 c() = + c(/2) c() = O() c(1) = 1 c() = + 2c(/2) c() = O(*log()) Mergesort c(1) = c(2) = 1 Fiboacci c() = c(-1)+c(-2)+1 c() = O(2^) 21