Outlie ad Readig Aalysis of Algorithms Iput Algorithm Output Ruig time ( 3.) Pseudo-code ( 3.2) Coutig primitive operatios ( 3.3-3.) Asymptotic otatio ( 3.6) Asymptotic aalysis ( 3.7) Case study Aalysis of Algorithms 2 Ruig Time Experimetal Studies The ruig time of a algorithm varies with the iput ad typically grows with the iput size Average case difficult to determie We focus o the worst case ruig time Easier to aalyze Crucial to applicatios such as games, fiace ad robotics Ruig Time 2 8 6 4 2 best case average case worst case 2 3 4 Iput Size Write a program implemetig the algorithm Ru the program with iputs of varyig size ad compositio Use a method like System.curretTimeMillis() to get a accurate measure of the actual ruig time Plot the results Time (ms) 9 8 7 6 4 3 2 Iput Size Aalysis of Algorithms 3 Aalysis of Algorithms 4 Limitatios of Experimets It is ecessary to implemet the algorithm, which may be difficult Results may ot be idicative of the ruig time o other iputs ot icluded i the experimet. I order to compare two algorithms, the same hardware ad software eviromets must be used Theoretical Aalysis Uses a high-level descriptio of the algorithm istead of a implemetatio Takes ito accout all possible iputs Allows us to evaluate the speed of a algorithm idepedet of the hardware/software eviromet Aalysis of Algorithms Aalysis of Algorithms 6
Pseudocode High-level descriptio of a algorithm More structured tha Eglish prose Less detailed tha a program Preferred otatio for describig algorithms Hides program desig issues Example: fid max elemet of a array AlgorithmarrayMax(A, ) Iput array A of itegers Outputmaximum elemet of A curretmax A[] for i to do if A[i] > curretmax the curretmax A[i] retur curretmax Pseudocode Details Cotrol flow if the [else ] while do repeat util for do Idetatio replaces braces Method declaratio Algorithm method (arg [, arg ]) Iput Output Method call var.method (arg [, arg ]) Retur value returexpressio Expressios Assigmet (like = i Java) = Equality testig (like == i Java) 2 Superscripts ad other mathematical formattig allowed Aalysis of Algorithms 7 Aalysis of Algorithms 8 Primitive Operatios Coutig Primitive Operatios Basic computatios performed by a algorithm Idetifiable i pseudocode Largely idepedet from the programmig laguage Exact defiitio ot importat (we will see why later) Examples: Evaluatig a expressio Assigig a value to a variable Idexig ito a array Callig a method Returig from a method By ispectig the pseudocode, we ca determie the maximum umber of primitive operatios executed by a algorithm, as a fuctio of the iput size AlgorithmarrayMax(A, ) # operatios curretmax A[] 2 for i to do 2 + if A[i] > curretmax the 2( ) curretmax A[i] 2( ) { icremet couter i } 2( ) retur curretmax Total 7 Aalysis of Algorithms 9 Aalysis of Algorithms Estimatig Ruig Time Algorithm arraymax executes 7 primitive operatios i the worst case Defie a Time take by the fastest primitive operatio b Time take by the slowest primitive operatio Let T() be the actual worst-case ruig time of arraymax. We have a (7 ) T() b(7 ) Hece, the ruig time T() is bouded by two liear fuctios Aalysis of Algorithms Growth Rate of Ruig Time Chagig the hardware/ software eviromet Affects T() by a costat factor, but Does ot alter the growth rate of T() The liear growth rate of the ruig time T() is a itrisic property of algorithm arraymax Aalysis of Algorithms 2 2
Growth Rates Growth rates of fuctios: Liear Quadratic 2 Cubic 3 T ( ) I a log-log chart, the slope of the lie correspods to the growth rate of the fuctio E+3 E+28 Cubic E+26 E+24 Quadratic E+22 E+2 Liear E+8 E+6 E+4 E+2 E+ E+8 E+6 E+4 E+2 E+ E+ E+2 E+4 E+6 E+8 E+ Costat Factors The growth rate is ot affected by costat factors or lower-order terms Examples 2 + is a liear fuctio 2 + 8 is a quadratic fuctio T ( ) E+26 E+24 E+22 E+2 E+8 E+6 E+4 E+2 E+ E+8 E+6 E+4 E+2 E+ Quadratic Quadratic Liear Liear E+ E+2 E+4 E+6 E+8 E+ Aalysis of Algorithms 3 Aalysis of Algorithms 4 Big-Oh Notatio Give fuctios f() ad g(), we say that f() is O(g()) if there are positive costats c ad such that f() cg() for Example: 2 + is O() 2 + c (c 2) /(c 2) Pick c = 3 ad =,, 3 2+, Aalysis of Algorithms Big-Oh Notatio (cot.) Example: the fuctio 2 is ot O() 2 c c The above iequality caot be satisfied sice c must be a costat,,,,, ^2, Aalysis of Algorithms 6 Big-Oh ad Growth Rate The big-oh otatio gives a upper boud o the growth rate of a fuctio The statemet f() is O(g()) meas that the growth rate of f() is o more tha the growth rate of g() We ca use the big -Oh otatio to rak fuctios accordig to their growth rate g() grows more f() grows more Same growth f() is O(g()) No g() is O(f()) No Aalysis of Algorithms 7 Classes of Fuctios Let {g()} deote the class (set) of fuctios that are O(g()) We have {} { 2 } { 3 } { 4 } { } where the cotaimet is strict { 3 } { 2 } {} Aalysis of Algorithms 8 3
Big-Oh Rules If is f() a polyomial of degree d, the f() is O( d ), i.e.,. Drop lower-order terms 2. Drop costat factors Use the smallest possible class of fuctios Say 2 is O() istead of 2 is O( 2 ) Use the simplest expressio of the class Say 3 + is O() istead of 3 + is O(3) Aalysis of Algorithms 9 Asymptotic Algorithm Aalysis The asymptotic aalysis of a algorithm determies the ruig time i big -Oh otatio To perform the asymptotic aalysis We fid the worst-case umber of primitive operatios executed as a fuctio of the iput size We express this fuctio with big-oh otatio Example: We determie that algorithm arraymax executes at most 7 primitive operatios We say that algorithm arraymax rus i O() time Sice costat factors ad lower-order terms are evetually dropped ayhow, we ca disregard them whe coutig primitive operatios Aalysis of Algorithms 2 Computig Prefix Averages We further illustrate asymptotic aalysis with two algorithms for prefix averages The i-th prefix average of a array X is average of the first (i + ) elemets of X A[i] = (X[] + X[] + + X[i])/(i+) Computig the array A of prefix averages of aother array X has applicatios to fiacial aalysis 3 3 2 2 X A 2 3 4 6 7 Aalysis of Algorithms 2 Prefix Averages (Quadratic) The followig algorithm computes prefix averages i quadratic time by applyig the defiitio AlgorithmprefixAverages(X, ) Iput array X of itegers Output array A of prefix averages of X #operatios A ew array of itegers for i to do s X[] for j to i do + 2 + + ( ) s s + X[j] + 2 + + ( ) A[i] s / (i + ) retur A Aalysis of Algorithms 22 Arithmetic Progressio 7 The ruig time of prefixaverages is 6 O( + 2 + + ) The sum of the first 4 itegers is ( + ) / 2 There is a simple visual 3 proof of this fact 2 Thus, algorithm prefixaverages rus i O( 2 ) time 2 3 4 6 Aalysis of Algorithms 23 Prefix Averages (Liear) The followig algorithm computes prefix averages i liear time by keepig a ruig sum AlgorithmprefixAverages2(X, ) Iput array X of itegers Output array A of prefix averages of X #operatios A ew array of itegers s for i to do s s + X[i] A[i] s / (i + ) retur A Algorithm prefixaverages2 rus i O() time Aalysis of Algorithms 24 4
Asymptotic Notatio (termiology): Special classes of algorithms: logarithmic: O(log ) liear: O() quadratic: O( 2 ) polyomial: O( k ), k = expoetial: O(a ), > Relatives of the Big-Oh Ω (f()): Big Omega--asymptotic lower boud Θ (f()): Big Theta--asymptotic tight boud A table of fuctios wrt iput, assume that each primitive operatio takes oe microsecod ( secod = 6 microsecod). O(g()) log 2 log 2 2 3 2 Secod 3 2 6 9 Hour 9.3* 9 3.6* 9 9 6* 4 3 Moth Cetury.8*2 6.8* 24 2.6* 2.6* 6 4 4 9.7* 3 3.2* 4.6* 7! 9 2 7 Aalysis of Algorithms 2 Aalysis of Algorithms 26