Pseudocode ( 1.1) Analysis of Algorithms. Primitive Operations. Pseudocode Details. Running Time ( 1.1) Estimating performance

Transcription

1 Aalysis of Algorithms Iput Algorithm Output A algorithm is a step-by-step procedure for solvig a problem i a fiite amout of time. Pseudocode ( 1.1) High-level descriptio of a algorithm More structured tha Eglish prose Less detailed tha a program Preferred otatio for describig algorithms Hides program desig issues Example: fid max elemet of a array Algorithm arraymax(a, ) Iput array A of itegers Output maximum elemet of A curretmax A[] for i 1 to 1 do if A[i] > curretmax the curretmax A[i] retur curretmax Aalysis of Algorithms v1.1 2 Pseudocode Details Primitive Operatios Cotrol flow if the [else ] while do repeat util for do Idetatio replaces braces Method declaratio Algorithm method (arg [, arg ]) Iput Output Method call var.method (arg [, arg ]) Retur value retur expressio Expressios Assigmet (like = i Java) = Equality testig (like == i Java) 2 Superscripts ad other mathematical formattig allowed Basic computatios performed by a algorithm Idetifiable i pseudocode Largely idepedet from the programmig laguage Examples: Evaluatig a expressio Assigig a value to a variable Idexig ito a array Callig a method Returig from a method Aalysis of Algorithms v1.1 3 Aalysis of Algorithms v1.1 4 Estimatig performace Cout Primitive Operatios = time eeded by RAM model Radom Access Machie (RAM) Model has: A CPU A potetially ubouded bak of memory cells Each cell ca hold a arbitrary umber or character Memory cells are umbered Accessig ay cell takes uit time 12 Ruig Time ( 1.1) The ruig time grows with the iput size. Ruig time varies with differet iput Worst-case: look at iput causig most operatios Best-case: look at iput causig least umber of operatios Average case: betwee best ad worst-case. Ruig Time best case average case worst case Iput Size Aalysis of Algorithms v1.1 5 Aalysis of Algorithms v1.1 6

2 Coutig Primitive Operatios ( 1.1) Worst-case primitive operatios cout, as a fuctio of the iput size Algorithm arraymax(a, ) # operatios curretmax A[] 2 for i 1 to 1 do 1 + if A[i] > curretmax the 2( 1) curretmax A[i] 2( 1) { icremet couter i } 2( 1) retur curretmax 1 Total 7 2 Aalysis of Algorithms v1.1 7 Coutig Primitive Operatios ( 1.1) Best-case primitive operatios cout, as a fuctio of the iput size Algorithm arraymax(a, ) # operatios curretmax A[] 2 for i 1 to 1 do 1 + if A[i] > curretmax the 2( 1) curretmax A[i] { icremet couter i } 2( 1) retur curretmax 1 Total 5 Aalysis of Algorithms v1.1 8 Defiig Worst [W()], Best [B(N)], ad Average [A()] Experimetal Studies ( 1.6) Let I = set of all iputs of size. Let t(i) = # of primitive ops by alg o iput i. W() = maximum t(i) take over all i i I B() = miimum t(i) take over all i i I A() = p ( i) t( i), p(i) = prob. of i occurrig. i I We focus o the worst case Easier to aalyze Usually wat to kow how bad ca algorithm be average-case requires kowig probability; ofte difficult to determie Implemet your algorithm Ru your implemetatio with iputs of varyig size ad compositio Measure ruig time of your implemetatio (e. g., with System.curretTimeMillis()) Plot the results Time (ms) Iput Size Aalysis of Algorithms v1.1 9 Aalysis of Algorithms v1.1 1 Limitatios of Experimets Implemet may be time-cosumig ad/or difficult Results may ot be idicative of the ruig time o other iputs ot icluded i the experimet. I order to compare two algorithms, the same hardware ad software eviromets must be used Ifeasible to test for correctess o all possible iputs. Theoretical Aalysis Uses a high-level descriptio of the algorithm istead of a implemetatio Characterizes ruig time as a fuctio of the iput size,. Takes ito accout all possible iputs Allows us to evaluate the speed of a algorithm idepedet of the hardware/software eviromet Ca prove correctess Aalysis of Algorithms v Aalysis of Algorithms v1.1 12

3 Growth Rate of Ruig Time Chagig the hardware/ software eviromet Affects ruig time by a costat factor; Does ot alter its growth rate Example: liear growth rate of arraymax is a itrisic property of algorithm. Growth Rates Growth rates of fuctios: Liear Quadratic 2 Cubic 3 T ( ) I a log-log chart, the slope of the lie correspods to the growth rate of the fuctio (for polyomials) 1E+3 1E+28 Cubic 1E+26 1E+24 Quadratic 1E+22 1E+2 Liear 1E+18 1E+16 1E+14 1E+12 1E+1 1E+8 1E+6 1E+4 1E+2 1E+ 1E+ 1E+2 1E+4 1E+6 1E+8 1E+1 Aalysis of Algorithms v Aalysis of Algorithms v Costat Factors The growth rate is ot affected by costat factors or lower-order terms Examples T ( ) is a liear fuctio is a quadratic fuctio 1E+26 1E+24 1E+22 1E+2 1E+18 1E+16 1E+14 1E+12 1E+1 1E+8 1E+6 1E+4 1E+2 1E+ Quadratic Quadratic Liear Liear 1E+ 1E+2 1E+4 1E+6 1E+8 1E+1 Aalysis of Algorithms v Big-Oh ad Growth Rate The big-oh otatio gives a upper boud o the growth rate of a fuctio The statemet f() is O(g()) meas that the growth rate of f() is o more tha the growth rate of g() We ca use the big-oh otatio to rak fuctios accordig to their growth rate g() grows more f() grows more Same growth f() is O(g()) No g() is O(f()) No Aalysis of Algorithms v Big-Oh Notatio ( 1.2) Give fuctios f() ad g(), we say that f() is O(g()) if there are positive costats c ad such that f() cg() for Example: is O() c (c 2) 1 1/(c 2) Pick c = 3 ad = 1 1, 1, , Big-Oh Example Example: the fuctio 2 is ot O() 2 c c The above iequality caot be satisfied sice c must be a costat 1,, 1, 1, 1, ^ , Aalysis of Algorithms v Aalysis of Algorithms v1.1 18

4 More Big-Oh Examples is O() eed c > ad 1 such that 7-2 c for this is true for c = 7 ad = is O( 3 ) eed c > ad 1 such that c 3 for this is true for c = 4 ad = 21 3 log + log log 3 log + log log is O(log ) eed c > ad 1 such that 3 log + log log c log for this is true for c = 4 ad = 2 Aalysis of Algorithms v Big-Oh Rules If is f() a polyomial of degree d, the f() is O( d ), i.e., 1. Drop lower-order terms 2. Drop costat factors Use the smallest possible class of fuctios Say 2 is O() istead of 2 is O( 2 ) Use the simplest expressio of the class Say is O() istead of is O(3) Aalysis of Algorithms v1.1 2 Asymptotic Algorithm Aalysis asymptotic aalysis = determiig a algorithms ruig time i big-oh otatio asymptotic aalysis steps: We fid the worst-case umber of primitive operatios executed as a fuctio of the iput size We express this fuctio with big-oh otatio Example: We determie that algorithm arraymax executes at most 7 2 primitive operatios We say that algorithm arraymax rus i O() time or rus i order time Sice costat factors ad lower-order terms are evetually dropped, we ca disregard them whe coutig primitive operatios! Aalysis of Algorithms v Ituitio for Asymptotic Notatio Big-Oh f() is O(g()) if f() is asymptotically less tha or equal to g() big-omega f() is Ω(g()) if f() is asymptotically greater tha or equal to g() big-theta f() is Θ(g()) if f() is asymptotically equal to g() little-oh f() is o(g()) if f() is asymptotically strictly less tha g() little-omega f() is ω(g()) if is asymptotically strictly greater tha g() Aalysis of Algorithms v Relatives of Big-Oh big-omega f() is Ω(g()) if there is a costat c > ad a iteger costat 1 such that f() c g() for big-theta f() is Θ(g()) if there are costats c > ad c > ad a iteger costat 1 such that c g() f() c g() for little-oh f() is o(g()) if, for ay costat c >, there is a iteger costat such that f() c g() for little-omega f() is ω(g()) if, for ay costat c >, there is a iteger costat such that f() c g() for Example Uses of the Relatives of Big-Oh 5 2 is Ω( 2 ) f() is Ω(g()) if there is a costat c > ad a iteger costat 1 such that f() c g() for let c = 5 ad = is Ω() f() is Ω(g()) if there is a costat c > ad a iteger costat 1 such that f() c g() for let c = 1 ad = is ω() f() is ω(g()) if, for ay costat c >, there is a iteger costat such that f() c g() for eed 5 2 c give c, the that satifies this is c/5 Aalysis of Algorithms v Aalysis of Algorithms v1.1 24

5 Math you eed to kow Summatios (Sec ) Logarithms ad Expoets (Sec ) Proof techiques (Sec ) Basic probability (Sec ) properties of logarithms: log b (xy) = log b x+ log b y log b (x/y) = log b x-log b y log b xa = alog b x log b a= log x a/log x b properties of expoetials: a (b+c) = a b a c a bc = (a b ) c a b /a c = a (b-c) b = a log a b b c = a c*log a b Math you eed to kow Proofs are a sequece of statemets Each statemet is true, based o Defiitios Hypotheses Well-kow math priciples Previous statemets Statemets lead towards coclusio Aalysis of Algorithms v Aalysis of Algorithms v Iductio proof Method of provig statemets for (ifiitely) large values of, ( is the iductio variable). Math way of usig a loop i a proof. Example iductio proof Prove: for all it x, for all it y, for all it, If is positive, the x y is divisible by x-y. Let S deote for all x ad y, x y is divisible by x- y Aalysis of Algorithms v Aalysis of Algorithms v Example iductio proof Example iductio proof Prove: for all it x, for all it y, for all it, If is positive, the x y is divisible by x-y. Let S deote for all x ad y, x y is divisible by x- y Proof with iductio: Base case: show S 1 Iductive Hypothesis (IH): for all k 1, if S k is true, tha S k+1 is true. OR Iductive Hypothesis (IH): for all k 2, if S k-1 is true, tha S k is true. Prove: for all it x, for all it y, for all it, If is positive, the x y is divisible by x-y. Let S deote for all x ad y, x y is divisible by x- y Proof with iductio: Aalysis of Algorithms v Aalysis of Algorithms v1.1 3

6 More math tools & proofs Computig Prefix Averages Correctess of computig average loop ivariats ad iductio Recurrece equatios Strog iductio Cost of recursive algorithms with recurrece equatios. asymptotic aalysis examples: two algorithms for prefix averages The i-th prefix average of a array X is average of the first (i + 1) elemets of X: A[i] = (X[] + X[1] + + X[i])/(i+1) Computig the array A of prefix averages of aother array X has applicatios to fiacial aalysis X A Aalysis of Algorithms v Aalysis of Algorithms v Prefix Averages (Quadratic) The followig algorithm computes prefix averages i quadratic time by applyig the defiitio Algorithm prefixaverages1(x, ) Iput array X of itegers Output array A of prefix averages of X #operatios A ew array of itegers for i to 1 do s X[] for j 1 to i do ( 1) s s + X[j] ( 1) A[i] s / (i + 1) retur A 1 Aalysis of Algorithms v Arithmetic Progressio The ruig time of prefixaverages1 is O( ) The sum of the first itegers is ( + 1) / 2 There is a simple visual proof of this fact Thus, algorithm prefixaverages1 rus i O( 2 ) time Aalysis of Algorithms v Prefix Averages (Liear) The followig algorithm computes prefix averages i liear time by computig prefix sums (ad averages) Algorithm recprefixsumadaverage(x, A, ) Iput array X of 1 iteger. Empty array A; A is same size as X. Output array A[] A[-1] chaged to hold prefix averages of X. returs sum of X[], X[1],,X[-1] if =1 A[] X[] retur A[] tot recprefixsumadaverage(x,a,-1) tot tot + X[-1] A[-1] tot / retur tot; Aalysis of Algorithms v Prefix Averages (Liear) The followig algorithm computes prefix averages i liear time by computig prefix sums (ad averages) Algorithm recprefixsumadaverage(x, A, ) T() operatios Iput array X of 1 iteger. Empty array A; A is same size as X. Output array A[] A[-1] chaged to hold prefix averages of X. returs sum of X[], X[1],,X[-1] #operatios if =1 1 A[] X[] 3 retur A[] 2 tot recprefixsumadaverage(x,a,-1) 3+T(-1) tot tot + X[-1] 4 A[-1] tot / 4 retur tot; 1 Aalysis of Algorithms v1.1 36

7 Prefix Averages, Liear Recurrece equatio T(1) = 6 T() = 13 + T(-1) for >1. Solutio of recurrece is T() = 13(-1) + 6 T() is O(). Aalysis of Algorithms v1.1 37