Algorithms Chapter 3 Growth of Functions

Transcription

1 Algorithms Chapter 3 Growth of Fuctios Istructor: Chig Chi Li 林清池助理教授 chigchi.li@gmail.com Departmet of Computer Sciece ad Egieerig Natioal Taiwa Ocea Uiversity

2 Outlie Asymptotic otatio Stadard otatios ad commo fuctios 2

3 The purpose of this chapter 1/3 The order of growth of the ruig time of a algorithm gives us some iformatio about: the algorithm s efficiecy the relative performace of alterative algorithms The merge sort, with its Θ( lg ) worst case ruig time, beats isertio sort, whose worst case ruig time is Θ( 2 ). For large eough iputs, the followig are domiated by the effects of the iput size itself. multiplicative costats lower order terms of a exact ruig time 3

4 The purpose of this chapter 2/3 Whe the iput size becomes large eough, we are studyig the asymptotic efficiecy of algorithms. That is, we are cocered with how the ruig time of a algorithm icreases with the size of the iput i the limit, as the size of the iput icreases without boud. Usually, a algorithm that is asymptotically more efficiet will be the best choice for all but very small iputs. 4

5 The purpose of this chapter 3/3 We will study how to measure ad aalyze a algorithm s efficiecy for large iputs. The ext sectio begis by defiig asymptotic otatios, Θ otatio O otatio Ω otatio The, we review the commoly used fuctios i the aalysis of algorithms. 5

6 Θ otatio For a give fuctio g(), we deote by Θ(g()) the set of fuctios Θ(g()) = { f(): there exist positive costats c 1, c 2, ad 0 such that 0 c 1 g() f () c 2 g() for all 0 }. For 0, the fuctio f() is equal to g() to withi a costat factor. Here, g() is a asymptotically tight boud for f(). Because Θ(g()) is a set, we could write f() Θ(g()). Usually, we write f() = Θ(g()). 6

7 A example To show that 2 /2 3 = Θ( 2 ), we must determie positive costats c 1, c 2, ad 0 such that Dividig by 2 yields c /2 3 c 2 2 for all 0. c 1 1/2 3/ c 2. c 1 1/2 3/ holds for 7byc 1 1/14 1/2 3/ c 2 holds for 1byc 2 1/2 Thus, choosig c 1 = 1/14, c 2 =1/2, ad 0 =7, we ca verify that 2 /2 3 = Θ( 2 ).

8 Aother example We show that 6 3 Θ( 2 ) by cotradictio. Suppose c 2 ad 0 exist such that 6 3 c 2 2 for all 0. The c 2 /6, a cotradictio. Sice c 2 is costat, it caot possibly hold for arbitrary large,

9 Summary The lower order terms ca be igored because they are isigificat for large. The coefficiet of the highest order term ca likewise be igored sice it oly chages c 1 ad c 2 by a costat factor equal to the coefficiet. I geeral, for ay polyomial p() = Σ i=0~d a i i, where a i are costats ad a d > 0, we have p() = Θ( d ). For example, f() = a 2 + b + c, where a, b, ad c are costats ad a > 0. The, we have f() = Θ( 2 ).

10 O otatio For a give fuctio g(), we deote by O(g()) the set of fuctios O(g()) = { f(): there exist positive costats c ad 0 such that 0 f () cg() for all 0 }. We write f() = O(g()) implies f()is a member of the set O(g()). Note that f() = Θ(g()) implies f() = O(g()). ay proof showig that f() = Θ(g()) also shows that f() = O(g()). Θ(g()) O(g()).

11 The meaig of O otatio 1/2 The Θ otatio asymptotically bouds a fuctio from above ad below. Whe we have oly a asymptotic upper boud, we use O otatio. Hece, Θ otatio is a stroger otatio tha O otatio.

12 The meaig of O otatio 2/2 Ay liear fuctio a + b is i O( 2 ), which is easily verified by takig c = a+ b ad 0 = 1. a+ b (a+ b ) 2 for 1 f() = O(g()) merely claims that g() is a asymptotic upper boud o f() does ot claim about how tight a upper boud it is I practical, O otatio is used to describe the worst case ruig time of a algorithm. a algorithm is O(g()) meas that the ruig time is at most costat times g(), for sufficietly large o matter what particular iput of size is chose for each value of

13 Ω otatio For a give fuctio g(), we deote by Ω(g()) the set of fuctios Ω(g()) = { f (): there exist positive costats c ad 0 such that 0 cg() f () for all 0 }. We write f() = Ω(g()) implies f()is a member of the set Ω(g()). Ω otatio provides asymptotic lower boud.

14 The relatioship betwee Θ, O, ad Ω Theorem 3.1 For ay two fuctios f() ad g(), we have f() = Θ(g()) if ad oly if f() = O(g()) ad f() = Ω(g()). For example: 2 /2 3 = Θ( 2 ) 2 /2 3 = O( 2 ) ad 2 /2 3 = Ω( 2 ) 2 /2 3 = O( 2 ) ad 2 /2 3 = Ω( 2 ) 2 /2 3 = Θ( 2 )

15 The meaig of Ω otatio The Ω otatio is used to boud the best case ruig time of a algorithm. a algorithm is Ω(g()) meas that the ruig time is at least costat times g(), for sufficietly large o matter what particular iput of size is chose for each value of

16 Asymptotic otatio i equatios ad iequalities 1/2 O the right had side of a equatio (or iequality) the equal sig meas set membership = O( 2 ) meas that O( 2 ) I a formula it is iterpreted as some aoymous fuctio that we do ot care to ame =2 2 +Θ() meas that =2 2 +f(), where f () Θ() O the left had side of a equatio 16 No matter how the aoymous fuctios are chose o the left of the equal sig, there is a way to choose the aoymous fuctios o the right of the equal sig to make the equatio valid 2 2 +Θ()=Θ( 2 ) meas that for ay fuctio f() Θ(), there is some fuctio g() Θ( 2 ) such that 2 2 +f ()=g() for all

17 Asymptotic otatio i equatios ad iequalities 2/2 A umber of such relatioships ca be chaied together, as i =2 2 +Θ() =Θ( 2 ) The first equatio says that there is some fuctio f() Θ() such that =2 2 +f() for all. The secod equatio says that for ay fuctio g() Θ(), there is some fuctio h() Θ( 2 ) such that 2 2 +g()=h() for all. Note that the iterpretatio implies )=Θ( 2 ), which is what the chaiig of equatios ituitively gives us. 17

18 o otatio For a give fuctio g(), we deote by o(g()) the set of fuctios o(g()) = {f(): for ay positive costat c>0, there exists a costat 0 >0 such that 0 f()<cg() for all 0 }. We use o otatio to deote a upper boud that is ot asymptotically tight. For example, 2=o( 2 ), but 2 2 o( 2 ). Ituitively, the fuctio f() becomes isigificat relative to g() as approaches ifiity; that is, lim f( ) = 0 g ( ) 18

19 ω otatio For a give fuctio g(), we deote by ω(g()) the set of fuctios ω(g())={f(): for ay positive costat c>0, there exists a costat 0 >0 such that 0 cg()<f() for all 0 }. We use ω otatio to deote a lower boud that is ot asymptotically tight. For example, 2 /2=ω(), but 2 /2 ω( 2 ). The relatio f()=ω(g()) implies that lim f( ) = g ( ) if the limit exists. 19

20 Compariso of fuctios 1/4 Trasitivity: f() = Θ(g()) ad g()= Θ(h()) imply f() = Θ(h()), f() = O(g()) ad g()= O(h()) imply f() = O(h()), f() = Ω(g())ad g()= Ω(h()) imply f() = Ω(h()), f() = o(g()) ad g()= o(h()) imply f() = o(h()), f() = ω(g()) ad g()= ω(h()) imply f() = ω(h()). 20

21 Compariso of fuctios 2/4 Reflexivity: f() = Θ(f()), f() = O(f()), f() = Ω(f()). Symmetry: f() = Θ(g()) if a oly if g() = Θ(f()). Traspose symmetry: f() = O(g()) if ad oly if g() = Ω(f()), f() = o(g()) if ad oly if g() = ω(f()). 21

22 Compariso of fuctios 3/4 Aalogy betwee the asymptotic compariso ad the real umber compariso: f() = Θ(g()) a = b f() = O(g()) a b f() = Ω(g()) a b f() = o(g()) a < b f() = ω(g()) a > b 22

23 Compariso of fuctios 4/4 Trichotomy property of real umbers does ot carry over to asymptotic otatio: Trichotomy: For ay two real umbers a ad b, exactly oe of the followig must hold: a < b, a = b, or a > b. Not all fuctios are asymptotically comparable. For two fuctios f() ad g(), it may be the case that either f()=o(g()) or f()=ω(g()). For example, the fuctio ad 1+si caot be compared, sice the value of 1+si oscillates betwee 0 ad 2. 23

24 Outlie Asymptotic otatio Stadard otatios ad commo fuctios 24

25 Mootoicity A fuctio f() is mootoically icreasig if m implies f(m) f(). A fuctio f() is mootoically decreasig if m implies f(m) f(). A fuctio f() is strictly icreasig if m< implies f(m)<f(). A fuctio f() is strictly decreasig if m> implies f(m)>f(). 25

26 Floors ad ceiligs For ay real umber x, we deote the greatest iteger less tha or equal to x by x ad the least iteger greater tha or equal to x by x. For all real x, For ay iteger, x-1< x x x <x+1 /2 + /2 = For ay real umber 0 ad itegers a, b>0 /a /b = /ab, ad /a /b = /ab a/b (a+(b-1))/b, ad a/b (a-(b-1))/b The floor ad ceilig fuctios are mootoically icreasig. 26

27 Modular arithmetic For ay iteger a ad ay positive iteger, the value a mod is the remaider (or residue) of the quotiet a/: a mod =a- a/ If (a mod )=(b mod ), we write a b (mod ) ad say that a is equivalet to b, modulo. If a b (mod ) If a ad b have the same remaider whe divided by If ad oly if is a divisor of b-a We write a b(mod ) if a is ot equivalet to b, modulo. 27

28 Polyomials A polyomial i of degree d is a fuctio P() = a d d + a d-1 d a a 1 + a 0 d is a oegative iteger a d,, a 0 are costats called the coefficiets of the polyomial a d 0 A asymptotically positive fuctio is oe that is positive for all sufficietly large. A polyomial is asymptotically positive if ad oly if a d > 0. For ay real costat a 0 (respectively, a 0), the fuctio a is mootoically icreasig (respectively, decreasig). A fuctio f() is polyomially bouded if f()=o( k ) for some costat k. 28

29 Expoetials 1/2 For all real a>0, m, ad, we have the followig idetities: a 0 = 1, a 1 = a, a 1 = 1/a (a m ) = (a ) m = a m a m a = a m+ 0 0 = 1 (for coveiet) For all real costats a ad b such that a > 1, lim b = 0, from a which we coclude that b = o(a ). Thus, ay expoetial fuctio with a base strictly greater tha 1 grows faster tha ay polyomial fuctio. 29

30 Expoetials 2/2 The atural logarithm fuctio for all real x, e e = x 2 3 x x = 1+ x + + 2! 3! + K = i= For all real x, we have e x 1+x equality holds oly whe x = 0 Whe x 1, we have 1+x e x 1+x+x 2 Whe x 0, we have e x =1+x+Θ(x 2 ) 0 i x i! For all x, lim 1 x + = e x 30

31 Logarithms 1/4 We shall use the followig otatios: lg = log 2 (biary logarithm) l = log e (atural logarithm) lg k = (lg) k lglg = lg(lg) (expoetiatio) (compositio) Note that lg+k meas (lg)+k, ot lg(+k). If we hold b > 1 costat, the for > 0, the fuctio log b is strictly icreasig. 31

32 Logarithms 2/4 For all real a, b, c > 0, ad, if the logarithm bases are ot 1, the, we have logc a log b a = logb logb a = log b c a log b c = c log b a a = b log b a 1 log b = logb a logb a = a 1 log a b c ( ab) = log a log b log + c c 32

33 Logarithms 3/4 If x < 1, the We also have the followig iequalities for x > 1: x 2 x 3 x 4 ( + x) = x + + L l 1 x l 1+ x the equality holds oly for x = 0 2 ( 1+ x) x 3 4 x

34 Logarithms 4/4 f() is called polylogarithmically bouded if f() = O(lg k ) for some costat k. By substitutig lg for ad 2 a for a i So, for ay costat a > 0 b lg a (2 ) lim = lg lim = a lg b = o( a ) b lg 0 lim b = 0 ay positive fuctio grows faster tha ay polylogarithmic fuctio a 34

35 Factorials Stirlig s approximatio: e is the base of the atural logarithm give a tighter upper boud, ad a tighter low boud Oe ca prove For all 1, we have 35 + = e 1 1 2! θ π > = = 0. if 1)! ( 0, if 1! ) (2! ) lg θ(!) lg( ) (! o ω = = = e e where, 2! < < + = α π α

36 Fuctioal iteratio Let f() be a fuctio over the reals. The, for oegative iteger i, we recursively defie f f ( f ( i) ( ) = ( i 1 ) ( )) For example, if f() = 2, the f (i) () = 2 i if if i = > 0, 0. 36

37 The iterated logarithm fuctio Let lg (i) be defied as above, with f() = lg Note that lg i = (lg) i lg (i) Because the logarithm of a opositive umber is udefied, lg (i) is defied oly if lg (i 1) >0 The iterated logarithm fuctio, is defied as lg* = mi{i 0: lg (i) 1} lg*2 = 1 lg*4 = 2 lg*16 = 3 lg*65536 = 4 lg* = 5 a very slowly growig fuctio 37

38 Fiboacci umbers 1/2 The Fiboacci umbers are defied by the recurrece relatio F F F 0 1 i = 0, = 1, = F i 1 + F i 2 for i 2. The Fiboacci umbers are : 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, Fiboacci umbers are related to the golde ratio φ ad to its cojugate φˆ. Oe ca prove that φ i φˆi 1+ 5 F i =, where φ = = ad ˆ φ = =

39 Fiboacci umbers 2/2 i φ 1 1 Sice ˆ φ < 1, we have < <. So that the ith Fiboacci umber roud to the earest iteger. ˆ F i = φ i ˆi φ 5 is equal to i φ 5 Thus, Fiboacci umber grow expoetially. 39