Measuring the Efficiency of Algorithms

Transcription

1 Algorithm analysis

2 Measuring the Efficiency of Algorithms When choosing algorithms, we often have to settle for a space/time tradeoff An algorithm can be designed to gain faster run times at the cost of using extra space (memory), or the other way around Memory is now quite inexpensive for desktop and laptop computers, but not yet for miniature devices 2

3 Measuring the Run Time of an Algorithm One way to measure the time cost of an algorithm is to use computer s clock to obtain actual run time Benchmarking or profiling Can use time() in time module Returns number of seconds that have elapsed between current time on the computer s clock and January 1,

4 Measuring the Run Time of an Example: Algorithm >>> import time >>> >>> N=10000 >>> b=time.time() >>> for i in range(n):... c=i... >>> e=time.time() >>> >>> print e b

5 List Insert vs append import time D=[10000,20000,30000,50000,60000,70000,80000,90000,100000] #print "append" t1=[] for N in D: A=[] b=time.time() for i in range(n): A.append(i) A.reverse() e=time.time() t1.append(e b) #print "insert" t2=[] for N in D: A=[] b=time.time() for i in range(n): A.insert(0,i) e=time.time() t2.append(e b) import matplotlib.pyplot as plt plt.plot(d,t1,'r o', D,t2,'b s') plt.show()

6 Measuring the Run Time of an Algorithm (continued) This method permits accurate predictions of the running times of many algorithms Problems: Different hardware platforms have different processing speeds, so the running times of an algorithm differ from machine to machine Running time varies with OS and programming language too It is impractical to determine the running time for some algorithms with very large data sets 6

7 Counting Instructions Another technique is to count the instructions executed with different problem sizes We count the instructions in the high-level code in which the algorithm is written, not instructions in the executable machine language program Distinguish between: Instructions that execute the same number of times regardless of problem size For now, we ignore instructions in this class Instructions whose execution count varies with problem size 7

8 Example of counting instruction def average_scale(sequence,scale,win=9): ''' average_scale(sequence,scale,win=9) computes the window averaging over the 'sequence' given the values obtained from the 'scale' dictionary and using an average window of length win. Returns the list of the averaged residues for each sequence position ''' L=len(sequence) #(1) border=int(win/2) # window border #(1') av_val=[0.0]*l #(2) for i in range(l): # for each residues #(3) left_border=max(0,i border) #(4) right_border=min(l,i+border+1) #(4') effective_window=right_border left_border #(4") for k in range(left_border,right_border): #(5) aa=sequence[k] #(6) av_val[i]=av_val[i]+scale[aa] #(6') av_val[i]=av_val[i]/effective_window #(7) return(av_val)

9 Example of counting instruction (1) and (1') executed only 1 time (2) implicit assignment of L values => L assignments (3) the for block is executed L times (4),(4),(4 ) are executed one time for each Loop => L (5) the for loop is executed roughly win time L => win*l (6) and (6') again win*l (7) it is executed L times Total(L) = 3*Win*L + 5*L + 2 Since Win is constant Total(L) < K*L for a given constant K

10 Asymptotic Notation Goal: to simplify analysis by getting rid of unneeded information (like rounding 1,000,001 1,000,000) We want to say in a formal way 3n 2 n 2 The Big-Oh Notation: given functions f(n) and g(n), we say that f(n) is O(g(n)) if and only if there are positive constants c and n 0 such that f(n) c g(n) for n n 0

11 Graphic Illustration f(n) = 2n+6 Conf. def: Need to find a function g(n) and a const. c and a constant n0 such as f(n) < cg(n) when n>n0 g(n) = n and c = 4 and n0=3 f(n) is O(n) The order of f(n) is n c g(n) 4n g(n) n f(n) = 2n + 6 n

12 More examples What about f(n) = 4n 2? Is it O(n)? Find a c and n0 such that 4n 2 < cn for any n > n0 50n n + 4 is O(n 3 ) Would be correct to say is O(n 3 +n) Not useful, as n 3 exceeds by far n, for large values Would be correct to say is O(n 5 ) OK, but g(n) should be as close as possible to f(n) 3log(n) + log (log (n)) = O(? ) Simple Rule: Drop lower order terms and constant factors

13 Big-Oh Rules If f 1 (n)=o(g 1 (n)) and f 2 (n)=o(g 2 (n)) f 1 (n)+f 2 (n)=o(g 1 (n)+g 2 (n))=max(o(g 1 (n)+g 2 (n)) f 1 (n)*f 2 (n)=o(g 1 (n)*g 2 (n)) The relative growth rate of two functions can always be determined by computing their limit But using this method is almost always an overkill lim f ( n) / f ( n) n 1 2

14 Big-Oh and Growth Rate The big-oh notation gives an upper bound on the growth rate of a function. The statement f(n) is O(g(n)) means that the growth rate of f(n) is no more than the growth rate of g(n). We can use the big-oh notation to rank functions according to their growth rate. f(n) is O(g(n)) g(n) is O(f(n)) g(n) grows faster Yes No f(n) grows faster No Yes Same growth Yes Yes

15 Big-Oh Rules If is f(n) a polynomial of degree d, then f(n) is O(n d ), i.e., 1. Drop lower-order terms 2. Drop constant factors Use the smallest possible class of functions Say 2n is O(n) instead of 2n is O(n 2 ) Use the simplest expression of the class Say 3n 5 is O(n) instead of 3n 5 is O(3n)

16 Common time complexities BETTER WORSE O(1) constant time O(log n) log time O(n) linear time O(n log n) log linear time O(n 2 ) quadratic time O(n 3 ) cubic time O(2 n ) exponential time 16

17 Exercise: write a code that compute the maximum subsequence sum of list of number

18 Code A : Big-O Complexity? def maxsum2(a=[]): Start=End=None lenlist=len(a) maxsum=0 for i in range(lenlist): thissum=0 for j in range(i,lenlist): thissum=thissum+a[j] if(thissum>maxsum): maxsum=thissum Start=i End=j return([start,end,maxsum])

19 Ca we do better?

20 Code B: Big-O? def maxsum1(a=[]): ''' maxsum1(a[]) finds the maximum subsequence sum in A[] returns the Start, the End positions and the sum value ''' Start=End=None # all negative numbers lenlist=len(a) maxsum=0 thissum=0 thisstart=0 for i in range(lenlist): #(1) thissum=thissum+a[i] # attempt to start if(thissum<0): # check if worth while #(2) thissum=0 # restart the next time thisstart=i+1 # the new possible begin will be i+1 elif(thissum > maxsum): # check if a new max is found #(3) maxsum=thissum # the new best sum Start=thisStart # the new best begin End=i # the new best end return([start,end,maxsum])