Algorithms and Complexity

Transcription

1 Algorithms and Algorithm An algorithm is a calculation procedure (composed by a finite number of steps) which solves a certain problem, working on a set of input values and producing a set of output values 2

2 Example The sorting problems is defined as follows: n Input set: sequence of numbers <a 1, a 2,,a n > n Output set: permutation <a 1, a 2,, a n > of input sequence so that a 1 a 2 a n. 3 Insertion Sort Insert A[ j ] in the sorted sequence A[1.. j --1] 4

3 Esempio 5 Analysis of algorithms Analysing algorithms means to predict the amount of resources (I/O, memory, time) required by an algorithm during its execution. Such analysis should be independent from the kind of hardware platform on which the algorithm is executed. We will assume in the rest of the course that the hardware platform is a common machine with a single CPU (Random Access Machine or RAM). 6

4 Problem Dimension Analysis of algorithms is usually performed with respect to one or more parameters which characterize the dimensions of the problem. Example n In the sorting algorithm such parameter is the number of elements of the input sequence. n In the multiplication between integer numbers, dimension is given by the number of bits used to represent the operands. 7 Hypothesis n Each statement in the pseudo-code requires a fixed time n Each statement has a different execution time. 8

5 Analysis of insertion sort t j is the number of times the statement is repeated, for a certain value of j times Insert A[ j ] in the ordered sequence A[1.. j -1 ] 9 Execution time of insertion sort 10

6 Best Case If the array is already sorted, the t j = 1 for every j. Then: Which can be written as follows T(n) = an + b 11 Worst Case If an array is sorted with inverse order at step j, j comparisons and swaps: t j = j. Remember that: 12

7 Worst Case (2) Therefore In the worst case: T(n) = an 2 +bn+c 13 Average Case Worst case analysis is important as it defines an upper bound to resources required by an algorithm. In some cases the average case (or mean value) is worthy of being analyzed. 14

8 Importance of complexity analysis Analysis of resources of an algorithm (also known as complexity analysis) allows describing performance of the algorithm depending on problem dimension. Choosing an algorithm with lower complexity is the best choice independently from the used technology and it can make a difference for problems with big dimensions. 15 Example Suppose you can get the solution of a problem with 2 algorithms: n One with complexity T(n) = 2n 2 n The other with complexity T(n) = 50n log 2 n Suppose you have these two machines: n Intel Core i7 Extreme 965EE performing MIPS at 3.2 GHz for the first algorithm n n AMD Athlon FX-60 performing MIPS at 2.6 GHz for the second algorithm. MIPS = Millions of Instructions Per Second 16

9 Example (II) Execution time is proportional to T(n) / MIPS: n When n = 1 K ( 10 3 ) Intel - T(n)/76000 = 2*10 6 / = ms AMD T(n)/18000 = 50*10 3 *log 2 (10 3 ) / =0.027 ms n When n = 1 M ( 10 6 ) Intel MIPS : ms AMD MIPS : 55,36 ms n When n = 1 G ( 10 9 ) Intel MIPS : 26,3 Ms = 7305 hours! = 304 days!!! AMD MIPS : 83,04 s 17 Asymptotic Notation In complexity analysis the asymptotic notation is often used to discover the complexity of an algorithm when problem dimension increases. The asymptotic notation is based on 3 notations: n Theta Notation Θ n Big O Notation O n Omega Notation Ω. 18

10 Θ Notation Given an algorithm of complexity T(n). T(n) = Θ( g(n) ) if and only if there exist three positive constants c 1, c 2 e n 0 so that 0 c 1 g(n) T(n) c 2 g(n) for each n n 0. In this case g(n) is the asymptotic limit for T(n). 19 Big O Notation T(n) = O( g(n) ) if and only if there exist two positive constants c and n 0 so that: 0 T(n) c g(n) for each n n 0. In this case g(n) is a upper asymptotic limit for T(n). 20

11 Ω Notation T(n) = Ω( g(n) ) if and only if there exist two positive constants c and n 0 so that 0 c g(n) T(n) for each n n 0. In this case g(n) is a lower asymptotic limit for T(n)

12 Theorem Given two functions g(n) and T(n), T(n) = Θ( g(n) ) if and only if n T(n) = O( g(n) ) and T(n) = Ω( g(n) ) 23 Problems 24

13 Empirical complexity measure Using the clock() function: #include <time.h> int main(){ clock_t b= clock(); } test_function(); clock_t e = clock(); double elapsed=(double)(e b)/clocks_per_sec; printf("elapsed: %f seconds\n",elapsed); return 0; 25 Example strcpy(destination, source); Vs. mycpy(destination, source); 26

14 Example O(n 2 ) strcpy(destination, void mycpy(char* source); d,char* s){ char* src; for(src=s;*src!='\0';++src){ Vs. char* org=s; char* dst=d; mycpy(destination, for(; source); origin<=src; ++org,++dst){ *dst = *origin; } } }; 27 Measurement code int main(){ int i,j; printf("function,n,clocks\n"); int n[10]={100,200,300,/* */,2000,2500,3000}; for(j=1;j<10;++j){ for(i=0; i<n_measure;++i){ clock_t b= clock(); test_strcpy(n[j]); clock_t e = clock(); printf("strcpy(),%d,%d\n",n[j],(int)(e-b)); }} /* */ } 28

15 Results s y = x x R² = y = x R² = strcpy() mycpy() 463 μs 29