Complexity, General Allmänt Find a function T(n), which behaves as the time it takes to execute the program for input of size n. Standard approach: count the number of primitive operations executed. Standard approximations: we are not interested of the precise count, only of the increase ratio as a function of the size n. For this we use the concept of Big O. LAD, complexity 1
Example Example Compute the sum of the numbers from 1 to n int sum = 0; ( 1 op) for ( int i = 1; i <= n; i++ ) ( 1 op, 1op, 2 op) sum =+ i; ( 2 op ) Gives the time function: T(n) = 1 + 1 + (n + 1) 1 + n (2 + 2) = 3 + 5 n LAD, complexity 2
Definition of Big O Big O Let g be a function whose domain and co-domain are the non-negative integer values. A function f is said to be of O(g) if for some pair of non-negative constants C and K, it holds: ( n K) f(n) C g(n) LAD, complexity 3
Common Big O functions Big O O(1) : The constant complexity O( 2 log n) : The logaritmic complexity O(n) : The linear complexity O(n 2 ) : The quadratic complexity O(n 3 ) : The cubic complexity O(2 n ) : The exponential complexity and combinations of the functions above, e.g. O(n 2 log n) LAD, complexity 4
Example above Big O Take the summation above for which we have f(n) == T(n) = 3 + 5 n. Let g be the function g(n) = n. Choose C = 6 and K = 3 then we have that: f(n) C g(n), for all n > K. i.e. f is of O(g), or f O(g) LAD, complexity 5
Example double loop Big O Consider the loop-statement: for ( int i = 0; i < n; i++ ) for (int j = 0; j < n; j++ ) // k primitive operations We find the complexity function by setting the work of the k primitive operations to one and use double summation: n n 1 = n n = n n O( n 2 ) i=0 j=0 i=0 LAD, complexity 6
Example: double loop Big O Now change the inner loop: for ( int i = 0; i < n; i++ ) for (int j = i; j < n; j++ ) // k primitive operations and we get n 1 i=0 n 1 j=i 1 = n 1 i=0 n i = n i=1 i = n (n + 1) 2 O( n 2 ) LAD, complexity 7
Example squareroot Big O public static int squareroot( int n ) { int i = 1, j = n+1; while ( i + 1!= j ) { int k = ( i + j ) / 2; if ( k*k <= n ) i = k; else j = k; } return i; } // squareroot LAD, complexity 8
Complexity of squareroot Big O The complexity varies depending on the size of the argument (n) We get the time function: T(n) = 1 + 2 + 2 + noofloops (2 + 3 + 2 + 1) or shorter T(n) = 5 + 8 noofloops Since the interval (i.. j )is decreased to half its size each loop, the number of loops becomes 2 log n why we draw the conclusion T(n) O( 2 log n ) LAD, complexity 9
Program squareroot Big O public static int noofloops; public static int squareroot( int n ) { while ( i + 1!= j ) { noofloops++;... }... } // squareroot public static void main(string[] args ) { int[] a = { 1, 8, 32, 128, 1024, 2048, 10000 }; for( int n : a ) { noofloops = 0; System.out.println( n + "\t" + squareroot(n) + "\t" + noofloops } LAD, complexity 10
Run squareroot Big O sed:bjerner:[~/dvds/program]$ java SquareRoot 1 1 1 8 2 3 32 5 5 128 11 7 1024 32 10 2048 45 11 10000 100 14 LAD, complexity 11
Algorithm Insertion sort Given an array a to be sorted. let i = smallest index + 1 while i highest index. if a[i] < a[i 1] let temp = a[i], a[i] = a[i 1] and j = i 1 while j > smallest index and a[j 1] > temp let a[j] = a[j 1] decrease j by 1 let a[j] = temp increase a[i] by 1 LAD, complexity 12
Correctness Predicates Insertion sort Loop invariant: The segment to the left of i in a is sorted. Loop condition: i is less than or equal to the highest index for a. Loop conclusion: The whole array is sorted. LAD, complexity 13
Java code Insertion sort public static void insertionsort( int[] a ) { for ( int i = 1; i < a.length; i++ ) if ( a[i] < a[i-1] ) { int tmp = a[i]; a[i] = a[i-1]; int j = i-1; while ( j > 0 && a[j-1] > temp ) { a[j] = a[j-1]; j--; } a[j] = temp; } }\\ insertionsort LAD, complexity 14
Complexity Insertion sort In analysing the comlexity of programs depending on probabilities, we analyse: the best case and the worst case. if they have different Big O complexity, we also analyse the: the average case LAD, complexity 15
Complexity Insertion sort The best case: Its obviously when the array is already sorted. Only the if-condition is computed. If size n = a.length, we get: n 1 i=0 1 = n O(n), i.e. linear The worst case: When the array is sorted reversely. Gives: n 1 i=0 i j=1 1 = n 1 i=0 i = n (n 1) 2 O( n 2 ), i.e. quadratic LAD, complexity 16
Complexity Insertion sort The average case: Assume that the inner loop only goes halfways, then we get: n 1 i=0 i 2 j=1 1 =... = N (N 1) 4 O(n 2 ), i.e. still quadratic LAD, complexity 17
Methods on stacks Stacks Historically, a stack is an object on which we can perform following methods: push(element) : void, puts an element onto the stack. pop() : void, takes the last pushed element away from the stack, i.e. in FILO order (or in LIFO order) top() : element, gives the last pushed element, but let it stay in the stack isempty : boolean: gives true if the stack is empty, otherwise false Sometimes size() : integer occurs. Cmp: with a plate holder in a restaurant. LAD, complexity 18
Exemple, pre-, in-, and post-fix Stacks In order to place a binary operator with respect to its operandes, there are three possibilities: preorder, i.e. first, + op 1 op 2 inorder, i.e. between, op 1 + op 2 postorder, i.e. last, op 1 op 2 + LAD, complexity 19
Exemple A, expressions of different forms Stacks We want to take the product of a and b and add it to the product of c and d preorder : + a b c d inorder : a b + c d postorder : a b c d + Note: For inorder we have to use priorty rules in order to perform the operations in right order! LAD, complexity 20
Exemple B, expressions of different forms Stacks Now, in stead we want to take the sum of a and b and multiply it with the sum of c and d preorder : + a b + c d inorder : (a + b) (c + d) postorder : a b + c d + Note: For inorder we have to use paranteses in order to perform the operations in right order! LAD, complexity 21
Compute expressions on postfix form Stacks Given a sequens of symbols, in postfix form, as an iterator it and a stack st As long as it not empty let sym be next symbol in it if sym is an operand put sym onto st else: if the operator is + Get the two uppermost operands from st, add them and put the result onto st else: Get the two uppermost operands from st, multiply them and put the result onto st The result is now on top of the stack st! LAD, complexity 22
Compute expressions on postfix form, Java Stacks Note: In most implementations pop gives the removed object as a result. Iterator<String> it =...; Stack<Integer> st = new Stack<Integer>(); while ( it.hasnext() ) { String sym = it.next(); switch ( sym.charat(0) ) { case + : st.push( st.pop() + st.pop() ); break; case * : st.push( st.pop() * st.pop() ); break; default : push( Integer.parsInt( sym ) ); } } LAD, complexity 23
Stack implementation Stacks May be implemented efficiantly either by array or as a linked structure. An array is allocated with expected maximum size of the stack, if possible, and a field keeping track of the stack top. All operations will then be of O(1). For the linked structure, a singly linked structure, is apropiate, since all operations only concern the first element in the list. Here also, the operations are of O(1). LAD, complexity 24