Data Structures and Algorithm Analysis

Size: px

Start display at page:

Download "Data Structures and Algorithm Analysis"

Maria McDaniel
6 years ago
Views:

1 Data Structures and Algorithm Analysis Dr. Malek Mouhoub Department of Computer Science University of Regina Winter 2005 Malek Mouhoub, CS340 Winter

2 1. Introduction 1. Introduction 1.1 Solving problems and algorithm analysis. 1.2 Features of C ADTs and Linear Structures. Malek Mouhoub, CS340 Winter

3 1.1 Solving problems and algorithm analysis 1.1 Solving problems and algorithm analysis Boss gives the following problem to Mr Dupont, a fresh hired BSc in computer science (to test him... or may be just for fun) : T (1) = 3 T (2) = 10 T (n) = 2T (n 1) T (n 2) What is T (100)? Malek Mouhoub, CS340 Winter

4 1.1 Solving problems and algorithm analysis Mr Dupont decides to directly code a recursive function tfn, in Java, to solve the problem : if (n==1) { return 3; } else if (n==2) {return 10;} else { return 2 tfn(n-1) - tfn(n-2); } 1. First mistake : no analysis of the problem risk to be fired! 2. Second mistake : bad choice of the programming language increasing the risk to be fired! Malek Mouhoub, CS340 Winter

5 1.1 Solving problems and algorithm analysis n = 1 3 n = 2 10 n = 3 17 n = 35 it takes 4.19 seconds n = 100 waits... and then kills the program! Mr Dupont decides then to use C : if (n==1) return 3; if (n==2) return 10; return 2 tfn(n-1) - tfn(n-2); n = 35 it takes only 1.25 seconds n = 50 waits and then kills the program! Malek Mouhoub, CS340 Winter

6 1.1 Solving problems and algorithm analysis Finally, Mr Dupont decides to (experimentally) analyze the problem : he times both programs and plots the results. 100 seconds 10 Java C N It seems that each time n increases by 1 the time increases by At n = 40 the C program took seconds, so for n = 100 he estimates : , 627, 995 years!!! Malek Mouhoub, CS340 Winter

7 1.1 Solving problems and algorithm analysis Mr Dupont remembers he has seen this kind of problem in one of the courses he has taken (data structures course). After consulting his course notes, Mr Dupont decides to use Dynamic Programming : int t[n+1]; t[1]=3; t[2]=10; for(i=3;i =n;i++) t[i] = 2 t[i-1] - t[i-2]; return t[n]; Malek Mouhoub, CS340 Winter

8 1.1 Solving problems and algorithm analysis This solution provides a much better complexity in time but despite of the space complexity : n = 100 takes only a fraction of a second, but for n = 10, 000, 000 (a test that may make the boss happy... if it succeeds) a segmentation fault occurs. Too much memory required. Malek Mouhoub, CS340 Winter

9 1.1 Solving problems and algorithm analysis Mr Dupont analyses the problem again : there is no reason to keep all the values, only the last 2 : if (n==1) return 3; last = 3; current = 10; for (i=3;i<=n;i++) { temp = current; current = 2 current - last; last = temp; } return current; At n = 100,000,000 it takes 3.00 seconds At n = 200,000,000 it takes 5,99 seconds at n = 300,000,000 it takes 8.99 seconds Malek Mouhoub, CS340 Winter

10 1.1 Solving problems and algorithm analysis How to solve such problems? 1. Analyze the problem on paper in order to find an efficient algorithm in terms of time and memory space complexity. (a) First look at the problem : T (1) = 3 T (2) = 10 T (3) = 17 T (4) = 24 T (5) = 31 Each step increases the result by 7. (b) Guess : T (n) = 7n 4 (c) Proof by induction 2. Code : return 7 n-4 Malek Mouhoub, CS340 Winter

11 1.1 Solving problems and algorithm analysis An algorithm analyst might ask : 1. What makes the first program so slow? 2. How fast are the 3 programs asymptotically? 3. Is the last version really the ultimate solution? Malek Mouhoub, CS340 Winter

12 1.1 Solving problems and algorithm analysis Let us look at the recursion tree for the first program at n= Here each circle represents one call to the routine tfn. So, for n=4 there are 5 such calls. In general, a call to tfn(n) requires a recursive call to tfn(n-1)(represented by the shaded region on the left) and a call to tfn(n-2)(shaded region on the right). Malek Mouhoub, CS340 Winter

13 1.1 Solving problems and algorithm analysis If we let f(n) represent the number of calls to compute T (n), then : f(n) = f(n 1) + f(n 2) + 1 f(1) = f(2) = 1 This is a version of the famous Fibonacci recurrence. It is known that f(n) n. This agrees very well with the times we presented earlier where each increase of n by 1 increases the time by a factor of a little under We say such growth is exponential with asymptotic growth rate O(1.618 n ). This answers question (1). Malek Mouhoub, CS340 Winter

14 1.1 Solving problems and algorithm analysis In the second and third program there was a loop for (i=3;i<=n;i++) This loop contained two or three assignments, a multiplication and a subtraction. We say such a loop takes O(n) time. This means that running time is proportional to n. Recall that increasing n from 100 million to 300 million increased the time from approximately 3 to approximately 9 seconds. The last program has one multiplication and one subtraction and takes O(1) or constant time. This answers question (2). Malek Mouhoub, CS340 Winter

15 1.1 Solving problems and algorithm analysis The answer to the last question is also NO. If the boss asked for T ( ) we would get integer overflow on most computers. Switching to a floating point representation would be of no value since we need to maintain all the significant digits in our results. The only alternative is to use a method to represent and manipulate large integers. Malek Mouhoub, CS340 Winter

16 1.1 Solving problems and algorithm analysis A straightforward way to represent a large integer is to use an array of integers, where each array slot stores one digit. The addition and subtraction require a linear-time algorithm. A simple algorithm for multiplication requires a quadratic-time cost. Malek Mouhoub, CS340 Winter

17 1.1 Solving problems and algorithm analysis The third question, is the last program the ultimate solution, begins to cross the line from algorithm analysis to computing science. A Computer Scientist might ask : 1. How do you justify counting function calls in the first case, counting array assignments in the second case, counting variable assignments in the third, and counting arithmetic operations in the last? 2. Is it really true that you can multiply two arbitrary large numbers together in constant time? 3. Is the last program really the ultimate one? Malek Mouhoub, CS340 Winter

18 1.1 Solving problems and algorithm analysis More CS questions that occur after thinking about the previous ones : What is computation? What is the simplest model of computation? How do various models relate - in terms of what can be computed and how efficiently? What are the theoretical limits of computation? What are the physical limits? What are the practical limits? How should we define a computational problem? Can we prove some problems are hard to compute for all possible programs? How can we classify problems as to how hard they are to compute? Malek Mouhoub, CS340 Winter

19 1.1 Solving problems and algorithm analysis CS questions with an engineering orientation : What general techniques can we use to solve computational problems? What data structures are best, and in what situations? Which models should we use to analyze algorithms in practice? When trying to improve the efficiency of a given program, which aspects should we focus on first? Malek Mouhoub, CS340 Winter

20 1.1 Solving problems and algorithm analysis Well... there is also CS and the BIG questions : Is the underlying foundation of the universe computational? Aristote, Newton, Lovelace, Einstein, Turing, Feymann Is computation sufficient to explain intelligence and consciousness? Babbage, Lovelace, Turing, Searle, Minsky, Penrose Is evolution a computational process? Holland, Kaufmann Malek Mouhoub, CS340 Winter

21 1.2 Features of C Features of C++ Default parameters Initializer list explicit constructor Constant member function Interface and Implementation Vectors, strings and pointers Function and class templates Malek Mouhoub, CS340 Winter

22 1.2 Features of C++ /* A class for simulating an integer memory cell */ class IntCell { public: /* Construct the IntCell. Initial value is 0. */ IntCell() { storedvalue = 0;} /* Construct the IntCell. Initial value is initialvalue. */ IntCell(int initialvalue) { storedvalue = initialvalue; } /* Return the stored value */ int read() { return storedvalue; } /* Change the stored value to x */ void write( int x ) { storedvalue = x; } private: int storedvalue; }; Malek Mouhoub, CS340 Winter

23 1.2 Features of C++ class IntCell { public: /*1*/ explicit IntCell( int initialvalue = 0 ) /*2*/ : storedvalue( initialvalue ) {} /*3*/ int read() const /*4*/ { return storedvalue; } /*5*/ void write( int x ) /*6*/ { storedvalue =x; } private: /*7*/ int storedvalue; }; Malek Mouhoub, CS340 Winter

24 1.2 Features of C++ File IntCell.h Separation of Interface and Implementation #ifndef _IntCell_H_ #define _IntCell_H_ class IntCell { public: explicit IntCell( int initialvalue = 0); int read() const; void write( int x ); private: int storedvalue; }; #endif Malek Mouhoub, CS340 Winter

25 1.2 Features of C++ File IntCell.cpp Separation of Interface and Implementation #include "IntCell.h" /* Construct the IntCell with initialvalue */ IntCell::IntCell( int initialvalue ) : storedvalue(initialvalue) {} /* Return the stored value */ int IntCell::read() const { return storedvalue;} /* Store x */ void IntCell::write( int x ) { storedvalue = x; } Malek Mouhoub, CS340 Winter

26 1.2 Features of C++ Separation of Interface and Implementation #include "IntCell.h" int main() { } IntCell m; //Or, IntCell m( 0 ); but not IntCell m(); m.write( 5 ); cout << "Cell contents: " << m.read() << endl; return 0; Malek Mouhoub, CS340 Winter

27 1.2 Features of C++ Separation of Interface and Implementation Preprocessor commands Scoping operator Signature must match exactly Objects are declared like primitive types IntCell obj1; // Zero parameter constructor obj2( 12 ); // One parameter constructor IntCell obj3 = 37; /* Error : Constructor is explicit */ IntCell obj4(); // Error : Function declaration Malek Mouhoub, CS340 Winter

28 1.2 Features of C++ #include <iostream.h> #include "vector.h" #include "mystring.h" Vectors and strings int main() { vector<string> v( 5 ); int itemread = 0; string x; while( cin >> x) { if( itemread == v.size()) v.resize(v.size() * 2); v[ itemread++] = x; } for (int i = itemsread - 1; i >=0; i--) cout << v[i] << endl; return 0; } Malek Mouhoub, CS340 Winter

29 1.2 Features of C++ int main() { /*1*/ IntCell *m; Pointers /*2*/ m=new IntCell( 0 ); /*3*/ m->write( 5 ); /*4*/ cout << "Cell contents:"<< m->read() << endl; /*5*/ delete m; /*6*/ return 0; } Malek Mouhoub, CS340 Winter

30 1.2 Features of C++ Function Templates template <class Comparable> const Comparable & findmax(const vector<comparable> & a) { /*1*/ int maxindex=0; /*2*/ for (int i=1;i <a.size();i++) /*3*/ if (a[maxindex] < a[i]) /*4*/ maxindex = i; /*5*/ return a[maxindex]; } Malek Mouhoub, CS340 Winter

31 1.2 Features of C++ Function Templates int main() { vector<int> v1( 37 ); vector<double> v2( 40 ); vector<string> v3( 80 ); vector<intcell> v4( 75 ); // Additional code to fill in the vectors cout << findmax( v1 ) << endl; cout << findmax( v2 ) << endl; cout << findmax( v3 ) << endl; cout << findmax( v4 ) << endl; // Error : operator < undefined return 0; } Malek Mouhoub, CS340 Winter

32 1.2 Features of C++ Class Templates // A class for simulating a memory cell. template <class Object> class MemoryCell { public: explicit MemoryCell (const Object &initialvalue=object()) : storedvalue ( initialvalue ) {} const Object & read() const { return storedvalue; } void write( const Object & x ) { storedvalue = x; } private: Object storedvalue; }; Malek Mouhoub, CS340 Winter

33 1.2 Features of C++ int main() { MemoryCell<int> m1; Class Templates MemoryCell<string> m2( hello ); m1.write( 37 ); m2.write( m2.read() + world ); cout << m1.read() << endl << m2.read() << endl; return 0; } Malek Mouhoub, CS340 Winter

34 1.3 ADTs and Linear Structures 1.3 ADTs and Linear Structures Abstract Data Types (ADTs) The List ADT The Stack ADT The Queue ADT Malek Mouhoub, CS340 Winter

35 Data Structures Data Structures A scalar item A sequential vector A linked list A n-dimentional space A hierarchical tree Malek Mouhoub, CS340 Winter

36 Data Structures The most important property to express of any entity in a system is its type. In this course we used entities that are structured objects (e.g., an object that is a collection of other objects). When determining its type, the kinds of distinguishing properties include : Ordering : are elements ordered or unordered? If ordering matters, is the order partial or total? Are elements removed FIFO (queues), LIFO (stacks), or by priority (priority queues)? Duplicates : are duplicates allowed? Boundedness : is the object bounded in size or unbounded? Can the bound change or it is fixed at creation time? Associative access : are elements retrieved by an index or key? Is the type of the index built-in (e.g. as for sequences and arrays) or user-definable (e.g. as for symbol tables and hash tables)? Shape : is the structure of the object linear, hierarchical, acyclic, n-dimensional, or arbitrarily complex (e.g. graphs, forests)? Malek Mouhoub, CS340 Winter

37 Data Structures Abstract Data Type (ADT) Set of data together with a set of operations. Definition of an ADT : [what to do?] Definition of data and the set of operations (functions). Implementation of an ADT : [how to do it?] How are the objects and operations implemented. use the C++ class. Malek Mouhoub, CS340 Winter

38 Data Structures ADT List Structure List is Data : A list of elements. Functions : for every list List, item Element List createlist() ::= return an empty list List insert(item,i,list) ::= add item at the position i and return the new list List remove(item,list) ::= remove item and return the new list Boolean isempty(list) ::= return list == empty find(item,list) ::= If item list return position of item else return ERROR findkth(i,list) ::= return the item at position i if it exists, ERROR otherwise. Malek Mouhoub, CS340 Winter

39 Data Structures Array Implementation of Lists Contiguous allocation of memory to store the elements of the list. Estimation (overestimation) of the maximum size of the list is required waste of memory space. O(N) for find, constant time for findkth But O(N) is required for insertion and deletion in the worst case. building a list by N successive inserts would require O(N 2 ) in the worst case. Malek Mouhoub, CS340 Winter

40 Data Structures List findkth(3)=52 find(52)=3 remove(34) insert(1,34) n n n n findkth=list[kth] O(C) find(x): O(n) removekth: O(n) remove(x):o(n) insert(kth,x) O(n) Figure 1: Contiguous allocation of memory to store the elements of the list Malek Mouhoub, CS340 Winter

41 Data Structures Linked Lists Non contiguous allocation of memory. O(N) for find O(N) for findkth (but better time in practice if the calls to findkth are in sorted order by the argument). Constant time for insertion and deletion. Malek Mouhoub, CS340 Winter

42 Data Structures Head Data Link \ 3 4 Head List = printlist():o(n) find(x):o(n) findkth(i):o(i) Figure 2: Non contiguous allocation of memory Malek Mouhoub, CS340 Winter

43 Data Structures A1 A2 A3 A4 A5 (a) A1 A2 A3 A4 A5 (b) Figure 3: (a) A double linked list (b) A double circular linked list Malek Mouhoub, CS340 Winter

44 Data Structures header header (a) A lilnked list remove(52) header (b) Deletion from a linked list insert(3,52) 52 (c) Insertion into a linked list A1 A2 A3 A4 A5 header (d) Linked list with a header header (e) Empty list with header Figure 4: Linked List Functions Malek Mouhoub, CS340 Winter

45 Data Structures ADT Stack Structure Stack is Data : A list of elements. Functions : for every stack Stack, item Element, capacity Natural Stack createstack(capacity) ::= return an empty stack Boolean isfull(stack) ::= return (number of items in stack == capacity) Boolean isempty(stack) ::= return (stack == createstack(capacity)) Stack push(item,stack)::= If isfull(stack) return stack else add item at the top of stack and return the new stack Stack pop(stack) ::= If isempty(stack) return stack else delete the element at the top of the stack and return the new stack Element topandpop(stack) ::= If isempty(stack) return ERROR else delete and return the element at the top of the stack Malek Mouhoub, CS340 Winter

46 Data Structures string input = "[()]" read "[" read "(" read ")" read "]" read "" Top Top [ ( [ Top [ Top push("[") push ("(") top()="(" pop() top()="[" pop() empty() correct Figure 5: Application: Balancing Symbols Malek Mouhoub, CS340 Winter

47 Data Structures intput infix = a+b*c+(d*e+f)*g read "a" read "+" read "b" read "*" read "c" output:a push ("+") + + output:ab * + push("*") * + output:abc read "+" read "(" read "d" read "*" read "e" + pop() output:abc*+ ( ( + + push ("(") output:abc*+d read "+" read "f" read ")" * ( + push("*") read "*" * ( + output:abc*+de read "g" + ( + + ( + pop() output:abc*+de*f pop() output:abc*+de* push("+") output:abc*+de*f+ pop() + * + push("*") * + output:abc*+def+g pop() output:abc*+def+g* pop() output:abc*+def+g*+ Figure 6: Application : Infix and Postfix Conversion Malek Mouhoub, CS340 Winter

48 Data Structures ADT Queue Structure Queue is Data : A list of elements. Functions : for every queue Queue, item Element, capacity Natural Queue createqueue(capacity)::= return an empty queue Boolean isfull(queue) ::= return (number of items in stack == capacity) Boolean isempty(queue) ::= return (queue == createqueue(capacity)) Queue enqueue(item,queue) ::= If isfull(queue) return queue else add item at the end of queue and return the new queue Queue dequeue(queue) ::= If isempty(queue) return queue else delete the element at the front of the queue and return the new queue Element getfront(stack) ::= If isempty(stack) return ERROR else delete and return the element at the front of queue Malek Mouhoub, CS340 Winter

49 Data Structures (1)Initial State (2)After enqueue(1) ^ front 2 4 ^ back 1 ^ back 2 4 ^ front (3)After enqueue(3) (4)After dequeue, which returns ^ back ^ front ^ back ^ front (5)After dequeue, which returns 4 (6)After dequeue, which returns ^ ^ front front ^ back back 1 3 (7)After dequeue, which return 3 and Makes the Queue Empty ^ ^ backfront 2 4 Figure 7: Array Implementation of Queues Malek Mouhoub, CS340 Winter

Purpose of Review. Review some basic C++ Familiarize us with Weiss s style Introduce specific constructs useful for implementing data structures

Purpose of Review. Review some basic C++ Familiarize us with Weiss s style Introduce specific constructs useful for implementing data structures C++ Review 1 Purpose of Review Review some basic C++ Familiarize us with Weiss s style Introduce specific constructs useful for implementing data structures 2 Class The Class defines the data structure