Data Structures & Algorithms (Cs-2133) By Dr. Qamas Gul Khan Safi Fall 2018 http://dsa17.weebly.com Department of Electrical Engineering University of Engineering & Technology, Taxila. Lecture No.01 Data Structures and Algorithms 1
TODAY S AGENDA Welcome to CS-2133 Why CS-2133 may change your life? Various administrative issues. What is algorithm? What is this course about? WELCOME TO CS-2133 About me: Dr. Gul Safi? Dr. Gul, doing research in Cloud Computing, Information Security and VANETs. Mostly all kinds of algorithms.. What you can expect from me: Helpful, encouraging; inspiring and enjoying class Good grades if u really work hard. What I expect from you: Turn in all homework and participate classes You learn some critical techniques from this course You show signs to be able to invent new algorithms Algorithms may change your life, don t think so? 2
On This Course Instructor: Dr. Qamas Gul Khan Safi qamas.gul@uettaxila.edu.pk Office Hours: 01:30 PM - 2:30 PM Course Webpage: http://dsa17.weebly.com check this regularly for important announcement related to this course Lecture notes, Labs and additional readings Code of Student Academic Responsibility Please check this for detailed requirements on academic integrity The departmental chair emphasized this issue and required all the violation behavior will be reported to the department chair I can easily use Google and a special software to figure your any plagiarism. 3
Why you want to study Algorithms? Making a lot of money out of a great algorithm $1,000,000,000? Example: PageRank algorithm by Larry Page The soul of Google search engine Google total assets: $31 billions on 2008 Why you want to study Algorithms? Make significant contribution to the society Viterbi algorithm -- How much it is worth? Dynamic programming algorithm for finding the most likely sequence of hidden states Cell Phone, wireless network, modem, etc 4
Why you want to study Algorithms? Viterbi algorithm conceived by Andrew Viterbi in 1967 as an error-correction scheme for noisy digital communication links, finding universal application in decoding the convolutional codes used in both Viterbi algorithm is a standard component of tens of millions of high-speed modems. It is a key building block of modern information infrastructure CDMA and GSM digital cellular, dial-up modems, satellite, deep-space communications, and 802.11 wireless LANs. It is now also commonly used in speech recognition, keyword spotting, computational linguistics, and bioinformatics. Why you want to study Algorithms? Simply to be cool to invent something in computer science Example: Shortest Path Problem and Algorithm Used in GPS and Mapquest or Google Maps 5
Algorithm and Data Structures An algorithm is a sequence of unambiguous instructions/operations for solving a problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time. Map Navigation AB problem input Graphs algorithm computer + programs Data Structures output Path How to study algorithms? Problem Representation/data structure in computer Operations on representations 6
Example: Sorting Statement of problem: Input: A sequence of numbers Output: A reordering of the input sequence so that a ' a' Instance: The sequence Algorithms: Selection sort Insertion sort Merge sort (many others) i j n whenever i j 5, 3, 2,8,3 a 1, a2,..., a n a ' 1, a' 2,..., a' n Selection Sort Input: array a[ 1], a[2],..., a[ n] Output: array a[1..n] sorted in non-decreasing order Algorithm: for i =1 to n swap a[i] with smallest of a[ i],..., a[ n] see also pseudocode, section 3.1 7
Some Important Points Each step of an algorithm is unambiguous The range of inputs has to be specified carefully The same algorithm can be represented in different ways The same problem may be solved by different algorithms Different algorithms may take different time to solve the same problem we may prefer one to the other Example: Finding gcd(m,n) Input: m and n are two nonnegative, not-both-zero integers (Note: the range of input is specified) Output: gcd(m,n), the greatest common divisor, i.e., the largest integer that divides both m and n Euclid algorithm: Based on gcd(m,n)=gcd(n, m mod n) For example: ALGORITHM gcd(60, 24) =gcd(24,12) while n 0 =gcd(12,0) r m mod n =12 m n Will this algorithm eventually comes to a stop? Why? n r return m Euclid ( m, n) 8
Another Algorithm for Finding gcd(m,n) Note that 0 gcd( m, n) min( m, n), the pseudocode is ALGORITHM t min( m, n) t t 1 return t gcd( m, n) while ( m mod t 0) or ( n mod t 0) What is the range of input for this algorithm? Can one of them to be zero? No, both m and n must be positive. In this course, you usually write the algorithm in pseudocode instead of the real code in some special language. Fundamentals of Algorithmic Problem Solving 1. Understanding the problem 2. Ascertaining the capabilities of a computational device Random-access machine (RAM) sequential algorithms 3. Choose between exact and approximate problem solving 4. Deciding on appropriate data structure 5. Algorithm design techniques 6. Methods of specifying an algorithm Pseudocode (for, if, while, //,, indentation ) 7. Prove an algorithm s correctness mathematic induction 8. Analyzing an algorithm Simplicity, efficiency, optimality 9. Coding an algorithm 9
In general A good algorithm is a result of repeated effort and rework Better data structure Better algorithm design Better time or space efficiency Easy to implement Optimal algorithm Some Well-known Computational Problems Sorting Searching Shortest paths in a graph Minimum spanning tree Primality testing Traveling salesman problem Knapsack problem Chess Towers of Hanoi 10
This Course is Focused on How to design algorithms How to express algorithms -- pseudocode Proving correctness Efficiency Analysis Theoretical analysis Empirical analysis Optimality Algorithm Design Strategies Brute force Divide and conquer Decrease and conquer Transform and conquer Greedy approach Dynamic programming Invented or applied by many genius in CS Backtracking and branch and bound Space and time tradeoffs 11
Analysis of Algorithms How good is the algorithm? Correctness Time efficiency Space efficiency Does there exist a better algorithm? Lower bounds Optimality In general: What is an Algorithm? Recipe, process, method, technique, procedure, routine, with following requirements: Finiteness: terminates after a finite number of steps Definiteness: rigorously and unambiguously specified Input: valid inputs are clearly specified Output: can be proved to produce the correct output given a valid input Effectiveness: steps are sufficiently simple and basic 12
Data Structures Prepares the students for (and is a prerequisite for) the more advanced material students will encounter in later courses. Cover well-known data structures such as dynamic arrays, linked lists, stacks, queues, tree and graphs. Implement data structures in C++ Need for Data Structures Data structures organize data more efficient programs. More powerful computers more complex applications. More complex applications demand more calculations. 13
Organizing Data Any organization for a collection of records that can be searched, processed in any order, or modified. The choice of data structure and algorithm can make the difference between a program running in a few seconds or many days. Efficiency A solution is said to be efficient if it solves the problem within its resource constraints. Space Time The cost of a solution is the amount of resources that the solution consumes. 14
Efficiency A solution is said to be efficient if it solves the problem within its resource constraints. Space Time The cost of a solution is the amount of resources that the solution consumes. Selecting a Data Structure Select a data structure as follows: 1. Analyze the problem to determine the resource constraints a solution must meet. 2. Determine the basic operations that must be supported. Quantify the resource constraints for each operation. 3. Select the data structure that best meets these requirements. 15
Data Structure Philosophy Each data structure has costs and benefits. Rarely is one data structure better than another in all situations. A data structure requires: space for each data item it stores, time to perform each basic operation, programming effort. Goals of this Course 1. Reinforce the concept that costs and benefits exist for every data structure. 2. Learn the commonly used data structures. These form a programmer's basic data structure toolkit. 3. Understand how to measure the cost of a data structure or program. These techniques also allow you to judge the merits of new data structures that you or others might invent. 16
Arrays Elementary data structure that exists as built-in in most programming languages. main( int argc, char** argv ) { int x[6]; int j; for(j=0; j < 6; j++) x[j] = 2*j; } Arrays Array declaration: int x[6]; An array is collection of cells of the same type. The collection has the name x. The cells are numbered with consecutive integers. To access a cell, use the array name and an index: x[0], x[1], x[2], x[3], x[4], x[5] 17
Array Layout Array cells are contiguous in computer memory The memory can be thought of as an array x[0] x[1] x[2] x[3] x[4] x[5] What is Array Name? x is an array name but there is no variable x. x is not an lvalue. For example, if we have the code int a, b; then we can write b = 2; a = b; a = 5; But we cannot write 2 = a; 18
Array Name x is not an lvalue int x[6]; int n; x[0] = 5; x[1] = 2; x = 3; x = a + b; x = &n; // not allowed // not allowed // not allowed Dynamic Arrays You would like to use an array data structure but you do not know the size of the array at compile time. You find out when the program executes that you need an integer array of size n=20. Allocate an array using the new operator: int* y = new int[20]; // or int* y = new int[n] y[0] = 10; y[1] = 15; // use is the same 19
Dynamic Arrays y is a lvalue; it is a pointer that holds the address of 20 consecutive cells in memory. It can be assigned a value. The new operator returns as address that is stored in y. We can write: y = &x[0]; y = x; // x can appear on the right // y gets the address of the // first cell of the x array Dynamic Arrays We must free the memory we got using the new operator once we are done with the y array. delete[ ] y; We would not do this to the x array because we did not use new to create it. 20
The LIST Data Structure The List is among the most generic of data structures. Real life: a. shopping list, b. groceries list, c. list of people to invite to dinner d. List of presents to get Lists A list is collection of items that are all of the same type (grocery items, integers, names) The items, or elements of the list, are stored in some particular order It is possible to insert new elements into various positions in the list and remove any element of the list 21
Lists List is a set of elements in a linear order. For example, data values a 1, a 2, a 3, a 4 can be arranged in a list: (a 3, a 1, a 2, a 4 ) In this list, a 3, is the first element, a 1 is the second element, and so on The order is important here; this is not just a random collection of elements, it is an ordered collection List Operations Useful operations createlist(): create a new list (presumably empty) copy(): set one list to be a copy of another clear(); clear a list (remove all elments) insert(x,?): Insert element X at a particular position in the list remove(?): Remove element at some position in the list get(?): Get element at a given position update(x,?): replace the element at a given position with X find(x): determine if the element X is in the list length(): return the length of the list. 22
List Operations We need to decide what is meant by particular position ; we have used? for this. There are two possibilities: 1. Use the actual index of element: insert after element 3, get element number 6. This approach is taken by arrays 2. Use a current marker or pointer to refer to a particular position in the list. List Operations If we use the current marker, the following four methods would be useful: start(): moves to current pointer to the very first element. tail(): moves to current pointer to the very last element. next(): move the current position forward one element back(): move the current position backward one element 23