The Running Time of Programs
|
|
- Andrea Gordon
- 6 years ago
- Views:
Transcription
1 The Running Time of Programs The Rule Many programs exhibit the property that most of their running time is spent in a small fraction of the source code. There is an informal rule that states 90% of the running time is spent in 10% of the code. While the exact percentage varies from program to program, the rule says that most programs exhibit significant and locality in where the running time is spent. Measuring the Running Time We can measure the running time of a program simply by using wall clock time. However, there are several disadvantages with this methodology, including: Input dependence Machine dependence We want a more analytical method for measuring the running time of a program instead of relying on the wall clock time. If we consider a program at the highest level, it will accept certain inputs, process them, and output the result. We want to come up with the running time measurement that depends only on the program's input size. In this way, the measurement is independent of the type of input and the hardware on which the program runs. What should be the standard measurement for the running time of a program that is independent of hardware and input type? Input Output Program Introducing Asymptotic Bound Analysis that relates the running time of a program with its input size. Such relation will be dependent on the algorithms and data structures that the program uses to process the input. Asymptotic Upper Bound (Big O) Big of f(n) is is a set of function T(n) with the property that there exists a positive constant and such that for : The below diagram shows how the above definition can be visualized: 1
2 C*f(n) T(n) N We normally use the overloaded notation: when we say that is order. When we prove a property about Big O, we have the freedom to choose arbitrary positive constants. So, we can see that: since we can chose N to be 0 and C to be Is in O(n)? We have to demonstrate that we can find C and N to satisfy the following condition: n 2 <= Cn for every n >= N But, no matter how large we choose C, there will always be an n value that is greater than C (since n is a variable) to make Example: 4n 3 + 3n 2 +4 is order O(n 3 ) 4n is order O(n 3 ) 2
3 When we use big O to analyze the running time of a program, the variable n is the size of the input and T(n) is the running time. As mentioned before the "shape" of the function T(n) will depend on the algorithms and data structures employed in the program. If T(n) = 4n 3 + n 2 +1, the running time of the program is 4n 3 + n 2 +1, which is O(n 3 ) (pronounced "big O of n cube) Asymptotic bound is the measure for the order of growth of the function T(n). We definitely want T(n) to have a small order so that the running time grows "reasonably" with the increase in the input size. And, what is a reasonable order of growth? It has been a commonly accepted wisdom in computer science that that order is polynomial order. That is T(n) is O(n d ) when d>=1 When we consider a polynomial function, the term with the highest exponent dominates. To appreciate why this is so consider T(n) = 4n 3 + n 2 +1 if n = 10 20, we have T(n) = 4(10 60 ) We can see that 4(10 60 ) >> (10 40 ) so the latter term becomes insignificant (20 order of magnitude difference!). So, the order of T(n) is determined by the highest exponent. Can we prove that T(n) is actually O(n 3 )? If we choose C to be 6, we ask if we can find N such that for all n >= N: 4n 3 + n 2 +1 <= 6n 3 4n 3 + n 2 +1 <= 4n 3 + n 3 + n 3 If we choose N = 1, we see that RHS is always greater than or equal LHS for n >= N. Tightness First, we generally want the tightest big oh upper bound we can prove. That is, if T (n) is O(n 2 ), we want to say so, rather than make the technically true but weaker statement that T (n) is O(n 3 ). On the other hand, this way lies madness, because if we like O(n 2 ) as an expression of running time, we should like O(0.5 n 2 ) even better, because it is tighter, and we should like O(.01 n 2 ) still more, and so on. However, since constant factors don t matter in big oh expressions, there is really no point in trying to make the estimate of running time tighter by shrinking the constant factor. Thus, whenever possible, we try to use a big oh expression that has a constant factor 1. More precisely, we shall say that f(n) is a tight big oh bound on T (n) if 1. T (n) is O(f(n)), and 2. f(n) is also O(T(n)) The following lists some of the more common running times for programs and their informal names Examples of Analysis of the Running Time 1. Finding closet pair of points Given n points in a co ordinate (x,y) plane, find two points with the closet distance from each other 3
4 We propose a program that implements the following algorithm to tackle this problem. The algorithm checks all pairs of points and updates the "minimum distance" value each time it calculates the distance between two points. What is the running time T(n) of this program? We will first assume that the calculation of the distance between two points take a constant amount of time, and, hence, it is independent of the number of input points. How many pairs of points do we need to consider? If there are 4 points, we start with the first one and check it against all the other three points. Then, we pick the second point and check it against all the other except the first and so on In general, for n points, we need to check (n 1) + (n 2) = pairs Therefore, T(n) = C*((n 1)*n)/2 = T(n) = O (n 2 ) Consider the pseudo code for solving this problem. for (i=o; i< numpoints; i++) { for (j=i+1; j <numpoints; j++) { Find distance from point i to point j and store in d If (d<minimum distance) update minimum distance in d We see that there are two loops where one is nested inside of the other. The outer loop will iterate numpoints time (which is n times if numpoints equals n). The inner loop will iterate less than numpoints time, but it will never go beyond numpoints. So, we can say that both loops have numpoints as their uppper bounds, and, hence, we can deduce the running them of this code by arguing that the code will iterate at most numpoints x numpoints times. Hence, it has a running time of T(n) of O(n 2 ) 4
5 2. Array processing: An input to a given program is an array of size n. We want to "compress" this array by reducing the number of duplicated adjacent elements to just one. For example, given: Input 1,1,2,2,2,3, 1, 1 the program returns: Return 1, 2, 3, 1 (The number of 1 is reduced from 2 two 1 and the number of 2 from 3 to 1) Consider the following pseudo code used to solve this problem: i= 0; j=0; while (i < arraylength) { array[j]=array[i]; do { i ++ ; while ((i<array.length) && (array[i]==array[j])); j++; It has got two nested loops just like in the first example. So, can we argue that the running time T(n) of this code is O(n 2 ) where n is the size of the array (array.length)? Not quite. If we look closely, the i variable increase every time, the code enters the innner loop and i cannot go above array.length. Therefore, this code has the running time T(n) = O(n) 3. Insertion Sort: Move the key forward starting with the first position. Maintain the invariant that the data before including at the key index is sorted Consider the two types of inputs to insertion sort: Best Case Analysis Case I: The data is already sorted, hence, there are no exchange operations. Case II: Worst Case Analysis The data is sorted in reverse order, so every pair of elements in the array must be exchanged for every key position. 5
6 The example case we gave earlier can be considered the average case (not best, not worst, but sits in between). When we analyze the running time, we always want to do it for the worst case since we want to bound the running time and be able to say that no matter what type of input a program accepts, its running time is never above this bound. Insertion Sort Input : Array A[0,1,,n-1] for(j=1 ; j<n ; j++){ key = A[j]; //part a i = j-1; //part a while((i>=0) &&(A[i]>key) ){ A[i+1]=A[i]; //part c i--; //part c A[i+1]=key; // part b Analyse the worst case runtime Part A + part B = k (constant time) Part C = m (constant time) Iteration 1 runtime k + m Iteration 2 runtime k + 2m (exchange twice when key moved to 2nd position) Iteration n 1 runtime k+(n 1)m k(n 1)+( n 1)m k(n 1)+m(n 1)(n/2) T(n) = k(n 1) + = = = O(n 2 ) 6
7 4. Merge sort: Input : Array A[0,1,,n 1] MS(A[0,1,..,n 1]) 1. if n = 1 return 2. Recursively sort A[0 ] (1) และ A [ ] (2) 3. Merge Array 1 and 2 together
8 How to merge two sorted array? l 1 r Maintain pointer l and r each pointed to an element in the left and right arrays, respectively. Compare the two pointed elements, if one is less than or equal to the other, bring it down to the third "merge" array r reaches the end of the array. Can bring down the rest of the elements after and including the one pointed to by l one by one in order
9 Since merge sort involves recursive function calls, to calculate its running time, we need a basis for calculating the running time of recursive functions. Consider a recursive factorial below: Since there is only one function, fact, involved, we shall use T (n) for the unknown running time of this function. We shall use n, the value of the argument, as the size of the argument. Clearly, recursive calls made by fact when the argument is n have a smaller argument, n 1 to be precise. For the basis of the inductive definition of T (n) we shall take n = 1, since no recursive call is made by fact when its argument is 1. With n = 1, the condition of line (1) is true, and so the call to fact executes lines (1) and (2). Each takes O(1) time, and so the running time of fact in the basis case is O(1). That is, T (1) is O(1). Now, consider what happens when n > 1. The condition of line (1) is false, and so we execute only lines (1) and (3). Line (1) takes O(1) time, and line (3) takes O(1) for the multiplication and assignment, plus T (n 1) for the recursive call to fact. That is, for n > 1, the running time of fact is O(1) + T (n 1). We can thus define T (n) by the following recurrence relation: BASIS. T (1) = O(1). INDUCTION. T (n) = O(1) + T (n 1), for n > 1. We now invent constant symbols to stand for those constants hidden within the various big oh expressions, as was suggested by rule (c) above. In this case, we can replace the O(1) in the basis by some constant a, and the O(1) in the induction by some constant b. These changes give us the following recurrence relation: BASIS. T (1) = a. INDUCTION. T (n) = b + T (n 1), for n > 1. Now we must solve this recurrence for T (n). We can calculate the first few values easily. T (1) = a by the basis. Thus, by the inductive rule, we have: T (2) = b + T (1) = a + b Continuing to use the inductive rule, we get T (3) = b + T (2) = b + (a + b) = a + 2b Then T (4) = b + T (3) = b + (a + 2b) = a + 3b 9
10 By this point, it should be no surprise if we guess that T (n) = a + (n 1)b, for all n >= 1. Indeed, computing some sample values, then guessing a solution, and finally proving our guess correct by an inductive proof is a common method of dealing with recurrences. Now we are ready to calculate the runtime of merge sort. First, we need to consider the running time of the merge procedure. Merging two sorted arrays each of size n/2 to get a resulting sorted array of size n takes the following runtime: T(n)= cn = O(n) This is so because we use a constant time to compare the two elements on the left and right arrays and bring the smaller of the two down to the merged array. And, we have at most n elements to bring down. We are now ready to set up the recurrence relation for the running time of merge sort as follow: T(n) = T(n/2) + T(n/2) + cn (1) Clearly, T(1) = c and for n > 1, we perform repeated substitution: T(n/2)=T(n/4)+T(n/4)+cn/2 T(n/4)=T(n/8)+T(n/8)+cn/4 cn c(n/2) c(n/2) c(n/4) c(n/4) c(n/4) c(n/4) cn cn cn We then sum all the nodes at the same level of the above tree. Then, we take the resulting sum at each level to produce the total sum for merge sort as a whole. As the tree grows to the level for T(2), the height of the tree is log 2 n T(n)= cn log 2 n Hence, T(n)= O(n log n). And, if we substitute T(n) = cn log 2 n to the equation (1) above, we can verify that this is indeed a correct solution to it. 10
11 5. Recursive Fibonacci: Fib(int n){ if(n==1) return 1; if(n==2) return 1; return Fib(n-1)+Fib(n-2); Fib(5) Fib(6) Fib(4) Fib(4) Fib(3) Fib(3) Fib(2) Fib(3) Fib(2) Fib(2) Fib(1) Fib(2) Fib(1) This tree is different from that of merge sort. Its height is n and not log 2 n as in merge sort. Therefore, the number of nodes at the lowest level of this tree is proportional to 2 n, and, hence, the running time of Fib() is exponential order. Calculating Fib(100) with this code on a state of the art machine takes around 100, 000 years! Notice that this tree is not quite a complete binary tree so the running time is not quite O(2 n ) since the base will be less than 2. In fact, the value of the base is calculated to be the value of the golden ratio, which is approximately So, to be more precise, the running time of this Fib is O(1.62 n ). The bloat in the running time is due to the many redundant repeated calculations. For example, the above tree depicts the same calculation of Fib(3) 3 times. 6. Fibonacci with Memoization To improve the running time of the recursive Fibonacci, we want to put the already calculated value resulting from a particular call in the memo. The next time the same call is encountered, just get the 11
12 resulting value from the memo instead of generating further recursive calls. Our memo is a static array A as shown. x 1 1 A[0] A[1] A[2] A[3] A[4].. The following is the code for Fibonacci with memoization. FibMemo(n){ If(n<=computed) return A[n]; A[n] = FibMemo(n-1) + FibMemo(n-2); computed = n; return A[n]; Initialize: A[1]=1; A[2]=1; computed = 2; FibMemo(6) FibMemo(5) FibMemo(4) FibMemo(4) FibMemo(3) FibMemo(3) FibMemo(2) FibMemo(3) FibMemo(2) FibMemo(2) FibMemo(1) FibMemo(2) FibMemo(2) FibMemo(1) Actual calculation Get these from the memo The running time of FibMemo is O(n) since effectively there are only n calls to FibMemo when calculating Fibonacci number for n. 12
PROGRAM EFFICIENCY & COMPLEXITY ANALYSIS
Lecture 03-04 PROGRAM EFFICIENCY & COMPLEXITY ANALYSIS By: Dr. Zahoor Jan 1 ALGORITHM DEFINITION A finite set of statements that guarantees an optimal solution in finite interval of time 2 GOOD ALGORITHMS?
More informationIntroduction to Analysis of Algorithms
Introduction to Analysis of Algorithms Analysis of Algorithms To determine how efficient an algorithm is we compute the amount of time that the algorithm needs to solve a problem. Given two algorithms
More informationPlotting run-time graphically. Plotting run-time graphically. CS241 Algorithmics - week 1 review. Prefix Averages - Algorithm #1
CS241 - week 1 review Special classes of algorithms: logarithmic: O(log n) linear: O(n) quadratic: O(n 2 ) polynomial: O(n k ), k 1 exponential: O(a n ), a > 1 Classifying algorithms is generally done
More informationRUNNING TIME ANALYSIS. Problem Solving with Computers-II
RUNNING TIME ANALYSIS Problem Solving with Computers-II Performance questions 4 How efficient is a particular algorithm? CPU time usage (Running time complexity) Memory usage Disk usage Network usage Why
More informationCITS3001. Algorithms, Agents and Artificial Intelligence. Semester 2, 2016
CITS3001 Algorithms, Agents and Artificial Intelligence Semester 2, 2016 Tim French School of Computer Science & Software Eng. The University of Western Australia 2. Review of algorithmic concepts CLRS,
More informationRecall from Last Time: Big-Oh Notation
CSE 326 Lecture 3: Analysis of Algorithms Today, we will review: Big-Oh, Little-Oh, Omega (Ω), and Theta (Θ): (Fraternities of functions ) Examples of time and space efficiency analysis Covered in Chapter
More informationUNIT 1 ANALYSIS OF ALGORITHMS
UNIT 1 ANALYSIS OF ALGORITHMS Analysis of Algorithms Structure Page Nos. 1.0 Introduction 7 1.1 Objectives 7 1.2 Mathematical Background 8 1.3 Process of Analysis 12 1.4 Calculation of Storage Complexity
More informationComplexity of Algorithms. Andreas Klappenecker
Complexity of Algorithms Andreas Klappenecker Example Fibonacci The sequence of Fibonacci numbers is defined as 0, 1, 1, 2, 3, 5, 8, 13, 21, 34,... F n 1 + F n 2 if n>1 F n = 1 if n =1 0 if n =0 Fibonacci
More informationComplexity of Algorithms
CSCE 222 Discrete Structures for Computing Complexity of Algorithms Dr. Hyunyoung Lee Based on slides by Andreas Klappenecker 1 Overview Example - Fibonacci Motivating Asymptotic Run Time Analysis Asymptotic
More informationCSE 143. Complexity Analysis. Program Efficiency. Constant Time Statements. Big Oh notation. Analyzing Loops. Constant Time Statements (2) CSE 143 1
CSE 1 Complexity Analysis Program Efficiency [Sections 12.1-12., 12., 12.9] Count number of instructions executed by program on inputs of a given size Express run time as a function of the input size Assume
More informationCS302 Topic: Algorithm Analysis #2. Thursday, Sept. 21, 2006
CS302 Topic: Algorithm Analysis #2 Thursday, Sept. 21, 2006 Analysis of Algorithms The theoretical study of computer program performance and resource usage What s also important (besides performance/resource
More informationLecture 2: Algorithm Analysis
ECE4050/CSC5050 Algorithms and Data Structures Lecture 2: Algorithm Analysis 1 Mathematical Background Logarithms Summations Recursion Induction Proofs Recurrence Relations 2 2 Logarithm Definition: 3
More informationRun Times. Efficiency Issues. Run Times cont d. More on O( ) notation
Comp2711 S1 2006 Correctness Oheads 1 Efficiency Issues Comp2711 S1 2006 Correctness Oheads 2 Run Times An implementation may be correct with respect to the Specification Pre- and Post-condition, but nevertheless
More informationCS 137 Part 7. Big-Oh Notation, Linear Searching and Basic Sorting Algorithms. November 10th, 2017
CS 137 Part 7 Big-Oh Notation, Linear Searching and Basic Sorting Algorithms November 10th, 2017 Big-Oh Notation Up to this point, we ve been writing code without any consideration for optimization. There
More informationComputer Algorithms. Introduction to Algorithm
Computer Algorithms Introduction to Algorithm CISC 4080 Yanjun Li 1 What is Algorithm? An Algorithm is a sequence of well-defined computational steps that transform the input into the output. These steps
More informationAlgorithm Analysis and Design
Algorithm Analysis and Design Dr. Truong Tuan Anh Faculty of Computer Science and Engineering Ho Chi Minh City University of Technology VNU- Ho Chi Minh City 1 References [1] Cormen, T. H., Leiserson,
More informationDESIGN AND ANALYSIS OF ALGORITHMS. Unit 1 Chapter 4 ITERATIVE ALGORITHM DESIGN ISSUES
DESIGN AND ANALYSIS OF ALGORITHMS Unit 1 Chapter 4 ITERATIVE ALGORITHM DESIGN ISSUES http://milanvachhani.blogspot.in USE OF LOOPS As we break down algorithm into sub-algorithms, sooner or later we shall
More informationAnalysis of Algorithms
ITP21 - Foundations of IT 1 Analysis of Algorithms Analysis of algorithms Analysis of algorithms is concerned with quantifying the efficiency of algorithms. The analysis may consider a variety of situations:
More informationAnalysis of Algorithms. CSE Data Structures April 10, 2002
Analysis of Algorithms CSE 373 - Data Structures April 10, 2002 Readings and References Reading Chapter 2, Data Structures and Algorithm Analysis in C, Weiss Other References 10-Apr-02 CSE 373 - Data Structures
More informationUnit 1 Chapter 4 ITERATIVE ALGORITHM DESIGN ISSUES
DESIGN AND ANALYSIS OF ALGORITHMS Unit 1 Chapter 4 ITERATIVE ALGORITHM DESIGN ISSUES http://milanvachhani.blogspot.in USE OF LOOPS As we break down algorithm into sub-algorithms, sooner or later we shall
More informationYou should know the first sum above. The rest will be given if you ever need them. However, you should remember that,, and.
Big-Oh Notation Formal Definitions A function is in (upper bound) iff there exist positive constants k and n 0 such that for all. A function is in (lower bound) iff there exist positive constants k and
More informationScientific Computing. Algorithm Analysis
ECE257 Numerical Methods and Scientific Computing Algorithm Analysis Today s s class: Introduction to algorithm analysis Growth of functions Introduction What is an algorithm? A sequence of computation
More informationData Structures and Algorithms. Part 2
1 Data Structures and Algorithms Part 2 Werner Nutt 2 Acknowledgments The course follows the book Introduction to Algorithms, by Cormen, Leiserson, Rivest and Stein, MIT Press [CLRST]. Many examples displayed
More informationAlgorithm Analysis. College of Computing & Information Technology King Abdulaziz University. CPCS-204 Data Structures I
Algorithm Analysis College of Computing & Information Technology King Abdulaziz University CPCS-204 Data Structures I Order Analysis Judging the Efficiency/Speed of an Algorithm Thus far, we ve looked
More informationAlgorithmic Analysis. Go go Big O(h)!
Algorithmic Analysis Go go Big O(h)! 1 Corresponding Book Sections Pearson: Chapter 6, Sections 1-3 Data Structures: 4.1-4.2.5 2 What is an Algorithm? Informally, any well defined computational procedure
More informationAlgorithms in Systems Engineering IE172. Midterm Review. Dr. Ted Ralphs
Algorithms in Systems Engineering IE172 Midterm Review Dr. Ted Ralphs IE172 Midterm Review 1 Textbook Sections Covered on Midterm Chapters 1-5 IE172 Review: Algorithms and Programming 2 Introduction to
More informationAlgorithm Analysis. CENG 707 Data Structures and Algorithms
Algorithm Analysis CENG 707 Data Structures and Algorithms 1 Algorithm An algorithm is a set of instructions to be followed to solve a problem. There can be more than one solution (more than one algorithm)
More informationIntroduction to the Analysis of Algorithms. Algorithm
Introduction to the Analysis of Algorithms Based on the notes from David Fernandez-Baca Bryn Mawr College CS206 Intro to Data Structures Algorithm An algorithm is a strategy (well-defined computational
More informationSorting. Bubble Sort. Selection Sort
Sorting In this class we will consider three sorting algorithms, that is, algorithms that will take as input an array of items, and then rearrange (sort) those items in increasing order within the array.
More informationMergeSort, Recurrences, Asymptotic Analysis Scribe: Michael P. Kim Date: September 28, 2016 Edited by Ofir Geri
CS161, Lecture 2 MergeSort, Recurrences, Asymptotic Analysis Scribe: Michael P. Kim Date: September 28, 2016 Edited by Ofir Geri 1 Introduction Today, we will introduce a fundamental algorithm design paradigm,
More informationCOE428 Lecture Notes Week 1 (Week of January 9, 2017)
COE428 Lecture Notes: Week 1 1 of 10 COE428 Lecture Notes Week 1 (Week of January 9, 2017) Table of Contents COE428 Lecture Notes Week 1 (Week of January 9, 2017)...1 Announcements...1 Topics...1 Informal
More informationAlgorithm efficiency can be measured in terms of: Time Space Other resources such as processors, network packets, etc.
Algorithms Analysis Algorithm efficiency can be measured in terms of: Time Space Other resources such as processors, network packets, etc. Algorithms analysis tends to focus on time: Techniques for measuring
More informationAgenda. The worst algorithm in the history of humanity. Asymptotic notations: Big-O, Big-Omega, Theta. An iterative solution
Agenda The worst algorithm in the history of humanity 1 Asymptotic notations: Big-O, Big-Omega, Theta An iterative solution A better iterative solution The repeated squaring trick Fibonacci sequence 2
More information10/5/2016. Comparing Algorithms. Analyzing Code ( worst case ) Example. Analyzing Code. Binary Search. Linear Search
10/5/2016 CSE373: Data Structures and Algorithms Asymptotic Analysis (Big O,, and ) Steve Tanimoto Autumn 2016 This lecture material represents the work of multiple instructors at the University of Washington.
More informationhttps://cs.uiowa.edu/resources/events Searching an array Let R(N) be the running time to search for an integer in an unsorted array. Can we find an f(n) such that R N O(f N )? Searching an array Let R(N)
More informationChapter 2: Complexity Analysis
Chapter 2: Complexity Analysis Objectives Looking ahead in this chapter, we ll consider: Computational and Asymptotic Complexity Big-O Notation Properties of the Big-O Notation Ω and Θ Notations Possible
More informationIT 4043 Data Structures and Algorithms
IT 4043 Data Structures and Algorithms Budditha Hettige Department of Computer Science 1 Syllabus Introduction to DSA Abstract Data Types Arrays List Operation Using Arrays Recursion Stacks Queues Link
More informationCS 506, Sect 002 Homework 5 Dr. David Nassimi Foundations of CS Due: Week 11, Mon. Apr. 7 Spring 2014
CS 506, Sect 002 Homework 5 Dr. David Nassimi Foundations of CS Due: Week 11, Mon. Apr. 7 Spring 2014 Study: Chapter 4 Analysis of Algorithms, Recursive Algorithms, and Recurrence Equations 1. Prove the
More informationCSCA48 Winter 2018 Week 10:Algorithm Analysis. Marzieh Ahmadzadeh, Nick Cheng University of Toronto Scarborough
CSCA48 Winter 2018 Week 10:Algorithm Analysis Marzieh Ahmadzadeh, Nick Cheng University of Toronto Scarborough Algorithm Definition: Solving a problem step-by-step in finite amount of time. Analysis: How
More informationCS 261 Data Structures. Big-Oh Analysis: A Review
CS 261 Data Structures Big-Oh Analysis: A Review Big-Oh: Purpose How can we characterize the runtime or space usage of an algorithm? We want a method that: doesn t depend upon hardware used (e.g., PC,
More informationLecture 5: Running Time Evaluation
Lecture 5: Running Time Evaluation Worst-case and average-case performance Georgy Gimel farb COMPSCI 220 Algorithms and Data Structures 1 / 13 1 Time complexity 2 Time growth 3 Worst-case 4 Average-case
More informationData Structures and Algorithms Chapter 2
1 Data Structures and Algorithms Chapter 2 Werner Nutt 2 Acknowledgments The course follows the book Introduction to Algorithms, by Cormen, Leiserson, Rivest and Stein, MIT Press [CLRST]. Many examples
More informationMergeSort, Recurrences, Asymptotic Analysis Scribe: Michael P. Kim Date: April 1, 2015
CS161, Lecture 2 MergeSort, Recurrences, Asymptotic Analysis Scribe: Michael P. Kim Date: April 1, 2015 1 Introduction Today, we will introduce a fundamental algorithm design paradigm, Divide-And-Conquer,
More informationAlgorithms A Look At Efficiency
Algorithms A Look At Efficiency 1B Big O Notation 15-121 Introduction to Data Structures, Carnegie Mellon University - CORTINA 1 Big O Instead of using the exact number of operations to express the complexity
More informationUnit #3: Recursion, Induction, and Loop Invariants
Unit #3: Recursion, Induction, and Loop Invariants CPSC 221: Basic Algorithms and Data Structures Jan Manuch 2017S1: May June 2017 Unit Outline Thinking Recursively Recursion Examples Analyzing Recursion:
More informationAlgorithms: Efficiency, Analysis, techniques for solving problems using computer approach or methodology but not programming
Chapter 1 Algorithms: Efficiency, Analysis, and dod Order Preface techniques for solving problems using computer approach or methodology but not programming language e.g., search Collie Collean in phone
More information1 Introduction. 2 InsertionSort. 2.1 Correctness of InsertionSort
CS 161, Lecture 2 MergeSort, Recurrences 101, and Asymptotic Analysis Scribes: Michael Kim (2015), Ofir Geri (2016), M. Wootters (2017) Date: September 27, 2017 Adapted From Virginia Williams lecture notes
More informationEECS 477: Introduction to algorithms. Lecture 6
EECS 477: Introduction to algorithms. Lecture 6 Prof. Igor Guskov guskov@eecs.umich.edu September 24, 2002 1 Lecture outline Finish up with asymptotic notation Asymptotic analysis of programs Analyzing
More informationSchool of Informatics, University of Edinburgh
CS1Bh Lecture Note 4 Dynamic Programming This lecture examines a problem solving technique known as dynamic programming. It is frequently used when a straightforward recursive solution to a problem has
More informationLecture 5 Sorting Arrays
Lecture 5 Sorting Arrays 15-122: Principles of Imperative Computation (Spring 2018) Frank Pfenning, Rob Simmons We begin this lecture by discussing how to compare running times of functions in an abstract,
More informationOrder Analysis of Algorithms. Sorting problem
Order Analysis of Algorithms Debdeep Mukhopadhyay IIT Kharagpur Sorting problem Input: A sequence of n numbers, a1,a2,,an Output: A permutation (reordering) (a1,a2,,an ) of the input sequence such that
More informationLecture Notes on Sorting
Lecture Notes on Sorting 15-122: Principles of Imperative Computation Frank Pfenning Lecture 4 September 2, 2010 1 Introduction Algorithms and data structures can be evaluated along a number of dimensions,
More informationChoice of C++ as Language
EECS 281: Data Structures and Algorithms Principles of Algorithm Analysis Choice of C++ as Language All algorithms implemented in this book are in C++, but principles are language independent That is,
More informationCS302 Topic: Algorithm Analysis. Thursday, Sept. 22, 2005
CS302 Topic: Algorithm Analysis Thursday, Sept. 22, 2005 Announcements Lab 3 (Stock Charts with graphical objects) is due this Friday, Sept. 23!! Lab 4 now available (Stock Reports); due Friday, Oct. 7
More informationAssignment 1 (concept): Solutions
CS10b Data Structures and Algorithms Due: Thursday, January 0th Assignment 1 (concept): Solutions Note, throughout Exercises 1 to 4, n denotes the input size of a problem. 1. (10%) Rank the following functions
More informationCSE 332 Spring 2013: Midterm Exam (closed book, closed notes, no calculators)
Name: Email address: Quiz Section: CSE 332 Spring 2013: Midterm Exam (closed book, closed notes, no calculators) Instructions: Read the directions for each question carefully before answering. We will
More informationMeasuring algorithm efficiency
CMPT 225 Measuring algorithm efficiency Timing Counting Cost functions Cases Best case Average case Worst case Searching Sorting O Notation O notation's mathematical basis O notation classes and notations
More informationComputer Science 210 Data Structures Siena College Fall Topic Notes: Complexity and Asymptotic Analysis
Computer Science 210 Data Structures Siena College Fall 2017 Topic Notes: Complexity and Asymptotic Analysis Consider the abstract data type, the Vector or ArrayList. This structure affords us the opportunity
More informationMidterm 1 Solutions. (i) False. One possible counterexample is the following: n n 3
CS 170 Efficient Algorithms & Intractable Problems, Spring 2006 Midterm 1 Solutions Note: These solutions are not necessarily model answers. Rather, they are designed to be tutorial in nature, and sometimes
More informationComplexity, Induction, and Recurrence Relations. CSE 373 Help Session 4/7/2016
Complexity, Induction, and Recurrence Relations CSE 373 Help Session 4/7/2016 Big-O Definition Definition: g(n) is in O( f(n) ) if there exist positive constants c and n0 such that g(n) c f(n) for all
More informationAnalysis of Algorithms - Introduction -
Analysis of Algorithms - Introduction - Andreas Ermedahl MRTC (Mälardalens Real-Time Research Center) andreas.ermedahl@mdh.se Autumn 004 Administrative stuff Course leader: Andreas Ermedahl Email: andreas.ermedahl@mdh.se
More informationLecture 3. Recurrences / Heapsort
Lecture 3. Recurrences / Heapsort T. H. Cormen, C. E. Leiserson and R. L. Rivest Introduction to Algorithms, 3rd Edition, MIT Press, 2009 Sungkyunkwan University Hyunseung Choo choo@skku.edu Copyright
More informationCSE373: Data Structures and Algorithms Lecture 4: Asymptotic Analysis. Aaron Bauer Winter 2014
CSE373: Data Structures and Algorithms Lecture 4: Asymptotic Analysis Aaron Bauer Winter 2014 Previously, on CSE 373 We want to analyze algorithms for efficiency (in time and space) And do so generally
More informationThe divide-and-conquer paradigm involves three steps at each level of the recursion: Divide the problem into a number of subproblems.
2.3 Designing algorithms There are many ways to design algorithms. Insertion sort uses an incremental approach: having sorted the subarray A[1 j - 1], we insert the single element A[j] into its proper
More informationCSC236 Week 5. Larry Zhang
CSC236 Week 5 Larry Zhang 1 Logistics Test 1 after lecture Location : IB110 (Last names A-S), IB 150 (Last names T-Z) Length of test: 50 minutes If you do really well... 2 Recap We learned two types of
More informationAlgorithm Complexity Analysis: Big-O Notation (Chapter 10.4) Dr. Yingwu Zhu
Algorithm Complexity Analysis: Big-O Notation (Chapter 10.4) Dr. Yingwu Zhu Measure Algorithm Efficiency Space utilization: amount of memory required Time efficiency: amount of time required to accomplish
More informationO(n): printing a list of n items to the screen, looking at each item once.
UNIT IV Sorting: O notation efficiency of sorting bubble sort quick sort selection sort heap sort insertion sort shell sort merge sort radix sort. O NOTATION BIG OH (O) NOTATION Big oh : the function f(n)=o(g(n))
More informationCSC148 Week 9. Larry Zhang
CSC148 Week 9 Larry Zhang 1 Announcement 2 More Announcements Assignment 2 out. You may work in groups of 1-3 students. Test 2 on next Friday (Mar 16). No lecture on that day. The test starts at 5:30 The
More informationSankalchand Patel College of Engineering - Visnagar Department of Computer Engineering and Information Technology. Assignment
Class: V - CE Sankalchand Patel College of Engineering - Visnagar Department of Computer Engineering and Information Technology Sub: Design and Analysis of Algorithms Analysis of Algorithm: Assignment
More informationAlgorithm Analysis. (Algorithm Analysis ) Data Structures and Programming Spring / 48
Algorithm Analysis (Algorithm Analysis ) Data Structures and Programming Spring 2018 1 / 48 What is an Algorithm? An algorithm is a clearly specified set of instructions to be followed to solve a problem
More informationRandomized Algorithms, Quicksort and Randomized Selection
CMPS 2200 Fall 2017 Randomized Algorithms, Quicksort and Randomized Selection Carola Wenk Slides by Carola Wenk and Charles Leiserson CMPS 2200 Intro. to Algorithms 1 Deterministic Algorithms Runtime for
More informationAnalysis of Algorithms & Big-O. CS16: Introduction to Algorithms & Data Structures Spring 2018
Analysis of Algorithms & Big-O CS16: Introduction to Algorithms & Data Structures Spring 2018 Outline Running time Big-O Big-Ω and Big-Θ Analyzing Seamcarve Dynamic programming Fibonacci sequence 2 Algorithms
More informationAdam Blank Lecture 2 Winter 2017 CSE 332. Data Structures and Parallelism
Adam Blank Lecture 2 Winter 2017 CSE 332 Data Structures and Parallelism CSE 332: Data Structures and Parallelism Algorithm Analysis 1 Outline 1 Comparing Algorithms 2 Asymptotic Analysis Comparing Programs
More informationModule 1: Asymptotic Time Complexity and Intro to Abstract Data Types
Module 1: Asymptotic Time Complexity and Intro to Abstract Data Types Dr. Natarajan Meghanathan Professor of Computer Science Jackson State University Jackson, MS 39217 E-mail: natarajan.meghanathan@jsums.edu
More informationDIVIDE & CONQUER. Problem of size n. Solution to sub problem 1
DIVIDE & CONQUER Definition: Divide & conquer is a general algorithm design strategy with a general plan as follows: 1. DIVIDE: A problem s instance is divided into several smaller instances of the same
More informationUNIT 1 BASICS OF AN ALGORITHM
UNIT 1 BASICS OF AN ALGORITHM Basics of an Algorithm Structure Page Nos. 1.0 Introduction 5 1.1 Objectives 6 1.2. Analysis and Complexity of Algorithms 6 1.3 Basic Technique for Design of Efficient Algorithms
More informationPseudo code of algorithms are to be read by.
Cs502 Quiz No1 Complete Solved File Pseudo code of algorithms are to be read by. People RAM Computer Compiler Approach of solving geometric problems by sweeping a line across the plane is called sweep.
More informationCSE 146. Asymptotic Analysis Interview Question of the Day Homework 1 & Project 1 Work Session
CSE 146 Asymptotic Analysis Interview Question of the Day Homework 1 & Project 1 Work Session Comparing Algorithms Rough Estimate Ignores Details Or really: independent of details What are some details
More informationWhat is an algorithm?
Reminders CS 142 Lecture 3 Analysis, ADTs & Objects Program 1 was assigned - Due on 1/27 by 11:55pm 2 Abstraction Measuring Algorithm Efficiency When you utilize the mylist.index(item) function you are
More informationFundamental mathematical techniques reviewed: Mathematical induction Recursion. Typically taught in courses such as Calculus and Discrete Mathematics.
Fundamental mathematical techniques reviewed: Mathematical induction Recursion Typically taught in courses such as Calculus and Discrete Mathematics. Techniques introduced: Divide-and-Conquer Algorithms
More information(Refer Slide Time: 1:27)
Data Structures and Algorithms Dr. Naveen Garg Department of Computer Science and Engineering Indian Institute of Technology, Delhi Lecture 1 Introduction to Data Structures and Algorithms Welcome to data
More informationTheory and Frontiers of Computer Science. Fall 2013 Carola Wenk
Theory and Frontiers of Computer Science Fall 2013 Carola Wenk We have seen so far Computer Architecture and Digital Logic (Von Neumann Architecture, binary numbers, circuits) Introduction to Python (if,
More informationSorting is a problem for which we can prove a non-trivial lower bound.
Sorting The sorting problem is defined as follows: Sorting: Given a list a with n elements possessing a total order, return a list with the same elements in non-decreasing order. Remember that total order
More informationUnit #2: Recursion, Induction, and Loop Invariants
Unit #2: Recursion, Induction, and Loop Invariants CPSC 221: Algorithms and Data Structures Will Evans 2012W1 Unit Outline Thinking Recursively Recursion Examples Analyzing Recursion: Induction and Recurrences
More informationSorting & Growth of Functions
Sorting & Growth of Functions CSci 588: Data Structures, Algorithms and Software Design Introduction to Algorithms, Cormen et al., Chapter 3 All material not from online sources or text copyright Travis
More informationANALYSIS OF ALGORITHMS
ANALYSIS OF ALGORITHMS Running Time Pseudo-Code Asymptotic Notation Asymptotic Analysis Mathematical facts T(n) n = 4 Input Algorithm Output 1 Average Case vs. Worst Case Running Time of an Algorithm An
More informationWeek - 03 Lecture - 18 Recursion. For the last lecture of this week, we will look at recursive functions. (Refer Slide Time: 00:05)
Programming, Data Structures and Algorithms in Python Prof. Madhavan Mukund Department of Computer Science and Engineering Indian Institute of Technology, Madras Week - 03 Lecture - 18 Recursion For the
More informationAnalysis of Algorithms Part I: Analyzing a pseudo-code
Analysis of Algorithms Part I: Analyzing a pseudo-code Introduction Pseudo-code representation of an algorithm Analyzing algorithms Measuring the running time and memory size of an algorithm Calculating
More informationIntroduction to Algorithms
Introduction to Algorithms An algorithm is any well-defined computational procedure that takes some value or set of values as input, and produces some value or set of values as output. 1 Why study algorithms?
More informationEECS 203 Spring 2016 Lecture 8 Page 1 of 6
EECS 203 Spring 2016 Lecture 8 Page 1 of 6 Algorithms (3.1-3.3) Algorithms are a huge topic. In CSE we have 2 theory classes purely dedicated to algorithms (EECS 477 and EECS 586) and a number of classes
More informationDATA STRUCTURES AND ALGORITHMS. Asymptotic analysis of the algorithms
DATA STRUCTURES AND ALGORITHMS Asymptotic analysis of the algorithms Summary of the previous lecture Algorithm definition Representation of the algorithms: Flowchart, Pseudocode Description Types of the
More informationData Structures Lecture 3 Order Notation and Recursion
Data Structures Lecture 3 Order Notation and Recursion 1 Overview The median grade.cpp program from Lecture 2 and background on constructing and using vectors. Algorithm analysis; order notation Recursion
More informationDesign and Analysis of Algorithms Prof. Madhavan Mukund Chennai Mathematical Institute. Module 02 Lecture - 45 Memoization
Design and Analysis of Algorithms Prof. Madhavan Mukund Chennai Mathematical Institute Module 02 Lecture - 45 Memoization Let us continue our discussion of inductive definitions. (Refer Slide Time: 00:05)
More informationQuiz 1 Solutions. (a) f(n) = n g(n) = log n Circle all that apply: f = O(g) f = Θ(g) f = Ω(g)
Introduction to Algorithms March 11, 2009 Massachusetts Institute of Technology 6.006 Spring 2009 Professors Sivan Toledo and Alan Edelman Quiz 1 Solutions Problem 1. Quiz 1 Solutions Asymptotic orders
More information6/12/2013. Introduction to Algorithms (2 nd edition) Overview. The Sorting Problem. Chapter 2: Getting Started. by Cormen, Leiserson, Rivest & Stein
Introduction to Algorithms (2 nd edition) by Cormen, Leiserson, Rivest & Stein Chapter 2: Getting Started (slides enhanced by N. Adlai A. DePano) Overview Aims to familiarize us with framework used throughout
More informationAlgorithm Analysis. Jordi Cortadella and Jordi Petit Department of Computer Science
Algorithm Analysis Jordi Cortadella and Jordi Petit Department of Computer Science What do we expect from an algorithm? Correct Easy to understand Easy to implement Efficient: Every algorithm requires
More informationNotes slides from before lecture. CSE 21, Winter 2017, Section A00. Lecture 4 Notes. Class URL:
Notes slides from before lecture CSE 21, Winter 2017, Section A00 Lecture 4 Notes Class URL: http://vlsicad.ucsd.edu/courses/cse21-w17/ Notes slides from before lecture Notes January 23 (1) HW2 due tomorrow
More informationThe Limits of Sorting Divide-and-Conquer Comparison Sorts II
The Limits of Sorting Divide-and-Conquer Comparison Sorts II CS 311 Data Structures and Algorithms Lecture Slides Monday, October 12, 2009 Glenn G. Chappell Department of Computer Science University of
More informationAnalyze the obvious algorithm, 5 points Here is the most obvious algorithm for this problem: (LastLargerElement[A[1..n]:
CSE 101 Homework 1 Background (Order and Recurrence Relations), correctness proofs, time analysis, and speeding up algorithms with restructuring, preprocessing and data structures. Due Thursday, April
More informationYour favorite blog : (popularly known as VIJAY JOTANI S BLOG..now in facebook.join ON FB VIJAY
Course Code : BCS-042 Course Title : Introduction to Algorithm Design Assignment Number : BCA(IV)-042/Assign/14-15 Maximum Marks : 80 Weightage : 25% Last Date of Submission : 15th October, 2014 (For July
More information