# DESIGN AND ANALYSIS OF ALGORITHMS. Unit 1 Chapter 4 ITERATIVE ALGORITHM DESIGN ISSUES

Save this PDF as:

Size: px
Start display at page:

Download "DESIGN AND ANALYSIS OF ALGORITHMS. Unit 1 Chapter 4 ITERATIVE ALGORITHM DESIGN ISSUES"

## Transcription

1 DESIGN AND ANALYSIS OF ALGORITHMS Unit 1 Chapter 4 ITERATIVE ALGORITHM DESIGN ISSUES

2 USE OF LOOPS As we break down algorithm into sub-algorithms, sooner or later we shall come across some iterative construct, that is, a loop, apart from conditional statements. A loop executing one more or one less time than the intended number is so common. To construct a loop we should be aware of three aspects: 1) The initial condition that needs to be true before the loop begins execution 2) The invariant relation that must hold before, during, and after each iteration of the loop 3) The condition under which the loop must terminate

3 The following design process is suggested to take care of these three aspects of algorithm development. 1) To Establish Initial Condition 2) To Find the Iterative Construct 3) Loop Termination

4 (1) TO ESTABLISH INITIAL CONDITION Set the loop control variables, to values which are appropriate for solving the smallest instance of the problem in question. For example, suppose we want to add elements of the given array using a loop. The loop variables are, i the loop control variable which is also used as the array index and s, the current value of the sum. The smallest problem in this case is the sum of 0 number of elements. In this case s must be 0. Thus the initial condition is: i = 0 s = 0

5 (2) TO FIND THE ITERATIVE CONSTRUCT Try to extend the smallest problem to the next smallest. In doing this, we will come to know what changes are to be done to the loop variables to achieve this change. For our example of summation of elements of an array, the next smallest is the case of n = 1, in which case s must be equal to a[i]. Thus the solution for n = 1 can be derived from the solution for n = 0 by first considering the solution for i = 1: i = 1 s = a[1] Then the same can be re-written as: i = i + 1 note: i on RHS is 0 s = s + a[i] i is now 1

6

7 (3) LOOP TERMINATION The simplest termination condition occurs when it is known in advance the number of times the loop is to be iterated. In that case we can use a FOR structured programming construct. For example, to execute a loop 20 times, we may write: for i = 1 to 20 do.. end for

8

9 EFFICIENCY OF ALGORITHMS The design and implementation of algorithms have a great influence on their efficiency. Every algorithm, when implemented must use some of the system resources to complete its task. The resources most relevant to the efficiency are the use of the central processor unit (CPU) time and internal memory (RAM). In the past, high cost of the computer resources was the driving force behind the desire to design algorithms that are economical in the use of CPU time and memory. As time passed, while on one hand, the cost of these resources has reduced and continues to do so, the requirement of more complex, and thereby time consuming algorithms has increased. Hence the need for designing efficient algorithms is present even today.

10 There is no standard method for designing efficient algorithms. Despite this, there are a few generalizations that can be made about the problem characteristics, while other characteristics would be specific to a particular problem. We shall discuss some means of improving the efficiency of an algorithm.

11 (1) REMOVING REDUNDANT COMPUTATIONS OUTSIDE LOOPS Most of the inefficiencies that creep into the implementation of algorithms are due to redundant computations or unnecessary storage. The effect of redundant computation is serious when it is embedded within a loop, which must be executed many a times. The most common mistake when using loops is to repeatedly re-calculate the part of an expression that remains constant throughout the execution phase of the entire loop. Let us take an example:

12 utmost care should be exercised in making sure that redundancies are removed in the innermost loops, as they are a major contributor to cost in terms of time.

13 (2) REFERENCING OF ARRAY ELEMENTS Generally, arrays are processed by iterative constructs. If care is not exercised while programming, redundant computations can creep into array processing. Consider, as an example, the following two versions of an algorithm in C, to find the maximum number and its position in an array of numbers:

14 Which of the two versions of the algorithm is preferable? Why?

15 Note that indexing of any kind, whether it be of simple data elements or of structured elements requires more time. Thus it can be said that the second version of the implementation is preferable because the condition test (that is, a[i] > max) which is the dominant operation is much more efficient to perform than the corresponding test in the version I of the program. By using the variable max, only one array reference is being made each time, whereas when using the variable p as index, two array references are required. References to array elements requires address arithmetic to arrive at the correct element and this requires time. Thus more time is spent in locating the specific value for comparison. Secondly, the second version of the program is better documented and the purpose of the code is much more explicit to the reader. This is because, by adding the variable called max we are providing a clue to the reader of the program about its operation.

16 (3) INEFFICIENCY DUE TO LATE TERMINATION Another possibility of inefficiency creeping into the implementation of an algorithm is, when considerably more tests are carried out, than are required to solve the problem at hand. example: Suppose we have to search an alphabetically ordered list of names for a particular name using linear search. An inefficient implementation for this case would be one where all names were examined even if a node in the list was reached where it could be definitely said that the name cannot occur later in the list. For example, suppose we are searching for Ketan, then, as soon as we reach a name that is alphabetically later than Ketan example, Lalit, we should not proceed further.

17 Let us take as an example, the Bubble Sort algorithm (sorting in ascending order):

18 BUBBLE SORT ALGORITHM

19

20 After the ith iteration all a[n - i n] are sorted and all a[1,..., n - i] are less than or equal to a[n - i + 1]. In addition, the inner loop causes a[1... n - i] elements to be sorted with i < n. For the inner loop, after the jth iteration the a[j + 1] element will be greater than or equal to all the elements in a[1... n]. The termination of the while loop is guaranteed since i increases by 1 with each iteration and a for loop must, by definition always terminate.

21 A weakness of this algorithm is that it relies more heavily on exchanges than most of the other sorting methods. Since exchanges are relatively time consuming, this particular characteristic makes the method costly for sorting a large random set. The importance of this method is that the algorithm is suitable for sorting a data set which has relatively a small percentage of its elements out of order, since it will require only a small number of comparisons and exchanges. An examination of the intermediate configuration for the sample data set suggests that the algorithm lacks balance and symmetry. While larger data elements make rapid progress towards the right end of the array, elements of small values only make a slow progress to the left end on the array.

22

23

24 (4) EARLY DETECTION OF DESIRED OUTPUT CONDITIONS It may sometimes happen that, due to the very nature of the input data a particular algorithm, for example, the Bubble Sort algorithm, establishes the desired output condition before the general condition for termination is met. For example, a bubble sort algorithm might receive a set of data that is already in sorted condition. When this is the case, it is obvious that the algorithm will have the data in sorted condition long before the general loop termination conditions are satisfied. It is therefore desirable to terminate the sorting as soon as it is established that the input data is already sorted. How do we determine during the course of the execution of the algorithm, whether the data is already sorted? To do this, all we need to do is to check whether there have been any exchanges in the current pass of the inner loop. If there have been no exchanges in the current pass, the data must be already sorted and the loop can be terminated early. See line no. 8 in the Bubble Sort Algorithm.

25 In general we must include additional tests or steps to determine or detect the conditions for early termination. These tests should be included only if they are inexpensive with regards to computer time as was the case in the Bubble sort algorithm. In general, it may be necessary to trade-off extra tests and storage versus CPU time to achieve early termination of algorithms.

26 ESTIMATING AND SPECIFYING EXECUTION TIMES To arrive at a quantitative measure of an algorithm's performance it is necessary to setup a computational model that reflects its behaviour under specified input conditions. This model must deal with the essence (the heart) of computation and at the same time be independent of any specific programming language. Thus we usually characterize an algorithm's performance in terms of the size of the problem being solved. It is obvious that more computing resources are required to solve larger problems in the same class of problems. An important question is "How does the "COST" of solving a problem vary as N increases. An intuitive response is The cost increases linearly with an increase in N.

27 Example, a problem with n = 200 takes twice as much time as a problem with n = 100 While this linear dependence on N is justifiable for some simple problems or algorithms, in general the relationship between complexity and the problem size follows a completely different pattern. At the lower end of the scale we have algorithms with logarithmic (that is, better) dependence on N, while on the other end of the scale we have algorithms with an exponential dependence on N. With an increase in N the relative difference in the cost of computation is enormous for these two extremes. Next Table illustrates the comparative computational cost for a range of N values.

28

29 ALGORITHM TO RAISE X TO SOME POWER N, CALLED POWER1 ()

30 Now suppose we are interested in finding the total running time for this algorithm. The time taken by each statement and the number of times it will be executed is shown by the side of each statement. Thus the total running time is: T(Power1, n) = t1 + (n - 1) * t2 + (n - 1) * t3 + t4 = (t1 + t4) + (n - 1) * (t2 + t3) Note that the first term (tl + t4) contributes only a constant amount of time, no matter whatever be the value of n. The behaviour of the algorithm as the problem size grows, that is, as n increases, is governed by the second term (t2 + t3). Also remember that we want to have a figure of merit that is independent of the software implementation or the specific hardware used. We thus disregard the multiplying factor (t2 + t3) and simply say that T(Power1, n) increases as n, that is, in direct proportion to n. So we refer to the rate of growth of the running time and not the actual running time of an algorithm.

31 Later, we shall define a notation to explicitly specify such growth in quantities, known as asymptotic running time (time complexity of algorithms). Since, we are interested only in this measure and not the actual running time, we can concentrate on the behaviour of the loops within the algorithm, as they, generally, are the major contributors to the execution time of a program and/or function. If it is a recursive algorithm, our emphasis will be on finding the number of times the algorithm will call itself

32 COMPUTATIONAL COST AS A FUNCTION OF PROBLEM SIZE FOR A RANGE OF COMPUTATIONAL COMPLEXITIES From previous table we observe that for problems that exhibit exponential behaviour, only limited input data size is allowable. Thus, we can solve only very small problems with an algorithm that exhibits exponential behaviour. Assume that a computer can perform one million operations per second. Then, an algorithm having exponential behaviour with a sufficiently large problem size N, will take extremely long time-even longer than the age of the Earth. On the other extreme, for an algorithm with logarithmic dependence on N, a problem with N = 10 4 would require only 13 steps which amounts to about 13 microseconds of computer time. These examples emphasize how important it is to have an understanding of the behaviour of the algorithms as a function of the problem size. Also by analyzing the algorithms, a theoretical model or the value of the inherent computational complexity can be gauged.

33 In analyzing an algorithm we must also look at the operations in the problem, to determine which is the dominant one. Sometimes, in a particular problem a specific arithmetic expression is evaluated a number of times or a number of comparisons and/ or exchanges are made and they increase non-linearly as N grows. For example, computations, exchanges, and comparisons characterize most sorting algorithms. In such algorithms, the number of comparisons dominate and therefore we use comparisons as the dominant operation in our computational model for sorting algorithms.

34 ORDER NOTATION The Order Notation is a standard notation developed to denote the computing time (or space) of algorithms and bounds on the upper limit or the lower limit on the computing time (or space) of algorithms. It refers collectively to the Big-Oh, the Theta, the Omega and the Small-Oh notations. An algorithm in which the dominant mechanism (one which is most executed) does not exceed c * n 2 times, where c is a constant and n is the problem size, is said to have an order n 2 complexity which is expressed as O(n 2 ).

35 BIG-OH NOTATION Formally, a function g(n) is O(f(n)) provided there exists a constant c for which the relationship g(n) <= c.f(n) holds for all values of n that are finite and positive. Thus we have a means of characterizing the asymptotic complexity of algorithms and hence of determining the size of problems that it can solve, using a conventional sequential computer. The order notation Big-Oh is defined as follows:

36

37

38 THETA NOTATION The Theta notation is defined as: Let g(n) be an asymptotically non-negative function on the set of natural numbers. In other words, function f(n) belongs to Θ(g(n)) if it can be sandwiched between c 1 g(n) and c 2 g(n) for some constants c l, c 2 for all n greater than some no. See Fig

39

40 PROPERTIES OF THETA Assume f(n) and g(n) are asymptotically positive. Then, If f(n) and g(n) are the running times of the two branches of an if statement, then the above result can be used to get a tight bound on the worst-case running time of the entire if statement, assuming that nothing is known about the condition test of if. Such a situation may arise, for example, if the condition test depends on an unknown input.

41 OMEGA NOTATION The Omega notation is defined as: Let g(n) be an asymptotically non-negative function on the set of natural numbers.

42

43

44

45 SMALL-OH NOTATION The upper bound provided by O may or may not be tight. We use Small-oh, o for an upper bound which is not tight. Let g(n) be an asymptotically non-negative function on the set of natural numbers. The implication of this definition is that f(n) becomes insignificant relative to g(n) as n approaches infinity. For example,

46 W NOTATION The lower bound provided by Ω may or may not be tight. We use ω for an lower bound which is not tight. Let g(n) be an asymptotically non-negative function on the set of natural numbers. The implication of this definition is that g(n) becomes insignificant relative to f(n) as n approaches infinity. For example,

47 EXAMPLE

48 MEASURING THE EXECUTION TIMES While working with the big-oh notation, we discarded the multiplicative constants. The functions f(n)=0.001n 2 and g(n)=1000n 2 are treated identically, even though g(n) is a million times larger than f(n) for all values of n. The rationale behind this attitude is illustrated by previous Table, which tabulates the growth rate of several functions arising in algorithm analysis, on problems of reasonable size, denoted by n. Specifically, assuming that each operation in previous Table takes 1 nano second, we can draw the following conclusions. 1) The algorithms of all complexity classes take roughly the same amount of time for n <= 10. 2) The algorithm whose running time is n! becomes useless well before n=20. 3) The algorithm whose running time is 2 n has a greater operating range than the one with running time of n!, but it becomes impractical for n > 40. 4) The algorithm whose running time is n 2 is perfectly reasonable up to about n = l00, but it quickly deteriorates with larger n. For n > it is likely to be unusable. 5) The algorithms whose complexity is n and n log n remain practically usable with the size of input up to ) All algorithms which exhibit log n behaviour are always usable.

49 This discussion shows that as far as selection of an algorithm is concerned, we can generally go by its complexity in terms of order notations, but to select between actual implementations, we should compare the actual execution times, or derive the multiplying constants, as suggested later in later chapter. For example, even though an algorithm with a complexity of n 3 will be considered to be slower compared to the one with a complexity of n 2, for n< the former will be faster, if the multiplying factor for the second algorithm is , that is, its complexity is n 2 Empirical execution times of the algorithms or parts thereof can be done by using various measuring devices-both hardware and software-that are available. Usually software solutions are used, though hardware solutions give better accuracy. These tools are generally called monitors or profiling software.

50 INTERATIVE AND RECURSIVE METHOD FOR X N Consider Algo. of power1 and power2, for calculating x n : Algo. Of power1 uses the definition of an integer power x n = x * x *... * x n times. Let us define f1(n) to be the number of multiplications required by Power1(n) to compute x n as a function of n. Thus f1(n) = n - 1 and these multiplications are always executed.

51 Second algorithm is a recursive algorithm and depends upon the fact that x n can be calculated by squaring x [n/2], and then multiplying the result by x if n was odd. If power1 was used to compute x [n/2], it will require only half the number of multiplications, then we require one more multiplication to square it. In fact this argument can be used recursively, to further divide [n/2] by two, and so on. See second Algorithm and the following trace of the working of power2(2, 5) is obtained:

52

53 How many multiplications are done? only 3, compared to the 4 that would have been required with power1(). Thus we define f2(n) to be the number of multiplications needed by power2(). If n is even on every recursive invocation of power2(), then the minimum of multiplications will be needed. On the other hand, if n is odd every time, then the maximum number of multiplications will be required. The first case occurs when n = 2 k, for some positive k

54

55 This discussion shows that the number of multiplications required log 2 n. We went through the detailed analysis of these two algorithms because what follows is important. The first is a simple straight forward algorithm, seemingly inefficient, the second is a complicated, but more efficient algorithm.

56 WHICH ONE SHOULD BE USED BY A USER? 1) If the user needs to compute the exponentials of small powers, that is, n is small, say 5 or less, then the number of multiplications saved by power2() compared to power1() will be quite small. Further, the programmer may require a significant amount of time to understand, implement, debug and test the more complicated power2(). As it is the programmer's (human) time that is valuable and not the computer's, use power1() in this case. 2) On the other hand, if exponentials to large integer powers are to be calculated, and there are many instances of these in a program, example, they are within a large loop, then power2() should be used. Suppose in a program X 256 is to be calculated times, then power1() will require multiplications, while power2() will require multiplications. That is where power2() will be needed to outperform power1().

57 CHANGES IN COMPLEXITY ACHIEVED BY A SMALL ADJUSTMENT IN THE ALGORITHM Check the example of bogo sort given in book, It shows how little change in algorithm increase the efficiency of program. In that example efficiency increase from O(n!) to O(n 2 )

58 THANK YOU Dr. Milan Vachhani Assistant Professor, Sunshine Group of Institutions, Rajkot

### Chapter 2: Complexity Analysis

Chapter 2: Complexity Analysis Objectives Looking ahead in this chapter, we ll consider: Computational and Asymptotic Complexity Big-O Notation Properties of the Big-O Notation Ω and Θ Notations Possible

### Introduction to Data Structure

Introduction to Data Structure CONTENTS 1.1 Basic Terminology 1. Elementary data structure organization 2. Classification of data structure 1.2 Operations on data structures 1.3 Different Approaches to

### Data Structure Lecture#5: Algorithm Analysis (Chapter 3) U Kang Seoul National University

Data Structure Lecture#5: Algorithm Analysis (Chapter 3) U Kang Seoul National University U Kang 1 In This Lecture Learn how to evaluate algorithm efficiency Learn the concept of average case, best case,

### Complexity of Algorithms. Andreas Klappenecker

Complexity of Algorithms Andreas Klappenecker Example Fibonacci The sequence of Fibonacci numbers is defined as 0, 1, 1, 2, 3, 5, 8, 13, 21, 34,... F n 1 + F n 2 if n>1 F n = 1 if n =1 0 if n =0 Fibonacci

### Complexity of Algorithms

CSCE 222 Discrete Structures for Computing Complexity of Algorithms Dr. Hyunyoung Lee Based on slides by Andreas Klappenecker 1 Overview Example - Fibonacci Motivating Asymptotic Run Time Analysis Asymptotic

### Algorithm Analysis. Gunnar Gotshalks. AlgAnalysis 1

Algorithm Analysis AlgAnalysis 1 How Fast is an Algorithm? 1 Measure the running time» Run the program for many data types > Use System.currentTimeMillis to record the time Worst Time Average Best» Usually

### Lecture 2: Algorithm Analysis

ECE4050/CSC5050 Algorithms and Data Structures Lecture 2: Algorithm Analysis 1 Mathematical Background Logarithms Summations Recursion Induction Proofs Recurrence Relations 2 2 Logarithm Definition: 3

### Data Structures Lecture 8

Fall 2017 Fang Yu Software Security Lab. Dept. Management Information Systems, National Chengchi University Data Structures Lecture 8 Recap What should you have learned? Basic java programming skills Object-oriented

### UNIT 1 ANALYSIS OF ALGORITHMS

UNIT 1 ANALYSIS OF ALGORITHMS Analysis of Algorithms Structure Page Nos. 1.0 Introduction 7 1.1 Objectives 7 1.2 Mathematical Background 8 1.3 Process of Analysis 12 1.4 Calculation of Storage Complexity

### Complexity Analysis of an Algorithm

Complexity Analysis of an Algorithm Algorithm Complexity-Why? Resources required for running that algorithm To estimate how long a program will run. To estimate the largest input that can reasonably be

### DATA STRUCTURES AND ALGORITHMS. Asymptotic analysis of the algorithms

DATA STRUCTURES AND ALGORITHMS Asymptotic analysis of the algorithms Summary of the previous lecture Algorithm definition Representation of the algorithms: Flowchart, Pseudocode Description Types of the

### Data Structures and Algorithms. Part 2

1 Data Structures and Algorithms Part 2 Werner Nutt 2 Acknowledgments The course follows the book Introduction to Algorithms, by Cormen, Leiserson, Rivest and Stein, MIT Press [CLRST]. Many examples displayed

### CSE373: Data Structures and Algorithms Lecture 4: Asymptotic Analysis. Aaron Bauer Winter 2014

CSE373: Data Structures and Algorithms Lecture 4: Asymptotic Analysis Aaron Bauer Winter 2014 Previously, on CSE 373 We want to analyze algorithms for efficiency (in time and space) And do so generally

### Introduction to Algorithms

Introduction to Algorithms An algorithm is any well-defined computational procedure that takes some value or set of values as input, and produces some value or set of values as output. 1 Why study algorithms?

### Algorithm. Lecture3: Algorithm Analysis. Empirical Analysis. Algorithm Performance

Algorithm (03F) Lecture3: Algorithm Analysis A step by step procedure to solve a problem Start from an initial state and input Proceed through a finite number of successive states Stop when reaching a

### Algorithm Efficiency & Sorting. Algorithm efficiency Big-O notation Searching algorithms Sorting algorithms

Algorithm Efficiency & Sorting Algorithm efficiency Big-O notation Searching algorithms Sorting algorithms Overview Writing programs to solve problem consists of a large number of decisions how to represent

### MergeSort, Recurrences, Asymptotic Analysis Scribe: Michael P. Kim Date: September 28, 2016 Edited by Ofir Geri

CS161, Lecture 2 MergeSort, Recurrences, Asymptotic Analysis Scribe: Michael P. Kim Date: September 28, 2016 Edited by Ofir Geri 1 Introduction Today, we will introduce a fundamental algorithm design paradigm,

### CS 261 Data Structures. Big-Oh Analysis: A Review

CS 261 Data Structures Big-Oh Analysis: A Review Big-Oh: Purpose How can we characterize the runtime or space usage of an algorithm? We want a method that: doesn t depend upon hardware used (e.g., PC,

### The Running Time of Programs

The Running Time of Programs The 90 10 Rule Many programs exhibit the property that most of their running time is spent in a small fraction of the source code. There is an informal rule that states 90%

### CS:3330 (22c:31) Algorithms

What s an Algorithm? CS:3330 (22c:31) Algorithms Introduction Computer Science is about problem solving using computers. Software is a solution to some problems. Algorithm is a design inside a software.

1 19 Sorting 2 19.1 Introduction (Cont.) Sorting data Place data in order Typically ascending or descending Based on one or more sort keys Algorithms Insertion sort Selection sort Merge sort More efficient,

### Another Sorting Algorithm

1 Another Sorting Algorithm What was the running time of insertion sort? Can we do better? 2 Designing Algorithms Many ways to design an algorithm: o Incremental: o Divide and Conquer: 3 Divide and Conquer

### MergeSort, Recurrences, Asymptotic Analysis Scribe: Michael P. Kim Date: April 1, 2015

CS161, Lecture 2 MergeSort, Recurrences, Asymptotic Analysis Scribe: Michael P. Kim Date: April 1, 2015 1 Introduction Today, we will introduce a fundamental algorithm design paradigm, Divide-And-Conquer,

### Run Times. Efficiency Issues. Run Times cont d. More on O( ) notation

Comp2711 S1 2006 Correctness Oheads 1 Efficiency Issues Comp2711 S1 2006 Correctness Oheads 2 Run Times An implementation may be correct with respect to the Specification Pre- and Post-condition, but nevertheless

### Algorithm efficiency can be measured in terms of: Time Space Other resources such as processors, network packets, etc.

Algorithms Analysis Algorithm efficiency can be measured in terms of: Time Space Other resources such as processors, network packets, etc. Algorithms analysis tends to focus on time: Techniques for measuring

### 1 Introduction. 2 InsertionSort. 2.1 Correctness of InsertionSort

CS 161, Lecture 2 MergeSort, Recurrences 101, and Asymptotic Analysis Scribes: Michael Kim (2015), Ofir Geri (2016), M. Wootters (2017) Date: September 27, 2017 Adapted From Virginia Williams lecture notes

### Algorithms in Systems Engineering IE172. Midterm Review. Dr. Ted Ralphs

Algorithms in Systems Engineering IE172 Midterm Review Dr. Ted Ralphs IE172 Midterm Review 1 Textbook Sections Covered on Midterm Chapters 1-5 IE172 Review: Algorithms and Programming 2 Introduction to

### Classic Data Structures Introduction UNIT I

ALGORITHM SPECIFICATION An algorithm is a finite set of instructions that, if followed, accomplishes a particular task. All algorithms must satisfy the following criteria: Input. An algorithm has zero

### asymptotic growth rate or order compare two functions, but ignore constant factors, small inputs

Big-Oh 1 asymptotic growth rate or order 2 compare two functions, but ignore constant factors, small inputs asymptotic growth rate or order 2 compare two functions, but ignore constant factors, small inputs

### Introduction to Computer Science

Introduction to Computer Science Program Analysis Ryan Stansifer Department of Computer Sciences Florida Institute of Technology Melbourne, Florida USA 32901 http://www.cs.fit.edu/ ryan/ 24 April 2017

### Machine-Independent Optimizations

Chapter 9 Machine-Independent Optimizations High-level language constructs can introduce substantial run-time overhead if we naively translate each construct independently into machine code. This chapter

### Sorting. Bubble Sort. Selection Sort

Sorting In this class we will consider three sorting algorithms, that is, algorithms that will take as input an array of items, and then rearrange (sort) those items in increasing order within the array.

### LECTURE 9 Data Structures: A systematic way of organizing and accessing data. --No single data structure works well for ALL purposes.

LECTURE 9 Data Structures: A systematic way of organizing and accessing data. --No single data structure works well for ALL purposes. Input Algorithm Output An algorithm is a step-by-step procedure for

### Assignment 1 (concept): Solutions

CS10b Data Structures and Algorithms Due: Thursday, January 0th Assignment 1 (concept): Solutions Note, throughout Exercises 1 to 4, n denotes the input size of a problem. 1. (10%) Rank the following functions

Java How to Program, 9/e Copyright 1992-2012 by Pearson Education, Inc. All Rights Reserved. Searching data involves determining whether a value (referred to as the search key) is present in the data

### Principles of Algorithm Analysis. Biostatistics 615/815

Principles of Algorithm Analysis Biostatistics 615/815 Lecture 3 Snapshot of Incoming Class 25 Programming Languages 20 15 10 5 0 R C/C++ MatLab SAS Java Other Can you describe the QuickSort Algorithm?

### Computer Science 210 Data Structures Siena College Fall Topic Notes: Complexity and Asymptotic Analysis

Computer Science 210 Data Structures Siena College Fall 2017 Topic Notes: Complexity and Asymptotic Analysis Consider the abstract data type, the Vector or ArrayList. This structure affords us the opportunity

### Data Structures Lecture 3 Order Notation and Recursion

Data Structures Lecture 3 Order Notation and Recursion 1 Overview The median grade.cpp program from Lecture 2 and background on constructing and using vectors. Algorithm analysis; order notation Recursion

### Chapter Fourteen Bonus Lessons: Algorithms and Efficiency

: Algorithms and Efficiency The following lessons take a deeper look at Chapter 14 topics regarding algorithms, efficiency, and Big O measurements. They can be completed by AP students after Chapter 14.

### Module 1: Asymptotic Time Complexity and Intro to Abstract Data Types

Module 1: Asymptotic Time Complexity and Intro to Abstract Data Types Dr. Natarajan Meghanathan Professor of Computer Science Jackson State University Jackson, MS 39217 E-mail: natarajan.meghanathan@jsums.edu

### Unit 1: Introduction to Algorithms

1 Unit 1: Introduction Unit 1: Introduction to Algorithms Steps involved in Problem Solving Problem solving by the computer involves the following steps 1. Problem definition 2. Analysis 3. Design 4. Coding

### EC121 Mathematical Techniques A Revision Notes

EC Mathematical Techniques A Revision Notes EC Mathematical Techniques A Revision Notes Mathematical Techniques A begins with two weeks of intensive revision of basic arithmetic and algebra, to the level

### Resources matter. Orders of Growth of Processes. R(n)= (n 2 ) Orders of growth of processes. Partial trace for (ifact 4) Partial trace for (fact 4)

Orders of Growth of Processes Today s topics Resources used by a program to solve a problem of size n Time Space Define order of growth Visualizing resources utilization using our model of evaluation Relating

### Why study algorithms? CS 561, Lecture 1. Today s Outline. Why study algorithms? (II)

Why study algorithms? CS 561, Lecture 1 Jared Saia University of New Mexico Seven years of College down the toilet - John Belushi in Animal House Q: Can I get a programming job without knowing something

### Position Sort. Anuj Kumar Developer PINGA Solution Pvt. Ltd. Noida, India ABSTRACT. Keywords 1. INTRODUCTION 2. METHODS AND MATERIALS

Position Sort International Journal of Computer Applications (0975 8887) Anuj Kumar Developer PINGA Solution Pvt. Ltd. Noida, India Mamta Former IT Faculty Ghaziabad, India ABSTRACT Computer science has

### Lecture 5 Sorting Arrays

Lecture 5 Sorting Arrays 15-122: Principles of Imperative Computation (Spring 2018) Frank Pfenning, Rob Simmons We begin this lecture by discussing how to compare running times of functions in an abstract,

### Algorithms and Data Structures

Algorithm Analysis Page 1 - Algorithm Analysis Dr. Fall 2008 Algorithm Analysis Page 2 Outline Textbook Overview Analysis of Algorithm Pseudo-Code and Primitive Operations Growth Rate and Big-Oh Notation

### Writing Parallel Programs; Cost Model.

CSE341T 08/30/2017 Lecture 2 Writing Parallel Programs; Cost Model. Due to physical and economical constraints, a typical machine we can buy now has 4 to 8 computing cores, and soon this number will be

### Introduction to Computer Science

Introduction to Computer Science Program Analysis Ryan Stansifer Department of Computer Sciences Florida Institute of Technology Melbourne, Florida USA 32901 http://www.cs.fit.edu/ ryan/ 8 December 2017

### Time and space behaviour. Chapter 20

Time and space behaviour Chapter 20 Algorithmic complexity How many steps does it take to execute an algorithm given an input of size n? Complexity of functions Consider: f n = 2 n 2 + 4 n + 13 This function

### 0.1 Welcome. 0.2 Insertion sort. Jessica Su (some portions copied from CLRS)

0.1 Welcome http://cs161.stanford.edu My contact info: Jessica Su, jtysu at stanford dot edu, office hours Monday 3-5 pm in Huang basement TA office hours: Monday, Tuesday, Wednesday 7-9 pm in Huang basement

### ANALYSIS OF ALGORITHMS

ANALYSIS OF ALGORITHMS Running Time Pseudo-Code Asymptotic Notation Asymptotic Analysis Mathematical facts T(n) n = 4 Input Algorithm Output 1 Average Case vs. Worst Case Running Time of an Algorithm An

### CSE 100 Advanced Data Structures

CSE 100 Advanced Data Structures Overview of course requirements Outline of CSE 100 topics Review of trees Helpful hints for team programming Information about computer accounts Page 1 of 25 CSE 100 web

### Recursion & Performance. Recursion. Recursion. Recursion. Where Recursion Shines. Breaking a Problem Down

Recursion & Performance Recursion Part 7 The best way to learn recursion is to, first, learn recursion! Recursion Recursion Recursion occurs when a function directly or indirectly calls itself This results

### A LISP programmer knows the value of everything, but the cost of nothing.

Chapter 6 Cost A LISP programmer knows the value of everything, but the cost of nothing. Alan Perlis The evaluation rules in Chapter 3 explain how to determine the value of every expression that can be

### Sorting: Given a list A with n elements possessing a total order, return a list with the same elements in non-decreasing order.

Sorting The sorting problem is defined as follows: Sorting: Given a list A with n elements possessing a total order, return a list with the same elements in non-decreasing order. Remember that total order

### INTRODUCTION. Analysis: Determination of time and space requirements of the algorithm

INTRODUCTION A. Preliminaries: Purpose: Learn the design and analysis of algorithms Definition of Algorithm: o A precise statement to solve a problem on a computer o A sequence of definite instructions

### Arrays. https://docs.oracle.com/javase/tutorial/java/nutsandbolts/arrays.html

1 Arrays Arrays in Java an array is a container object that holds a fixed number of values of a single type the length of an array is established when the array is created 2 https://docs.oracle.com/javase/tutorial/java/nutsandbolts/arrays.html

### Algorithm Complexity Analysis: Big-O Notation (Chapter 10.4) Dr. Yingwu Zhu

Algorithm Complexity Analysis: Big-O Notation (Chapter 10.4) Dr. Yingwu Zhu Measure Algorithm Efficiency Space utilization: amount of memory required Time efficiency: amount of time required to accomplish

### Math 501 Solutions to Homework Assignment 10

Department of Mathematics Discrete Mathematics Math 501 Solutions to Homework Assignment 10 Section 7.3 Additional Exercises 1. int CountValuesLessThanT(int a[], int n, int T) { int count = 0; for (int

### Table : IEEE Single Format ± a a 2 a 3 :::a 8 b b 2 b 3 :::b 23 If exponent bitstring a :::a 8 is Then numerical value represented is ( ) 2 = (

Floating Point Numbers in Java by Michael L. Overton Virtually all modern computers follow the IEEE 2 floating point standard in their representation of floating point numbers. The Java programming language

### Chapter 12: Indexing and Hashing

Chapter 12: Indexing and Hashing Basic Concepts Ordered Indices B+-Tree Index Files B-Tree Index Files Static Hashing Dynamic Hashing Comparison of Ordered Indexing and Hashing Index Definition in SQL

### ENGI 4892: Data Structures Assignment 2 SOLUTIONS

ENGI 4892: Data Structures Assignment 2 SOLUTIONS Due May 30 in class at 1:00 Total marks: 50 Notes: You will lose marks for giving only the final answer for any question on this assignment. Some steps

### Recitation 9. Prelim Review

Recitation 9 Prelim Review 1 Heaps 2 Review: Binary heap min heap 1 2 99 4 3 PriorityQueue Maintains max or min of collection (no duplicates) Follows heap order invariant at every level Always balanced!

### Algorithm Analysis. Big Oh

Algorithm Analysis with Big Oh Data Structures and Design with Java and JUnit Chapter 12 Rick Mercer Algorithm Analysis w Objectives Analyze the efficiency of algorithms Analyze a few classic algorithms

### Algorithms and complexity analysis

Chapter 2 Algorithms and complexity analysis 2.1 Relationship between data structures and algorithms The primary objective of programming is to efficiently process the input to generate the desired output.

### 19 Much that I bound, I could not free; Much that I freed returned to me. Lee Wilson Dodd

19 Much that I bound, I could not free; Much that I freed returned to me. Lee Wilson Dodd Will you walk a little faster? said a whiting to a snail, There s a porpoise close behind us, and he s treading

### Data Structures and Algorithms Key to Homework 1

Data Structures and Algorithms Key to Homework 1 January 31, 2005 15 Define an ADT for a set of integers (remember that a set may not contain duplicates) Your ADT should consist of the functions that can

### TREES AND ORDERS OF GROWTH 8

TREES AND ORDERS OF GROWTH 8 COMPUTER SCIENCE 61A October 23, 2014 1 Trees in OOP 1.1 Our Implementation Previously, we have seen trees defined as an abstract data type using lists. This time, we will

### 3.2 Recursions One-term recursion, searching for a first occurrence, two-term recursion. 2 sin.

Chapter 3 Sequences 3.1 Summation Nested loops, while-loops with compound termination criteria 3.2 Recursions One-term recursion, searching for a first occurrence, two-term recursion. In Chapter 2 we played

### Data Structures and Algorithms

Data Structures and Algorithms About the course (objectives, outline, recommended reading) Problem solving Notions of Algorithmics (growth of functions, efficiency, programming model, example analysis)

### Computational complexity

Computational complexity Heuristic Algorithms Giovanni Righini University of Milan Department of Computer Science (Crema) Definitions: problems and instances A problem is a general question expressed in

### CS/COE 1501

CS/COE 1501 www.cs.pitt.edu/~nlf4/cs1501/ Introduction Meta-notes These notes are intended for use by students in CS1501 at the University of Pittsburgh. They are provided free of charge and may not be

### Algorithm Design and Recursion. Search and Sort Algorithms

Algorithm Design and Recursion Search and Sort Algorithms Objectives To understand the basic techniques for analyzing the efficiency of algorithms. To know what searching is and understand the algorithms

### CSCI-1200 Data Structures Spring 2018 Lecture 7 Order Notation & Basic Recursion

CSCI-1200 Data Structures Spring 2018 Lecture 7 Order Notation & Basic Recursion Review from Lectures 5 & 6 Arrays and pointers, Pointer arithmetic and dereferencing, Types of memory ( automatic, static,

### Sorting is a problem for which we can prove a non-trivial lower bound.

Sorting The sorting problem is defined as follows: Sorting: Given a list a with n elements possessing a total order, return a list with the same elements in non-decreasing order. Remember that total order

### Algorithmic Analysis. Go go Big O(h)!

Algorithmic Analysis Go go Big O(h)! 1 Corresponding Book Sections Pearson: Chapter 6, Sections 1-3 Data Structures: 4.1-4.2.5 2 What is an Algorithm? Informally, any well defined computational procedure

### In this chapter you ll learn:

Much that I bound, I could not free; Much that I freed returned to me. Lee Wilson Dodd Will you walk a little faster? said a whiting to a snail, There s a porpoise close behind us, and he s treading on

### Problem with Scanning an Infix Expression

Operator Notation Consider the infix expression (X Y) + (W U), with parentheses added to make the evaluation order perfectly obvious. This is an arithmetic expression written in standard form, called infix

### Programming Basics and Practice GEDB029 Decision Making, Branching and Looping. Prof. Dr. Mannan Saeed Muhammad bit.ly/gedb029

Programming Basics and Practice GEDB029 Decision Making, Branching and Looping Prof. Dr. Mannan Saeed Muhammad bit.ly/gedb029 Decision Making and Branching C language possesses such decision-making capabilities

### UNIT 1 BASICS OF AN ALGORITHM

UNIT 1 BASICS OF AN ALGORITHM Basics of an Algorithm Structure Page Nos. 1.0 Introduction 5 1.1 Objectives 6 1.2. Analysis and Complexity of Algorithms 6 1.3 Basic Technique for Design of Efficient Algorithms

### 10/11/2013. Chapter 3. Objectives. Objectives (continued) Introduction. Attributes of Algorithms. Introduction. The Efficiency of Algorithms

Chapter 3 The Efficiency of Algorithms Objectives INVITATION TO Computer Science 1 After studying this chapter, students will be able to: Describe algorithm attributes and why they are important Explain

### EE 368. Week 6 (Notes)

EE 368 Week 6 (Notes) 1 Expression Trees Binary trees provide an efficient data structure for representing expressions with binary operators. Root contains the operator Left and right children contain

### Analysis of algorithms

Analysis of algorithms Time and space To analyze an algorithm means: developing a formula for predicting how fast an algorithm is, based on the size of the input (time complexity), and/or developing a

### University of Toronto Department of Electrical and Computer Engineering. Midterm Examination. ECE 345 Algorithms and Data Structures Fall 2010

University of Toronto Department of Electrical and Computer Engineering Midterm Examination ECE 345 Algorithms and Data Structures Fall 2010 Print your name and ID number neatly in the space provided below;

### 6/12/2013. Introduction to Algorithms (2 nd edition) Overview. The Sorting Problem. Chapter 2: Getting Started. by Cormen, Leiserson, Rivest & Stein

Introduction to Algorithms (2 nd edition) by Cormen, Leiserson, Rivest & Stein Chapter 2: Getting Started (slides enhanced by N. Adlai A. DePano) Overview Aims to familiarize us with framework used throughout

### Outline. runtime of programs algorithm efficiency Big-O notation List interface Array lists

Outline runtime of programs algorithm efficiency Big-O notation List interface Array lists Runtime of Programs compare the following two program fragments: int result = 1; int result = 1; for (int i=2;

### Chapter 5: Recursion

Chapter 5: Recursion Objectives Looking ahead in this chapter, we ll consider Recursive Definitions Function Calls and Recursive Implementation Anatomy of a Recursive Call Tail Recursion Nontail Recursion

### Introduction to Algorithms

Lecture 1 Introduction to Algorithms 1.1 Overview The purpose of this lecture is to give a brief overview of the topic of Algorithms and the kind of thinking it involves: why we focus on the subjects that

### Chapter 11: Indexing and Hashing

Chapter 11: Indexing and Hashing Basic Concepts Ordered Indices B + -Tree Index Files B-Tree Index Files Static Hashing Dynamic Hashing Comparison of Ordered Indexing and Hashing Index Definition in SQL

### Divisibility Rules and Their Explanations

Divisibility Rules and Their Explanations Increase Your Number Sense These divisibility rules apply to determining the divisibility of a positive integer (1, 2, 3, ) by another positive integer or 0 (although

### Two hours UNIVERSITY OF MANCHESTER SCHOOL OF COMPUTER SCIENCE. Date: Thursday 1st June 2017 Time: 14:00-16:00

COMP 26120 Two hours UNIVERSITY OF MANCHESTER SCHOOL OF COMPUTER SCIENCE Algorithms and Imperative Programming Date: Thursday 1st June 2017 Time: 14:00-16:00 Please answer THREE Questions from the FOUR

### CSCA48 Winter 2018 Week 10:Algorithm Analysis. Marzieh Ahmadzadeh, Nick Cheng University of Toronto Scarborough

CSCA48 Winter 2018 Week 10:Algorithm Analysis Marzieh Ahmadzadeh, Nick Cheng University of Toronto Scarborough Algorithm Definition: Solving a problem step-by-step in finite amount of time. Analysis: How

### This chapter covers recursive definition, including finding closed forms.

Chapter 12 Recursive Definition This chapter covers recursive definition, including finding closed forms. 12.1 Recursive definitions Thus far, we have defined objects of variable length using semi-formal

### CLASS: II YEAR / IV SEMESTER CSE CS 6402-DESIGN AND ANALYSIS OF ALGORITHM UNIT I INTRODUCTION

CLASS: II YEAR / IV SEMESTER CSE CS 6402-DESIGN AND ANALYSIS OF ALGORITHM UNIT I INTRODUCTION 1. What is performance measurement? 2. What is an algorithm? 3. How the algorithm is good? 4. What are the

### Chapter 3. Set Theory. 3.1 What is a Set?

Chapter 3 Set Theory 3.1 What is a Set? A set is a well-defined collection of objects called elements or members of the set. Here, well-defined means accurately and unambiguously stated or described. Any

### Counting. Chapter Basic Counting. The Sum Principle. We begin with an example that illustrates a fundamental principle.

Chapter 1 Counting 1.1 Basic Counting The Sum Principle We begin with an example that illustrates a fundamental principle. Exercise 1.1-1 The loop below is part of an implementation of selection sort,

### 1 Summation Formulae, Induction and Bounding. 2 How to compare functions: o, ω, O, Ω, Θ. 3 How to count the running time of algorithms

What We ve Learned IE170: Algorithms in Systems Engineering: Lecture 10 Jeff Linderoth Department of Industrial and Systems Engineering Lehigh University February 5, 2007 1 Summation Formulae, Induction