COMP6700/2140 Correctness and Efficiency
|
|
- Meredith Simmons
- 6 years ago
- Views:
Transcription
1 COMP6700/2140 Correctness and Efficiency Alexei B Khorev and Josh Milthorpe Research School of Computer Science, ANU May 2017 Alexei B Khorev and Josh Milthorpe (RSCS, ANU) COMP6700/2140 Correctness and Efficiency May / 17
2 Topics 1 Concept of an Algorithm 2 Correctness 3 Cost of Computation: Time, T Space (Memory, P) Energy 4 Efficiency and Complexity of Algorithms 5 O-expression Notations 6 Complexity Classes 7 Linear Search and Selection Sort 8 Performance Measurement 9 Bubble Sort and Gap Sort Alexei B Khorev and Josh Milthorpe (RSCS, ANU) COMP6700/2140 Correctness and Efficiency May / 17
3 Algorithms: Top-Down Design (Again) Algorithms occupy the intermediate position between a solution and a method, both in terms of generality and detail The origin mathematical (algebraic), the essence transformation of data (input output) in a finite number of steps Writing a computer code often involves solving algorithmic problems Some of these problems can be handled by the top-down design approach: 1 Clearly state the problem that you are trying to solve 2 Define the inputs required by the program and the outputs to be produced 3 Decompose the program into classes and their associated methods 4 Design the algorithm that you intend to implement for each method Repeat every step for devising the solution used at the higher level (step-wise refinement) 5 Turn the algorithm into pseudo-code (easily translatable into Java statements) 6 Test the resulting program The top-down design is often used together with the bottom-up design translating an existing solution (like a mathematical solution into the code and integration separate parts into one working code) The bottom-up design may also involve the data structure selection (or construction); this is called the data-driven design Alexei B Khorev and Josh Milthorpe (RSCS, ANU) COMP6700/2140 Correctness and Efficiency May / 17
4 Algorithms: generality A good algorithm always has a virtue of generality: it can be used on a family of data structures with specific characteristics reduced to minimum An example sorting an array of N floats: This problem is a particular case of a more general sorting problem: Given an array of N > 0 objects, return an array which contains these N objects in ascending (descending) order It is assumed that array elements can be compared, using unspecified criterion (in Java, it means that the object class implements the Comparable interface) When implemented with such generality, the algorithm is said to be generic Computer algorithms are characterised by: 1 a large number of repetitive operations: large size array, looping constructs, pipelines 2 a particular data model in case of sorting and searching ( data model here means type of operations which can be performed of the DS) 3 correctness and efficiency, where correctness means that a right result is obtained for all possible problem instances (size of the data structure, state of the input sorted, unsorted, random, nearly-sorted etc), while efficiency means how the time and computer resources needed to successfully carry out the algorithm application scale with the problem size N (number of elements in the DS) Alexei B Khorev and Josh Milthorpe (RSCS, ANU) COMP6700/2140 Correctness and Efficiency May / 17
5 Algorithms: Complexity To quantify the algorithm efficiency, one has to establish how T(N) the number atomic operations (they do not depend on N), needed to perform the algorithm scales with the size of the problem N The time of execution depends on hardware platform and state of the execution environment; it is not an invariant measure The computer science uses the big-o notations (pronounced big-omicron by eggheads, big-o by pub frequenters) to characterise the efficiency (complexity) of an algorithm It tells what is the leading term of the scaling dependency: if lim T(N) = const f (N), thent(n) = O (f (N)), N when N becomes large enough The dependence T(N) of an algorithm, viewed as a problem, is referred to as complexity viewed as a solution, is referred as efficiency Alexei B Khorev and Josh Milthorpe (RSCS, ANU) COMP6700/2140 Correctness and Efficiency May / 17
6 Complexity classes The function f(n) usually has one of the following types of behaviour: the power law: f (N) = N β, where β is a number equals to 2 for the Selection Sort, and to 2376 for smart matrix multiplication, etc Such polynomial problems are tractable the logarithm law: f (N) = (log N) β, where β is 1 for a binary search (in a sorted array, in a binary search tree) These are easy problems the exponential law: f (N) = 10 β N, where β is 1/3 for the factorisation problem Such problems with exponential behaviour are hard (intractable) Examples: the traveling salesman problem, set partitioning etc A combination (as factors) of the above, eg, f (N) = N log 2 N, for the QuickSort algorithm f (N) = N 3 (log N) k, for the quantum Shor algorithm Alexei B Khorev and Josh Milthorpe (RSCS, ANU) COMP6700/2140 Correctness and Efficiency May / 17
7 How complexity matters Logarithmic scale in the ordinate axis! Alexei B Khorev and Josh Milthorpe (RSCS, ANU) COMP6700/2140 Correctness and Efficiency May / 17
8 Ο, Ω and Θ Some rigour To proper understand the O notations one should remember that O(f(N)) denotes not a single function, but a class of functions with the designated asymptotic behaviour If we say that a function g(n) has an asymptotic of O(f(N)), g(n) = O(f(N)), it means not equality, but rather the inclusion of g(n) into the same class The g(n) = O(f(N)) has meaning of the upper-bound, ie that its magnitude is at most as big as const f(n) (when N is large) At most! but it could also be (substantially) smaller Hence, an algorithm should not be dismissed outright if it is characterised by the efficiency of, say, O(N 2 ) (like in sorting), just because its actual performance can be better Opposite to upper-bound is an lower-bound estimate, ie that the function magnitude is at least as large as const f(n) (N is large) This asymptotic behaviour (different set) is notated with another symbol, Omega: g(n) = Ω(f (N)) The Ω-properties of algorithms are harder to establish, but they are more valuable Even more valuable (and still harder to come by) are asymptotic with both lower- and upper bounds: that a function g(n) is such that for sufficiently large N, there exist two constants, C 1 and C 2 (C 1 > C 2 ), that C 2 f(n) < g(n) < C 2 F(N) Such asymptotic sets are notated with the symbol Θ : g(n) = Θ(f(N)) Alexei B Khorev and Josh Milthorpe (RSCS, ANU) COMP6700/2140 Correctness and Efficiency May / 17
9 Greek Asymptotes bounded from above g(n) = O(f(n)) bounded from below g(n) = Ω(f(n)) bounded in between g(n) = Θ(f(n)) One can proof that (something which is intuitively obvious): g(n) = O(f(n)) and g(n) = Ω(f(n)) g(n) = Θ(f(n)) The reason (not because programmers are stupid ) of why the O-notations are used predominantly (when Ω would be more accurate) is explained in a classic Donald Knuth s paper Big Omicron and Big Omega and Big Theta, in SIGACT NEWS, Apr-June 1967, pp18 24 Alexei B Khorev and Josh Milthorpe (RSCS, ANU) COMP6700/2140 Correctness and Efficiency May / 17
10 Linear Search Problem: Given unsorted array of N elements (double s), find the smallest element in a subarray Algorithm: (trivial, correctness follows from the facts that each element of the array will undergo a comparison with the tentative result and comparisons of integers and real numbers are transitive, ie from a> b and b > c follows a > c) 1 Assume tentatively that the first number in the array is the smallest element 2 Compare this number with every element in succession If one of these subsequent numbers is smaller, replace the tentative choice by this element 3 When all of the index locations have been exhausted, return the tentative result data an array of doubles value of smallest element */ public static double smallest(double[] data, int lo, int hi) { assert (hi > lo); int ismallest = lo; //Initial value for (int i=lo+1; i<=hi; i++) if (data[i] < data[ismallest]) ismallest = i; return data[ismallest]; } Complexity of this algorithm is O(N): we have to go from the first element to the very last one in search of the smallest (unsorted nature of the array is essential) Alexei B Khorev and Josh Milthorpe (RSCS, ANU) COMP6700/2140 Correctness and Efficiency May / 17
11 SelectionSort 1 In a loop which iterates a counter, i, over every index of an array in ascending order 2 At every step, do the following Compute the smallestinrange (the index of the smallest element) between the present array index, i, and the maximum array index Call this index ismallest Interchange the array elements at i and ismallest data an array of doubles */ public static void selectionsort(double[] data) for (int i=0; i<datalength-1; i++) { int ismallest = SelectionSortTest smallestinrange(data, i, datalength-1); // perform a swap if data[i] is smaller if (data[i] < data[ismallest]) { double temp = data[i]; data[i] = data[ismallest]; data[ismallest] = temp; } } } How array changes in-place step 1: step 2: step 3: step 4: Complexity number of iterations, ~N smallestinrange() is called on average N times at each step 2 which results into O(N 2 ) for an almost sorted array it is O(N) (as the majority of swaps are not performed) Alexei B Khorev and Josh Milthorpe (RSCS, ANU) COMP6700/2140 Correctness and Efficiency May / 17
12 Algorithm correctness Correctness and efficiency of algorithm implementations must be tested both theoretically (by examining the code), and experimentally (by running comprehensive test harnesses with all possible types of input data, and measuring the time-size dependence) The test harnesses (special testing programs not necessarily written in Java and input/output data) represent assets of the development as much as the code itself Correctness is the primary goal of testing, but the efficiency can also be an important (eg, to check a theoretical conclusion regarding the algorithm complexity) To check the correctness of our implementation of SelectionSort do the following (example SelectionSortTestjava): 1 Create an array and initialise it with N numbers (include cases 0 and 1!) 2 Print the initial array, or run a routine to check that if it s ordered 3 Run SelectionSort 4 Print the sorted array, or run a routine to check that if it s ordered 5 Repeat the above for many different sizes and initialisations: sorted arrays unsorted arrays almost sorted, reversed sorted arrays with randomly chosen elements boundary cases 0 or 1 element long arrays, very long arrays Alexei B Khorev and Josh Milthorpe (RSCS, ANU) COMP6700/2140 Correctness and Efficiency May / 17
13 Algorithm efficiency To check the efficiency one has to: 1 Work with large N (the upper limit is determined by the memory constraints) 2 Measure how the time taken by the sorting method scales with N 3 Consider randomised arrays (initialised by using a pseudo-random number generator) and limiting cases (fully and nearly sorted arrays, all and almost all equal elements etc) Often, the efficiency changes (deteriorates) in the limiting cases, eg, QuickSort (discussed later) for sorted array (N log 2 N N 2 ), which may force to reconsider the algorithm implementation Time measurements 1 OK: use the Unix system utility command time -p when running the Java program (the full path is needed for some shells): /usr/bin/time -p java SelectionSortTest Of three times printed, user time gives the time in seconds taken by the program including paging interrupts 2 Better: to call the javautildate class, and to read times in milliseconds The overhead here is the creation and call to Date objects Also, in a time-sharing system, there is no way to find the time for your particular process alone (In both pervious approaches, the overhead effect flattens out for large N) 3 Best: call the system clock via SystemcurrentTimeMillis(), or create a utility class similar to StopWatchjava to include such calls for calculating elapsed times Alexei B Khorev and Josh Milthorpe (RSCS, ANU) COMP6700/2140 Correctness and Efficiency May / 17
14 Example of Time Measurements A measurement example program (written by C Hortsmann years ago) SelectionSortWithTimingjava the original sorting program with timing based on SystemcurrentTimeMillis() calls ArrayUtiljava a utility class to create random array and also perform element s swapping StopWatchjava the helper class to set the time counting on/off To study the performance of a particular selection sort implementation, one should run this program multiple times, with the length of input arrays changing from 0, 1, few and progressively larger and ultimately very large (how large?), using different filling for each case: random (several, with different random seed), sorted (both descended and ascended), all equal, and almost sorted Why? Because, even the most accurate theoretical estimate will not tell you how different parts (search, writing temporary data and reading it etc) of execution will be carried out in a real CPU, its cache(s) and the main memory Alexei B Khorev and Josh Milthorpe (RSCS, ANU) COMP6700/2140 Correctness and Efficiency May / 17
15 Bubble Sort and Gap Sort These are two elegant algorithms for solving the sorting problem Neither is as naïvely intuitive as the selection sort or the insertion sort in the next section The gap sort is an amazingly fast variant of the bubble sort The essence of the BubbleSort algorithm: 1 Run an inner loop which repeatedly steps through every element in the array from left to right comparing adjacent elements, and if the left element is larger than the right element, then the elements are swapped The (new) right element then becomes the left element of the next comparison and so on 2 After one inner loop, the largest element is floated to the right hand side The order of the other elements may have changed 3 After the second inner loop, the second largest element has floated right to be the datalength-2 element The algorithm terminates when it makes one pass with no interchanges The gist of this (and other algorithms which try to improve the SelectionSort efficiency) is to minimise repetitive swaps Yet, BubbleSort is still O(N 2 ) (demonstrate BubbleSortTestjava) The analogous GapSort improves the efficiency, but the reason for this isn t well understood The algorithm achieves an intermediate performance between a standard O(N 2 ) and O(N log 2 N) (the actual asymptotic fluctuates) GapSort compares elements which are separated by a gap number of elements (rather than one element as in the bubble sort) The initial gap is taken pretty large approximately 3/4 of the size of the array It is reduced in size every outer iteration by a magic factor (13) Intuitively, the gapping is effective in reducing the number of unnecessary swaps (demo in GapSortTestjava) Alexei B Khorev and Josh Milthorpe (RSCS, ANU) COMP6700/2140 Correctness and Efficiency May / 17
16 Insertion Sort A simplified algorithm is the insertion sort (inspired by card playing): 1 In a loop from left to right (i = 0(length-1)), set a temporary variable tmp = data[i] 2 As we go along, the subarray on the left grows from 0 to lenght-1 and always remains sorted 3 Insert tmp in to the appropriate place k in the left subarray, k = 0(i - 1) in such a way that it remains sorted 4 Shift array items to the right when necessary The left subarray is enlarged by 1 The insertion can be done naively, or via binary search (because the array on left, in which the insertion is done, is sorted) The naïve version is demonstrated in InsertionSortTestjava The efficiency of InsertionSort is still O(N 2 ), for generic (random) inputs But for almost sorted inputs, it becomes almost O(N) This property of InsertionSort is valuable, since the most efficient famous QuickSort (see below) suffers deterioration in such cases By combining the two algorithms, an improved QuickSort can be implemented with much more robust performance An algorithm must be seen to be believed (D Knuth) demo SortingApplet A useful site with major sorting algorithms (animated like the above) including both time and space complexity analyses is Sorting Algorithm Animations Note on sorting an important version of sorting is a so called stable sort: when you must preserve the order of any pair of elements which are equal for the purpose of sorting Stable sorting is more complex and may have worser performance Alexei B Khorev and Josh Milthorpe (RSCS, ANU) COMP6700/2140 Correctness and Efficiency May / 17
17 Where to look for this topic in the textbook? Hortsmann s Core Java for the Impatient (not discussed) Oracle s Java Tutorial chapter Algorithms has only passing reference to O-expression notation (but does discuss sorting and searching algorithms somewhat) To study algorithms properly, one should take a whole course and/or use of those bodice-ripper books: Introduction to Algorithms, by Thomas H Cormen, Sharles E Leiserson, Ronald L Rivest and Clifford Stein, McGraw-Hill, 3d Ed, 2007 Fundamentals of Algorithms, by Gilles Brassard and Paul Bratley, Prentice-Hall, 1996 The Art of Computer Programming 3rd Edition, by Donald Knuth, Addison-Wesley, ; volumes: 1 Fundamental Algorithms 2 Seminumerical Algorithms 3 Sorting and Searching Alexei B Khorev and Josh Milthorpe (RSCS, ANU) COMP6700/2140 Correctness and Efficiency May / 17
COMP6700/2140 Recursive Operations
COMP6700/2140 Recursive Operations Alexei B Khorev and Josh Milthorpe Research School of Computer Science, ANU May 2017 Alexei B Khorev and Josh Milthorpe (RSCS, ANU) COMP6700/2140 Recursive Operations
COMP6700/2140 Sorting and Searching
COMP6700/2140 Sorting and Searching Alexei B Khorev and Josh Milthorpe Research School of Computer Science, ANU May 2017 Alexei B Khorev and Josh Milthorpe (RSCS, ANU) COMP6700/2140 Sorting and Searching
Using Templates to Introduce Time Efficiency Analysis in an Algorithms Course
Using Templates to Introduce Time Efficiency Analysis in an Algorithms Course Irena Pevac Department of Computer Science Central Connecticut State University, New Britain, CT, USA Abstract: We propose
Measuring algorithm efficiency
CMPT 225 Measuring algorithm efficiency Timing Counting Cost functions Cases Best case Average case Worst case Searching Sorting O Notation O notation's mathematical basis O notation classes and notations
Data Structures and Algorithms Chapter 2
1 Data Structures and Algorithms Chapter 2 Werner Nutt 2 Acknowledgments The course follows the book Introduction to Algorithms, by Cormen, Leiserson, Rivest and Stein, MIT Press [CLRST]. Many examples
Software Analysis. Asymptotic Performance Analysis
Software Analysis Performance Analysis Presenter: Jonathan Aldrich Performance Analysis Software Analysis 1 Asymptotic Performance Analysis How do we compare algorithm performance? Abstract away low-level
O(n): printing a list of n items to the screen, looking at each item once.
UNIT IV Sorting: O notation efficiency of sorting bubble sort quick sort selection sort heap sort insertion sort shell sort merge sort radix sort. O NOTATION BIG OH (O) NOTATION Big oh : the function f(n)=o(g(n))
PROGRAM EFFICIENCY & COMPLEXITY ANALYSIS
Lecture 03-04 PROGRAM EFFICIENCY & COMPLEXITY ANALYSIS By: Dr. Zahoor Jan 1 ALGORITHM DEFINITION A finite set of statements that guarantees an optimal solution in finite interval of time 2 GOOD ALGORITHMS?
International Journal of Scientific & Engineering Research, Volume 4, Issue 7, July ISSN
International Journal of Scientific & Engineering Research, Volume 4, Issue 7, July-201 971 Comparative Performance Analysis Of Sorting Algorithms Abhinav Yadav, Dr. Sanjeev Bansal Abstract Sorting Algorithms
Introduction to Computer Science
Introduction to Computer Science Program Analysis Ryan Stansifer Department of Computer Sciences Florida Institute of Technology Melbourne, Florida USA 32901 http://www.cs.fit.edu/ ryan/ 24 April 2017
Sorting. Order in the court! sorting 1
Sorting Order in the court! sorting 1 Importance of sorting Sorting a list of values is a fundamental task of computers - this task is one of the primary reasons why people use computers in the first place
Classic Data Structures Introduction UNIT I
ALGORITHM SPECIFICATION An algorithm is a finite set of instructions that, if followed, accomplishes a particular task. All algorithms must satisfy the following criteria: Input. An algorithm has zero
Position Sort. Anuj Kumar Developer PINGA Solution Pvt. Ltd. Noida, India ABSTRACT. Keywords 1. INTRODUCTION 2. METHODS AND MATERIALS
Position Sort International Journal of Computer Applications (0975 8887) Anuj Kumar Developer PINGA Solution Pvt. Ltd. Noida, India Mamta Former IT Faculty Ghaziabad, India ABSTRACT Computer science has
The p-sized partitioning algorithm for fast computation of factorials of numbers
J Supercomput (2006) 38:73 82 DOI 10.1007/s11227-006-7285-5 The p-sized partitioning algorithm for fast computation of factorials of numbers Ahmet Ugur Henry Thompson C Science + Business Media, LLC 2006
A Pragmatic Approach to Compare Various Sorting Techniques
International Journal of Computational Engineering & Management, Vol. 14, October 2011 72 A Pragmatic Approach to Compare Various Sorting Techniques Tanu Babbar 1 and Mandeep Juneja 2 1 Computer Department,
Sorting. Order in the court! sorting 1
Sorting Order in the court! sorting 1 Importance of sorting Sorting a list of values is a fundamental task of computers - this task is one of the primary reasons why people use computers in the first place
CS2 Algorithms and Data Structures Note 1
CS2 Algorithms and Data Structures Note 1 Analysing Algorithms This thread of the course is concerned with the design and analysis of good algorithms and data structures. Intuitively speaking, an algorithm
MergeSort, Recurrences, Asymptotic Analysis Scribe: Michael P. Kim Date: September 28, 2016 Edited by Ofir Geri
CS161, Lecture 2 MergeSort, Recurrences, Asymptotic Analysis Scribe: Michael P. Kim Date: September 28, 2016 Edited by Ofir Geri 1 Introduction Today, we will introduce a fundamental algorithm design paradigm,
COMP Data Structures
COMP 2140 - Data Structures Shahin Kamali Topic 5 - Sorting University of Manitoba Based on notes by S. Durocher. COMP 2140 - Data Structures 1 / 55 Overview Review: Insertion Sort Merge Sort Quicksort
Run Times. Efficiency Issues. Run Times cont d. More on O( ) notation
Comp2711 S1 2006 Correctness Oheads 1 Efficiency Issues Comp2711 S1 2006 Correctness Oheads 2 Run Times An implementation may be correct with respect to the Specification Pre- and Post-condition, but nevertheless
DESIGN AND ANALYSIS OF ALGORITHMS. Unit 1 Chapter 4 ITERATIVE ALGORITHM DESIGN ISSUES
DESIGN AND ANALYSIS OF ALGORITHMS Unit 1 Chapter 4 ITERATIVE ALGORITHM DESIGN ISSUES http://milanvachhani.blogspot.in USE OF LOOPS As we break down algorithm into sub-algorithms, sooner or later we shall
Analysis of Algorithms Part I: Analyzing a pseudo-code
Analysis of Algorithms Part I: Analyzing a pseudo-code Introduction Pseudo-code representation of an algorithm Analyzing algorithms Measuring the running time and memory size of an algorithm Calculating
COT 5407: Introduction to Algorithms. Giri Narasimhan. ECS 254A; Phone: x3748
COT 5407: Introduction to Algorithms Giri Narasimhan ECS 254A; Phone: x3748 giri@cis.fiu.edu http://www.cis.fiu.edu/~giri/teach/5407s17.html https://moodle.cis.fiu.edu/v3.1/course/view.php?id=1494 8/28/07
Plotting run-time graphically. Plotting run-time graphically. CS241 Algorithmics - week 1 review. Prefix Averages - Algorithm #1
CS241 - week 1 review Special classes of algorithms: logarithmic: O(log n) linear: O(n) quadratic: O(n 2 ) polynomial: O(n k ), k 1 exponential: O(a n ), a > 1 Classifying algorithms is generally done
Outline. Computer Science 331. Three Classical Algorithms. The Sorting Problem. Classical Sorting Algorithms. Mike Jacobson. Description Analysis
Outline Computer Science 331 Classical Sorting Algorithms Mike Jacobson Department of Computer Science University of Calgary Lecture #22 1 Introduction 2 3 4 5 Comparisons Mike Jacobson (University of
Chapter 2: Complexity Analysis
Chapter 2: Complexity Analysis Objectives Looking ahead in this chapter, we ll consider: Computational and Asymptotic Complexity Big-O Notation Properties of the Big-O Notation Ω and Θ Notations Possible
Unit 1 Chapter 4 ITERATIVE ALGORITHM DESIGN ISSUES
DESIGN AND ANALYSIS OF ALGORITHMS Unit 1 Chapter 4 ITERATIVE ALGORITHM DESIGN ISSUES http://milanvachhani.blogspot.in USE OF LOOPS As we break down algorithm into sub-algorithms, sooner or later we shall
Lecture Notes on Sorting
Lecture Notes on Sorting 15-122: Principles of Imperative Computation Frank Pfenning Lecture 4 September 2, 2010 1 Introduction Algorithms and data structures can be evaluated along a number of dimensions,
Algorithmic Analysis. Go go Big O(h)!
Algorithmic Analysis Go go Big O(h)! 1 Corresponding Book Sections Pearson: Chapter 6, Sections 1-3 Data Structures: 4.1-4.2.5 2 What is an Algorithm? Informally, any well defined computational procedure
CSE 146. Asymptotic Analysis Interview Question of the Day Homework 1 & Project 1 Work Session
CSE 146 Asymptotic Analysis Interview Question of the Day Homework 1 & Project 1 Work Session Comparing Algorithms Rough Estimate Ignores Details Or really: independent of details What are some details
MergeSort, Recurrences, Asymptotic Analysis Scribe: Michael P. Kim Date: April 1, 2015
CS161, Lecture 2 MergeSort, Recurrences, Asymptotic Analysis Scribe: Michael P. Kim Date: April 1, 2015 1 Introduction Today, we will introduce a fundamental algorithm design paradigm, Divide-And-Conquer,
COMP6700/2140 Abstract Data Types: Queue, Set, Map
COMP6700/2140 Abstract Data Types: Queue, Set, Map Alexei B Khorev and Josh Milthorpe Research School of Computer Science, ANU 19 April 2017 Alexei B Khorev and Josh Milthorpe (RSCS, ANU) COMP6700/2140
Homogeneous and Non Homogeneous Algorithms
Homogeneous and Non Homogeneous Algorithms Paparrizos K. Ioannis Department of Informatics Aristotle University of Thessaloniki 54124 Thessaloniki, Greece E-mail: ipapa@csd.auth.gr Abstract Motivated by
Data Structures and Algorithms. Part 2
1 Data Structures and Algorithms Part 2 Werner Nutt 2 Acknowledgments The course follows the book Introduction to Algorithms, by Cormen, Leiserson, Rivest and Stein, MIT Press [CLRST]. Many examples displayed
Class Note #02. [Overall Information] [During the Lecture]
Class Note #02 Date: 01/11/2006 [Overall Information] In this class, after a few additional announcements, we study the worst-case running time of Insertion Sort. The asymptotic notation (also called,
Introduction to Computer Science
Introduction to Computer Science Program Analysis Ryan Stansifer Department of Computer Sciences Florida Institute of Technology Melbourne, Florida USA 32901 http://www.cs.fit.edu/ ryan/ 8 December 2017
The growth of functions. (Chapter 3)
The growth of functions. (Chapter 3) Runtime Growth Rates (I) The runtimes of some (most?) algorithms are clean curves, though some do oscillate: It can be useful when describing algorithms (or even problems)
Lecture 5 Sorting Arrays
Lecture 5 Sorting Arrays 15-122: Principles of Imperative Computation (Spring 2018) Frank Pfenning, Rob Simmons We begin this lecture by discussing how to compare running times of functions in an abstract,
Analysis of Algorithms
ITP21 - Foundations of IT 1 Analysis of Algorithms Analysis of algorithms Analysis of algorithms is concerned with quantifying the efficiency of algorithms. The analysis may consider a variety of situations:
Algorithms: Efficiency, Analysis, techniques for solving problems using computer approach or methodology but not programming
Chapter 1 Algorithms: Efficiency, Analysis, and dod Order Preface techniques for solving problems using computer approach or methodology but not programming language e.g., search Collie Collean in phone
CSE 373 APRIL 3 RD ALGORITHM ANALYSIS
CSE 373 APRIL 3 RD ALGORITHM ANALYSIS ASSORTED MINUTIAE HW1P1 due tonight at midnight HW1P2 due Friday at midnight HW2 out tonight Second Java review session: Friday 10:30 ARC 147 TODAY S SCHEDULE Algorithm
Data Structures, Algorithms & Data Science Platforms
Indian Institute of Science Bangalore, India भ रत य व ज ञ न स स थ न ब गल र, भ रत Department of Computational and Data Sciences DS221 19 Sep 19 Oct, 2017 Data Structures, Algorithms & Data Science Platforms
CSE373: Data Structures and Algorithms Lecture 4: Asymptotic Analysis. Aaron Bauer Winter 2014
CSE373: Data Structures and Algorithms Lecture 4: Asymptotic Analysis Aaron Bauer Winter 2014 Previously, on CSE 373 We want to analyze algorithms for efficiency (in time and space) And do so generally
Objectives. Chapter 23 Sorting. Why study sorting? What data to sort? Insertion Sort. CS1: Java Programming Colorado State University
Chapter 3 Sorting Objectives To study and analyze time complexity of various sorting algorithms ( 3. 3.7). To design, implement, and analyze insertion sort ( 3.). To design, implement, and analyze bubble
Complexity of Algorithms. Andreas Klappenecker
Complexity of Algorithms Andreas Klappenecker Example Fibonacci The sequence of Fibonacci numbers is defined as 0, 1, 1, 2, 3, 5, 8, 13, 21, 34,... F n 1 + F n 2 if n>1 F n = 1 if n =1 0 if n =0 Fibonacci
CS583 Lecture 01. Jana Kosecka. some materials here are based on Profs. E. Demaine, D. Luebke A.Shehu, J-M. Lien and Prof. Wang s past lecture notes
CS583 Lecture 01 Jana Kosecka some materials here are based on Profs. E. Demaine, D. Luebke A.Shehu, J-M. Lien and Prof. Wang s past lecture notes Course Info course webpage: - from the syllabus on http://cs.gmu.edu/
Lecture 9: Sorting Algorithms
Lecture 9: Sorting Algorithms Bo Tang @ SUSTech, Spring 2018 Sorting problem Sorting Problem Input: an array A[1..n] with n integers Output: a sorted array A (in ascending order) Problem is: sort A[1..n]
Complexity of Algorithms
CSCE 222 Discrete Structures for Computing Complexity of Algorithms Dr. Hyunyoung Lee Based on slides by Andreas Klappenecker 1 Overview Example - Fibonacci Motivating Asymptotic Run Time Analysis Asymptotic
CS 137 Part 7. Big-Oh Notation, Linear Searching and Basic Sorting Algorithms. November 10th, 2017
CS 137 Part 7 Big-Oh Notation, Linear Searching and Basic Sorting Algorithms November 10th, 2017 Big-Oh Notation Up to this point, we ve been writing code without any consideration for optimization. There
Choice of C++ as Language
EECS 281: Data Structures and Algorithms Principles of Algorithm Analysis Choice of C++ as Language All algorithms implemented in this book are in C++, but principles are language independent That is,
ANALYSIS OF ALGORITHMS
ANALYSIS OF ALGORITHMS Running Time Pseudo-Code Asymptotic Notation Asymptotic Analysis Mathematical facts T(n) n = 4 Input Algorithm Output 1 Average Case vs. Worst Case Running Time of an Algorithm An
Lecture 5: Running Time Evaluation
Lecture 5: Running Time Evaluation Worst-case and average-case performance Georgy Gimel farb COMPSCI 220 Algorithms and Data Structures 1 / 13 1 Time complexity 2 Time growth 3 Worst-case 4 Average-case
Sorting. Task Description. Selection Sort. Should we worry about speed?
Sorting Should we worry about speed? Task Description We have an array of n values in any order We need to have the array sorted in ascending or descending order of values 2 Selection Sort Select the smallest
Algorithm. Algorithm Analysis. Algorithm. Algorithm. Analyzing Sorting Algorithms (Insertion Sort) Analyzing Algorithms 8/31/2017
8/3/07 Analysis Introduction to Analysis Model of Analysis Mathematical Preliminaries for Analysis Set Notation Asymptotic Analysis What is an algorithm? An algorithm is any well-defined computational
Data Structures and Algorithms
Data Structures and Algorithms About the course (objectives, outline, recommended reading) Problem solving Notions of Algorithmics (growth of functions, efficiency, programming model, example analysis)
Lecture 11: In-Place Sorting and Loop Invariants 10:00 AM, Feb 16, 2018
Integrated Introduction to Computer Science Fisler, Nelson Lecture 11: In-Place Sorting and Loop Invariants 10:00 AM, Feb 16, 2018 Contents 1 In-Place Sorting 1 2 Swapping Elements in an Array 1 3 Bubble
Introduction to Data Structure
Introduction to Data Structure CONTENTS 1.1 Basic Terminology 1. Elementary data structure organization 2. Classification of data structure 1.2 Operations on data structures 1.3 Different Approaches to
DATA STRUCTURES AND ALGORITHMS. Sorting algorithms (insertion sort, bubble sort, selection sort)
DATA STRUCTURES AND ALGORITHMS Sorting algorithms (insertion sort, bubble sort, selection sort) Summary of the previous lecture Recursion Definition Examples Factorial Fibonacci Hanoi tower Printing of
Outline and Reading. Analysis of Algorithms 1
Outline and Reading Algorithms Running time ( 3.1) Pseudo-code ( 3.2) Counting primitive operations ( 3.4) Asymptotic notation ( 3.4.1) Asymptotic analysis ( 3.4.2) Case study ( 3.4.3) Analysis of Algorithms
Algorithm efficiency can be measured in terms of: Time Space Other resources such as processors, network packets, etc.
Algorithms Analysis Algorithm efficiency can be measured in terms of: Time Space Other resources such as processors, network packets, etc. Algorithms analysis tends to focus on time: Techniques for measuring
12/1/2016. Sorting. Savitch Chapter 7.4. Why sort. Easier to search (binary search) Sorting used as a step in many algorithms
Sorting Savitch Chapter. Why sort Easier to search (binary search) Sorting used as a step in many algorithms Sorting algorithms There are many algorithms for sorting: Selection sort Insertion sort Bubble
The Limits of Sorting Divide-and-Conquer Comparison Sorts II
The Limits of Sorting Divide-and-Conquer Comparison Sorts II CS 311 Data Structures and Algorithms Lecture Slides Monday, October 12, 2009 Glenn G. Chappell Department of Computer Science University of
Algorithm Analysis. Performance Factors
Algorithm Analysis How can we demonstrate that one algorithm is superior to another without being misled by any of the following problems: Special cases Every algorithm has certain inputs that allow it
6/12/2013. Introduction to Algorithms (2 nd edition) Overview. The Sorting Problem. Chapter 2: Getting Started. by Cormen, Leiserson, Rivest & Stein
Introduction to Algorithms (2 nd edition) by Cormen, Leiserson, Rivest & Stein Chapter 2: Getting Started (slides enhanced by N. Adlai A. DePano) Overview Aims to familiarize us with framework used throughout
Algorithm Efficiency & Sorting. Algorithm efficiency Big-O notation Searching algorithms Sorting algorithms
Algorithm Efficiency & Sorting Algorithm efficiency Big-O notation Searching algorithms Sorting algorithms Overview Writing programs to solve problem consists of a large number of decisions how to represent
SAMPLE OF THE STUDY MATERIAL PART OF CHAPTER 6. Sorting Algorithms
SAMPLE OF THE STUDY MATERIAL PART OF CHAPTER 6 6.0 Introduction Sorting algorithms used in computer science are often classified by: Computational complexity (worst, average and best behavior) of element
UNIT 1 ANALYSIS OF ALGORITHMS
UNIT 1 ANALYSIS OF ALGORITHMS Analysis of Algorithms Structure Page Nos. 1.0 Introduction 7 1.1 Objectives 7 1.2 Mathematical Background 8 1.3 Process of Analysis 12 1.4 Calculation of Storage Complexity
Computer Science Approach to problem solving
Computer Science Approach to problem solving If my boss / supervisor / teacher formulates a problem to be solved urgently, can I write a program to efficiently solve this problem??? Polynomial-Time Brute
Chapter Contents. An Introduction to Sorting. Selection Sort. Selection Sort. Selection Sort. Iterative Selection Sort. Chapter 9
An Introduction to Sorting Chapter 9 Chapter Contents Iterative Recursive The Efficiency of Iterative Recursive The Efficiency of of a Chain of Linked Nodes The Java Code The Efficiency of Comparing the
An algorithm is a sequence of instructions that one must perform in order to solve a wellformulated
1 An algorithm is a sequence of instructions that one must perform in order to solve a wellformulated problem. input algorithm problem output Problem: Complexity Algorithm: Correctness Complexity 2 Algorithm
CS Sorting Terms & Definitions. Comparing Sorting Algorithms. Bubble Sort. Bubble Sort: Graphical Trace
CS 704 Introduction to Data Structures and Software Engineering Sorting Terms & Definitions Internal sorts holds all data in RAM External sorts use Files Ascending : Low to High Descending : High to Low
Same idea, different implementation. Is it important?
Analysis of algorithms Major field that provides tools for evaluating the efficiency of different solutions What is an efficient algorithm? Faster is better How do you measure time? Wall clock? Computer
Computer Science 4U Unit 1. Programming Concepts and Skills Algorithms
Computer Science 4U Unit 1 Programming Concepts and Skills Algorithms Algorithm In mathematics and computer science, an algorithm is a step-by-step procedure for calculations. Algorithms are used for calculation,
CS 157: Assignment 5
Problem : Printing Neatly CS 157: Assignment 5 Douglas R. Lanman 4 April 006 In a word processor or in L A TEX, one routinely encounters the pretty printing problem. That is, how does one transform text
Algorithm Analysis. (Algorithm Analysis ) Data Structures and Programming Spring / 48
Algorithm Analysis (Algorithm Analysis ) Data Structures and Programming Spring 2018 1 / 48 What is an Algorithm? An algorithm is a clearly specified set of instructions to be followed to solve a problem
CS126 Final Exam Review
CS126 Final Exam Review Fall 2007 1 Asymptotic Analysis (Big-O) Definition. f(n) is O(g(n)) if there exists constants c, n 0 > 0 such that f(n) c g(n) n n 0 We have not formed any theorems dealing with
STUDENT LESSON A17 Quadratic Sorting Algorithms
STUDENT LESSON A17 Quadratic Sorting Algorithms Java Curriculum for AP Computer Science, Student Lesson A17 1 STUDENT LESSON A17 Quadratic Sorting Algorithms INTRODUCTION: In this lesson, you will learn
D.Abhyankar 1, M.Ingle 2. -id: 2 M.Ingle, School of Computer Science, D.A. University, Indore M.P.
A Novel Mergesort D.Abhyankar 1, M.Ingle 2 1 D. Abhyankar, School of Computer Science, D.A. University, Indore M.P. India Email-id: deepak.scsit@gmail.com 2 M.Ingle, School of Computer Science, D.A. University,
Algorithms and Data Structures
Algorithm Analysis Page 1 - Algorithm Analysis Dr. Fall 2008 Algorithm Analysis Page 2 Outline Textbook Overview Analysis of Algorithm Pseudo-Code and Primitive Operations Growth Rate and Big-Oh Notation
Elementary maths for GMT. Algorithm analysis Part II
Elementary maths for GMT Algorithm analysis Part II Algorithms, Big-Oh and Big-Omega An algorithm has a O( ) and Ω( ) running time By default, we mean the worst case running time A worst case O running
Recursion. COMS W1007 Introduction to Computer Science. Christopher Conway 26 June 2003
Recursion COMS W1007 Introduction to Computer Science Christopher Conway 26 June 2003 The Fibonacci Sequence The Fibonacci numbers are: 1, 1, 2, 3, 5, 8, 13, 21, 34,... We can calculate the nth Fibonacci
Algorithm Analysis. Applied Algorithmics COMP526. Algorithm Analysis. Algorithm Analysis via experiments
Applied Algorithmics COMP526 Lecturer: Leszek Gąsieniec, 321 (Ashton Bldg), L.A.Gasieniec@liverpool.ac.uk Lectures: Mondays 4pm (BROD-107), and Tuesdays 3+4pm (BROD-305a) Office hours: TBA, 321 (Ashton)
We will use the following code as an example throughout the next two topics:
2.3 Asymptotic Analysis It has already been described qualitatively that if we intend to store nothing but objects, that this can be done quickly using a hash table; however, if we wish to store relationships
Data Structures and Algorithms (DSA) Course 13 Algorithms
Data Structures and Algorithms (DSA) Course 13 Algorithms Iulian Năstac Special binary trees (Recapitulation) P: A binary tree with the height i could have a maximum number of 2 i+1-1 nodes. Notes: A level
Algorithm Analysis. Gunnar Gotshalks. AlgAnalysis 1
Algorithm Analysis AlgAnalysis 1 How Fast is an Algorithm? 1 Measure the running time» Run the program for many data types > Use System.currentTimeMillis to record the time Worst Time Average Best» Usually
CITS3001. Algorithms, Agents and Artificial Intelligence. Semester 2, 2016
CITS3001 Algorithms, Agents and Artificial Intelligence Semester 2, 2016 Tim French School of Computer Science & Software Eng. The University of Western Australia 2. Review of algorithmic concepts CLRS,
Fundamental problem in computing science. putting a collection of items in order. Often used as part of another algorithm
cmpt-225 Sorting Sorting Fundamental problem in computing science putting a collection of items in order Often used as part of another algorithm e.g. sort a list, then do many binary searches e.g. looking
Algorithm Analysis. Algorithm Efficiency Best, Worst, Average Cases Asymptotic Analysis (Big Oh) Space Complexity
Algorithm Analysis Algorithm Efficiency Best, Worst, Average Cases Asymptotic Analysis (Big Oh) Space Complexity ITCS 2214:Data Structures 1 Mathematical Fundamentals Measuring Algorithm Efficiency Empirical
asymptotic growth rate or order compare two functions, but ignore constant factors, small inputs
Big-Oh 1 asymptotic growth rate or order 2 compare two functions, but ignore constant factors, small inputs asymptotic growth rate or order 2 compare two functions, but ignore constant factors, small inputs
10/5/2016. Comparing Algorithms. Analyzing Code ( worst case ) Example. Analyzing Code. Binary Search. Linear Search
10/5/2016 CSE373: Data Structures and Algorithms Asymptotic Analysis (Big O,, and ) Steve Tanimoto Autumn 2016 This lecture material represents the work of multiple instructors at the University of Washington.
Analysis of Algorithms. CS 1037a Topic 13
Analysis of Algorithms CS 1037a Topic 13 Overview Time complexity - exact count of operations T(n) as a function of input size n - complexity analysis using O(...) bounds - constant time, linear, logarithmic,
Analytical Modeling of Parallel Programs
Analytical Modeling of Parallel Programs Alexandre David Introduction to Parallel Computing 1 Topic overview Sources of overhead in parallel programs. Performance metrics for parallel systems. Effect of
CS240 Fall Mike Lam, Professor. Algorithm Analysis
CS240 Fall 2014 Mike Lam, Professor Algorithm Analysis Algorithm Analysis Motivation: what and why Mathematical functions Comparative & asymptotic analysis Big-O notation ("Big-Oh" in textbook) Analyzing
Analysis of Algorithm. Chapter 2
Analysis of Algorithm Chapter 2 Outline Efficiency of algorithm Apriori of analysis Asymptotic notation The complexity of algorithm using Big-O notation Polynomial vs Exponential algorithm Average, best
The divide and conquer strategy has three basic parts. For a given problem of size n,
1 Divide & Conquer One strategy for designing efficient algorithms is the divide and conquer approach, which is also called, more simply, a recursive approach. The analysis of recursive algorithms often
(Refer Slide Time: 1:27)
Data Structures and Algorithms Dr. Naveen Garg Department of Computer Science and Engineering Indian Institute of Technology, Delhi Lecture 1 Introduction to Data Structures and Algorithms Welcome to data
Elementary Sorting Algorithms
Elementary Sorting Algorithms COMP1927 16x1 Sedgewick Chapter 6 WARM UP EXERCISE: HANDSHAKE PROBLEM In a room of n people, how many different handshakes are possible? 0 + 1 + 2 + + (n-1) Using Maths formula:
CS 6402 DESIGN AND ANALYSIS OF ALGORITHMS QUESTION BANK
CS 6402 DESIGN AND ANALYSIS OF ALGORITHMS QUESTION BANK Page 1 UNIT I INTRODUCTION 2 marks 1. Why is the need of studying algorithms? From a practical standpoint, a standard set of algorithms from different
DATA STRUCTURES AND ALGORITHMS. Asymptotic analysis of the algorithms
DATA STRUCTURES AND ALGORITHMS Asymptotic analysis of the algorithms Summary of the previous lecture Algorithm definition Representation of the algorithms: Flowchart, Pseudocode Description Types of the
Java How to Program, 9/e. Copyright by Pearson Education, Inc. All Rights Reserved.
Java How to Program, 9/e Copyright 1992-2012 by Pearson Education, Inc. All Rights Reserved. Searching data involves determining whether a value (referred to as the search key) is present in the data