Fundamental mathematical techniques reviewed: Mathematical induction Recursion. Typically taught in courses such as Calculus and Discrete Mathematics.

Save this PDF as:

Size: px
Start display at page:

Download "Fundamental mathematical techniques reviewed: Mathematical induction Recursion. Typically taught in courses such as Calculus and Discrete Mathematics."

Transcription

1 Fundamental mathematical techniques reviewed: Mathematical induction Recursion Typically taught in courses such as Calculus and Discrete Mathematics. Techniques introduced: Divide-and-Conquer Algorithms Covered: Binary Search Merge Sort Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 1

2 Solutions From Simpler Cases Mathematical induction is a technique for proving statements about sets of consecutive integers. One can view this as being done by inducing our knowledge of the next case from that of its predecessor. Recursion is a technique where a solution to a problem consists of utilizing solutions to smaller versions of the problem. Divide-and-Conquer is a recursive technique: i. Divide a large problem into smaller subproblems. ii. iii. Solve the subproblems recursively, unless the problems are small enough to be solved directly. Combine the solutions to the subproblems in order to obtain a solution to the original problem. Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 2

3 Mathematical Induction Given a statement about positive integers, show that the statement is always true. Let P(n) be a predicate, a statement that is true or false, depending on its argument n. Assume n is a positive integer. Example of Predicate: The product of the positive integers from 1 to n is divisible by 10. This predicate is true for n = 5 since = 120, which is divisible by 10. This predicate is false for n = 4 since = 24, which is not divisible by 10. Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 3

4 Principle of Mathematical Induction Given a predicate that it is true for all positive integers n, the predicate can typically be proved as follows. Let P(n) be a predicate, where n is an arbitrary positive integer. Suppose the following can be accomplished. 1. Show that P(1) is true. 2. Show that whenever P(k) is true, we can derive that P(k + 1) is also true. If these two goals can be achieved, then it follows that P(n) is true for all positive integers n. Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 4

5 Why Does Mathematical Induction Work? Suppose the two steps on the previous slide have been proven. Then from step 1, P(1) is true, and thus by step 2, P = P(2) is true, and thus by step 2, P = P(3) is true, and thus by step 2, P = P(4) is true, and so forth. That is, step 2 allows for the induction of the truth of P(n) for every positive integer n from the truth of P(1). Base Case: The statement P 1 = true is referred to as the base case of the problem being considered. Inductive Hypothesis: The assumption in Step 2 that P k = true is called the Inductive Hypothesis. This is due to the fact that Step 2 is typically used to induce the conclusion that the statement P k + 1 = true. Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 5

6 Example: Prove That For All Positive Integers n, n i=1 i = n(n+1). 2 n First, it might be guessed that i n i=1. i=1 = n(n+1), as follows. 2 Let S = i Then S = n 1 + n. Write S in reverse order: S = n + (n 1) So, if these two equations are added by combining the first terms of the right sides, the second terms of the right sides, and so on, the result is 2S = n n n + 1 = n(n + 1), or S = n(n+1). Again, note that this exposition is not a proof, due to the imprecision of the notation. 2 Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 6

7 (continued) n Now, a formal proof is given of i=1 i = n(n+1). 2 The equation claims that the sum of the first n positive integers is n(n+1). 2 Base case: For n = 1, the left side of the asserted equation is 1 i = 1 and the right side of the asserted equation is i=1 1(1+1) 2 = 1. Thus, for n = 1, the asserted equation is true. Therefore, the base case of the induction proof is achieved. Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 7

8 (continued) Inductive case: Suppose the asserted equation is valid for n = k, for some positive integer k. Notice that it is justifiable to state this assumption due to the demonstration above that the case n = k = 1 is an instance for which the assumption is valid. Now, the asserted equation must be proven true for the next case, namely, n = k + 1. That is, by using the assumption for n = k, it must be proven that k+1 i=1 i. 2 Notice that the left side of the latter equation can be rewritten as k+1 k i=1 i = i=1 i + (k + 1). Substituting from the inductive hypothesis gives k+1 i=1 i = k(k+1) + (k + 1) = (k+1)(k+2), 2 2 as desired. Thus, the proof is complete. = (k+1)(k+2) Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 8

9 Recursion A subprogram that calls upon itself, either directly or indirectly, is called recursive. Formally, an algorithm exhibits recursive behavior when it can be defined by two properties. A simple base case or cases. A set of rules that reduce all other cases towards the base case. Recursive calls are made with a smaller/simpler set of data. When a call is made with a sufficiently small/simple enough set of data, the call is resolved directly. Notice the similarity of mathematical induction and recursion. Just as mathematical induction is a technique for inducing conclusions for large n from our knowledge of small n, recursion allows for the processing of large or complex data sets based on the ability to process smaller or less complex data sets. Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 13

10 Recursively Defined Factorial Function Definition of n!: Let n be a nonnegative integer. Then n! is defined as n! = 1 if n = 0; n n 1! otherwise. Theorem: For n > 0, n! is the product of the integers from 1 to n. So, n! can be computed using a tight loop. Note that the definition of n! is recursive and lends itself to a recursive calculation. Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 14

11 Example: Compute 3! The definition of n! is used to compute 3! as follows. From the recursive definition, 3! = 3 2!. Thus, the value of 2! needs to be determined. Using the second line of the recursive definition, 3! = 3 2! = 3 2 1! = !. Notice that the first line of the Definition of n Factorial yields 0! = 1. This is the simplest case of n considered by the definition of n!, a case that does not require further use of recursion and therefore is a base case. Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 15

12 Example: Compute 3! (continued) A recursive definition or algorithm may have more than one base case. It is the existence of one or more base cases, and logic that drives the computation toward base cases, that prevent recursion from producing an infinite loop. In the example, substitute 1 for 0! in order to resolve our calculations. If one were to proceed in the typical fashion of a person calculating with pencil and paper, this would yield 3! = ! = = 6. Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 16

13 Pseudocode for n! Integer function factorial (integer n) Input: n is assumed to be a nonnegative integer. Algorithm: Produce the value of n! by using recursion. Action: If n = 0, then return 1 Else return n factorial n How does one analyze the running time of this function? Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 17

14 Analysis of Factorial Function Let T(n) denote the running time of the procedure with input value n. From the base case of the recursion, T 0 = Θ(1), since the time to compute 0! is constant. From the recurrence given above, the time to compute n!, for n > 0, can be defined as T n = T n 1 + Θ(1). The conditions T 0 = Θ(1); T n = T n 1 + Θ 1 form a recursive relation. Now, evaluate T(n) in such a way as to express T(n) without recursion. (1) Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 18

15 Analysis of Factorial Function (continued) Repeated substitution of the recursive relation results in T n = T n 1 + Θ 1 = T n 2 + Θ 1 + Θ 1 = T n 2 + 2Θ 1 = T n 3 + Θ 1 + 2Θ 1 = T n 3 + 3Θ 1. The emerging pattern is T n = T n k + kθ 1. Such a pattern will lead us to conjecture that T n = T 0 + nθ 1, which, by the base case of the recursive definition, yields T n = Θ 1 + nθ 1 = Θ n. Note the above is not a proof. Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 19

16 Analysis of factorial function (continued) Observe that the Θ-notation in condition (1) is a generalization of proportionality. Suppose the simplified recursive relation is considered T 0 = 1; (2) T n = T n Previous observations leads one to suspect that this yields T n proof by mathematical induction follows. For n = 0, the assertion is T 0 = 1, which is true. = n + 1. A Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 20

17 Analysis of factorial function (continued) Suppose the assertion is true for some nonnegative integer k. Thus, the inductive hypothesis is the equation T k = k + 1. Now, one needs to show T k + 1 = k + 2. Using the recursive relation (2) and the inductive hypothesis yields T k + 1 = T k + 1 = k = k + 2, as desired. Since condition (1) is a generalization of (2), in which the Θ- interpretation is not affected by the differences between (1) and (2), it follows that condition (1) satisfies T n = Θ(n). Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 21

18 Computing Fibonacci Numbers RecursiveFibonacci(n) Input: a non-negative number n Output: the Fibonacci number with index n If n=0 then return 1 If n=1 then return 1 f =RecursiveFibonacci(n-1) + RecursiveFibonacci(n-2) Return(f) Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 22

19 Recursion Tree RF(5) RF(4) RF(3) RF(3) RF(2) RF(2) RF(1) RF(2) RF(1) RF(1) RF(0) RF(1) RF(0) RF(1) RF(0) Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 23

20 Sequential Search (Non-Recursive) The traditional non-recursive sequential search algorithm is presented so that it can be compared to the recursive implementation of binary search that is given later. Consider the problem of searching an unordered set of data by a traditional sequential search. In the worst case, every item must be examined, since the item being sought i) might not exist or ii) might be the last item listed. So, without loss of generality, it is assumed that the sequential search starts at the beginning of the unordered data set and concludes based on one of the following conditions. The search succeeds when the required item is located. The search fails after every item has been examined without finding the item being sought. Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 24

21 Sequential Search (continued) Since the data is not known to be ordered, the sequential examination of data items is necessary. Note that if any data item is skipped, that item could be the one being sought. Figure 2-1. An example of sequential search. Given the array of data, a search for the value 4 requires five key comparisons. A search for the value 9 requires three key comparisons. A search for the value 1 requires seven key comparisons in order to determine that the requested value is not present. Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 25

22 Subprogram SequentialSearch (X, searchvalue, success, foundat) (continued) Action: position = 1 Do success = (searchvalue = X[position]) If success, then foundat = position Else position = position + 1 While (Not success) and (position n ) {End Do} Return success, foundat End Search Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 27

23 Analysis of Sequential Search The set of instructions inside the loop runs in Θ 1 time since each instruction runs in Θ 1 time. In the worst case, the body of the loop will be executed n times. This occurs when either the search is unsuccessful or when the item we seek is the last item in the array X. Thus, the worst-case sequential search runs in Θ n time. Assuming that the data is ordered in a truly random fashion, then a successful search will, on average, succeed after examining half of the entries. So, the average successful search runs in Θ n time. Finally, since the data is presented in a random fashion, it is possible that the item being sought is found immediately. So, the time required for the best-case search is Θ 1. Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 28

24 Binary Search Recursion is commonly used when every recursive call involves a significant reduction in the size of the current instance of the problem. An example of such a recursive algorithm is Binary Search. Consider the impact of performing a search on a sorted set of data. Think about designing an algorithm that mimics what you would do to find Miller in a hardcopy phone book. That is, grab a bunch of pages and flip back and forth, each time grabbing fewer and fewer pages, until the desired item is located. Notice that this method considers very few data values relative to the number considered by a sequential search. Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 29

25 Subprogram BinarySearch(X, searchvalue, success, foundat, minindex, maxindex) Algorithm: Binary search algorithm to search ordered subarray for a key field equal to searchvalue. The algorithm is recursive. In order to search the entire array, assuming indices 1,, n, the initial call is of the form Search(X, searchvalue, success, foundat, 1, n). If searchvalue is found, return success = true and foundat as an index at which searchvalue is found; otherwise, return success = false. Local variable: index midindex Action: If minindex > maxindex, then {The subarray is empty} success = false, foundat = 0 Else {The subarray is nonempty} Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 30

26 BinarySearch Algorithm (continued) midindex = (minindex + maxindex) / 2 If searchvalue = X[midIndex].key, then success = true, foundat = midindex Else {searchvalue X[midIndex].key} If searchvalue < X[midIndex].key, then BinarySearch(X, searchvalue, success, foundat, minindex, midindex - 1) Else { searchvalue > X[midIndex].key} BinarySearch(X, searchvalue, success, foundat, midindex + 1, maxindex); End {Else searchvalue X[midIndex].key} End {Subarray is nonempty} End Search Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 31

27 Examples of Binary Search Figure 2-2. Given the array of data, a search for the value 4 requires two key comparisons (6,4). A search for the value 9 requires three key comparisons (6,8,9). A search for the value 1 requires three key comparisons (6,4,3) in order to determine that the value is not present. Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 32

28 Analysis of Binary Search Each recursive call processes an interval of about ½ the length of the interval processed by the previous call, so the running time, T(n), of the binary search algorithm satisfies the recursive relation T 1 = Θ(1); T n T n 2 + Θ(1). Seeking a pattern: Notice in the worst case, T n = T n 2 + Θ 1 = T n 4 + Θ 1 + Θ 1 = T n 4 + 2Θ 1 = T n 8 + Θ 1 + 2Θ 1 = T n 8 + 3Θ 1. It appears T n = T n/2 k + kθ 1, where the argument of T reaches the base value 1 = n/2 k when 2 k = n, or k = log 2 n. Such a pattern leads us to the conjecture that T n = T 1 + log 2 n Θ 1 = Θ log n. Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 33

29 Analysis of Binary Search (continued) The conjecture that in the worst case, T n = Θ log n, and thus that in general, T n = O log n, is established by showing, using mathematical induction, that the worst-case recursion derived from the previous discussion, T 1 = 1; resolves as T n T n = T n 2 + 1, = 1 + log 2 n. The proof is left as an exercise. Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 34

30 Comparison of Sequential and Binary Searches Sequential search Binary search Data Not ordered Ordered by search key Time for search that fails Θ(n) Θ(log n) Worst-case time for search that succeeds Expected-case time for search that succeeds Best-case time for search that succeeds Θ(n) Θ(n) Θ(1) Θ(log n) Θ(log n) Θ(1) Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 35

31 Recursive Sorting Algorithms Figure 2-3. Recursively sorting a set of data. Take the initial list and divide it into two lists, each roughly half the size of the original list. Recursively sort each of the sublists. Merge these sorted sublists to create the final sorted list. Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 36

32 Analyzing a Recursive Sorting Algorithm The recursive relation that describes the running time of such an algorithm is given by T 1 = Θ 1 ; T n = S n + 2T(n 2) + C n, where S n is the time used by the algorithm to split a list of n entries into two sublists of approximately n/2 entries apiece, and C n is the time used by the algorithm to combine two sorted lists of approximately n/2 entries apiece into a single sorted list. Discuss S n = Θ(n) since S = Split = Θ(n) and C = Merge = Θ(n) Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 37

33 Merge Sort: The Split Step Divide the input list X into 2 other lists Y and Z in a simple fashion. A commonly used method: mimic how you might partition a deck of cards into 2 piles of equal size. Elements of X alternate between going into Y and going into Z. E.g., a member whose initial rank in X is odd goes into Y and a member whose initial rank in X is even goes into Z. This is illustrated in Figure 2-3, above. Analysis: This step is essentially a scan of X, hence is performed in S n = Θ(n) time. Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 38

34 Merge Sort: The Combine Step Figure 2-5. An example of merging two ordered lists, initially indexed by head1 and head2, to create an ordered list headmerge. Snapshots are presented at various stages of the algorithm. As the merge progresses, head1 and head2 each indexes the first unmerged node in their respective list. Combine step: since the recursive calls have produced Y and Z as sorted lists, merge these sorted lists into a sorted list X. Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 39

35 Merging Two Lists Input: lists Y and Z with a total of n members, each in ascending order Output: list X, initially empty, is output with members that were the input members of Y and Z, in ascending order Action: While Y and Z are both non-empty, do the following. Compare the first members of Y and Z. Whichever has the smaller key value is removed from its list and placed at the end of X. Note one of Y and Z shrinks, and X grows. This is done in Θ(1) time. End While. The time required for this loop is bounded above by O(n) and below by the length of the shorter of the input lists. In a general merge, the best case runs in Θ(1) time. In Merge Sort, the input Y and Z have approximately n/2 members each. Thus, the loop runs in Θ(n) time. Now, one of Y and Z is empty, and the other isn t. Concatenate X and the non-empty one of Y and Z in Θ(1) time. Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 40

36 Merge Sort: Analysis The running time of Merge Sort satisfies T 1 = Θ 1 ; T n = S n + 2T(n 2) + C n, where the splitting time is S n = Θ(n) and the combine time is the time for a merge operation, C n = Θ(n). This yields T 1 = Θ 1 ; T n = 2T(n 2) + Θ n. The recursive relation is resolved as T 1 = 1; T n = 2T(n 2) + n, by substituting n = 2 k. This gives T 1 = 1; T 2 k = 2T 2 k k. } Θ(n log n) Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 41

37 Merge Sort Analysis (continued) Using mathematical induction on k yields T n = n 1 + log 2 n. The details of how this is done are left as an exercise. Hence, Merge Sort runs in Θ n log n time. Russ Miller & Laurence Boxer Algorithms Sequential & Parallel: A Unified Approach, 3E Chapter 2 / Slide 42

Recursion defining an object (or function, algorithm, etc.) in terms of itself. Recursion can be used to define sequences

Section 5.3 1 Recursion 2 Recursion Recursion defining an object (or function, algorithm, etc.) in terms of itself. Recursion can be used to define sequences Previously sequences were defined using a specific

Lecture 15 : Review DRAFT

CS/Math 240: Introduction to Discrete Mathematics 3/10/2011 Lecture 15 : Review Instructor: Dieter van Melkebeek Scribe: Dalibor Zelený DRAFT Today slectureservesasareviewofthematerialthatwillappearonyoursecondmidtermexam.

Lecture 6: Sequential Sorting

15-150 Lecture 6: Sequential Sorting Lecture by Dan Licata February 2, 2012 Today s lecture is about sorting. Along the way, we ll learn about divide and conquer algorithms, the tree method, and complete

CS Data Structures and Algorithm Analysis

CS 483 - Data Structures and Algorithm Analysis Lecture VI: Chapter 5, part 2; Chapter 6, part 1 R. Paul Wiegand George Mason University, Department of Computer Science March 8, 2006 Outline 1 Topological

Introduction to Algorithms / Algorithms I Lecturer: Michael Dinitz Topic: Shortest Paths Date: 10/13/15

600.363 Introduction to Algorithms / 600.463 Algorithms I Lecturer: Michael Dinitz Topic: Shortest Paths Date: 10/13/15 14.1 Introduction Today we re going to talk about algorithms for computing shortest

Scan and Quicksort. 1 Scan. 1.1 Contraction CSE341T 09/20/2017. Lecture 7

CSE341T 09/20/2017 Lecture 7 Scan and Quicksort 1 Scan Scan is a very useful primitive for parallel programming. We will use it all the time in this class. First, lets start by thinking about what other

Time and space behaviour. Chapter 20

Time and space behaviour Chapter 20 Algorithmic complexity How many steps does it take to execute an algorithm given an input of size n? Complexity of functions Consider: f n = 2 n 2 + 4 n + 13 This function

CSC 505, Spring 2005 Week 6 Lectures page 1 of 9

CSC 505, Spring 2005 Week 6 Lectures page 1 of 9 Objectives: learn general strategies for problems about order statistics learn how to find the median (or k-th largest) in linear average-case number of

n 1 x i = xn 1 x 1. i= (2n 1) = n 2.

Questions 1. Use mathematical induction to prove that, for any n 1 n 1 x i = xn 1 x 1. (Recall that I proved it in a different way back in lecture 2.) 2. Use mathematical induction to prove that, for all

Sorting. There exist sorting algorithms which have shown to be more efficient in practice.

Sorting Next to storing and retrieving data, sorting of data is one of the more common algorithmic tasks, with many different ways to perform it. Whenever we perform a web search and/or view statistics

This chapter covers recursive definition, including finding closed forms.

Chapter 12 Recursive Definition This chapter covers recursive definition, including finding closed forms. 12.1 Recursive definitions Thus far, we have defined objects of variable length using semi-formal

Problem Set 5 Solutions

Introduction to Algorithms November 4, 2005 Massachusetts Institute of Technology 6.046J/18.410J Professors Erik D. Demaine and Charles E. Leiserson Handout 21 Problem Set 5 Solutions Problem 5-1. Skip

Lecture 8: Mergesort / Quicksort Steven Skiena

Lecture 8: Mergesort / Quicksort Steven Skiena Department of Computer Science State University of New York Stony Brook, NY 11794 4400 http://www.cs.stonybrook.edu/ skiena Problem of the Day Give an efficient

Analysis of Algorithms - Quicksort -

Analysis of Algorithms - Quicksort - Andreas Ermedahl MRTC (Mälardalens Real-Time Reseach Center) andreas.ermedahl@mdh.se Autumn 2003 Quicksort Proposed by C.A.R. Hoare in 962. Divide- and- conquer algorithm

Announcements. CS243: Discrete Structures. Strong Induction and Recursively Defined Structures. Review. Example (review) Example (review), cont.

Announcements CS43: Discrete Structures Strong Induction and Recursively Defined Structures Işıl Dillig Homework 4 is due today Homework 5 is out today Covers induction (last lecture, this lecture, and

Maximal Monochromatic Geodesics in an Antipodal Coloring of Hypercube

Maximal Monochromatic Geodesics in an Antipodal Coloring of Hypercube Kavish Gandhi April 4, 2015 Abstract A geodesic in the hypercube is the shortest possible path between two vertices. Leader and Long

Introduction. e.g., the item could be an entire block of information about a student, while the search key might be only the student's name

Chapter 7 Sorting 2 Introduction sorting fundamental task in data management well-studied problem in computer science basic problem given an of items where each item contains a key, rearrange the items

Searching Algorithms/Time Analysis

Searching Algorithms/Time Analysis CSE21 Fall 2017, Day 8 Oct 16, 2017 https://sites.google.com/a/eng.ucsd.edu/cse21-fall-2017-miles-jones/ (MinSort) loop invariant induction Loop invariant: After the

Cache-Oblivious Traversals of an Array s Pairs

Cache-Oblivious Traversals of an Array s Pairs Tobias Johnson May 7, 2007 Abstract Cache-obliviousness is a concept first introduced by Frigo et al. in [1]. We follow their model and develop a cache-oblivious

Assignment 1. Stefano Guerra. October 11, The following observation follows directly from the definition of in order and pre order traversal:

Assignment 1 Stefano Guerra October 11, 2016 1 Problem 1 Describe a recursive algorithm to reconstruct an arbitrary binary tree, given its preorder and inorder node sequences as input. First, recall that

Scan and its Uses. 1 Scan. 1.1 Contraction CSE341T/CSE549T 09/17/2014. Lecture 8

CSE341T/CSE549T 09/17/2014 Lecture 8 Scan and its Uses 1 Scan Today, we start by learning a very useful primitive. First, lets start by thinking about what other primitives we have learned so far? The

The Running Time of Programs

The Running Time of Programs The 90 10 Rule Many programs exhibit the property that most of their running time is spent in a small fraction of the source code. There is an informal rule that states 90%

Lecture 5: Sorting Part A

Lecture 5: Sorting Part A Heapsort Running time O(n lg n), like merge sort Sorts in place (as insertion sort), only constant number of array elements are stored outside the input array at any time Combines

MergeSort, Recurrences, Asymptotic Analysis Scribe: Michael P. Kim Date: September 28, 2016 Edited by Ofir Geri

CS161, Lecture 2 MergeSort, Recurrences, Asymptotic Analysis Scribe: Michael P. Kim Date: September 28, 2016 Edited by Ofir Geri 1 Introduction Today, we will introduce a fundamental algorithm design paradigm,

Overview. A mathema5cal proof technique Proves statements about natural numbers 0,1,2,... (or more generally, induc+vely defined objects) Merge Sort

Goals for Today Induc+on Lecture 22 Spring 2011 Be able to state the principle of induc+on Iden+fy its rela+onship to recursion State how it is different from recursion Be able to understand induc+ve proofs

Principles of Algorithm Design

Principles of Algorithm Design When you are trying to design an algorithm or a data structure, it s often hard to see how to accomplish the task. The following techniques can often be useful: 1. Experiment

Sorting. Sorting in Arrays. SelectionSort. SelectionSort. Binary search works great, but how do we create a sorted array in the first place?

Sorting Binary search works great, but how do we create a sorted array in the first place? Sorting in Arrays Sorting algorithms: Selection sort: O(n 2 ) time Merge sort: O(nlog 2 (n)) time Quicksort: O(n

UNIT 1 BASICS OF AN ALGORITHM

UNIT 1 BASICS OF AN ALGORITHM Basics of an Algorithm Structure Page Nos. 1.0 Introduction 5 1.1 Objectives 6 1.2. Analysis and Complexity of Algorithms 6 1.3 Basic Technique for Design of Efficient Algorithms

Recursion Chapter 3.5

Recursion Chapter 3.5-1 - Outline Induction Linear recursion Example 1: Factorials Example 2: Powers Example 3: Reversing an array Binary recursion Example 1: The Fibonacci sequence Example 2: The Tower

Divide-and-Conquer. Dr. Yingwu Zhu

Divide-and-Conquer Dr. Yingwu Zhu Divide-and-Conquer The most-well known algorithm design technique: 1. Divide instance of problem into two or more smaller instances 2. Solve smaller instances independently

CS:3330 (22c:31) Algorithms

What s an Algorithm? CS:3330 (22c:31) Algorithms Introduction Computer Science is about problem solving using computers. Software is a solution to some problems. Algorithm is a design inside a software.

Figure 4.1: The evolution of a rooted tree.

106 CHAPTER 4. INDUCTION, RECURSION AND RECURRENCES 4.6 Rooted Trees 4.6.1 The idea of a rooted tree We talked about how a tree diagram helps us visualize merge sort or other divide and conquer algorithms.

6/12/2013. Introduction to Algorithms (2 nd edition) Overview. The Sorting Problem. Chapter 2: Getting Started. by Cormen, Leiserson, Rivest & Stein

Introduction to Algorithms (2 nd edition) by Cormen, Leiserson, Rivest & Stein Chapter 2: Getting Started (slides enhanced by N. Adlai A. DePano) Overview Aims to familiarize us with framework used throughout

Searching a Sorted Set of Strings

Department of Mathematics and Computer Science January 24, 2017 University of Southern Denmark RF Searching a Sorted Set of Strings Assume we have a set of n strings in RAM, and know their sorted order

TECH. Recurrence Equations vs. Recursive Procedures. Recursion and Induction. e.g. Fibonacci Function. The working of recursive procedure

Recursion and Induction For advanced algorithm development, recursion is an essential design technique Recursive Procedures What is a Proof? Induction Proofs Proving Correctness of Procedures Recurrence

Chapter 12 Supplement: Recursion with Java 1.5. Mr. Dave Clausen La Cañada High School

Chapter 12 Supplement: Recursion with Java 1.5 La Cañada High School Recursion: Definitions Recursion The process of a subprogram (method) calling itself. A clearly defined stopping state must exist. The

Algorithms for Data Science

Algorithms for Data Science CSOR W4246 Eleni Drinea Computer Science Department Columbia University Thursday, October 1, 2015 Outline 1 Recap 2 Shortest paths in graphs with non-negative edge weights (Dijkstra

Solutions to relevant spring 2000 exam problems

Problem 2, exam Here s Prim s algorithm, modified slightly to use C syntax. MSTPrim (G, w, r): Q = V[G]; for (each u Q) { key[u] = ; key[r] = 0; π[r] = 0; while (Q not empty) { u = ExtractMin (Q); for

1 Introduction. 2 InsertionSort. 2.1 Correctness of InsertionSort

CS 161, Lecture 2 MergeSort, Recurrences 101, and Asymptotic Analysis Scribes: Michael Kim (2015), Ofir Geri (2016), M. Wootters (2017) Date: September 27, 2017 Adapted From Virginia Williams lecture notes

¼ À ÈÌ Ê ÈÊÇ Ä ÅË ½µ º½¹ ¾µ º¾¹ µ º ¹½ µ º ¹ µ º ¹ µ º ¹¾ µ ¹ µ ¹ ½ ECE608 - Chapter 8 answers (1) CLR 8.1-3 If the sort runs in linear time for m input permutations, then the height h of those paths of

Parallel Sorting Algorithms

CSC 391/691: GPU Programming Fall 015 Parallel Sorting Algorithms Copyright 015 Samuel S. Cho Sorting Algorithms Review Bubble Sort: O(n ) Insertion Sort: O(n ) Quick Sort: O(n log n) Heap Sort: O(n log

Data Structures and Algorithms. Part 2

1 Data Structures and Algorithms Part 2 Werner Nutt 2 Acknowledgments The course follows the book Introduction to Algorithms, by Cormen, Leiserson, Rivest and Stein, MIT Press [CLRST]. Many examples displayed

Algorithm Design and Recursion. Search and Sort Algorithms

Algorithm Design and Recursion Search and Sort Algorithms Objectives To understand the basic techniques for analyzing the efficiency of algorithms. To know what searching is and understand the algorithms

Cuckoo Hashing for Undergraduates Rasmus Pagh IT University of Copenhagen March 27, 2006 Abstract This lecture note presents and analyses two simple hashing algorithms: Hashing with Chaining, and Cuckoo

An Optimal Parallel Algorithm for Merging using Multiselection

An Optimal Parallel Algorithm for Merging using Multiselection Narsingh Deo Amit Jain Muralidhar Medidi Department of Computer Science, University of Central Florida, Orlando, FL 32816 Keywords: selection,

Algorithm Efficiency & Sorting. Algorithm efficiency Big-O notation Searching algorithms Sorting algorithms

Algorithm Efficiency & Sorting Algorithm efficiency Big-O notation Searching algorithms Sorting algorithms Overview Writing programs to solve problem consists of a large number of decisions how to represent

We can use a max-heap to sort data.

Sorting 7B N log N Sorts 1 Heap Sort We can use a max-heap to sort data. Convert an array to a max-heap. Remove the root from the heap and store it in its proper position in the same array. Repeat until

Counting. Chapter Basic Counting. The Sum Principle. We begin with an example that illustrates a fundamental principle.

Chapter 1 Counting 1.1 Basic Counting The Sum Principle We begin with an example that illustrates a fundamental principle. Exercise 1.1-1 The loop below is part of an implementation of selection sort,

Lecture 7 Quicksort : Principles of Imperative Computation (Spring 2018) Frank Pfenning

Lecture 7 Quicksort 15-122: Principles of Imperative Computation (Spring 2018) Frank Pfenning In this lecture we consider two related algorithms for sorting that achieve a much better running time than

CSE 3101: Introduction to the Design and Analysis of Algorithms. Office hours: Wed 4-6 pm (CSEB 3043), or by appointment.

CSE 3101: Introduction to the Design and Analysis of Algorithms Instructor: Suprakash Datta (datta[at]cse.yorku.ca) ext 77875 Lectures: Tues, BC 215, 7 10 PM Office hours: Wed 4-6 pm (CSEB 3043), or by

CS5114: Theory of Algorithms CS5114: Theory of Algorithms lots lots Review of Mathematical Induction Mathematical Induction Purpose

Department of Computer Science Virginia Tech Blacksburg, Virginia Copyright c 00 by Clifford A. Shaffer : Theory of Algorithms Title page : Theory of Algorithms Clifford A. Shaffer Spring 00 Clifford A.

EECS 477: Introduction to algorithms. Lecture 7

EECS 477: Introduction to algorithms. Lecture 7 Prof. Igor Guskov guskov@eecs.umich.edu September 26, 2002 1 Lecture outline Recursion issues Recurrence relations Examples 2 Recursion: factorial unsigned

Lecture 2: Algorithm Analysis

ECE4050/CSC5050 Algorithms and Data Structures Lecture 2: Algorithm Analysis 1 Mathematical Background Logarithms Summations Recursion Induction Proofs Recurrence Relations 2 2 Logarithm Definition: 3

Hashing. Yufei Tao. Department of Computer Science and Engineering Chinese University of Hong Kong

Department of Computer Science and Engineering Chinese University of Hong Kong In this lecture, we will revisit the dictionary search problem, where we want to locate an integer v in a set of size n or

Outline: Search and Recursion (Ch13)

Search and Recursion Michael Mandel Lecture 12 Methods in Computational Linguistics I The City University of New York, Graduate Center https://github.com/ling78100/lectureexamples/blob/master/lecture12final.ipynb

Lecture Notes on Quicksort

Lecture Notes on Quicksort 15-122: Principles of Imperative Computation Frank Pfenning Lecture 8 February 5, 2015 1 Introduction In this lecture we consider two related algorithms for sorting that achieve

Solving Linear Recurrence Relations (8.2)

EECS 203 Spring 2016 Lecture 18 Page 1 of 10 Review: Recurrence relations (Chapter 8) Last time we started in on recurrence relations. In computer science, one of the primary reasons we look at solving

Chapter 3: The Efficiency of Algorithms Invitation to Computer Science,

Chapter 3: The Efficiency of Algorithms Invitation to Computer Science, Objectives In this chapter, you will learn about Attributes of algorithms Measuring efficiency Analysis of algorithms When things

MergeSort, Recurrences, Asymptotic Analysis Scribe: Michael P. Kim Date: April 1, 2015

CS161, Lecture 2 MergeSort, Recurrences, Asymptotic Analysis Scribe: Michael P. Kim Date: April 1, 2015 1 Introduction Today, we will introduce a fundamental algorithm design paradigm, Divide-And-Conquer,

6/4/12. Recursive void Methods. Chapter 11. Vertical Numbers. Vertical Numbers. Vertical Numbers. Algorithm for Vertical Numbers

Recursive void Methods Chapter 11 Recursion Slides prepared by Rose Williams, Binghamton University A recursive method is a method that includes a call to itself Recursion is based on the general problem

CSE Winter 2015 Quiz 2 Solutions

CSE 101 - Winter 2015 Quiz 2 s January 27, 2015 1. True or False: For any DAG G = (V, E) with at least one vertex v V, there must exist at least one topological ordering. (Answer: True) Fact (from class)

ADAPTIVE SORTING WITH AVL TREES Amr Elmasry Computer Science Department Alexandria University Alexandria, Egypt elmasry@alexeng.edu.eg Abstract A new adaptive sorting algorithm is introduced. The new implementation

Module 1: Asymptotic Time Complexity and Intro to Abstract Data Types

Module 1: Asymptotic Time Complexity and Intro to Abstract Data Types Dr. Natarajan Meghanathan Professor of Computer Science Jackson State University Jackson, MS 39217 E-mail: natarajan.meghanathan@jsums.edu

Recursion. Let s start by looking at some problems that are nicely solved using recursion. First, let s look at generating The Fibonacci series.

Recursion The programs we have discussed so far have been primarily iterative and procedural. Code calls other methods in a hierarchical manner. For some problems, it is very useful to have the methods

CS/Math 40: Introduction to Discrete Mathematics Fall 015 Instructors: Beck Hasti, Gautam Prakriya Reading 8 : Recursion 8.1 Recursion Recursion in computer science and mathematics refers to the idea of

Data Structure Lecture#17: Internal Sorting 2 (Chapter 7) U Kang Seoul National University

Data Structure Lecture#17: Internal Sorting 2 (Chapter 7) U Kang Seoul National University U Kang 1 In This Lecture Main ideas and analysis of Merge sort Main ideas and analysis of Quicksort U Kang 2 Merge

Chapter 4: Trees. 4.2 For node B :

Chapter : Trees. (a) A. (b) G, H, I, L, M, and K.. For node B : (a) A. (b) D and E. (c) C. (d). (e).... There are N nodes. Each node has two pointers, so there are N pointers. Each node but the root has

CS 3512, Spring Instructor: Doug Dunham. Textbook: James L. Hein, Discrete Structures, Logic, and Computability, 3rd Ed. Jones and Barlett, 2010

CS 3512, Spring 2011 Instructor: Doug Dunham Textbook: James L. Hein, Discrete Structures, Logic, and Computability, 3rd Ed. Jones and Barlett, 2010 Prerequisites: Calc I, CS2511 Rough course outline:

CS 173 [A]: Discrete Structures, Fall 2012 Homework 8 Solutions

CS 173 [A]: Discrete Structures, Fall 01 Homework 8 Solutions This homework contains 4 problems worth a total of 35 points. It is due on Wednesday, November 14th, at 5pm. 1 Induction Proofs With Inequalities

1 The Shortest Path Problem

CS161 Lecture 11 Single Source Shortest Paths Scribed by: Romil Verma (2015), Virginia Date: May 8, 2016 1 The Shortest Path Problem In this lecture, we ll discuss the shortest path problem. Assume we

Greedy algorithms is another useful way for solving optimization problems.

Greedy Algorithms Greedy algorithms is another useful way for solving optimization problems. Optimization Problems For the given input, we are seeking solutions that must satisfy certain conditions. These

Range Minimum Queries Part Two

Range Minimum Queries Part Two Recap from Last Time The RMQ Problem The Range Minimum Query (RMQ) problem is the following: Given a fixed array A and two indices i j, what is the smallest element out of

0.1 Welcome. 0.2 Insertion sort. Jessica Su (some portions copied from CLRS)

0.1 Welcome http://cs161.stanford.edu My contact info: Jessica Su, jtysu at stanford dot edu, office hours Monday 3-5 pm in Huang basement TA office hours: Monday, Tuesday, Wednesday 7-9 pm in Huang basement

CSCE 411 Design and Analysis of Algorithms

CSCE 411 Design and Analysis of Algorithms Set 3: Divide and Conquer Slides by Prof. Jennifer Welch Spring 2014 CSCE 411, Spring 2014: Set 3 1 General Idea of Divide & Conquer 1. Take your problem and

The Encoding Complexity of Network Coding

The Encoding Complexity of Network Coding Michael Langberg Alexander Sprintson Jehoshua Bruck California Institute of Technology Email: mikel,spalex,bruck @caltech.edu Abstract In the multicast network

Ramsey s Theorem on Graphs

Ramsey s Theorem on Graphs 1 Introduction Exposition by William Gasarch Imagine that you have 6 people at a party. We assume that, for every pair of them, either THEY KNOW EACH OTHER or NEITHER OF THEM

CSE 21 Spring 2016 Homework 4 Answer Key

1 CSE 21 Spring 2016 Homework 4 Answer Key 1. In each situation, write a recurrence relation, including base case(s), that describes the recursive structure of the problem. You do not need to solve the

Definition For vertices u, v V (G), the distance from u to v, denoted d(u, v), in G is the length of a shortest u, v-path. 1

Graph fundamentals Bipartite graph characterization Lemma. If a graph contains an odd closed walk, then it contains an odd cycle. Proof strategy: Consider a shortest closed odd walk W. If W is not a cycle,

Assignment 1 (concept): Solutions

CS10b Data Structures and Algorithms Due: Thursday, January 0th Assignment 1 (concept): Solutions Note, throughout Exercises 1 to 4, n denotes the input size of a problem. 1. (10%) Rank the following functions

Fast Sorting and Selection. A Lower Bound for Worst Case

Lists and Iterators 0//06 Presentation for use with the textbook, Algorithm Design and Applications, by M. T. Goodrich and R. Tamassia, Wiley, 0 Fast Sorting and Selection USGS NEIC. Public domain government

Agenda. We ve done Growth of functions Asymptotic Notations (O, o,,!, ) Now Binary search as an example Recurrence relations, solving them

Agenda We ve done Growth of functions Asymptotic Notations (O, o,,!, ) Now Binary search as an example Recurrence relations, solving them c Hung Q. Ngo (SUNY at Buffalo) CSE 250 Data Structures in C++

Introduction to Asymptotic Running Time Analysis CMPSC 122

Introduction to Asymptotic Running Time Analysis CMPSC 122 I. Comparing Two Searching Algorithms Let's begin by considering two searching algorithms you know, both of which will have input parameters of

Practical Session #11 - Sort properties, QuickSort algorithm, Selection

Practical Session #11 - Sort properties, QuickSort algorithm, Selection Quicksort quicksort( A, low, high ) if( high > low ) pivot partition( A, low, high ) // quicksort( A, low, pivot-1 ) quicksort( A,

Formal Model. Figure 1: The target concept T is a subset of the concept S = [0, 1]. The search agent needs to search S for a point in T.

Although this paper analyzes shaping with respect to its benefits on search problems, the reader should recognize that shaping is often intimately related to reinforcement learning. The objective in reinforcement

Recursive Algorithms. CS 180 Sunil Prabhakar Department of Computer Science Purdue University

Recursive Algorithms CS 180 Sunil Prabhakar Department of Computer Science Purdue University Recursive Algorithms Within a given method, we are allowed to call other accessible methods. It is also possible

DO NOT RE-DISTRIBUTE THIS SOLUTION FILE

Professor Kindred Math 104, Graph Theory Homework 2 Solutions February 7, 2013 Introduction to Graph Theory, West Section 1.2: 26, 38, 42 Section 1.3: 14, 18 Section 2.1: 26, 29, 30 DO NOT RE-DISTRIBUTE

15.4 Longest common subsequence

15.4 Longest common subsequence Biological applications often need to compare the DNA of two (or more) different organisms A strand of DNA consists of a string of molecules called bases, where the possible

P Is Not Equal to NP. ScholarlyCommons. University of Pennsylvania. Jon Freeman University of Pennsylvania. October 1989

University of Pennsylvania ScholarlyCommons Technical Reports (CIS) Department of Computer & Information Science October 1989 P Is Not Equal to NP Jon Freeman University of Pennsylvania Follow this and

CS301 - Data Structures Glossary By

CS301 - Data Structures Glossary By Abstract Data Type : A set of data values and associated operations that are precisely specified independent of any particular implementation. Also known as ADT Algorithm

I/O Model. Cache-Oblivious Algorithms : Algorithms in the Real World. Advantages of Cache-Oblivious Algorithms 4/9/13

I/O Model 15-853: Algorithms in the Real World Locality II: Cache-oblivious algorithms Matrix multiplication Distribution sort Static searching Abstracts a single level of the memory hierarchy Fast memory

Lecture 3. Recurrences / Heapsort

Lecture 3. Recurrences / Heapsort T. H. Cormen, C. E. Leiserson and R. L. Rivest Introduction to Algorithms, 3rd Edition, MIT Press, 2009 Sungkyunkwan University Hyunseung Choo choo@skku.edu Copyright

A Genus Bound for Digital Image Boundaries

A Genus Bound for Digital Image Boundaries Lowell Abrams and Donniell E. Fishkind March 9, 2005 Abstract Shattuck and Leahy [4] conjectured and Abrams, Fishkind, and Priebe [1],[2] proved that the boundary

11. Divide and Conquer

The whole is of necessity prior to the part. Aristotle 11. Divide and Conquer 11.1. Divide and Conquer Many recursive algorithms follow the divide and conquer philosophy. They divide the problem into smaller

Report on Cache-Oblivious Priority Queue and Graph Algorithm Applications[1]

Report on Cache-Oblivious Priority Queue and Graph Algorithm Applications[1] Marc André Tanner May 30, 2014 Abstract This report contains two main sections: In section 1 the cache-oblivious computational

CIS 121 Data Structures and Algorithms with Java Spring Code Snippets and Recurrences Monday, January 29/Tuesday, January 30

CIS 11 Data Structures and Algorithms with Java Spring 018 Code Snippets and Recurrences Monday, January 9/Tuesday, January 30 Learning Goals Practice solving recurrences and proving asymptotic bounds

1 Dijkstra s Algorithm

Lecture 11 Dijkstra s Algorithm Scribes: Himanshu Bhandoh (2015), Virginia Williams, and Date: November 1, 2017 Anthony Kim (2016), G. Valiant (2017), M. Wootters (2017) (Adapted from Virginia Williams

Chapter 2: Complexity Analysis

Chapter 2: Complexity Analysis Objectives Looking ahead in this chapter, we ll consider: Computational and Asymptotic Complexity Big-O Notation Properties of the Big-O Notation Ω and Θ Notations Possible