Slide Set 18. for ENCM 339 Fall 2017 Section 01. Steve Norman, PhD, PEng

Transcription

1 Slide Set 18 for ENCM 339 Fall 2017 Section 01 Steve Norman, PhD, PEng Electrical & Computer Engineering Schulich School of Engineering University of Calgary December 2017

2 ENCM 339 Fall 2017 Section 01 Slide Set 18 slide 2/20 Contents Attention! Performance of some recursive functions

3 ENCM 339 Fall 2017 Section 01 Slide Set 18 slide 3/20 Outline of Slide Set 18 Attention! Performance of some recursive functions

4 ENCM 339 Fall 2017 Section 01 Slide Set 18 slide 4/20 Attention! All of the content in this slide set is useful to know. But none of the content will be tested on the Fall 2017 ENCM 339 Section 01 final examination.

5 ENCM 339 Fall 2017 Section 01 Slide Set 18 slide 5/20 Outline of Slide Set 18 Attention! Performance of some recursive functions

6 ENCM 339 Fall 2017 Section 01 Slide Set 18 slide 6/20 Performance of some recursive functions In Slide Set 17, I had this to say: Mergesort and quicksort, if coded properly, are both astonishingly faster for sorting large arrays than are easier-to-code sorting algorithms, such as bubble sort and insertion sort. Let s look at the above claim in more detail. Then, let s look at a simple recursive function that is astonishingly slow.

7 ENCM 339 Fall 2017 Section 01 Slide Set 18 slide 7/20 Performance measurement of some sort functions The slides that follow were written a few years ago, when it could be assumed that all ENCM 339 students would know what a vector<double> is in C++. Here s what students in Section 01 of ENCM 339 in Fall 2017 need to know to understand the slides: The problem of sorting a vector<double> in C++ is essentially the same as sorting an array with elements of type double in C.

8 ENCM 339 Fall 2017 Section 01 Slide Set 18 slide 8/20 I wrote a C++ program to measure the time it took to sort a vector<double>, with each of five functions: the C++ library sort function, which uses a quicksort-based algorithm the C++ library stable_sort function, which uses mergesort my own implementation of mergesort a good implementation of insertion sort an implementation of bubble sort (The C++ library sort function uses an algorithm called introsort, which, oversimplifying a little, is quicksort with a guard against worst-case performance of quicksort.)

9 ENCM 339 Fall 2017 Section 01 Slide Set 18 slide 9/20 The text on the next page is copy-pasted from a terminal window on my Mac, and shows three timing runs for vectors of 1,000 elements. Times vary from one run to the next, which is to be expected, for many reasons. Two important reasons are: Each program starts with a different randomly-ordered vector. For some algorithms, sort speed may be faster for some orders of numbers than for other orders. Modern computer memory systems are very complex. The average time needed to read or write an array element may vary from one run of a program to the next.

10 $./a.out 1e3 each vector will have 1000 elements time for library sort was 5.5e-05 seconds time for library stable_sort was 6e-05 seconds time for sn merge sort was 6.5e-05 seconds time for insertion sort was seconds time for bubble sort was seconds $./a.out 1e3 each vector will have 1000 elements time for library sort was 4.8e-05 seconds time for library stable_sort was 5.6e-05 seconds time for sn merge sort was 5.8e-05 seconds time for insertion sort was seconds time for bubble sort was seconds $./a.out 1e3 each vector will have 1000 elements time for library sort was 5.5e-05 seconds time for library stable_sort was 6.1e-05 seconds time for sn merge sort was 6.5e-05 seconds time for insertion sort was seconds time for bubble sort was seconds

11 ENCM 339 Fall 2017 Section 01 Slide Set 18 slide 11/20 Results for 1,000 to 100,000 elements Times are in microseconds. Numbers are averages from 3 runs for each array size, and are rounded to 2 or 3 significant digits. number of elements algorithm 1,000 10, ,000 sort ,910 stable sort ,310 SN merge sort ,420 insertion sort ,700 1,410,000 bubble sort 1, ,000 13,700,000

12 ENCM 339 Fall 2017 Section 01 Slide Set 18 slide 12/20 Results for 300,000 to 10 million elements Times are in seconds. Numbers are averages from 3 runs for each array size, and are rounded to 3 significant digits. number of elements algorithm 300,000 1 million 3 million 10 million sort stable sort SN merge sort insertion sort too long too long Reasonable estimates for insertion sort would be 22 minutes for 3 million elements, and 4.1 hours for 10 million elements.

13 ENCM 339 Fall 2017 Section 01 Slide Set 18 slide 13/20 Why is there such a huge difference in running times? Computer scientists have shown that for N array elements, starting in random order, average-case running times are approximately as follows: K B N 2 for bubble sort and K I N 2 for insertion sort; K Q N log 2 N for quicksort and K M N log 2 N for mergesort. K B, K I, K Q, and K M are numbers that depend on processor speed, memory speed, quality of source code, and efficiency of machine code generated by the compiler. Crucial fact: log 2 N grows very slowly as a function of N. For example, log 2 1,000,

14 ENCM 339 Fall 2017 Section 01 Slide Set 18 slide 14/20 Key point: With large data sets, it may be really important to choose an algorithm that is not only correct, but also fast! Speed justifies the choice of mergesort (with slightly tricky merge code) or quicksort (with significantly tricky partition code), over simple algorithms like bubble sort and insertion sort. In 2017, fortunately, most good languages have libraries that make fast sort algorithms available to you without your having to code them. C++ is excellent in this respect; C is less excellent because C does not have a feature called templates.

15 ENCM 339 Fall 2017 Section 01 Slide Set 18 slide 15/20 A terribly slow recursive function Recursion happens to be very helpful in implementing quicksort and mergesort. But do not make the inference that use of recursion automatically results in speed! We ll now look at a recursive function to compute F n, where 0 if n = 0 F n = 1 if n = 1 F n 1 + F n 2 if n 2 This formula generates a sequence famous in mathematics, called the Fibonacci numbers: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34,...

16 ENCM 339 Fall 2017 Section 01 Slide Set 18 slide 16/20 Consider the following function prototype: unsigned int fibo(unsigned int n); // PROMISES: // Return value is the nth Fibonacci number. Let s write out a recursive function definition in C, using the formula for F n given on the previous slide.

17 ENCM 339 Fall 2017 Section 01 Slide Set 18 slide 17/20 Time to repeatedly compute F million times on my Mac, using a simple loop: 3.0 seconds. Average time to compute F 50 once: 30 nanoseconds. Time to compute F 50 once on my Mac, using the obvious recursive function: 68.5 seconds. The recursive function is slower by a factor of about 2.3 billion. What is going on with the recursive function?! To get some understanding, let s determine the number of calls that result when n == 5 in the original call to the recursive version of fibo.

18 ENCM 339 Fall 2017 Section 01 Slide Set 18 slide 18/20 The problem with the recursive definition is that it does a huge amount of redundant work. This table shows how quickly things go bad as n increases... original value of n # of calls to fibo , ,692, ,160, ,730,022,147

19 ENCM 339 Fall 2017 Section 01 Slide Set 18 slide 19/20 Review: Fundamental rules of recursion 1. Base cases: There must be at least one version of a problem that is so simple that it can be solved without a recursive call. 2. Always make progress: A recursive call must in some sense move in the direction of a base case. 3. Design rule: When designing a recursive function, believe that the recursive calls will work. 4. Performance rule: Performance may be very poor if a recursive function solves exactly the same problem many times.

20 ENCM 339 Fall 2017 Section 01 Slide Set 18 slide 20/20 Fast recursive solutions for computing F n There are some fast recursive ways to compute F n, based on this formula: F n = [ 0 1 ] [ ] n [ ] What does it mean to raise a matrix to a power? Here s an example: [ ] 3 = [ ] [ ] [ That leads back to the first example of recursion in Slide Set 17, efficient computation of a power function, but we won t go into details... ]