Performance analysis.

Size: px

Start display at page:

Download "Performance analysis."

Bernice Lane
5 years ago
Views:

2 Performance analysis Overview Why we care What we measure and how How functions grow Empirical analysis The doubling hypothesis Performance lab Mystery methods 2

Compare algorithms and data structures Change to a more

3 Why performance analysis Predicting performance When will my program finish? Will my program finish? Compare algorithms and data structures Change to a more complicated sorting algorithm? Change from a linked list to a doubling array? Will it be worth the trouble? 3

4 What do we care about? Performance metrics Time, how long do I have to wait to solve it? Measure with a stop watch (real or virtual) Run in a performance profiler Often part of an IDE (e.g. Microsoft Visual Studio) Sometimes standalone (e.g. gprof) Helps you determine bottleneck in your code long start = System.currentTimeMillis(); // Do some stuff long now = System.currentTimeMillis(); double elapsedsecs = (now - start) / ; Measuring how long some code takes. 4

5 What do we care about? Performance metrics Space, do I have the resources to solve it? Usually we care about physical memory 8 GB = 8.6 billion places to store a byte (stores 256 possibilities) Java double, 64-bits = 8 bytes 8 GB / 8 bytes = over 1 million doubles! Can swap to disk for some extra space But much much slower Stats.java class provides measurement of time and memory usage. 5

6 Breaking the predictive keyboard... file training words mobydick.txt 209K wiki_200k.txt 3.7M lm_train_blog_0.txt 73M % java PredictiveKeyboard mobydick.txt elapsed time (s) : 0.49 heap memory used (MB) : max memory (MB) : Training words : Unique words : % java PredictiveKeyboard wiki_200k.txt elapsed time (s) : 3.90 heap memory used (MB) : max memory (MB) : Training words : Unique words :

7 Breaking the predictive keyboard... file training words mobydick.txt 209K wiki_200k.txt 3.7M lm_train_blog_0.txt 73M % java PredictiveKeyboard lm_train_blog_0.txt Exception in thread "main" java.lang.outofmemoryerror: Java heap space at java.util.arrays.copyofrange(arrays.java:3209) at java.lang.string.<init>(string.java:215) at java.lang.stringbuffer.tostring(stringbuffer.java:585) at WordPredictor.build(WordPredictor.java:96) at PredictiveKeyboard.main(PredictiveKeyboard.java:25) % java Xmx512m PredictiveKeyboard lm_train_blog_0.txt elapsed time (s) : heap memory used (MB) : max memory (MB) : Training words : Unique words : Use -Xmx to ask for 512MB of heap instead of the default (128MB on my macbook) 7

8 How you solve it matters Discrete Fourier transform Break waveform of N samples into periodic components Applications: DVD, JPEG, MRI, astrophysics Bruce force algorithm: N 2 steps Cooley-Tukey FFT algorithm: N log N steps 8

9 Three-sum problem A "simple" problem Given N integers, find all triples that sum to 0 % more 8ints.txt % java ThreeSum < 8ints.txt Brute force algorithm: Try all possible triples and see if they sum to 0. 9

10 Three sum program public class ThreeSum { public static void main(string [] args) { int N = StdIn.readInt(); int [] nums = new int[n]; for (int i = 0; i < N; i++) nums[i] = StdIn.readInt(); } } All possible triples i < j < k from the set of integers. for (int i = 0; i < N; i++) for (int j = i + 1; j < N; j++) for (int k = j + 1; k < N; k++) if (nums[i] + nums[j] + nums[k] == 0) System.out.println(nums[i] + " " + nums[j] + " " + nums[k]); 10

11 Empirical analysis: three sum Run program for various input sizes, 2 machines: My first laptop: Pentium 1, 150Mhz, 80MB RAM My desktop: Phenom II, 3.2Ghz (3.6Ghz turbo), 16GB RAM VS 11

12 Empirical analysis: three sum Run program for various input sizes, 2 machines: My first laptop: Pentium 1, 150Mhz, 80MB RAM My desktop: Phenom II, 3.2Ghz (3.6Ghz turbo), 16GB RAM N ancient laptop modern desktop

13 Doubling hypothesis Cheap and cheerful analysis Time program for input size N Time program for input size 2N Time program for input size 4N Ratio T(2N) / T(N) approaches a constant Constant tells you the exponent in T = an b N T(N) ratio Desktop data Constant from ratio Hypothesis Order of growth 2 T = a N linear, O(N) 4 T = a N 2 quadratic, O(N 2 ) 8 T = a N 3 cubic, O(N 3 ) 16 T = a N 4 O(N 4 ) 13

14 Estimating constant, making predictions N T(N) ratio Desktop data T = a N = a (6400) 3 a = 1.01 x How long for desktop to solve a 100,000 integer problem? 1.01 x (100000) 3 = secs = 280 hours N T(N) ratio Laptop data T = a N = a (6400) 3 a=1.80 x How long for laptop to solve a 100,000 integer problem? 1.80 x (100000) 3 = 1.80 x secs = hours 14

15 Bottom line My three sum algorithm sucks Does not scale to large problems an algorithm problem 15 years of computer progress didn't help much My algorithm: O(N 3 ) A slightly more complicated algorithm: O(N 2 log N) Using the better algorithm, how long does it take the desktop to solve a 100,000 integer problem? 1.01 x (100000) 2 log(100000) = 168 secs This assumes the same constant. Really should do the doubling experiment again with the new algorithm. Using the better algorithm, how long does it take the ancient laptop to solve a 100,000 integer problem? 1.80 x (100000) 2 log(100000) = secs 15

16 Programming activity Determine order of growth: 4 mystery methods, MysteryA-D Add code to time one of the methods Avoid timing input / output Test using doubling hypothesis Files rand*.txt, from 250 to 64k numbers Start small and work bigger! N T(N) ratio k 2k 4k Constant from ratio Hypothesis Order of growth 2 T = a N linear, O(N) 4 T = a N 2 quadratic, O(N 2 ) 8 T = a N 3 cubic, O(N 3 ) 16 T = a N 4 O(N 4 ) Fill out a table like this for each method! Use ratio to determine order of growth. 16

17 Summary Introduction to Analysis of Algorithms Next year: an entire semester course Today: simple empirical estimation The algorithm matters Fast computer only buys you out of trouble temporarily Doubling hypothesis Measure time ratio as you double the input size If the ratio = 2 b, runtime of algorithm T(N) = a N b 17

Performance. CSCI 135: Fundamentals of Computer Science Keith Vertanen. h"p://

Performance. CSCI 135: Fundamentals of Computer Science Keith Vertanen. hp:// Performance h"p://www.flickr.com/photos/exo3ccarlife/3270764550/ h"p://www.flickr.com/photos/roel1943/5436844655/ CSCI 135: Fundamentals of Computer Science Keith Vertanen Performance analysis Overview