Algorithms. Algorithms. Algorithms 1.4 ANALYSIS OF ALGORITHMS

Transcription

1 Announcements Algorithms ROBERT SEDGEWICK KEVI WAYE First programming assignment. Due Tomorrow at :00pm. Try electronic submission system today. "Check All Submitted Files. will perform checks on your code. You may use this up to 0 times. Can still submit after you use up your checks. Should not be your primary testing technique! Registration. Register for Piazza. Register for Coursera. Register for Poll Everywhere. ROBERT Algorithms F O U R T H E D I T I O SEDGEWICK KEVI WAYE AALYSIS OF ALGORITHMS introduction empirical observations mathematical models order-of-growth classifications theory of algorithms memory Efficiency 6 vs. 6 6: Techniques for solving problems. 6: Techniques for solving problems efficiently. Algorithms ROBERT SEDGEWICK KEVI WAYE AALYSIS OF ALGORITHMS introduction empirical observations mathematical models order-of-growth classifications theory of algorithms memory Simple Example: Checking symmetry of an x matrix aive: Scan all elements. Better: Scan only elements above the diagonal (>x speedup). 4

2 Efficiency (more insidious example) Running time public static String concatenateospace(string s, String s) for (int i = 0; i < s.length(); i++) if (s.charat(i)!= ) s = s + s.charat(i); return s; As soon as an Analytic Engine exists, it will necessarily guide the future course of the science. Whenever any result is sought by its aid, the question will arise By what course of calculation can these results be arrived at by the machine in the shortest time? Charles Babbage (864) Common Problem. ovice programmer does not understand performance characteristics how many times do you have to turn the crank? of data structure. Results in poor performance that gets WORSE with input size. Today Precise definitions of program performance. Experimental and theoretical techniques for measuring performance. Analytic Engine 5 The Life of the Philosopher 6 Reasons to analyze algorithms Predict performance. The iron folding-doors of the small-room or oven were opened. Captain Kater and myself entered, and they were closed upon us... The thermometer marked, if I recollect rightly, 65 degrees. The pulse was quickened, and I ought to have to have counted but did not count the number of inspirations per minute. Perspiration commenced immediately and was very copious. We remained, I believe, about five or six minutes without very great discomfort, and I experienced no subsequent inconvenience from the result of the experiment Charles Babbage, From the Life of the Philosopher this course Compare algorithms. Provide guarantees. Understand theoretical basis. theory of algorithms Primary practical reasons: avoid performance bugs enable new technologies example:simulation of galaxy formation client gets poor performance because programmer did not understand performance characteristics 65 Fahrenheit / 30 Celsius 7 8

3 Running time of programs The challenge Q. Will my program be able to solve a large practical input? Why is my program so slow? Why does it run out of memory? Programs Mathematical objects. Running on physical hardware. Mathematical model Left program runtime: c Right program runtime: c( / - /) Empirical observations Runtime of a program varies even when run on the same input. 9 Insight. [Knuth 970s] Use scientific method to understand performance. 0 Scientific method applied to analysis of algorithms A framework for predicting performance and comparing algorithms. Scientific method. Observe some feature of the natural world. Hypothesize a model that is consistent with the observations. Predict events using the hypothesis. Verify the predictions by making further observations. Validate by repeating until the hypothesis and observations agree. Principles. Experiments must be reproducible. Hypotheses must be falsifiable. Algorithms ROBERT SEDGEWICK KEVI WAYE AALYSIS OF ALGORITHMS introduction empirical observations mathematical models order-of-growth classifications theory of algorithms memory Feature of the natural world. Computer itself.

4 Example: 3-SUM 3-SUM: brute-force algorithm 3-SUM. Given distinct integers, how many triples sum to exactly zero? % more 8ints.txt % java ThreeSum 8ints.txt a[i] a[j] a[k] sum public class ThreeSum public static int count(int[] a) int = a.length; int count = 0; for (int i = 0; i < ; i++) for (int j = i+; j < ; j++) for (int k = j+; k < ; k++) if (a[i] + a[j] + a[k] == 0) count++; return count; check each triple for simplicity, ignore integer overflow Context. Deeply related to problems in computational geometry. public static void main(string[] args) int[] a = In.readInts(args[0]); StdOut.println(count(a)); 3 4 Measuring the running time Measuring the running time Q. How to time a program? A. Manual. % java ThreeSum Kints.txt tick tick tick Q. How to time a program? A. Automatic. 70 % java ThreeSum Kints.txt 58 % java ThreeSum 4Kints.txt 4039 public class Stopwatch Stopwatch() (part of stdlib.jar ) create a new stopwatch double elapsedtime() time since creation (in seconds) public static void main(string[] args) int[] a = In.readInts(args[0]); Stopwatch stopwatch = new Stopwatch(); StdOut.println(ThreeSum.count(a)); double time = stopwatch.elapsedtime(); client code 5 6

5 Empirical analysis Empirical analysis Run the program for various input sizes and measure running time. Run the program for various input sizes and measure running time. time (seconds) ,000 0., , , ,000? 7 8 Data analysis Data analysis Standard plot. Plot running time T () vs. input size. Hard to form a useful hypothesis. standard plot running time T() Log-log plot. Plot running time T () vs. input size using log-log scale. log-log plot 3 orders of magnitude lg(t()) straight line of slope 3 T() = a b power law lg(t ()) = b lg + lg a lg(t ()) = b lg + c b =.999 c = a = K K 4K 8K lg K K 4K 8K problem size Regression. Fit straight line through data points: lg(t()) y = b x + = cb lg() + c Interpretation. T() = a b, where a = c Hypothesis. The running time is about seconds. slope 9 0

6 Prediction and validation Doubling hypothesis Hypothesis. The running time is about seconds. Predictions. 5.0 seconds for = 8, seconds for = 6,000. "order of growth" of running time is about 3 [stay tuned] Doubling hypothesis. Another way to build models of the form T() = a b Run program, doubling the size of the input. time (seconds) ratio lg ratio Observations. time (seconds) 8, , ,000 5., , , , lg (5. / 6.40) = 3.0 6, validates hypothesis! Hypothesis. Running time is about a b with b = lg ratio. seems to converge to a constant b 3 Doubling hypothesis Experimental algorithmics Doubling hypothesis. Quick way to estimate b in a power-law relationship. Q. How to estimate a (assuming we know b)? A. Run the program (for a sufficient large value of ) and solve for a. time (seconds) 8, = a , a = System independent effects. Algorithm. Input data. System dependent effects. Hardware: CPU, memory, cache, Software: compiler, interpreter, garbage collector, determines exponent b in power law System: operating system, network, other apps, determines constant a in power law 8, Caveat. In some cases, b can depend on system (e.g. virtualization) Bad news. Difficult to get precise measurements. Hypothesis. Running time is about seconds. Good news. Much easier and cheaper than other sciences. almost identical hypothesis to one obtained via linear regression 3 e.g., can run huge number of experiments 4

7 Mathematical models for running time.4 AALYSIS OF ALGORITHMS Total running time: sum of cost frequency for all operations. eed to analyze program to determine set of operations. Cost depends on machine, compiler. Frequency depends on algorithm, input data. Algorithms ROBERT SEDGEWICK KEVI WAYE introduction empirical observations mathematical models order-of-growth classifications theory of algorithms memory Donald Knuth 974 Turing Award In principle, accurate mathematical models are available. 6 Timing basic operations (a hopeless endeavor) Cost of basic operations operation example nanoseconds integer add a + b. operation example nanoseconds integer multiply a * b.4 variable declaration int a c integer divide a / b 5.4 assignment statement a = b c floating-point add a + b 4.6 integer compare a < b c3 floating-point multiply a * b 4. array element access a[i] c4 floating-point divide a / b 3.5 sine Math.sin(theta) 9.3 arctangent Math.atan(y, x) 9.0 array length a.length c5 D array allocation new int[] c6 D array allocation new int[][] c string length s.length() c8 substring extraction s.substring(/, ) c9 Running OS X on Macbook Pro.GHz with GB RAM string concatenation s + t c0 Computer Architecture Caveats (see COS 475). Most computers are more like assembly lines than oracles (pipelining). ovice Register vs. cache vs. RAM vs. hard disk (Java is a high level language) 7 mistake. Abusive string concatenation. 8

8 Example: -SUM Example: -SUM Q. How many instructions as a function of input size? Q. How many instructions as a function of input size? int count = 0; for (int i = 0; i < ; i++) if (a[i] == 0) count++; int count = 0; for (int i = 0; i < ; i++) for (int j = i+; j < ; j++) if (a[i] + a[j] == 0) count++; array accesses operation frequency Frequency, =0000 variable declaration assignment statement less than compare equal to compare 0000 Alternate Pf ( ) = ( ) = array access 0000 increment to 0000 to ( ) = half of square half of diagonal 30 Example: -SUM Simplifying the calculations Q. How many instructions as a function of input size? int count = 0; for (int i = 0; i < ; i++) for (int j = i+; j < ; j++) if (a[i] + a[j] == 0) count++; operation frequency ( ) = ( ) = It is convenient to have a measure of the amount of work involved in a computing process, even though it be a very crude one. We may count up the number of times that various elementary operations are applied in the whole process and then given them various weights. We might, for instance, count the number of additions, subtractions, multiplications, divisions, recording of numbers, and extractions of figures from tables. In the case of computing with matrices most of the work consists of multiplications and writing down numbers, and we shall therefore only attempt to count the number of multiplications and recordings. Alan Turing variable declaration + assignment statement + less than compare ½ ( + ) ( + ) equal to compare ½ ( ) array access ( ) increment ½ ( ) to ( ) tedious to count exactly ROUDIG-OFF ERRORS I MATRIX PROCESSES By A. M. TURIG ational Physical Laboratory, Teddington, Middlesex) [Received 4 ovember 947] SUMMARY A number of methods of solving sets of linear equations and inverting matrices are discussed. The theory of the rounding-off errors involved is investigated for some of the methods. In all cases examined, including the well-known 'Gauss elimination process', it is found that the errors are normally quite moderate: no exponential build-up need occur. 3 3

9 Simplification : cost model Cost model. Use some basic operation as a proxy for running time. int count = 0; for (int i = 0; i < ; i++) for (int j = i+; j < ; j++) if (a[i] + a[j] == 0) count++; operation frequency variable declaration + assignment statement ( ) = ( ) = Simplification : tilde notation Estimate running time (or memory) as a function of input size. Ignore lower order terms. when is large, terms are negligible when is small, we don't care Ex. ⅙ ! ~ ⅙ 3 Ex. ⅙ /3 + 56! ~ ⅙ 3 Ex 3. ⅙ 3 - ½ + ⅓! ~ ⅙ 3 discard lower-order terms (e.g., = 000: million vs million) 66,666,667,000 3 /6 3 /6 / + /3 66,67,000 Leading-term approximation less than compare ½ ( + ) ( + ) equal to compare ½ ( ) array access ( ) increment ½ ( ) to ( ) cost model = array accesses (we assume compiler/jvm do not optimize any array accesses away!) Technical definition. f() ~ g() means lim f () g() = Simplification : tilde notation Estimate running time (or memory) as a function of input size. Ignore lower order terms. when is large, terms are negligible when is small, we don't care Example: -SUM Q. Approximately how many array accesses as a function of input size? int count = 0; for (int i = 0; i < ; i++) for (int j = i+; j < ; j++) if (a[i] + a[j] == 0) count++; "inner loop" operation frequency tilde notation variable declaration + ~ assignment statement + ~ less than compare ½ ( + ) ( + ) ~ ½ equal to compare ½ ( ) ~ ½ array access ( ) ~ A. ~ array accesses. Because (½ ) = ( ) = ( ) = increment ½ ( ) to ( ) ~ ½ to ~ Bottom line. Use cost model and tilde notation to simplify counts

10 Example: 3-SUM Estimating a discrete sum Q. Approximately how many array accesses as a function of input size? int count = 0; for (int i = 0; i < ; i++) for (int j = i+; j < ; j++) for (int k = j+; k < ; k++) if (a[i] + a[j] + a[k] == 0) count++; "inner loop" Q. How to estimate a discrete sum? A. Take discrete mathematics course. A. Replace the sum with an integral, and use calculus! Ex i= i x= xdx A. ~ ½ 3 array accesses. Because (3/6 3 ) = ½ 3 3 = ( )( ) 3! 6 3 Ex. k + k + + k. Ex 3. + / + /3 + + /. i= i= i k i x= x= x k dx dx x = ln k + k+ Bottom line. Use cost model and tilde notation to simplify counts. Ex 4. 3-sum triple loop. i= j=i k=j x= y=x z=y dz dy dx Estimating a discrete sum Estimating a discrete sum Q. How to estimate a discrete sum? A. Take discrete mathematics course. A. Replace the sum with an integral, and use calculus! Q. How to estimate a discrete sum? A3. Use Maple or Wolfram Alpha. Ex 4. + ½ + ¼ + ⅛ + i=0 i = x=0 x dx = ln.447 wolframalpha.com [wayne:nobel.princeton.edu] > maple5 \^/ Maple 5 (X86 64 LIUX)._ \ / _. Copyright (c) Maplesoft, a division of Waterloo Maple Inc. 0 \ MAPLE / All rights reserved. Maple is a trademark of < > Waterloo Maple Inc. Type? for help. > factor(sum(sum(sum(, k=j+..), j = i+..), i =..)); Caveat. Integral trick doesn't always work! ( - ) ( - )

11 Mathematical models for running time In principle, accurate mathematical models are available. In practice, Formulas can be complicated. Realities of hardware impact accuracy of formulas. Advanced mathematics might be required. Exact models best left for experts. costs (depend on machine, compiler) T = c A + c B + c3 C + c4 D + c5 E A = array access B = integer add C = integer compare D = increment E = variable assignment frequencies (depend on algorithm, input) Algorithms ROBERT SEDGEWICK KEVI WAYE AALYSIS OF ALGORITHMS introduction empirical observations mathematical models order-of-growth classifications theory of algorithms memory Bottom line. We use approximate models in this course: T() ~ c 3. 4 Order-of-growth Definition. If f() ~ a g(), then the order-of-growth of f() is just g() Example: Runtime of 3SUM: ~ /6 t 3 [see page 8] Order-of-growth of the runtime of 3SUM: 3 We often say order-of-growth of 3SUM as shorthand for the runtime. Common order-of-growth classifications Good news. the small set of functions, log,, log,, 3, and suffices to describe order-of-growth of typical algorithms. log-log plot 5T exponential cubic quadratic order of growth discards leading coefficient linearithmic linear 64T time int count = 0; for (int i = 0; i < ; i++) for (int j = i+; j < ; j++) for (int k = j+; k < ; k++) if (a[i] + a[j] + a[k] == 0) count++; Time to execute: t 8T 4T T T logarithmic constant K K 4K 8K 5K size Typical orders of growth 43 44

12 Common order-of-growth classifications Practical implications of order-of-growth order of growth name typical code framework description example T() / T() constant a = b[0] + b[]; statement log logarithmic linear while ( > ) = / ;... for (int i = 0; i < ; i++)... log linearithmic [see mergesort lecture] quadratic for (int i = 0; i < ; i++) for (int j = 0; j < ; j++)... add two array elements divide in half binary search ~ loop divide and conquer double loop find the maximum mergesort ~ check all pairs 4 problem size solvable in minutes growth rate 970s 980s 990s 000s any any any any log any any any any millions tens of hundreds of millions millions billions log hundreds of hundreds of millions millions thousands millions hundreds thousand thousands tens of thousands game changer 3 cubic for (int i = 0; i < ; i++) for (int j = 0; j < ; j++) for (int k = 0; k < ; k++)... triple loop check all triples 8 3 hundred hundreds thousand thousands 0 0s 0s 30 exponential [see combinatorial search lecture] exhaustive search check all subsets T() Bottom line. eed linear or linearithmic alg to keep pace with Moore's law Some algorithmic successes Binary search demo -body simulation. Simulate gravitational interactions among bodies. Brute force: steps. Barnes-Hut algorithm: log steps, enables new research. Andrew time 64T quadratic Appel PU '8 Goal. Given a sorted array and a key, find index of the key in the array? Binary search. Compare key against middle entry. Too small, go left. Too big, go right. Equal, found. successful search for 33 3T T linearithmic T linear lo hi size K K 4K 8K Worst case: lg c 47 see Coursera for rigorous proof 48

13 Binary search: Java implementation An log algorithm for 3-SUM Trivial to implement? First binary search published in 946; first bug-free one in 96. Bug in Java's Arrays.binarySearch() discovered in 006. Sorting-based algorithm. Step : Sort the (distinct) numbers. Step : For each pair of numbers a[i] and a[j], binary search for -(a[i] + a[j]). input sort public static int binarysearch(int[] a, int key) int lo = 0, hi = a.length-; while (lo <= hi) int mid = lo + (hi - lo) / ; if (key < a[mid]) hi = mid - ; else if (key > a[mid]) lo = mid + ; else return mid; return -; Invariant. If key appears in the array a[], then a[lo] key a[hi]. one "3-way compare" Analysis. Order of growth is log. Step : with insertion sort. Step : log with binary search. binary searches, each log Remark. Can achieve by modifying binary search step. binary search (-40, -0) 60 (-40, -0) 50 (-40, 0) 40 (-40, 5) 35 (-40, 0) 30 (-40, 40) 0 (-0, -0) 30 (-0, 0) 0 ( 0, 30) -40 ( 0, 40) -50 ( 30, 40) -70 only count if a[i] < a[j] < a[k] to avoid double counting Comparing programs Hypothesis. The sorting-based log algorithm for 3-SUM is significantly faster in practice than the brute-force 3 algorithm. time (seconds), time (seconds), AALYSIS OF ALGORITHMS, , , ThreeSum.java, , , , , , Algorithms ROBERT SEDGEWICK KEVI WAYE introduction empirical observations mathematical models order-of-growth classifications theory of algorithms memory ThreeSumDeluxe.java Guiding principle. Typically, better order of growth faster in practice. 5

14 Types of analyses: Performance depends on input Types of analyses: Performance depends on input Best case. Lower bound on cost. Determined by easiest input. Provides a goal for all inputs. Worst case. Upper bound on cost. Determined by most difficult input. Provides a guarantee for all inputs. Average case. Expected cost for random input. eed a model for random input. Provides a way to predict performance. Best case. Lower bound on cost. Determined by easiest input. Provides a goal for all inputs. Worst case. Upper bound on cost. Determined by most difficult input. Provides a guarantee for all inputs. Average case. Expected cost for random input. eed a model for random input. Provides a way to predict performance. Where you lie depends on your input! Ex. Compares for binary search. Best: Ex. Array accesses for brute-force 3-SUM. Best: 3 Ex. Compares for binary search. Ex. Array accesses for brute-force 3-SUM. Average: lg Average: 3 Worst: lg Worst: 3 log log Types of analyses Best case. Lower bound on cost. Worst case. Upper bound on cost (guarantee). Average case. Expected cost. Primary practical reason: avoid performance bugs. Example: Algorithm selection Given arbitrary data, performance may be anywhere in our bounds. Approach : depend on worst case guarantee. Example: Use Mergesort instead of Quicksort Approach : randomize, depend on probabilistic guarantee. Example: Randomize input before giving to Quicksort Mergesort. [next week] Quicksort. [next week] Theory of algorithms Previous slides Best, average, and worst case for a specific algorithm. ew goals. Establish difficulty of a problem, e.g. how hard is 3SUM? Develop optimal algorithm. Approach: Use order-of-growth in worst case Use order-of-growth (just like we ve been doing). Analysis is asymptotic, i.e. for very large. Analysis is to within a constant factor, using OaG instead of Tilde. Consider only worst case. Analysis avoids messy input models. Analysis focuses on guarantees. log log log log log 55 56

15 Theory of algorithms Theory of algorithms Testing optimality of algorithm A for problem P Find worst case order of growth guarantee for specific algorithm A, g() Find lower bound on guarantee for any algorithm that solves P, B() If they match, i.e. g() = B(), then: Worst case performance of A is asymptotically optimal. Optimal algorithm for P has order of growth g() If they don t, g() at least provides an upper bound. Example: The -SUM problem (how many 0s?) Let A be the brute force algorithm where we simply look at each entry and count the zeros. Worst case order of growth: g() = Of any algorithm that solves -SUM, must at least examine every entry. Lower bound on worst case order of growth: B() = g() = B(). A is optimal! Algorithm A g() Brute force algorithm???? g() don t care don t care Lower bound for best algorithm B() Lower bound for best algorithm???? B() Worst case performance for optimal algorithm 57 don t care don t care Worst case performance for optimal -SUM algorithm 58 Theory of algorithms: example Theory of algorithms: example Example: The 3-SUM problem (how many 0s?) Let A be the brute force algorithm where we look at each triple. Worst case order of growth: g() = 3 Of any algorithm that solves 3-SUM, must at least examine every entry. Lower bound on worst case order of growth: B() = g() B() Brute force algorithm don t care don t care 3 Lower bound for best algorithm 3 Brute force algorithm don t care don t care 3 3 don t care don t care Worst case performance for optimal -SUM algorithm What this tells us It is possible to solve 3SUM in 3 time in the worst case. The optimal algorithm has worst case running time OaG between and 3. Lower bound for best algorithm don t care don t care Worst case performance for optimal 3-SUM algorithm 59 What this doesn t tell us Is there an algorithm with better running time than 3 in the worst case? Is there some clever way of finding a better lower bound than? 60

16 Theory of algorithms terminology Theory of algorithms terminology Testing optimality of algorithm A for problem P Find worst case order of growth guarantee for specific algorithm A, g() Find lower bound on guarantee for any algorithm that solves P, B() Testing optimality of brute force algorithm for 3-SUM Find worst case order of growth guarantee for brute force, 3 Find lower bound on guarantee for any algorithm that solves P, Algorithm A g() O(g()) Algorithm A 3 O( 3 ) don t care don t care g() don t care don t care 3 Lower bound for best algorithm B() Ω(B()) Lower bound for best algorithm Ω() don t care don t care B() Worst case performance for optimal algorithm don t care don t care Worst case performance for optimal algorithm Standard terminology The running time of the optimal algorithm for problem P is O(g()) The running time of the optimal algorithm for problem P is Ω(B()) 6 Standard terminology The running time of the optimal algorithm for problem P is O(3 ) The running time of the optimal algorithm for problem P is Ω() 6 Commonly-used notations in the theory of algorithms Theory of algorithm: 3SUM notation provides example shorthand for used to Big Theta asymptotic order of growth Θ( ) ½ log + 3 classify algorithms The run time of 3SUM s optimal solution... =O(3 ) based on brute force solution. =O( log ) based on binary search based solution. =O( ) based on solution developed in precept this week. =Ω() based on a simple argument about accessing all data. Grows at least as slow as, and at least as fast as. Big Oh Θ( ) and smaller O( ) Big Omega Θ( ) and larger Ω( ) 0 00 develop log + 3 upper bounds ½ 5 develop 3 + log + 3 lower bounds Example: Optimal algorithm of 3-SUM is O( 3 ) based on brute force solution. (i.e. its order of growth is 3 or less) Open questions What is the order of growth of the optimal solution for 3-SUM? Equivalent question: If it is Θ(B(n)) in the worst case, what is B(n)? We know it is between and. Does there exist an algorithm with worst case run time better than? i.e. an algorithm that is better than Θ( ) in the worst case? Does there exist a way to provide a quadratic lower bound on 3SUM? i.e. can we prove that the optimal algorithm for 3-SUM is Ω( )? 64 63

17 Algorithm design approach To-within a constant factor Start. Develop an algorithm. Prove a lower bound. Gap? Lower the upper bound (discover a new algorithm). Raise the lower bound (more difficult). Golden Age of Algorithm Design (970s-Present*). Steadily decreasing upper bounds for many important problems. Many known optimal algorithms. Caveats. Overly pessimistic to focus on worst case? Can do better than to within a constant factor to predict performance. Worst case performance for optimal algorithm Asymptotic performance not always useful (e.g. matrix multiplication). O Ω 65 notation provides example shorthand for used to Tilde leading term ~ log 0 Big Theta asymptotic growth rate Θ( ) Big Oh Θ( ) and smaller O( ) Big Omega Θ( ) and larger Ω( ) ½ log log + 3 ½ Common practice. Analysis to within a constant factor. provide approximate model classify algorithms develop upper bounds Easy to be more precise. Use Tilde-notation instead of Big Theta (or Big Oh) log + 3 develop lower bounds 66 Order of growth isn t everything - Matrix multiplication year algorithm order of growth? brute force Strassen Pan Bini.780 Only faster for huge matrices!.4 AALYSIS OF ALGORITHMS 98 Schönhage.5 98 Romani Coppersmith-Winograd Strassen Coppersmith-Winograd Strother.3737 Algorithms ROBERT SEDGEWICK KEVI WAYE introduction empirical observations mathematical models order-of-growth classifications theory of algorithms memory 0 Williams.377 number of floating-point operations to multiply two -by- matrices Asymptotic performance not always useful (e.g. matrix multiplication). 67

18 Basics Typical memory usage for primitive types and arrays Bit. 0 or. Byte. 8 bits. Megabyte (MB). million or 0 bytes. Gigabyte (GB). hard drives IST billion or 30 bytes. 64-bit machine. We assume a 64-bit machine with 8 byte pointers. Can address more memory. Pointers use more space. most computer scientists some JVMs "compress" ordinary object pointers to 4 bytes to avoid this cost type bytes boolean byte char int 4 float 4 long 8 double 8 for primitive types type bytes char[] + 4 int[] double[] for one-dimensional arrays type bytes char[][] ~ M int[][] ~ 4 M double[][] ~ 8 M for two-dimensional arrays Typical memory usage for objects in Java Typical memory usage summary Object overhead. 6 bytes (+8 if inner class). Reference. 8 bytes. Padding. Each object uses a multiple of 8 bytes. Ex. A Date object uses 3 bytes of memory. Total memory usage for a data type value: Primitive type: 4 bytes for int, 8 bytes for double, Object reference: 8 bytes. Array: 4 bytes + memory for each array entry. Object: 6 bytes + memory for each instance variable + 8 bytes if inner class (for pointer to enclosing class). Padding: round up to multiple of 8 bytes. public class Date private int day; private int month; private int year;... object overhead day month year padding int values 6 bytes (object overhead) 4 bytes (padding) Shallow memory usage: Don't count referenced objects. Deep memory usage: If array entry or instance variable is a reference, add memory (recursively) for referenced object. 3 bytes 7 7

19 Typical memory usage for objects in Java Typical memory usage for objects in Java Total memory usage for a data type value: Primitive type: 4 bytes for int, 8 bytes for double, Object reference: 8 bytes. Array: 4 bytes + memory for each array entry. Object: 6 bytes + memory for each instance variable + 8 bytes if inner class (for pointer to enclosing class). Padding: round up to multiple of 8 bytes. Total memory usage for a data type value: Primitive type: 4 bytes for int, 8 bytes for double, Object reference: 8 bytes. Array: 4 bytes + memory for each array entry. Object: 6 bytes + memory for each instance variable + 8 bytes if inner class (for pointer to enclosing class). Padding: round up to multiple of 8 bytes. public class String private char[] value; private int offset; private int count; private int hash;... object overhead value offset count hash padding reference int values 6 bytes (object overhead) 8 bytes (reference to array) + 4 bytes (char[] array) 4 bytes (padding) public class String private char[] value; private int offset; private int count; private int hash;... object overhead value offset count hash padding reference int values 6 bytes (object overhead) 8 bytes (reference to array) + 4 bytes (char[] array) 4 bytes (padding) Ex. A Java 6 string object uses ~ bytes (deep). Deep memory: + 64 bytes ~ bytes 73 Ex. A Java 6 string object uses 40 bytes (shallow memory). Shallow memory: 40 bytes 74 Example Turning the crank: summary Q. How much memory does WeightedQuickUnionUF use as a function of? Use tilde notation to simplify your answer. public class WeightedQuickUnionUF private int[] id; private int[] sz; private int count; public WeightedQuickUnionUF(int ) id = new int[]; sz = new int[]; for (int i = 0; i < ; i++) id[i] = i; for (int i = 0; i < ; i++) sz[i] = ;... A ~ 8 bytes. 6 bytes (object overhead) 8 + (4 + 4) each reference + int[] array 4 bytes (padding) Empirical analysis. Execute program to perform experiments. Assume power law and formulate a hypothesis for running time. Model enables us to make predictions. Mathematical analysis. Analyze algorithm to count frequency of operations. Use tilde notation or order of growth to simplify analysis. Model enables us to explain behavior. Scientific method. Mathematical model is independent of a particular system; applies to machines not yet built. Empirical analysis is necessary to validate mathematical models and to make predictions