CS : Data Structures

Transcription

1 CS : Data Structures Michael Schatz Oct 7, 2016 Lecture 15: More Machine Code Optimization ;-)

2 Assignment 5: Due Sunday Oct 10pm Remember: javac Xlint:all & checkstyle *.java & JUnit Solutions should be independently written!

3 Bit Hacking $ xxd -b -c 9 solutions.txt cut -f d ' ' : : : b: : d: : f: : : a: : c: : e: : : : a2: ab: b4: bd: c6: cf: d8: e1: ea: f3: fc:

4 Bit Hacking $ xxd -b -c 9 solutions.txt cut -f d ' ' : : : b: : d: : f: : : a: : c: : No peeking J e: : : : a2: ab: b4: bd: c6: cf: d8: e1: ea: f3: fc:

5 Midterm Review

6 Midterm Review *You are not responsible for knowing the k-d tree

7 ADT: Arrays n-3 n-2 n-1 a: t t t t t t get(2) put(n-2, X) Fixed length data structure Constant time get() and put() methods Definitely needs to be generic J

8 Accessing RAM myarray[0] 0x0000 0x0001 0x0002 0x0003 0x0004 0x0005 0x0006 0x0007 0x0008 0x0009 0x000A 0x000B 0x000C 0x000D 0x000E 0x000F 0x0010 0x0011 0x0012 0x0013 0x0014 0x0015 0x0016 0x0017 0x0018 0x0019 0x001A 0x001B 0x001C 0x001D 0x001E 0x001F Byte [] myarray = new myarray[5000]; // myarray now starts at 0x0004 myarray[5] => offset for myarray + 5 * (sizeof type) => 0x04 + 0x05 * 1 => movl 0x09, %eax

9 Array Growing If the array size starts at 1, how expensive will it be to grow to 1M if we copy one element at a time? M push()s will require a total of ,999 copies = 0.5MM steps! O(n 2 ) performance L 5 6

10 Array Doubling If the array size starts at 1, how expensive will it be to grow to 1M? How many doublings will it take? How many times will an item be copied? How many rounds of doubling? lg(1m) = 20 How many total copies? k = 1M Whats the total runtime for n pushes? O(n) J This single push was O(n), but the next O(n) pushes are O(1) J 32

11 Sums of Powers of Two , ,048, ,048,575

12 ADT: Linked List (Singly Linked List) Node Node Node Node Node list data 0 data 1 data 2 data n-2 data n-1 null next next next next next Variable length data structure Constant time to head of list Linear time seek

13 ADT: Linked List (Singly Linked List) Node Node Node Node Node list data 0 data 1 data 2 data n-2 data n-1 null next next next next next Variable length data structure Constant time to head of list Linear time seek

14 Doubly Linked List List Node Node Node Node first 1 next prev 2 next prev 3 next prev 4 next prev null last null Very similar to a singly linked list, except each node has a reference to both the next and previous node in the list A little more overhead, but significantly increased flexibility: supports insertfront, insertback, removefront, removeback, insertbefore, removemiddle

15 Stacks Stacks are very simple but surprisingly useful data structures for storing a collection of information Any number of items can be stored, but you can only manipulate the top of the stack: Push: adds a new element to the top Pop: takes off the top element Top: Lets you peek at top element s value without removing it from stack Many Applications In hardware call stack Memory management systems Parsing arithmetic instructions: ((x+3) / (x+9)) * (42 * sin(x)) Back-tracing, such as searching within a maze

16 ListStack vs ArrayStack ListStack Node Node Node Node head value next value next value next value next null ArrayStack int [] arr int top

17 ArrayStack versus ListStack ListStack has some nice properties: Unbounded: keep adding items to the stack until you run out of memory Push, pop, and top are all constant time (Pretty) Simple implementation But also has significant overhead: You may end up dedicating more memory to node references (8 bytes each) than to the values that you are storing (4 bytes per int, 1 byte for char, etc) The values may be distributed all over RAM which can incur a penalty on some hardware See computer architecture class for details ArrayStack has some nice properties: Minimal overhead to storing items: 8 bytes for a reference to the entire array with N elements (Pretty) simple implementation: pop just decrements one number, top is a quick array lookup But also has a significant challenge/limitation Sometimes push() may be impossible (throw a StackOverflow exception), or expensive (copy entire array to a new larger array)

18 Queue Applications Whenever a resource is shared among multiple jobs: accessing the CPU accessing the disk Fair scheduling (ticketmaster, printing) Whenever data is transferred asynchronously (data not necessarily received at same rate as it is sent): Sending data over the network Working with UNIX pipes:./slow./fast./medium Also many applications to searching graphs (see 3-4 weeks) FIFO: First-In-First-Out Add to back + Remove from front

19 ListQueue vs ArrayQueue ListQueue Node Node Node Node first value next value next value next value next null last ArrayQueue int [] arr int top Many of the same tradeoffs as ListStack vs ArrayStack

20 ArrayQueue ArrayQueue int [] arr int top = 1 1 queue.enqueue(1);

21 ArrayQueue ArrayQueue int [] arr int top = queue.enqueue(1); queue.enqueue(2);

22 ArrayQueue ArrayQueue int [] arr int top = queue.enqueue(1); queue.enqueue(2); queue.enqueue(3);

23 ArrayQueue ArrayQueue int [] arr int top = queue.enqueue(1); queue.enqueue(2); queue.enqueue(3); queue.enqueue(4);

24 ArrayQueue ArrayQueue int [] arr int top = queue.dequeue()

25 ArrayQueue ArrayQueue int [] arr int top = queue.dequeue()

26 ArrayQueue ArrayQueue int [] arr int top = Whats wrong with copying? How could we fix it?

27 ArrayQueue ArrayQueue int [] arr int f = 0 int b = 0 f b Use a separate index for the front and back of the queue We know the queue is empty when f == b

28 ArrayQueue ArrayQueue int [] arr int f = 0 int b = 1 1 f b queue.enqueue(1);

29 ArrayQueue ArrayQueue int [] arr int f = 0 int b = f b queue.enqueue(1); queue.enqueue(2);

30 ArrayQueue ArrayQueue int [] arr int f = 0 int b = f b queue.enqueue(1); queue.enqueue(2); queue.enqueue(3);

31 ArrayQueue ArrayQueue int [] arr int f = 0 int b = f b Notice: queuelen = b - f queue.enqueue(1); queue.enqueue(2); queue.enqueue(3); queue.enqueue(4);

32 ArrayQueue ArrayQueue int [] arr int f = 1 int b = f b queue.dequeue() Hooray, enqueue and dequeue are O(1) J We don t even need to clear out the old front of the list

33 ArrayQueue ArrayQueue int [] arr int f = 13 int b = f b What happens when we get to the end of the array? Queuelen = = 4 J

34 ArrayQueue ArrayQueue int [] arr int f = 13 int b = f b queue.enqueue(17); Should we double the array? Nah, the array is mostly empty. Lets use it up first!

35 ArrayQueue ArrayQueue int [] arr int f = 13 int b = b f queue.enqueue(17);

36 ArrayQueue ArrayQueue int [] arr int f = 13 int b = b f queue.enqueue(17); queue.enqueue(18);

37 ArrayQueue ArrayQueue int [] arr int f = 14 int b = b f queue.dequeue()

38 ArrayQueue ArrayQueue int [] arr int f = 15 int b = b f queue.dequeue() queue.dequeue()

39 ArrayQueue ArrayQueue int [] arr int f = 16 int b = b f queue.dequeue() queue.dequeue() queue.dequeue()

40 ArrayQueue ArrayQueue int [] arr int f = 0 int b = f b How can we implement the wrap around? queue.dequeue() queue.dequeue() queue.dequeue() queue.dequeue()

41 Mod.java Modular Arithmetic m = a % b means to set m to be the remainder when dividing a by b m is guaranteed to fall between 0 and b public class Mod { public static void main(string [] args){ System.out.println("i\ti%2\ti%5\ti%10\ti%16"); } } for (int i = 0; i < 20; i++) { System.out.println(i + "\t" + i % 2 + "\t" + i % 5 + "\t" + i % 10 + "\t" + i % 16); } back = (back + 1) % arr.length How do we compute length or know when it is full? Use a separate counter. When array is totally full, double the size and copy into new array starting at 0 $ java Mod i i%2 i%5 i%10 i%

42 Dequeues insertfront() insertback() front back removefront() removeback() Dynamic Data Structure used for storing sequences of data Insert/Remove at either end in O(1) If you exclusively add/remove at one end, then it becomes a stack If you exclusive add to one end and remove from other, then it becomes a queue Many other applications: browser history: deque of last 100 webpages visited

43 ListDeque vs ArrayDeque ListDeque Node Node Node Node first value next value next value next value next null last ArrayDeque int [] arr int f int b Many of the same tradeoffs as ListQueue vs ArrayQueue

44 ArrayDequeue ArrayDeque int [] arr int f = 0 int b = 0 f b deque = new ArrayDequeue();

45 ArrayDequeue ArrayDeque int [] arr int f = 0 int b = 1 1 f b deque = new ArrayDequeue(); deque.insertback(1);

46 ArrayDequeue ArrayDeque int [] arr int f = 0 int b = f b deque = new ArrayDequeue(); deque.insertback(1); deque.insertback(2);

47 ArrayDequeue ArrayDeque int [] arr int f = 0 int b = f b deque = new ArrayDequeue(); deque.insertback(1); deque.insertback(2); deque.insertfront(3);

48 ArrayDequeue ArrayDeque int [] arr int f = 16 int b = b f deque = new ArrayDequeue(); deque.insertback(1); deque.insertback(2); deque.insertfront(3);

49 ArrayDequeue ArrayDeque int [] arr int f = 15 int b = Inserting at front usually means subtract, but gets tricky when we wrap around. f = (f-1) % arr.length; // depends on how this is implemented for negative numbers f = (f -1+arr.length) % arr.length; // does the right thing b deque = new ArrayDequeue(); deque.insertback(1); deque.insertback(2); deque.insertfront(3); deque.insertfront(4); f

50 Advanced Data Structure #1: K-d tree Balanced Binary Search Tree invented by Jon Louis Bentley in 1975 Generalization of the ubiquitous binary search tree Very fast to build & search almost any type of spatial data

51 Complexity Analysis How long will the algorithm take when run on inputs of different sizes: If it takes X seconds to process 1000 items, how long will it take to process twice as many (2000 items) or ten times as many (10,000 items)? Generally looking for an order of magnitude estimate: Constant time O(1): It takes the same amount of time to do it for 1 or for 1 million items Example: Getting a value from any position in an array Linear time O(n): If you double the number of elements it will take twice as long Example: scanning a linked list Logarithmic time O(lg(n)): Doubling the number of elements only makes it a tiny bit harder because the total number of steps is log(n) Example: Finding the nearest Pokemon stop using a k-d tree Also very important for space characterization: Sometimes doubling the number of elements will more than double the amount of space needed

52 FindMax Analysis public static int findmaximum(int [] myarray) { int max = myarray[0]; for (int i = 1; i < myarray.length; i++) { if (myarray[i] > max) { max = myarray[i]; } } } return max; $ java ArrayFind Scanning the array of size: 10,000,000 The max is: Search took: 11,666,963 nanoseconds $ java ArrayFind Scanning the array of size: 100,000,000 The max is: Search took: 71,270,945 nanoseconds Why isnt ArrayFind 100M exactly 10 times longer than ArrayFind 10M?

53 FindMax Analysis public static int findmaximum(int [] myarray) { int max = myarray[0]; for (int i = 1; i < myarray.length; i++) { if (myarray[i] > max) { max = myarray[i]; } } } return max; How many comparisons are done? i < myarray.length n myarray[i] > max n-1 C(n) = 2n-1 How many assignments are done (worst case)? max = myarray[0] 1 for i =1; i < myarray.length; i++ n val = myarray[i] n-1 max = myarray[i] n-1 A(n) = 1+ n + 2(n-1) = 3n-1

54 FindMax Analysis public static int findmaximum(int [] myarray) { int max = myarray[0]; for (int i = 1; i < myarray.length; i++) { if (myarray[i] > max) { max = myarray[i]; } } } return max; What is the total amount of work done? T(n) = C(n) + A(n) = (2n-2) + (3n 1) = 5n-3 Should we worry about the -3? Nah, for sufficiently large inputs will make a tiny difference Should we worry about the 5n? Nah, the runtime is linearly proportional to the length of the array

55 Big-O Notation Formally, algorithms that run in O(X) time means that the total number of steps (comparisons and assignments) is a polynomial whose largest term is X, aka asymptotic behavior f(x) O(g(x)) if there exists c > 0 (e.g., c = 1) and x 0 (e.g., x 0 = 5) such that f(x) cg(x) whenever x x 0 T(n) = 33 => O(1) T(n) = 5n-2 => O(n) T(n) = 37n n 8 => O(n 2 ) T(n) = 99n n n + 2 => O(n 3 ) T(n) = 127n log (n) + log(n) + 16 => O(n lg n) T(n) = 33 log (n) + 8 => O(lg n) T(n) = 900*2 n + 12n n + 54 => O(2 n ) Informally, you can read Big-O(X) as On the order of X O(1) => On the order of constant time O(n) => On the order of linear time O(n 2 ) => On the order of quadratic time O(n 3 ) => On the order of cubic time O(lg n) => On the order of logarithmic time O(n lg n) => On the order of n log n time

56 Growth of functions x^2 5x x A quadratic function isnt necessarily larger than a linear function for all possible inputs, but eventually will be x That largest term defines the Big-O complexity

57 Growth of functions

58 Trying every possible permutation Growth of functions Trying every possible subset Processing every element in a square array, Comparing every element to every other element Finding the nearest Pokemon stop from every other stop with a k-d tree Linear Search Finding an element in a balanced tree, Simple arithmetic, simple comparison, array access

59 Part 2: Midterm Review!

60 Next Steps 1. Reflect on the magic and power of Sentinels! 2. Work on Assignment 5: Due Sunday Oct 10:00 pm 3. Start to review for Midterm on Monday Oct Your notes from class 2. Lecture Notes on Piazza 3. Slides on course webpage 4. Online & printed textbooks 5. Sample Midterm!!!

61 Welcome to CS Questions?