CSE 101. Algorithm Design and Analysis Miles Jones Office 4208 CSE Building Lecture 2: Asymptotics

Transcription

1 CSE 101 Algorithm Design and Analysis Miles Jones Office 4208 CSE Building Lecture 2: Asymptotics

2 A BETTER APPROACH? function fib2(n) if n = 0 then return 0 create array f[0 n] f[0] := 0 f[1] := 1 for i = 2 n: f[i]:=f[i 1] + f[i 2] return f[n] The for loop consists of a single computer step so in order to compute f[n], you need n computer steps!!!!! This is a huge improvement

3 EXPONENTIAL VS POLYNOMIAL Polynomial Exponential Θ(1) Θ(log(n)) Θ(n) Θ(nlog(n)) Θ(n^2) Θ(n^k) Θ(2^n) Θ(n!)

4 BASIC COMPUTER STEPS Primitives: Branching Storing Comparing Simple addition Array look up

5 ADDITION AS A COMPUTER STEP Adding small numbers together should be one computer step. Adding n bit numbers in general takes Θ(n) steps if n is large (like 1000 digits long).

6 RUNTIME OF FIB1 AND FIB2 The nth Fibonacci number is F(n)<2^n so it has less than n bits and the procedure fib1 takes O(2^n) additions and each addition takes O(n) time so the runtime of fib1 is O(n2^n). the procedure fib2 takes O(n) additions and each addition takes O(n) time so the runtime of fib2 is O(n*n).

7 AMBIGUITY IN COMPUTER STEPS. We say that the time it takes to execute fib2 is proportional to n^2. What does that mean?? 2n^2? 3n^2?100n^2? May depend on the computer.

8 THOUGHT EXPERIMENT Suppose there was an alien species that was way more advanced than us. They had computers that could run 1 billion times faster than our computers. We were racing to see who could figure out how the planets in our solar system interacted. Our algorithm takes O(n 2 ) operations and their algorithm Takes O(2 n ). More specifically, our algorithm takes n 2 computer steps and theirs takes 2 n computer steps.

9 THOUGHT EXPERIMENT Let s say our computers can do 1 computer step in 1 nanosecond. That means the aliens can do 1 million computer steps in 1 nanosecond or 1 in 10 9 nanoseconds. That means that to figure out the interaction within our solar system, Our computer would take = 81 nanoseconds and theirs would take = nanoseconds. There is a point where our computers would run faster than the aliens. How many planets must there be for this to happen?

10 BIG-O BIG-Θ BIG-Ω f(n) = O(g(n)) means that There exists a constant c such that f(n) cg(n) for all n large enough. f(n) = Ω(g(n)) means that g(n) = O(f(n)) f(n) = Θ(g(n)) means that f(n) = O(g(n)) and g(n) = O(f(n))

11 QUICK RULES FOR BIG-O Any polynomial is Big-O of its highest power Exponentials dominate polynomials. If f/g goes to 0 then f=o(g) If f/g goes to infinity then f=ω(g) If f/g goes to c>0 then f=θ(g), f=o(g) and f=ω(g)

12 GRAPHS

13 GRAPHS VT NH ME MA CT RI Graphs: collections of vertices and edges

14 UNDIRECTED GRAPHS G=(V,E) where V is a set of vertices and E is a set of edges or pairs of vertices. VT NH ME G(new England)= ({CT,MA,ME,NH,RI,VT}, {(CT,MA),(CT,RI),(MA,NH),(MA,RI),(MA,VT),(ME,NH),(NH,VT)}) MA CT RI

15 DIRECTED GRAPHS G=(V,E) where V is a set of vertices and E is a set of edges or ordered pairs of vertices. Me Mom Aunt Dad Bro Any undirected graph can be thought of as a directed graph with each edge being counted twice, i.e. (x,y),(y,x) A graph is simple if it does not have double edges or self-loops.

16 ADJACENCY MATRIX VT NH ME MA CT RI If your graph is dense then you need close to this amount of memory anyway so it works well for dense graphs

17 CT: MA, RI MA: CT, NH, RI, VT ME: NH NH: ME, VT RI: CT, MA VT: MA,NH ADJACENCY LIST VT CT NH MA RI ME PROS: Takes up only O( E ) memory CONS: Takes O( V ) time to look up an edge If your graph is sparse then E = O( V ) and so the amount of memory used is on the same order as the number of vertices.

18 ADJACENCY MATRIX AND LIST FOR DIRECTED GRAPHS Me Mom Aunt Me Mom Bro Aunt Dad Me Mom Bro Aunt Dad Dad Me: Mom, Bro Mom: Me, Bro, Aunt Bro: Mom Aunt: Dad: Me Bro

19 CLASS CONVENTIONS (IMPORTANT!!) (Unless otherwise stated:) When I say graph I mean to say simple graph That means that in this class no graph has self loops or double edges. All of our graphs will come in Adjacency list format.

20 TYPES OF GRAPHS

21 SPARSE AND DENSE For every undirected simple graph, 0 E For every directed simple graph, 0 E V 2 V 2 If E is close to V 2 then the graph is called dense Otherwise it is called sparse.

22 ADJACENCY LIST Adjacency Lists can be implemented in various ways including arrays and linked lists. In this class, I want you to think of an adjacency list as an array of arrays. a:[b,e,f] b:[a,c] c:[b,d,f] d:[c,g] e:[a,i] f:[a,c,j] g:[d,h,j] h:[g,o] i:[e,j,m,n] j:[f,g,i] k:[g,o] l:[p] m:[i] n:[i,o] o:[h,k,n,p] p:[l,o] [[b,e,f],[a,c],[b,d,f],[c,g],[a,i],[a,c,j],[d,h,j],[g,o],[e,j,m,n],[f,g,i],[g,o],[p],[i],[i,o],[h,k,n,p],[l,o]]

23 DEPTH FIRST SEARCH (DFS) What vertices are reachable from a given starting point? Think of a robot bird who is trying to find its way to a tree. How would you program the robot to find the tree? What if you wanted to know all the intersections the bird could reach?

24 PATH Definition of a path: A path is a sequence of vertices and edges: v 1, e 1, v 2, e 2 v n 1, e n 1, v n Such that e i = v i, v i+1 When we say path in this class, we are talking about simple paths which means that no two edges are the same. Note that a single vertex v 1 is a trivial path from the vertex to itself.

25 DFS (CONNECTEDNESS) The goal of Depth-First-Search is to start at a vertex s and determine for each vertex v if there exists a path from s to v. This property is called connectedness: Two vertices s and v in an undirected graph G are called connected if there exists a path from s to v.

26 BALL OF STRING AND PIECE OF CHALK

27 CHALK AND STRING The bird could unravel the string until it reaches a dead end or an intersection with some chalk on it. When it reaches a dead end, it reravels the string until it gets back to the previous intersection and goes to the next unexplored path.

28 CHALK/STRING IN A COMPUTER How would we simulate these actions of raveling and unraveling on a computer? The intersections are vertices and paths are edges. Chalk Label each vertex with a Boolean variable indicating if it has been visited or not String

29 CHALK/STRING IN A COMPUTER How would we simulate these actions of raveling and unraveling on a computer? The intersections are vertices and paths are edges. Chalk Label each vertex with a Boolean variable indicating if it has been visited or not String Use a stack. A stack is a data structure that you can add to and take away from The rule is that the last object in is the first object out

30 What is important to us is that each action push and pop each take a constant amount of computer steps. STACKS

31 EXPLORE Let us consider the simple algorithm that finds each vertex reachable from a given vertex. procedure explore(g = (V,E), v) visited(v)=true for each edge (v,u): if not visited(u), then explore( G,u) This is a recursive algorithm in that it calls upon itself. Does it work? Does it terminate? How long does it take?

32 DOES IT WORK? If it doesn t work then there is a vertex u that is reachable from v but does not become true. procedure explore(g = (V,E), v) visited(v)=true for each edge (v,u): if not visited(u), then explore( G,u) let w be the first reachable vertex that is not true and so z is the last reachable vertex that is true.

33 DOES IT WORK? At some point, z was set to true and then you loop through all vertices including w. procedure explore(g = (V,E), v) visited(v)=true for each edge (v,u): if not visited(u), then explore( G,u) w has not been set to true so explore(g,w) and set w to true.

34 DOES IT TERMINATE? Does it terminate? procedure explore(g = (V,E), v) visited(v)=true for each edge (v,u): if not visited(u), then explore( G,u) There are only a finite number of vertices and you only call on explore if a vertex has not been visited.

35 HOW LONG DOES IT TAKE? Worst case scenario, you pass through all vertices and set them to true which takes O( V ) then after you set it to true you look at each edge incident with it which will take O(deg(v)) for each vertex v. procedure explore(g = (V,E), v) visited(v)=true for each edge (v,u): if not visited(u), then explore( G,u) What is deg (v) v V

36 HOW LONG DOES IT TAKE? Worst case scenario, you pass through all vertices and set them to true which takes O( V ) then after you set it to true you look at each edge incident with it which will take O(deg(v)) for each vertex v. procedure explore(g = (V,E), v) visited(v)=true for each edge (v,u): if not visited(u), then explore( G,u) What is σ v V deg (v) = 2 E for undirected σ v V outdeg(v) = E for directed

37 HOW LONG DOES IT TAKE? So the procedure assigns true to V vertices and checks E edges when it has visited all vertices. procedure explore(g = (V,E), v) visited(v)=true for each edge (v,u): if not visited(u), then explore( G,u) This algorithm has a runtime of O( V + E ). For graph algorithms, this is called linear time, since the runtime grows proportional to the number of vertices and edges.

38 HOW LONG DOES IT TAKE? Note that procedure explore(g = (V,E), v) visited(v)=true for each edge (v,u): if not visited(u), then explore( G,u) O( V )<O( V + E )<O( V ^2) depending on if the graph is sparse or dense.