CSE 101 Algorithm Design and Analysis Miles Jones mej016@eng.ucsd.edu Office 4208 CSE Building Lecture 6: BFS and Dijkstra s
BREADTH FIRST SEARCH (BFS) Given a graph G and a starting vertex s, BFS computes distances from s to every other node. It keeps this information in an array dist. To do this, BFS computes distances layer by layer. It sets dist(s)=0 and the next layer is all the vertices adjacent to s. If v is adjacent to s then it sets dist(v)=1. Once it has assigned distance values 0,1,2, d. The nodes at distance d+1 are the undiscovered nodes adjacent to the d nodes.
STACK VERSUS QUEUE The main difference between DFS and BFS is that DFS uses a stack and BFS uses a queue. The queue gives us some extra information. The queue starts with just the node s, the only one that has distance 0. Then for each subsequent distance, there is a point when the queue contains all the nodes at distance d and nothing else. As these nodes are ejected, their undiscovered neighbors are the next nodes injected into the end of the queue.
BFS procedure BFS(G, s) Input: Graph G = (V,E), (directed or undirected) and a vertex s in V. Output: For all vertices u reachable from s, dist(u) is the distance from s to u. and for all vertices u not reachable from s, dist(u) = for each vertex u in V: dist(u)= dist(s) = 0 Q = [s] (queue just containing s) while Q is not empty u = eject(q) for all edges (u,v) in E if dist(v)= then inject(q,v) dist(v)=dist(u) + 1
EJECT (DEQUEUE) u = eject(q) sets u equal to the first entry of Q and then removes that value from Q. Example: If Q = [B,D,A] then u = eject(q) means that now u has the value B and Q = [D,A]
INJECT (ENQUEUE) inject(q,v) puts v at the end of the list Q. Example: if Q = [B,A,D] then inject(q,f) means that now Q = [B,A,D,F]
EXAMPLE C A F B E D
CORRECTNESS OF BFS We want to show that BFS assigns dist() correctly to all vertices reachable from s. Proof by induction on distance. Base Case: dist(s) = 0. Inductive hypothesis: For any vertex v that is distance k from s, dist(v) = k. Inductive step: Suppose u is distance k + 1 (WTS: BFS assigns dist(u) = k + 1.)
CORRECTNESS OF BFS (PROOF CONT.) Inductive Step: Suppose u is distance k + 1 from s. dist(u) starts at infinity There exists a vertex w such that w is distance k from s and ( w,u) is in E. In the last for loop when w is ejected, we encounter the edge ( w,u) and dist(u) is infinity so we inject u into Q and set dist(u) = dist(w) + 1 = k + 1.
RUNTIME Each vertex is initially set to infinity. Then each vertex reachable by s is put into Q and taken out of Q so there are V initializations and at most 2 V queue operations. In the while loop, it eventually visits all edges reachable from s (once in directed graphs and twice in undirected graphs.) so there are at most E, (2 E ) times it visits an edge. So, worst case it takes 3 V + 2 E = O V + E operations. procedure BFS(G,s) for each vertex u in V: distance(u)=i nfinity dist(s) = 0 Q = [s] (queue just containing s) while Q is not empty u = eject(q) for all edges (u,v) in E if dist(v)=infi nity then inject(q,v) dist(v)=dist(u) + 1
SHORTEST PATH TREE Is there a way to output the dist values and the shortest path tree? procedure BFS(G,s) for each vertex u in V: dist(u)=infini ty prev(u)=nil dist(s) = 0 Q = [s] (queue just containing s) while Q is not empty u = eject(q) for all edges (u,v) in E if dist(v)=infi nity then inject(q,v) dist(v)=dis t(u) + 1 prev(v)=u
BFS A B C D E H F I G J K L M
DFS VS BFS DFS gives info about the whole graph BFS gives info related to a given vertex within the graph BFS uses a queue DFS uses a stack BFS does not restart at other connected components since all vertices not connected to your starting vertex are distance infinity away.
GRAPHS WITH EDGE LENGTHS
EDGE LENGTHS (WEIGHTS) edges can be given values such as distance cost time we will denote the length (weight) of edge e = (u, v) as l e, l (u,v), l e, l(u, v)
BFS ON WEIGHTED GRAPHS. BFS only works to find shortest distances on graphs in which each edge has equal weight. A 3 B 2 C 1 1 D 4 E F
BFS ON WEIGHTED GRAPHS. On a graph G with integer edge lengths, form G by adding l e 1 many new vertices between u and v for every edge e = (u, v). Then run BFS on G. A 3 B 2 C A B C 1 1 D 4 E F D E F G G
A 0 1 B 2 C 4 3 5 6 D 7 8 9 E F EXAMPLE
PROBLEMS WITH THIS METHOD If the edge lengths (weights) are large integers then it is impractical. In this example with 10 vertices, we must add 1,783 more vertices!!!!!!!!
ALARM CLOCK. ORD To find the shortest path to every vertex starting from SAN, we set an alarm clock for each edge coming out of SAN. So, we set the alarm for DEN for 100 seconds and ORD for 250 seconds. We let time pass and wait for the next alarm to go off. The first alarm to go off is DEN. So we look at all outgoing edges from DEN. If the time it takes to get to a vertex is faster through DEN, then we replace the alarm. In this case, there are still 150 seconds remaining but there is an edge that is 100 seconds away so we reset the edge to ORD to be 200. SAN DEN
ALARM CLOCK ALGORITHM Set alarm clock for vertex s at time 0 If next alarm goes off at time T for a vertex u then distance(u) = T for each edge (u,v) in E if no alarm is set for v then set it for T + l(u,v) if there is an alarm for v that is later than T + l(u,v) then reset the alarm for the earlier time if there is an alarm for v that is earlier than T + l(u,v) then leave the alarm as it is.
HOW TO IMPLEMENT This alarm clock algorithm works great. There is a problem. How do we implement it and teach a computer to do it. (We must learn how to quickly add vertices and take them out of a structure depending on their alarm setting.) The structure type we want to use is called a priority queue.
PRIORITY QUEUE A priority queue is a data structure of a set of objects (vertices) along with key values for each object that can be changed (alarm settings.) And it can support the following operations. insert(h,u) deletemin(h) decreasekey(h,u) makequeue(s)
DIJKSTRA S SHORTEST PATH ALGORITHM procedure dijkstra(g,l,s) Input: a graph G = (V,E) (directed or undirected.) a set of edge lengths l = {l(e):e is in E} s is the starting vertex Output: for all vertices u reachable from s, dist(u) is set to the distance from s to u and for all vertices unreachable dist(u) is set to infinity.
DIJKSTRA S SHORTEST PATH ALGORITHM procedure dijkstra(g,l,s) for all u in V dist(u) := infinity prev(u) := nil dist(s) := 0 H := makequeue(v) while H is not empty u := deletemin(h) for all edges (u,v) in E if dist(v) > dist(u) + l(u,v) then dist(v):= dist(u) + l(u,v) prev(v):=u decreasekey(h,v)
DIJKSTRA S SHORTEST PATH ALGORITHM The dist() value refers to the current alarm setting. A value of infinity means that an alarm has not been set yet. The prev(u) is set to the vertex before u in the current shortest path to u from s. By following these prev values, we can reconstruct the shortest paths. Dijkstra s algorithm is the same as BFS but it uses a priority queue instead of a regular queue.
ORD DIJKSTRA EXAMPLE IAD DEN SAN DFW ATL
RUNTIME OF DIJKSTRA S procedure dijkstra(g,l,s) for all u in V dist(u) := infinity prev(u) := nil dist(s) := 0 H := makequeue(v) while H is not empty u := deletemin(h) for all edges (u,v) in E if dist(v) > dist(u) + l(u,v) then dist(v):= dist(u) + l(u,v) prev(v):=u decreasekey(h,v) makequeue(v) deletemin(h) decreasekey(h,v)
RUNTIME OF DIJKSTRA S procedure dijkstra(g,l,s) for all u in V dist(u) := infinity prev(u) := nil dist(s) := 0 H := makequeue(v) while H is not empty u := deletemin(h) for all edges (u,v) in E if dist(v) > dist(u) + l(u,v) then dist(v):= dist(u) + l(u,v) prev(v):=u decreasekey(h,v) deletemin V decreasekey E
ARRAY AS A PRIORITY QUEUE Array: an ordered list of vertices each with a key value. [key(a),key(b),key(c),key(d),key(e),key(f)] makequeue: deletemin: decreasekey:
ARRAY AS A PRIORITY QUEUE Dijkstra s takes time makequeue + deletemin V + decreasekey E If we use an array then it will take V + O V V + O 1 E = O V 2 + E = O V 2
BINARY HEAP F 6 D 8 A 10 A complete binary tree of objects (vertices) with the property that each key value of an object is less than the key value of its parent. B 10 H 8 E 12 K 12 J 11 C 12 L 15 Q 14 N 14 G 22 O 26 I 16 P 26 M 13 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 [F 6, D 8, A 10, B 10, H 8, E 12, K 12, J 11, C 12, L 15, Q 14, N 14, G 22, O 26, I 16, P 26, M 13 ]
BINARY HEAP 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 [F 6, D 8, A 10, B 10, H 8, E 12, K 12, J 11, C 12, L 15, Q 14, N 14, G 22, O 26, I 16, P 26, M 13 ] The Binary heap can be implemented with an array a[n] of vertices. The children of a i are a 2i and a 2i+1. The parent of a i is a i/2.
BINARY HEAP (DELETEMIN) F 6 D 8 A 10 The object with the minimum key value is guaranteed to be the root. Once you take it out, you must reorder the tree. You replace the root with the last object and let it trickle down. J 11 P 26 M 13 B 10 H 8 E 12 K 12 C 12 L 9 Q 14 N 14 G 22 O 26 I 16 [F 6, D 8, A 10, B 10, H 8, E 12, K 12, J 21, C 12, L 9, Q 14, N 14, G 22, O 26, I 16, P 26, M 13 ] 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 M 13
M 13 BINARY HEAP (DELETEMIN) F 6 D 8 A 10 The object with the minimum key value is guaranteed to be the root. Once you take it out, you must reorder the tree. You replace the root with the last object and let it trickle down. J 11 B 10 H 8 E 12 K 12 C 12 L 15 Q 14 N 14 G 22 O 26 I 16 P 26 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 [F M 136, D 8, A 10, B 10, H 8, E 12, K 12, J 21, C 12, L 9, Q 14, N 14, G 22, O 26, I 16, P 26 ]
BINARY HEAP (DELETEMIN) D 8 F 6 The object with the minimum key value is guaranteed to be the root. Once you take it out, you must reorder the tree. You replace the root with the last object and let it trickle down. J 11 B 10 A M 10 13 H 8 E 12 K 12 C 12 L 15 Q 14 N 14 G 22 O 26 I 16 P 26 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 [D 8, MD 138, A 10, B 10, H 8, E 12, K 12, J 21, C 12, L 9, Q 14, N 14, G 22, O 26, I 16, P 26 ]
BINARY HEAP (DELETEMIN) D 8 F 6 The object with the minimum key value is guaranteed to be the root. Once you take it out, you must reorder the tree. You replace the root with the last object and let it trickle down. J 11 H 8 A 10 B 10 M 13 E 12 K 12 C 12 L 15 Q 14 N 14 G 22 O 26 I 16 P 26 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 [D 8, H 8, A 10, B 10, MH 138, E 12, K 12, J 21, C 12, L 9, Q 14, N 14, G 22, O 26, I 16, P 26 ]
BINARY HEAP (DELETEMIN) When the last object is put in as the root, it may trickle down the entire length of the heap, the time taken is O(log n ) where n is the number of objects in the heap. When performing Dijkstra s algorithm, the number of objects in the heap is V so the time taken is O(log V ).
BINARY HEAP (DECREASEKEY) F 6 D 8 A 10 When you decrease a key, you may have to adjust the heap by having the decreased key object bubble up. B 10 H 8 E 12 K 12 J 11 C 12 L 15 Q 14 N 14 G 22 O 26 I 16 P 26 M 13 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 [F 6, D 8, A 10, B 10, H 8, E 12, K 12, J 11, C 12, L 15, Q 14, N 14, G 22, O 26, I 16, P 26, M 13 ]
BINARY HEAP (DECREASEKEY) F 6 D 8 A 10 When you decrease a key, you may have to adjust the heap by having the decreased key object bubble up. B 10 H 8 E 12 K 12 J 11 C 12 L 15 Q 14 N 14 G 22 O 8 I 16 P 26 M 13 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 [F 6, D 8, A 10, B 10, H 8, E 12, K 12, J 11, C 12, L 15, Q 14, N 14, G 22, O 26 8, I 16, P 26, M 13 ]
BINARY HEAP (DECREASEKEY) F 6 D 8 A 10 When you decrease a key, you may have to adjust the heap by having the decreased key object bubble up. B 10 H 8 E 12 O 8 J 11 C 12 L 15 Q 14 N 14 G 22 K 12 I 16 P 26 M 13 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 [F 6, D 8, A 10, B 10, H 8, E 12, KO 12 8, J 11, C 12, L 15, Q 14, N 14, G 22, K 12, I 16, P 26, M 13 ]
BINARY HEAP (DECREASEKEY) F 6 D 8 O 8 When you decrease a key, you may have to adjust the heap by having the decreased key object bubble up. B 10 H 8 E 12 A 10 J 11 C 12 L 15 Q 14 N 14 G 22 K 12 I 16 P 26 M 13 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 [F 6, D 8, AO 10 8, B 10, H 8, E 12, A 10, J 11, C 12, L 15, Q 14, N 14, G 22, K 12, I 16, P 26, M 13 ]
BINARY HEAP (DECREASEKEY) When the object decreases key, it may bubble up the entire heap, the time taken is O(log n ) where n is the number of objects in the heap. When performing Dijkstra s algorithm, the number of objects in the heap is V so the time taken is O(log V ).
BINARY HEAP makequeue: O( V ) deletemin: O(log V ). decreasekey: O(log V ) Dijkstra s takes time makequeue + deletemin V + decreasekey E If we use an array then it will take O( V ) + O log( V ) V + O log( V ) E = O( V + E )log( V ) )
ARRAY VS BINARY HEAP Array O( V 2 ) Binary Heap O( V + E )log( V ) ) Sparse graphs: E = O( V ) Dense graphs: E = O( V 2 )
DIJKSTRA S ALGORITHM WITH DIFFERENT PRIORITY QUEUES. Runtime of Array: O V 2 Runtime of Binary heap: O V + E log V Fibonacci Heap!!!: O V log V + E