Additional Divide and Conquer Algorithms. Skipping from chapter 4: Quicksort Binary Search Binary Tree Traversal Matrix Multiplication

Transcription

1 Aitional Divie an Conquer Algorithms Skipping from chapter 4: Quicksort Binary Search Binary Tree Traversal Matrix Multiplication Divie an Conquer Closest Pair Let s revisit the closest pair problem. Last time we looke at a brute force solution that ran in O(n 2 ) time. Let P 1 = (x 1, y 1 ), P n = (x n, y n ) be a set S of n points in the plane, where n, for simplicity is a power of two. We can sort these points in ascening orer of their x coorinate using a O(nlgn) sorting algorithm such as mergesort. Next we ivie the points into two subsets S 1 an S 2 where each have n/2 points by rawing a line through the meian of all points. We can fin the meian in O(n) time. This is shown below: S 1 S 2 Next, we recursively fin the closest pairs for the left subset S 1 an for the right subset S 2. If the set consists of just two points, then there is only one solution (that connects those two points). Let 1 an 2 be the smallest istances between pairs of points in S 1 an S 2 as shown below: 1 2

2 Let = the minimum of ( 1 an 2 ). This is not necessarily our answer, because the shortest istance between points might connect a pair of points on opposite sies of the line. Consequently, we must compare to the pairs that cross the line. We can limit our attention to points in the symmetrical vertical strip of with 2 since the istance between any other pair of points is greater than : 1 2 For every point between the left ashe line an the symmetrical line in the mile we must inspect the istance to points on the right sie of the line to the right ashe line. However, we only nee to look at those points that are within a istance of from the current point. The key here is that there can be no more than six such points because any pair of points in the right half is at least apart from each other. The worst case is shown below: If we maintain an aitional list of the points sorte by y coorinate then we can limit the points we examine to +/- in both the x an y irection for these borer points. To perform the step of comparing istances between points on both sies of the line requires O(n) runtime. For each of up to n points we have a constant number (up to 6) of other points to examine. This process is similar to the merge time require for MergeSort. We then compare the smallest of these istances to, an choose the smallest istance as the solution to the problem.

3 Our total runtime is then: T(n) = 2T(n/2) + O(n) Time to split in half plus line comparison We can solve this using a variety of methos as O(nlgn). This is the best we can o as an efficiency class, since it has been proven that this problem is Ω(nlgn). Divie an Conquer Convex Hull Problem Earlier we saw a brute force O(n 3 ) algorithm to fin the convex hull. Let s look at an expecte O(nlgn) algorithm calle QuickHull that is somewhat similar to Quicksort. Once again, we have a set of points P locate in a 2D plane. First, sort the points in increasing orer of their x coorinate. This can be one in O(nlgn) time. It shoul be obvious that the leftmost point P 1 an the rightmost point P n must belong to the set s convex hull. Let P 1 P n be the line rawn from P 1 to P n. This line separates P into two sets, S 1 to the left of the line, an S 2 to the right of the line. (left is counter clockwise when connecting the points). S 1 constitutes the bounary of the upper convex hull an S 2 the bounary of the lower convex hull: S 1 P n P 1 S 2 If we re lucky, the line exactly separates the points in half, so half are in S 1 an half are in S 2. In the worst case, all other points are in S 1 or S 2. If there is some ranomness the on general we can expect to cut the problem close to in half as we repeat the process. Next we ll fin the convex hull for S 1. We can repeat the same exact process to solve S 2. To fin the convex hull for S 1, we fin the point in S 1 that is farthest from the line segment P 1 P n. Let s say this point is P max. If we examine the line segment from P 1 to P max then we can recursively repeat the algorithm for all points to the left of this line (The set S 11 in the iagram below). Similarly, if we examine the line segment from P n to P max then we can recursively repeat the algorithm for all points to the right of this line (The set S 12 in the iagram below). All points insie the triangle can be iscare, since they are in the triangle an can t be part of the convex hull.

4 P max S 12 S 1 P n S 11 P 1 S 2 If we try to fin the points in the convex hull for a set with only one point, then that point must be in the set. At this point we have etermine the upper convex hull: S 12 S 1 P n P 1 S 2 If we repeat the process on the lower convex hull we will complete the problem an fin all of the points for the entire convex hull. For an animate applet that illustrates this algorithm, see: What is the runtime? Let s say that we always split the set of points in half each time we raw the line segment from P 1 to P n. This means we split the problem up into two subproblems of size n/2. Fining the most istant point from the line segment takes O(n) time. This leas to our familiar recurrence relation of: T(n) = 2T(n/2) + O(n) which is O(nlgn).

5 Decrease an Conquer The ecrease an conquer technique is similar to ivie an conquer, except instea of partitioning a problem into multiple subproblems of smaller size, we use some technique to reuce our problem into a single problem that is smaller than the original. The smaller problem might be: Decrease by some constant Decrease by some constant factor Variable ecrease Decrease an conquer is also referre to as inuctive or incremental approach. Here are some examples of Decrease an Conquer algorithms: Decrease by one: Insertion sort Graph search algorithms: DFS BFS Topological sorting Algorithms for generating permutations, subsets Decrease by a constant factor Binary search Fake-coin problems Josephus problem Variable-size ecrease Eucli s algorithm : gc(m,n) = gc(n, m mo n) Selection by partition Review of Basic Graph Algorithms A graph is compose of eges E an vertices V that link the noes together. A graph G is often enote G=(V,E) where V is the set of vertices an E the set of eges. Two types of graphs: 1. Directe graphs: G=(V,E) where E is compose of orere pairs of vertices; i.e. the eges have irection an point from one vertex to another. 2. Unirecte graphs: G=(V,E) where E is compose of unorere pairs of vertices; i.e. the eges are biirectional.

6 Directe Graph: Unirecte Graph: The egree of a vertex in an unirecte graph is the number of eges that leave/enter the vertex. The egree of a vertex in a irecte graph is the same,but we istinguish between in-egree an out-egree. Degree = in-egree + out-egree. The running time of a graph algorithm expresse in terms of E an V, where E = E an V= V ; e.g. G=O(EV) is E * V We can implement a graph in three ways: 1. Ajacency List 2. Ajacency-Matrix 3. Pointers/memory for each noe (actually a form of ajacency list) Using an Ajacency List: List of pointers for each vertex A 1 2 3

7 A The sum of the lengths of the ajacency lists is 2 E in an unirecte graph, an E in a irecte graph. The amount of memory to store the array for the ajacency list is O(max(V,E))=O(V+E).

8 Using an Ajacency Matrix: When is using an ajacency matrix a goo iea? A ba iea? The matrix always uses Θ( v 2 ) memory. Usually easier to implement an perform lookup than an ajacency list. Sparse graph: very few eges. Dense graph: lots of eges. Up to O(v 2 ) eges if fully connecte. The ajacency matrix is a goo way to represent a weighte graph. In a weighte graph, the eges have weights associate with them. Upate matrix entry to contain the weight. Weights coul inicate istance, cost, etc. Search: The goal is to methoically explore every vertex an every ege; perhaps to o some processing on each. Will assume ajacency-list representation of the input graph.

9 Breath-First-Search (BFS) : Example 1: Binary Tree. This is a special case of a graph. The orer of search is across levels. The root is examine first; then both chilren of the root; then the chilren of those noes, etc Sometimes we can stop the algorithm if we are looking for a particular element, but the general BFS algorithm runs through every noe. Example 2: irecte graph: 1. Pick a source vertex S to start. 2. Fin (or iscover) the vertices that are ajacent to S. 3. Pick each chil of S in turn an iscover their vertices ajacent to that chil. 4. Done when all chilren have been iscovere an examine. This results in a tree that is roote at the source vertex S. The iea is to fin the istance from some Source vertex by expaning the frontier of what we have visite. Pseuocoe: Uses FIFO Queue Q BFS(s) ; s is our source vertex for each u V - {s} ; Initialize unvisite vertices to o [u] [s] 0 ; istance to source vertex is 0 Q {s} ; Queue of vertices to visit while Q<>0 o remove u from Q for each v Aj[u] o ; Get ajacent vertices if [v]= then [v] [u]+1 ; Increment epth put v onto Q ; A to noes to explore

10 Example: (this is the final state, start with 0 an infinity as values) a b 2 c f 2 e i 3 g 3 3 h Initially, [a] is set to 0 an the rest to. Q [a]. Remove hea: Q [] chilren of a are c,b [c]=, [b]= so [c] [a]+1=1, [b] [a]+1=1 Q [c b] Remove hea: Q [b] chilren of c are e,f [e]=, [f]= so [e] [c]+1=2, [f] [c]+1=2 Q [b e f] Remove hea: Q [e f] chilren of b is f [f] <>, nothing one with it Remove hea: Q [f] chilren of e is, i, h []=, [i]=, [h]= so [] [i] [h] [e]+1=3 Q [f i h] Remove hea: Q [ i h] chilren of is g [g]=, so [g] []+1 = 3 Q [ i h g] Each of these has chilren that are alreay has a value less than, so these will not set any further values an we are one with the BFS. Can create a tree out of the orer we visit the noes:

11 a 0 b c 1 e f 2 h i g 3 Memory require: Nee to maintain Q, which contains a list of all fringe vertices we nee to explore. Runtime: O(V+E) ; O(E) to scan through ajacency list an O(V) to visit each vertex. This is consiere linear time in the size of G. Claim: BFS always computes the shortest path istance in [I] between S an vertex I. We will skip the proof for now. What if some noes are unreachable from the source? (reverse c-e,f-h eges). What values o these noes get? Depth First Search: Another metho to search graphs. Example 1: DFS on binary tree. Specialize case of more general graph. The orer of the search is own paths an from left to right. The root is examine first; then the left chil of the root; then the left chil of this noe, etc. until a leaf is foun. At a leaf, backtrack to the lowest right chil an repeat Example 2: DFS on irecte graph. 1. Start at some source vertex S. 2. Fin (or explore) the first vertex that is ajacent to S. 3. Repeat with this vertex an explore the first vertex that is ajacent to it.

12 Pseuocoe: 4. When a vertex is foun that has no unexplore vertices ajacent to it then backtrack up one level 5. Done when all chilren have been iscovere an examine. Results in a forest of trees. DFS(s) for each vertex u V o color[u] White time 1 for each vertex u V o if color[u]=white then DFS-Visit(u,time) DFS-Visit(u,time) color[u] Gray [u] time time time+1 for each v Aj[u] o if color[u]=white then DFS-Visit(v,time) color[u] Black f[u] time time+1 ; not visite ; time stamp ; in progress noes ; =iscover time ; f=finish time Example: 1/18 a 2/17 10/11 b 5/6 c 3/16 f e 9/14 12/13 i g 4/7 8/15 h Numbers are Discover/Finish times. We coul have ifferent visit times epening on which eges we pick to traverse uring the DFS. The tree built by this search looks like:

13 a c f g h b e i What if some noes are unreachable? We still visit those noes in DFS. Consier if c-e, f-h links were reverse. Then we en up with two separate trees: 1/10 a 2/9 11/12 b 5/6 c 3/8 f e 13/18 15/16 i g 4/7 14/17 h Still visit all vertices an get a forest: a set of unconnecte graphs without cycles (a tree is a connecte graph without cycles). Time for DFS: O(V 2 ) - DFS loop goes O(V) times once for each vertex (can t be more than once, because a vertex oes not stay white), an the loop over Aj runs up to V times. But The for loop in DFS-Visit looks at every element in Aj once. It is charge once per ege for a irecte graph, or twice if unirecte. A small part of Aj is looke at uring each recursive call but over the entire time the for loop is execute only the same number of times as the size of the ajacency list which is Θ (E). Since the initial loop takes Θ (V) time, the total runtime is Θ (V+E). This is consiere linear in terms of the size of the input ajacency-list representation. So if there are lots of eges then E ominates the runtime, otherwise V oes.

14 Note: Don t have to track the backtracking/fringe as in BFS since this is one for us in the recursive calls an the stack. The stack makes the noes orere LIFO. The amount of storage neee is linear in terms of the epth of the tree. Types of Eges: There are 4 types. DFS can be moifie to classify eges as being of the correct type: 1. Tree Ege: An ege in a epth-first forest. Ege(u,v) is a tree ege if v was first iscovere from u. 2. Back Ege: An ege that connects some vertex to an ancestor in a epth-first tree. Self-loops are back eges. 3. Forwar Ege: An ege that connects some vertex to a escenant in a epth-first tree. 4. Cross Ege: Any other ege. DAG s Nothing to o with sheep! A DAG is a Directe Acyclic Graph. This is a irecte graph that contains no cycles. Examples: a c e b f A irecte graph D is acyclic iff a DFS of G yiels no back eges. Proof: Trivial. Acyclic means no back ege because a back ege makes a cycle. Suppose we have a back ege (u,v). Then v is an ancestor of u in the epth-first forest. But then there is alreay a path from v to u an the back ege makes a cycle. Suppose G has a cycle c. But then DFS of G will have a back ege. Let v be the first vertex in c foun by DFS an let u be a vertex in c that is connecte back to v. Then

15 when DFS expans the chilren of u, the vertex v is foun. Since v is an ancestor of u the ege (u,v) is a back ege. v u Topological Sort of a ag A topological sort of a ag is an orering of all the vertices of G so that if (u,v) is an ege then u is liste (sorte) before v. This is a ifferent notion of sorting than we are use to. a,b,f,e,,c an f,a,e,b,,c are both topological sorts of the above ag. There may be multiple sorts; this is okay since a is not relate to f, either vertex can come first. Main use: Inicate orer of events, what shoul happen first Algorithm for Topological-Sort: 1. Call DFS(G) to compute f(v), the finish time for each vertex. 2. As each vertex is finishe insert it onto the front of the list. 3. Return the list. Time is Θ (V+E), time for DFS. Example: Pizza irecte graph 1/12 sauce 2/3 bake 4/5 cheese 6/11 sausage 7/10 8/9 oregano olives crust 13/14 DFS: Start with sauce. The numbers inicate start/finish time. We insert into the list in reverse orer of finish time. Crust, Sauce, Sausage, Olives, Oregano, Cheese, Bake

16 Why oes this work? Because we on t have any back eges in a ag, so we won t return to process a parent until after processing the chilren. We can orer by finish times because a vertex that finishes earlier will be epenent on a vertex that finishes later.