Chapter 1 Divide and Conquer Algorithm Theory WS 2015/16 Fabian Kuhn
Divide And Conquer Principle Important algorithm design method Examples from Informatik 2: Sorting: Mergesort, Quicksort Binary search can be considered as a divide and conquer algorithm Further examples Median Compairing orders Delaunay triangulation / Voronoi diagram Closest pairs Line intersections Polynomial multiplication / FFT... Algorithm Theory, WS 2015/16 Fabian Kuhn 2
Example 1: Quicksort l function Quick (: sequence): sequence; {returns the sorted sequence } begin if # 1 then return else { choose pivot element in ; partition into l with elements, and with elements return Quick( l ) Quick( ) end; Algorithm Theory, WS 2015/16 Fabian Kuhn 3
Example 2: Mergesort l sort recursively sort recursively l merge Algorithm Theory, WS 2015/16 Fabian Kuhn 4
Formulation of the D&C principle Divide and conquer method for solving a problem instance of size : 1. Divide : Solve the problem directly. : Divide the problem into subproblems of sizes 1,, ( 2). 2. Conquer Solve the subproblems in the same way (recursively). 3. Combine Combine the partial solutions to generate a solution for the original instance. Algorithm Theory, WS 2015/16 Fabian Kuhn 5
Analysis Recurrence relation: : max. number of steps necessary for solving an instance of size Special case:, cost for divide and combine: DC 1 2/2 DC Algorithm Theory, WS 2015/16 Fabian Kuhn 6
Analysis, Example Recurrence relation: 2 2, 1 Guess the solution by repeated substitution: Algorithm Theory, WS 2015/16 Fabian Kuhn 7
Analysis, Example Recurrence relation: Verify by induction: 2 2, 1 Algorithm Theory, WS 2015/16 Fabian Kuhn 8
Analysis, Example Recurrence relation: 2 2, 1 Guess the solution by drawing the recursion tree: Algorithm Theory, WS 2015/16 Fabian Kuhn 9
Comparing Orders Many web systems maintain user preferences / rankings on things like books, movies, restaurants, Collaborative filtering: Predict user taste by comparing rankings of different users. If the system finds users with similar tastes, it can make recommendations (e.g., Amazon) Core issue: Compare two rankings Intuitively, two rankings (of movies) are more similar, the more pairs are ordered in the same way Label the first user s movies from 1 to according to ranking Order labels according to second user s ranking How far is this from the ascending order (of the first user)? Algorithm Theory, WS 2015/16 Fabian Kuhn 10
Number of Inversions Formal problem: Given: array,,,, of distinct elements Objective: Compute number of inversions 0 Example: 4, 1, 5, 2, 7, 10, 6 Naive solution: Algorithm Theory, WS 2015/16 Fabian Kuhn 11
Divide and conquer l 1. Divide array into 2 equal parts l and 2. Recursively compute #inversions in l and 3. Combine: add #pairs l, such that l Algorithm Theory, WS 2015/16 Fabian Kuhn 12
Combine Step: Example Assume l and are sorted 3 5 8 13 14 18 24 25 30 6 7 9 19 21 23 28 32 33 Algorithm Theory, WS 2015/16 Fabian Kuhn 13
Combine Step Assume l and are sorted l Idea: Maintain pointers and to go through the sorted parts While going through the sorted parts, we merge the two parts into one sorted part (like in MergeSort) and we count the number of inversions between the parts Invariant: At each point in time, all inversions involving some element left of (in l ) or left of (in ) are counted and all others still have to be counted... Algorithm Theory, WS 2015/16 Fabian Kuhn 14
Combine Step Assume l and are sorted l Pointers and, initially pointing to first elements of l and If : is smallest among the remaining elements No inversion of and one of the remaining elements Do not change count If : is smallest among the remaining elements is smaller than all remaining elements in l Add number of remaining elements in l to count Increment point, pointing to smaller element Algorithm Theory, WS 2015/16 Fabian Kuhn 15
Combine Step Need sub sequences in sorted order Then, combine step is like merging in merge sort Idea: Solve sorting and #inversions at the same time! 1. Partition into two equal parts l and 2. Recursively compute #inversions and sort l and 3. Merge l and to sorted sequence, at the same time, compute number of inversions between elements in l and in Algorithm Theory, WS 2015/16 Fabian Kuhn 16
Analysis, Guessing Recurrence relation: Repeated substitution: 2 2, 1 Algorithm Theory, WS 2015/16 Fabian Kuhn 17
Analysis, Induction Recurrence relation: Verify by induction: 2 2, 1 Algorithm Theory, WS 2015/16 Fabian Kuhn 18
Geometric divide and conquer Closest Pair Problem: Given a set of points, find a pair of points with the smallest distance. Naive solution: Algorithm Theory, WS 2015/16 Fabian Kuhn 19
Divide and conquer solution 1. Divide: Divide into two equal sized sets l und. 2. Conquer: l mindist l mindist 3. Combine: l min l, l l, return min l,, l l l l Algorithm Theory, WS 2015/16 Fabian Kuhn 20
Divide and conquer solution 1. Divide: Divide into two equal sized sets l und. 2. Conquer: l mindist l mindist 3. Combine: l min l, l l, return min l,, l Computation of l : min l, l Algorithm Theory, WS 2015/16 Fabian Kuhn 21
Merge step 1. Consider only points within distance of the bisection line, in the order of increasing coordinates. 2. For each point consider all points with 3. There are at most 7 such points. Algorithm Theory, WS 2015/16 Fabian Kuhn 22
Combine step 4 2 3 1 l min l, Algorithm Theory, WS 2015/16 Fabian Kuhn 23
Implementation Initially sort the points in in order of increasing coordinates While computing closest pair, also sort according to coord. Partition into l and, solve and sort sub problems recursively Merge to get sorted according to coordinates Center points: points within distance min l, of center Go through center points in in order of incr. coordinates Algorithm Theory, WS 2015/16 Fabian Kuhn 24
Running Time Recurrence relation: 2 2, 1 Solution: Same as for computing number of number of inversions, merge sort (and many others ) log Algorithm Theory, WS 2015/16 Fabian Kuhn 25
Recurrence Relations: Master Theorem Recurrence relation, for Cases, log Ω,log Θ log,log Algorithm Theory, WS 2015/16 Fabian Kuhn 26