Welcome to the course Algorithm Design Summer Term 2011 Friedhelm Meyer auf der Heide Lecture 13, 15.7.2011 Friedhelm Meyer auf der Heide 1
Topics - Divide & conquer - Dynamic programming - Greedy Algorithms - Randomized Algorithms - Approximation Algorithms - Online Algorithms Friedhelm Meyer auf der Heide 2
Chapter 1: Divide & Conquer Friedhelm Meyer auf der Heide 3
Divide & Conquer Divide the problem in a number of smaller subproblems. Conquer the subproblems by solving them recursively. Combine the results of the subproblem into a solution of the original problem. Our Examples: Integer multiplication, matrix multiplication (HW), closest pair of points in the plane, Fast Fourier transform,.(hw) Friedhelm Meyer auf der Heide 4
Chapter 2: Dynamic Progamming Friedhelm Meyer auf der Heide 5
Dynamic programming You probably know dynamic programming algorithms for optimal evaluation of the product of many matrices, and for the word problem for context-free languages (CYK-algorithm) Examples: String Distance, Shortest Paths, Knapsack, Minimum Weight Triangulation (HW), Optimal Search Trees (HW) Friedhelm Meyer auf der Heide 6
Friedhelm Meyer auf der Heide 7
Friedhelm Meyer auf der Heide 8
Friedhelm Meyer auf der Heide 9
Friedhelm Meyer auf der Heide 10
Chapter 3: Greedy Algorithms Friedhelm Meyer auf der Heide 11
Greedy Algorithms Examples: Activity Selection (or Interval Scheduling), Scheduling to Minimize Lateness Theoretical Foundations of the Greedy Method: Matroid Theory Examples for Matroids: Linear independent vectors, Minimum Spanning Trees, Bipartite matching with maximum left side Friedhelm Meyer auf der Heide 12
For which problems are Greedy-Algorithms optimal? Consider a finite set E and a system U of subsets of E. (E,U) is called a subset-system, if the following holds: (i)? 2 U (ii) For each B 2 U, also each subset of B is in U. B2 U is called maximal, if no proper superset of B is in U. The Optimization problem corresponding to (E,U) is : Given a weight function w:e Q +,compute a maximal set B 2 U with maximizes w(b) = e2 B w(e). (Minimization problems are described analagously.) Friedhelm Meyer auf der Heide 13
Systems of Subsets and the Canonical Greedy Algorithm Canonical Greedy ((E,U)) (1) Sort E such that w(e 1 ) w(e n ). (2) B?. (3) For k=1 to n if B [ {e k } 2 U then B B [ {e k } (4) Return B For which types of subset-systems is the canonical Greedy algorithm optimal? Friedhelm Meyer auf der Heide 14
Example 2 Problem: Minimum spanning tree (MST) Input: a graph G=(V,E) with positive edge weights w:e Q +. Output: a spanning tree with minimum weight. MST is a subset-system: - E : the set of edges, - U : all subsets of E that do not contain a cycle form a subset-system Friedhelm Meyer auf der Heide 15
Greedy-Algorithms und Matroids A system of subsets (E,U) is a matroid, if in addition, the following exchange property holds: (iii) For all A,B 2 U with A < B, there is x2 B-A such that A[{x} 2 U. Remark: All maximal sets of a matroid have the same size. (homework) We will see: - The system of subsets for 0-1 knapsack (Example 1) is not a matroid. - The systems of subsets for examples 0,2,3 are a matroids. Theorem: Consider a system of subsets (E,U). canonical Greedy is optimal for (E,U) for every weight function w, if and only if (E,U) is a matroid. Kruskal s algorithm is optimal for MST (example 2)! Friedhelm Meyer auf der Heide 16
Chapter 4. Randomised Algorithms Friedhelm Meyer auf der Heide 17
Randomized Algorithms use the outcome of random experiments ( coin flips ) to guide the execution of the algorithm. For a fixed input, runtime and output may be random variables. Friedhelm Meyer auf der Heide 18
Randomized Algorithms Examples: Quicksort, Skip Lists, Contention Resolution (symmetry -breaking), Global Minimum Cut (contraction algorithm), Random Variables and their Expectations (Guessing Cards, Coupon Collector) Chernoff Bounds, example Load Balancing Friedhelm Meyer auf der Heide 19
Cuts in graphs Let G=(V,E) be a connected, undirected graph. A cut in G is defined by a disjoint partition of V into two sets A and B. Its size is the number of edges between A and B. Goal: Compute a cut with minimum size. A cut of size 6. A B Friedhelm Meyer auf der Heide 20
Friedhelm Meyer auf der Heide 21
Friedhelm Meyer auf der Heide 22
Tools from Probability Theory: Chernoff Bounds Prob( X-¹ ) ± ¹) is inverse exponential in n!!!! Friedhelm Meyer auf der Heide 23
Chapter 5. Approximation Algorithms Friedhelm Meyer auf der Heide 24
Friedhelm Meyer auf der Heide 25
Approximation Algorithms Examples: Greedy Techniques (Load-Balancing, Center Selection ) Pricing Method (Vertex Cover) Linear Programming and Rounding (Vertex Cover, generalized Load-Balancing ) Polynomial Time Approximation Scheme (Knapsack ) 26 Friedhelm Meyer auf der Heide 26
Friedhelm Meyer auf der Heide 27
Friedhelm Meyer auf der Heide 28
15.07.2011 29 Friedhelm Meyer auf der Heide 29
Friedhelm Meyer auf der Heide 30
Friedhelm Meyer auf der Heide 31
Approximation via rounding linear programs Friedhelm Meyer auf der Heide 32
Friedhelm Meyer auf der Heide 33
minimize Friedhelm Meyer auf der Heide 34
Friedhelm Meyer auf der Heide 35
Friedhelm Meyer auf der Heide 36
Friedhelm Meyer auf der Heide 37
Chapter 6. Online Algorithms Friedhelm Meyer auf der Heide 38
Online Algorithms An online algorithm is one that can process its input piece-by-piece, without having the entire input available from the start. In contrast, an offline algorithm is given the whole problem data from the beginning and is required to output an answer which solves the problem at hand. Input: a sequence of requests. Task: process the requests as efficiently as possible Online: i th request has to be processed before future requests are known Offline: All requests are known in advance Friedhelm Meyer auf der Heide 39
How to measure quality of online algorithms? 1. Assume some a priori knowledge about request sequence, e.g., requests are chosen randomly 2. Assume worst case measure, compare online cost to offline cost Online : standard competitive analysis competitive ratio Online randomized: Friedhelm Meyer auf der Heide 40
Examples: Paging, page migration, data management in networks Friedhelm Meyer auf der Heide 41
Page Migration Model (1) Page migration Classical online problem processors connected by a network v 3 v 2 v 4 v 5 v 1 v 6 v 7 There are costs of communication associated with each edge. Cost of communication between pair of nodes = cost of the cheapest path between these nodes. Costs of communication fulfill the triangle inequality. Friedhelm Meyer auf der Heide 42
Page Migration Model (2) Alternative view: processors in a metric space v 3 v 2 v 4 v 5 v 1 v 6 v 7 Indivisible memory page of size one processor (initially at ) in the local memory of Friedhelm Meyer auf der Heide 43
Page Migration Model (3) Input: sequence of processors, dictated by a request adversary - processor which wants to access (read or write) one unit of data from the memory page. v 3 v 2 v 4 v 5 v 1 v 7 v 6 After serving a request an algorithm may move the page to a new processor. Friedhelm Meyer auf der Heide 44
Page Migration (cost model) Cost model: The page is at node. Serving a request issued at costs. Moving the page to node costs. Friedhelm Meyer auf der Heide 45
A randomized algorithm Memoryless coin-flipping algorithm CF [Westbrook 92] In each step, after serving a request issued at, move page to with probability. Theorem: CF is 3-competitive against an adaptive-online adversary (may see the outcomes of the coinflips). Remark: This ratio is optimal against adaptive-online adversary. Friedhelm Meyer auf der Heide 46
Deterministic algorithm Algorithm Move-To-Min (MTM) [Awerbuch, Bartal, Fiat 93] After each steps, choose to be the node which minimizes, and move to. ( is the best place for the page in the last steps) Theorem: MTM is 7-competitive Remark: The currently best deterministic algorithm achieves competitive ratio of 4.086 Friedhelm Meyer auf der Heide 47
Data management in networks Friedhelm Meyer auf der Heide 48
Scenario Networks have low bandwidth, global objects are small, access is fine grained. typical for parallel processor networks, partially also for the internet. bottleneck: link-congestion task: distribute global objects (maybe dynamically) among processors such that an application (sequence of read/write access to global variables) can be executed using small link-congestion storage overhead is small. - Exploit Locality - Friedhelm Meyer auf der Heide 49
Basic Strategy Design strategy for trees Produce strategy for target-network by tree embedding Friedhelm Meyer auf der Heide 50
Dynamic Model Application: Sequence of read / write requests from processors to objects. Each processor decides solely based on its local knowledge. distributed online-strategy Goal: Develop strategy that produces only by a factor c more congestion than an optimal offline strategy. c-competitive strategy (and by a factor m more storage per processor (m, c) competitive strategy ) Friedhelm Meyer auf der Heide 51
Dynamic strategy for trees v writes to x : v creates (or updates) copy of x in v, and invalidates all other copies (consistency!) v reads x: v reads the closest copy of x and creates copies in every processor on the path back to v. (Remark: Data Tracking in trees is easy!) Friedhelm Meyer auf der Heide 52
Example and Analysis Consider phase write (v 0 ), read (v 1 ), read (v 2 ),..., read (v k-1 ), write (v k ) v 0 Friedhelm Meyer auf der Heide 53
Example and Analysis Consider phase write (v 0 ), read (v 1 ), read (v 2 ),..., read (v k-1 ), write (v k ) v 1 v 0 Friedhelm Meyer auf der Heide 54
Example and Analysis Consider phase write (v 0 ), read (v 1 ), read (v 2 ),..., read (v k-1 ), write (v k ) v 1 v 0 Friedhelm Meyer auf der Heide 55
Example and Analysis Consider phase write (v 0 ), read (v 1 ), read (v 2 ),..., read (v k-1 ), write (v k ) v 1 v 0 v 2 Friedhelm Meyer auf der Heide 56
Example and Analysis Consider phase write (v 0 ), read (v 1 ), read (v 2 ),..., read (v k-1 ), write (v k ) v 1 v 0 v 2 Friedhelm Meyer auf der Heide 57
Example and Analysis Consider phase write (v 0 ), read (v 1 ), read (v 2 ),..., read (v k-1 ), write (v k ) v 1 v 0 v 2 v 3 Friedhelm Meyer auf der Heide 58
Example and Analysis Consider phase write (v 0 ), read (v 1 ), read (v 2 ),..., read (v k-1 ), write (v k ) v 1 v k v 0 v 2 v 3 Friedhelm Meyer auf der Heide 59
Example and Analysis Consider phase write (v 0 ), read (v 1 ), read (v 2 ),..., read (v k-1 ), write (v k ) v 1 v k v 0 v 2 3 Each strategy has to use each link of the red subtree at least once. Our strategy uses each of these links at most three times. Strategy is 3-competitive for trees v Friedhelm Meyer auf der Heide 60
Other networks Idea: Simulate suitable tree in target-network M. tree embedding: Goals: - small dilation (in order to reduce overall load) - randomized embedding (in order to reduce congestion) Goals contradict?!? Friedhelm Meyer auf der Heide 61
Tree embedding Randomized, locality preserving embedding! Example: nxn-mesh 0 1 2 3 M v leaves: nodes of the mesh link-capacity: # links leaving the submesh Friedhelm Meyer auf der Heide 62
Result for meshes The static and dynamic strategies are O (log(n))-competitive in nxn-meshes, w.h.p. Finding an optimal static placement for several variables is NP-hard already on 3x3-meshes. Friedhelm Meyer auf der Heide 63
Some Results d-dimensional meshes O(d log n) competitive ratio w.h.p. Fat Trees O(log n) Hypercubes O(log n) SE Networks De Bruijn Networks Direct Butterflies Indirect Butterflies Arbitrary Networks O(log n) O(log n) O(log n) O(1) polylog n (Räcke 2002) Friedhelm Meyer auf der Heide 64
I wish you. not too much stress with exams and some time for relaxing holydays. Friedhelm Meyer auf der Heide 65
Thank you for your attention! Friedhelm Meyer auf der Heide Heinz Nixdorf Institute & Computer Science Department Fürstenallee 11 33102 Paderborn, Germany Tel.: +49 (0) 52 51/60 64 80 Fax: +49 (0) 52 51/62 64 82 E-Mail: fmadh@upb.de http://www.upb.de/cs/ag-madh Friedhelm Meyer auf der Heide 66