Welcome to the course Algorithm Design

Transcription

1 Welcome to the course Algorithm Design Summer Term 2011 Friedhelm Meyer auf der Heide Lecture 13, Friedhelm Meyer auf der Heide 1

2 Topics - Divide & conquer - Dynamic programming - Greedy Algorithms - Randomized Algorithms - Approximation Algorithms - Online Algorithms Friedhelm Meyer auf der Heide 2

3 Chapter 1: Divide & Conquer Friedhelm Meyer auf der Heide 3

4 Divide & Conquer Divide the problem in a number of smaller subproblems. Conquer the subproblems by solving them recursively. Combine the results of the subproblem into a solution of the original problem. Our Examples: Integer multiplication, matrix multiplication (HW), closest pair of points in the plane, Fast Fourier transform,.(hw) Friedhelm Meyer auf der Heide 4

5 Chapter 2: Dynamic Progamming Friedhelm Meyer auf der Heide 5

6 Dynamic programming You probably know dynamic programming algorithms for optimal evaluation of the product of many matrices, and for the word problem for context-free languages (CYK-algorithm) Examples: String Distance, Shortest Paths, Knapsack, Minimum Weight Triangulation (HW), Optimal Search Trees (HW) Friedhelm Meyer auf der Heide 6

7 Friedhelm Meyer auf der Heide 7

8 Friedhelm Meyer auf der Heide 8

9 Friedhelm Meyer auf der Heide 9

10 Friedhelm Meyer auf der Heide 10

11 Chapter 3: Greedy Algorithms Friedhelm Meyer auf der Heide 11

12 Greedy Algorithms Examples: Activity Selection (or Interval Scheduling), Scheduling to Minimize Lateness Theoretical Foundations of the Greedy Method: Matroid Theory Examples for Matroids: Linear independent vectors, Minimum Spanning Trees, Bipartite matching with maximum left side Friedhelm Meyer auf der Heide 12

13 For which problems are Greedy-Algorithms optimal? Consider a finite set E and a system U of subsets of E. (E,U) is called a subset-system, if the following holds: (i)? 2 U (ii) For each B 2 U, also each subset of B is in U. B2 U is called maximal, if no proper superset of B is in U. The Optimization problem corresponding to (E,U) is : Given a weight function w:e Q +,compute a maximal set B 2 U with maximizes w(b) = e2 B w(e). (Minimization problems are described analagously.) Friedhelm Meyer auf der Heide 13

14 Systems of Subsets and the Canonical Greedy Algorithm Canonical Greedy ((E,U)) (1) Sort E such that w(e 1 ) w(e n ). (2) B?. (3) For k=1 to n if B [ {e k } 2 U then B B [ {e k } (4) Return B For which types of subset-systems is the canonical Greedy algorithm optimal? Friedhelm Meyer auf der Heide 14

15 Example 2 Problem: Minimum spanning tree (MST) Input: a graph G=(V,E) with positive edge weights w:e Q +. Output: a spanning tree with minimum weight. MST is a subset-system: - E : the set of edges, - U : all subsets of E that do not contain a cycle form a subset-system Friedhelm Meyer auf der Heide 15

16 Greedy-Algorithms und Matroids A system of subsets (E,U) is a matroid, if in addition, the following exchange property holds: (iii) For all A,B 2 U with A < B, there is x2 B-A such that A[{x} 2 U. Remark: All maximal sets of a matroid have the same size. (homework) We will see: - The system of subsets for 0-1 knapsack (Example 1) is not a matroid. - The systems of subsets for examples 0,2,3 are a matroids. Theorem: Consider a system of subsets (E,U). canonical Greedy is optimal for (E,U) for every weight function w, if and only if (E,U) is a matroid. Kruskal s algorithm is optimal for MST (example 2)! Friedhelm Meyer auf der Heide 16

17 Chapter 4. Randomised Algorithms Friedhelm Meyer auf der Heide 17

18 Randomized Algorithms use the outcome of random experiments ( coin flips ) to guide the execution of the algorithm. For a fixed input, runtime and output may be random variables. Friedhelm Meyer auf der Heide 18

19 Randomized Algorithms Examples: Quicksort, Skip Lists, Contention Resolution (symmetry -breaking), Global Minimum Cut (contraction algorithm), Random Variables and their Expectations (Guessing Cards, Coupon Collector) Chernoff Bounds, example Load Balancing Friedhelm Meyer auf der Heide 19

20 Cuts in graphs Let G=(V,E) be a connected, undirected graph. A cut in G is defined by a disjoint partition of V into two sets A and B. Its size is the number of edges between A and B. Goal: Compute a cut with minimum size. A cut of size 6. A B Friedhelm Meyer auf der Heide 20

21 Friedhelm Meyer auf der Heide 21

22 Friedhelm Meyer auf der Heide 22

23 Tools from Probability Theory: Chernoff Bounds Prob( X-¹ ) ± ¹) is inverse exponential in n!!!! Friedhelm Meyer auf der Heide 23

24 Chapter 5. Approximation Algorithms Friedhelm Meyer auf der Heide 24

25 Friedhelm Meyer auf der Heide 25

26 Approximation Algorithms Examples: Greedy Techniques (Load-Balancing, Center Selection ) Pricing Method (Vertex Cover) Linear Programming and Rounding (Vertex Cover, generalized Load-Balancing ) Polynomial Time Approximation Scheme (Knapsack ) 26 Friedhelm Meyer auf der Heide 26

27 Friedhelm Meyer auf der Heide 27

28 Friedhelm Meyer auf der Heide 28

29 Friedhelm Meyer auf der Heide 29

30 Friedhelm Meyer auf der Heide 30

31 Friedhelm Meyer auf der Heide 31

32 Approximation via rounding linear programs Friedhelm Meyer auf der Heide 32

33 Friedhelm Meyer auf der Heide 33

34 minimize Friedhelm Meyer auf der Heide 34

35 Friedhelm Meyer auf der Heide 35

36 Friedhelm Meyer auf der Heide 36

37 Friedhelm Meyer auf der Heide 37

38 Chapter 6. Online Algorithms Friedhelm Meyer auf der Heide 38

39 Online Algorithms An online algorithm is one that can process its input piece-by-piece, without having the entire input available from the start. In contrast, an offline algorithm is given the whole problem data from the beginning and is required to output an answer which solves the problem at hand. Input: a sequence of requests. Task: process the requests as efficiently as possible Online: i th request has to be processed before future requests are known Offline: All requests are known in advance Friedhelm Meyer auf der Heide 39

40 How to measure quality of online algorithms? 1. Assume some a priori knowledge about request sequence, e.g., requests are chosen randomly 2. Assume worst case measure, compare online cost to offline cost Online : standard competitive analysis competitive ratio Online randomized: Friedhelm Meyer auf der Heide 40

41 Examples: Paging, page migration, data management in networks Friedhelm Meyer auf der Heide 41

42 Page Migration Model (1) Page migration Classical online problem processors connected by a network v 3 v 2 v 4 v 5 v 1 v 6 v 7 There are costs of communication associated with each edge. Cost of communication between pair of nodes = cost of the cheapest path between these nodes. Costs of communication fulfill the triangle inequality. Friedhelm Meyer auf der Heide 42

43 Page Migration Model (2) Alternative view: processors in a metric space v 3 v 2 v 4 v 5 v 1 v 6 v 7 Indivisible memory page of size one processor (initially at ) in the local memory of Friedhelm Meyer auf der Heide 43

44 Page Migration Model (3) Input: sequence of processors, dictated by a request adversary - processor which wants to access (read or write) one unit of data from the memory page. v 3 v 2 v 4 v 5 v 1 v 7 v 6 After serving a request an algorithm may move the page to a new processor. Friedhelm Meyer auf der Heide 44

45 Page Migration (cost model) Cost model: The page is at node. Serving a request issued at costs. Moving the page to node costs. Friedhelm Meyer auf der Heide 45

46 A randomized algorithm Memoryless coin-flipping algorithm CF [Westbrook 92] In each step, after serving a request issued at, move page to with probability. Theorem: CF is 3-competitive against an adaptive-online adversary (may see the outcomes of the coinflips). Remark: This ratio is optimal against adaptive-online adversary. Friedhelm Meyer auf der Heide 46

47 Deterministic algorithm Algorithm Move-To-Min (MTM) [Awerbuch, Bartal, Fiat 93] After each steps, choose to be the node which minimizes, and move to. ( is the best place for the page in the last steps) Theorem: MTM is 7-competitive Remark: The currently best deterministic algorithm achieves competitive ratio of Friedhelm Meyer auf der Heide 47

48 Data management in networks Friedhelm Meyer auf der Heide 48

49 Scenario Networks have low bandwidth, global objects are small, access is fine grained. typical for parallel processor networks, partially also for the internet. bottleneck: link-congestion task: distribute global objects (maybe dynamically) among processors such that an application (sequence of read/write access to global variables) can be executed using small link-congestion storage overhead is small. - Exploit Locality - Friedhelm Meyer auf der Heide 49

50 Basic Strategy Design strategy for trees Produce strategy for target-network by tree embedding Friedhelm Meyer auf der Heide 50

51 Dynamic Model Application: Sequence of read / write requests from processors to objects. Each processor decides solely based on its local knowledge. distributed online-strategy Goal: Develop strategy that produces only by a factor c more congestion than an optimal offline strategy. c-competitive strategy (and by a factor m more storage per processor (m, c) competitive strategy ) Friedhelm Meyer auf der Heide 51

52 Dynamic strategy for trees v writes to x : v creates (or updates) copy of x in v, and invalidates all other copies (consistency!) v reads x: v reads the closest copy of x and creates copies in every processor on the path back to v. (Remark: Data Tracking in trees is easy!) Friedhelm Meyer auf der Heide 52

53 Example and Analysis Consider phase write (v 0 ), read (v 1 ), read (v 2 ),..., read (v k-1 ), write (v k ) v 0 Friedhelm Meyer auf der Heide 53

54 Example and Analysis Consider phase write (v 0 ), read (v 1 ), read (v 2 ),..., read (v k-1 ), write (v k ) v 1 v 0 Friedhelm Meyer auf der Heide 54

55 Example and Analysis Consider phase write (v 0 ), read (v 1 ), read (v 2 ),..., read (v k-1 ), write (v k ) v 1 v 0 Friedhelm Meyer auf der Heide 55

56 Example and Analysis Consider phase write (v 0 ), read (v 1 ), read (v 2 ),..., read (v k-1 ), write (v k ) v 1 v 0 v 2 Friedhelm Meyer auf der Heide 56

57 Example and Analysis Consider phase write (v 0 ), read (v 1 ), read (v 2 ),..., read (v k-1 ), write (v k ) v 1 v 0 v 2 Friedhelm Meyer auf der Heide 57

58 Example and Analysis Consider phase write (v 0 ), read (v 1 ), read (v 2 ),..., read (v k-1 ), write (v k ) v 1 v 0 v 2 v 3 Friedhelm Meyer auf der Heide 58

59 Example and Analysis Consider phase write (v 0 ), read (v 1 ), read (v 2 ),..., read (v k-1 ), write (v k ) v 1 v k v 0 v 2 v 3 Friedhelm Meyer auf der Heide 59

60 Example and Analysis Consider phase write (v 0 ), read (v 1 ), read (v 2 ),..., read (v k-1 ), write (v k ) v 1 v k v 0 v 2 3 Each strategy has to use each link of the red subtree at least once. Our strategy uses each of these links at most three times. Strategy is 3-competitive for trees v Friedhelm Meyer auf der Heide 60

61 Other networks Idea: Simulate suitable tree in target-network M. tree embedding: Goals: - small dilation (in order to reduce overall load) - randomized embedding (in order to reduce congestion) Goals contradict?!? Friedhelm Meyer auf der Heide 61

62 Tree embedding Randomized, locality preserving embedding! Example: nxn-mesh M v leaves: nodes of the mesh link-capacity: # links leaving the submesh Friedhelm Meyer auf der Heide 62

63 Result for meshes The static and dynamic strategies are O (log(n))-competitive in nxn-meshes, w.h.p. Finding an optimal static placement for several variables is NP-hard already on 3x3-meshes. Friedhelm Meyer auf der Heide 63

64 Some Results d-dimensional meshes O(d log n) competitive ratio w.h.p. Fat Trees O(log n) Hypercubes O(log n) SE Networks De Bruijn Networks Direct Butterflies Indirect Butterflies Arbitrary Networks O(log n) O(log n) O(log n) O(1) polylog n (Räcke 2002) Friedhelm Meyer auf der Heide 64

65 I wish you. not too much stress with exams and some time for relaxing holydays. Friedhelm Meyer auf der Heide 65

66 Thank you for your attention! Friedhelm Meyer auf der Heide Heinz Nixdorf Institute & Computer Science Department Fürstenallee Paderborn, Germany Tel.: +49 (0) 52 51/ Fax: +49 (0) 52 51/ Friedhelm Meyer auf der Heide 66