Algorithms (IX) Guoqiang Li. School of Software, Shanghai Jiao Tong University

Transcription

1 Algorithms (IX) Guoqiang Li School of Software, Shanghai Jiao Tong University

2 Q: What we have learned in Algorithm?

3 Algorithm Design

4 Algorithm Design

5 Basic methodologies: Algorithm Design

6 Algorithm Design Basic methodologies: Divide and Conquer (Chapter 2)

7 Algorithm Design Basic methodologies: Divide and Conquer (Chapter 2) Master Theorem

8 Algorithm Design Basic methodologies: Divide and Conquer (Chapter 2) Master Theorem Recursion (Chapter 0 & 1)

9 Algorithm Design Basic methodologies: Divide and Conquer (Chapter 2) Master Theorem Recursion (Chapter 0 & 1) Advanced topics:

10 Algorithm Design Basic methodologies: Divide and Conquer (Chapter 2) Master Theorem Recursion (Chapter 0 & 1) Advanced topics: Dynamic Programming (Chapter 6)

11 Algorithm Design Basic methodologies: Divide and Conquer (Chapter 2) Master Theorem Recursion (Chapter 0 & 1) Advanced topics: Dynamic Programming (Chapter 6) Filling in Squares.

12 Algorithm Design Basic methodologies: Divide and Conquer (Chapter 2) Master Theorem Recursion (Chapter 0 & 1) Advanced topics: Dynamic Programming (Chapter 6) Filling in Squares. Complexity.

13 Algorithm Design Basic methodologies: Divide and Conquer (Chapter 2) Master Theorem Recursion (Chapter 0 & 1) Advanced topics: Dynamic Programming (Chapter 6) Filling in Squares. Complexity. Greedy Algorithm (Chapter 5)

14 Algorithm Design Basic methodologies: Divide and Conquer (Chapter 2) Master Theorem Recursion (Chapter 0 & 1) Advanced topics: Dynamic Programming (Chapter 6) Filling in Squares. Complexity. Greedy Algorithm (Chapter 5) Linear Programming (Chapter 7)

15 Algorithm Design Basic methodologies: Divide and Conquer (Chapter 2) Master Theorem Recursion (Chapter 0 & 1) Advanced topics: Dynamic Programming (Chapter 6) Filling in Squares. Complexity. Greedy Algorithm (Chapter 5) Linear Programming (Chapter 7) Simplex.

16 Algorithm Design Basic methodologies: Divide and Conquer (Chapter 2) Master Theorem Recursion (Chapter 0 & 1) Advanced topics: Dynamic Programming (Chapter 6) Filling in Squares. Complexity. Greedy Algorithm (Chapter 5) Linear Programming (Chapter 7) Simplex. System Design.

17 Algorithm Design Basic methodologies: Divide and Conquer (Chapter 2) Master Theorem Recursion (Chapter 0 & 1) Advanced topics: Dynamic Programming (Chapter 6) Filling in Squares. Complexity. Greedy Algorithm (Chapter 5) Linear Programming (Chapter 7) Simplex. System Design. Duality.

18 Algorithm Design Basic methodologies: Divide and Conquer (Chapter 2) Master Theorem Recursion (Chapter 0 & 1) Advanced topics: Dynamic Programming (Chapter 6) Filling in Squares. Complexity. Greedy Algorithm (Chapter 5) Linear Programming (Chapter 7) Simplex. System Design. Duality. Randomized Algorithm

19 Algorithm Design Basic methodologies: Divide and Conquer (Chapter 2) Master Theorem Recursion (Chapter 0 & 1) Advanced topics: Dynamic Programming (Chapter 6) Filling in Squares. Complexity. Greedy Algorithm (Chapter 5) Linear Programming (Chapter 7) Simplex. System Design. Duality. Randomized Algorithm Approximation Algorithm

20 Algorithm Analysis

21 Algorithm Analysis

22 Basic Knowledge Algorithm Analysis

23 Algorithm Analysis Basic Knowledge Big-O Notation (Chapter 0)

24 Algorithm Analysis Basic Knowledge Big-O Notation (Chapter 0) Recursion and Loop (Chapter 1)

25 Algorithm Analysis Basic Knowledge Big-O Notation (Chapter 0) Recursion and Loop (Chapter 1) geometrical Analysis (Chapter 2)

26 Algorithm Analysis Basic Knowledge Big-O Notation (Chapter 0) Recursion and Loop (Chapter 1) geometrical Analysis (Chapter 2) Advanced Methodology:

27 Algorithm Analysis Basic Knowledge Big-O Notation (Chapter 0) Recursion and Loop (Chapter 1) geometrical Analysis (Chapter 2) Advanced Methodology: Probability Analysis (Chapter 1)

28 Algorithm Analysis Basic Knowledge Big-O Notation (Chapter 0) Recursion and Loop (Chapter 1) geometrical Analysis (Chapter 2) Advanced Methodology: Probability Analysis (Chapter 1) Amortized Analysis (Chapter 5)

29 Algorithm Analysis Basic Knowledge Big-O Notation (Chapter 0) Recursion and Loop (Chapter 1) geometrical Analysis (Chapter 2) Advanced Methodology: Probability Analysis (Chapter 1) Amortized Analysis (Chapter 5) Competition Analysis (Chapter 9)

30 Standard Algorithm

31 Standard Algorithms

32 Standard Algorithms Arithmetic Algorithm

33 Standard Algorithms Arithmetic Algorithm Sorting

34 Standard Algorithms Arithmetic Algorithm Sorting Searching Strongly Connected Components

35 Standard Algorithms Arithmetic Algorithm Sorting Searching Strongly Connected Components Finding shortest paths in graphs

36 Standard Algorithms Arithmetic Algorithm Sorting Searching Strongly Connected Components Finding shortest paths in graphs Minimum spanning trees in graphs

37 Standard Algorithms Arithmetic Algorithm Sorting Searching Strongly Connected Components Finding shortest paths in graphs Minimum spanning trees in graphs Matchings in bipartite graphs

38 Standard Algorithms Arithmetic Algorithm Sorting Searching Strongly Connected Components Finding shortest paths in graphs Minimum spanning trees in graphs Matchings in bipartite graphs Maximum flows in networks

39 Advanced Data Structure

40 Advanced Data Structure

41 Advanced Data Structure Trees, Graphs

42 Advanced Data Structure Trees, Graphs Kripke Structure, Automata

43 Advanced Data Structure Trees, Graphs Kripke Structure, Automata Priority Queue (Chapter 4)

44 Advanced Data Structure Trees, Graphs Kripke Structure, Automata Priority Queue (Chapter 4) Disjoint Set (Chapter 5)

45 What Is Important in Algorithm under My Concern?

46 I. Proof and Witness

47 Mathematical Induction. Proof and Witness

48 Proof and Witness Mathematical Induction. Proof of iff, =,

49 Proof and Witness Mathematical Induction. Proof of iff, =, Deduction.

50 Examples log n! = Θ(n log n)

51 Examples log n! = Θ(n log n) A directed graph has a cycle if and only if its depth-first search reveals a back edge.

52 Examples log n! = Θ(n log n) A directed graph has a cycle if and only if its depth-first search reveals a back edge. An undirected graph is bipartite if and only if it contains no cycles of odd length.

53 II. Reductions

54 Bipartite Matching BOYS Al Bob Chet Dan GIRLS Alice Beatrice Carol Danielle

55 Bipartite Matching Al Alice s Bob Chet Beatrice Carol t Dan Danielle

56 Reductions We want to solve Problem P. We already have an algorithm that solves Problem Q. If any subroutine for Q can also be used to solve P, we say P reduces to Q. Often, P is solvable by a single call to Q s subroutine, which means any instance x of P can be transformed into an instance y of Q such that P(x) can be deduced from Q(y).

57 III. Divide-and-Conquer Vs. Dynamic Programming

58 Master Theorem If T(n) = at( n/b ) + O(n d ) for some constants a > 0, b > 1 and d 0, then O(n d ) if d > log b a T(n) = O(n d log n) if d = log b a O(n log b a ) if d < log b a

59 The Proof of the Theorem Proof: Assume that n is a power of b. The size of the subproblems decreases by a factor of b with each level of recursion, and therefore reaches the base case after log b n levels - the the height of the recursion tree. Its branching factor is a, so the k-th level of the tree is made up of a k subproblems, each of size n/b k. a k O( n b k )d = O(n d ) ( a b d )k k goes from 0 to log b n, these numbers form a geometric series with ratio a/b d, comes down to three cases.

60 The Proof of the Theorem The ratio is less than 1. Then the series is decreasing, and its sum is just given by its first term, O(n d ). The ratio is greater than 1. The series is increasing and its sum is given by its last term, O(n log b a ) The ratio is exactly 1. In this case all O(log n) terms of the series are equal to O(n d ).

61 The Problem When a spell checker encounters a possible misspelling, it looks in its dictionary for other words that are close by. S N O W Y S U N N Y Cost: 3 S N O W Y S U N N Y Cost: 5

62 The Problem When a spell checker encounters a possible misspelling, it looks in its dictionary for other words that are close by. Q: What is the appropriate notion of closeness in this case? S N O W Y S U N N Y Cost: 3 S N O W Y S U N N Y Cost: 5

63 The Problem When a spell checker encounters a possible misspelling, it looks in its dictionary for other words that are close by. Q: What is the appropriate notion of closeness in this case? A natural measure of the distance between two strings is the extent to which they can be aligned, or matched up. S N O W Y S U N N Y Cost: 3 S N O W Y S U N N Y Cost: 5

64 The Problem When a spell checker encounters a possible misspelling, it looks in its dictionary for other words that are close by. Q: What is the appropriate notion of closeness in this case? A natural measure of the distance between two strings is the extent to which they can be aligned, or matched up. Technically, an alignment is simply a way of writing the strings one above the other. S N O W Y S U N N Y Cost: 3 S N O W Y S U N N Y Cost: 5

65 The Problem The cost of an alignment is the number of columns in which the letters differ. S N O W Y S U N N Y Cost: 3 S N O W Y S U N N Y Cost: 5

66 The Problem The cost of an alignment is the number of columns in which the letters differ. And the edit distance between two strings is the cost of their best possible alignment. S N O W Y S U N N Y Cost: 3 S N O W Y S U N N Y Cost: 5

67 The Problem The cost of an alignment is the number of columns in which the letters differ. And the edit distance between two strings is the cost of their best possible alignment. Edit distance is so named because it can also be thought of as the minimum number of edits S N O W Y S U N N Y Cost: 3 S N O W Y S U N N Y Cost: 5

68 The Problem The cost of an alignment is the number of columns in which the letters differ. And the edit distance between two strings is the cost of their best possible alignment. Edit distance is so named because it can also be thought of as the minimum number of edits insertions, deletions, and substitutions of characters needed to transform the first string into the second. S N O W Y S U N N Y Cost: 3 S N O W Y S U N N Y Cost: 5

69 A Dynamic Programming Solution When solving a problem by dynamic programming, the most crucial question is,

70 A Dynamic Programming Solution When solving a problem by dynamic programming, the most crucial question is, What are the subproblems?

71 A Dynamic Programming Solution When solving a problem by dynamic programming, the most crucial question is, What are the subproblems? Our goal is to find the edit distance between two strings x[1,..., m] and y[1,..., n].

72 A Dynamic Programming Solution When solving a problem by dynamic programming, the most crucial question is, What are the subproblems? Our goal is to find the edit distance between two strings x[1,..., m] and y[1,..., n]. For every i, j with 1 i m and 1 j n, let

73 A Dynamic Programming Solution When solving a problem by dynamic programming, the most crucial question is, What are the subproblems? Our goal is to find the edit distance between two strings x[1,..., m] and y[1,..., n]. For every i, j with 1 i m and 1 j n, let E(i, j): the edit distance between some prefix of the first string, x[1,..., i], and some prefix of the second, y[1,..., j].

74 A Dynamic Programming Solution When solving a problem by dynamic programming, the most crucial question is, What are the subproblems? Our goal is to find the edit distance between two strings x[1,..., m] and y[1,..., n]. For every i, j with 1 i m and 1 j n, let E(i, j): the edit distance between some prefix of the first string, x[1,..., i], and some prefix of the second, y[1,..., j]. E(i, j) = min{1 + E(i 1, j), 1 + E(i, j 1), diff(i, j) + E(i 1, j 1)}, where diff(i, j) is defined to be 0 if x[i] = y[j] and 1 otherwise.

75 An Example Edit distance between EXPONENTIAL and POLYNOMIAL, subproblem E(4, 3) corresponds to the prefixes EXPO and POL. The rightmost column of their best alignment must be one of the following: O or Thus, E(4, 3) = min{1 + E(3, 3), 1 + E(4, 2); 1 + E(3, 2)}. (a) i 1 i m j 1 j n GOAL L (b) or O L P O L Y N O M I A L E X P O N E N T I A L

76 Chapter by Chapter

77 Chapter 0 Prologue

78 Big-O Notation Multiplicative constants can be omitted: 14n 2 becomes n 2. n a dominates n b if a > b, for instance, n 2 dominates n a n dominates b n if a > b, for instance, 3 n dominates 2 n Any exponential dominates any polynomial: 3 n dominates n 5 Any polynomial dominates any logarithm: n dominates (log n) 3. This also means, for example, that n 2 dominates n log n.

79 An Example 0.1. Ineachofthefollowingsituations,indicatewhether f = O(g),or f = Ω(g),orboth(inwhichcase f = Θ(g)). f(n) g(n) (a) n 100 n 200 (b) n 1/2 n 2/3 (c) 100n + log n n + (log n) 2 (d) n log n 10n log10n (e) log 2n log 3n (f) 10 logn log(n 2 ) (g) n 1.01 n log 2 n (h) n 2 / logn n(log n) 2

80 An Example (i) n 0.1 (log n) 10 (j) (log n) log n n/ logn (k) n (log n) 3 (l) n 1/2 5 log 2 n (m) n2 n 3 n (n) 2 n 2 n+1 (o) n! 2 n (p) (log n) log n 2 (log 2 n)2 n (q) i=1 ik n k+1

81 Chapter 1 Algorithms with Numbers

82 Recursion Analysis MODEXP(x, y, N); Two n-bit integers x and N, and an integer exponent y; if y = 0 then return 1; z=modexp (x, y/2, N); if y is even then return z 2 mod N; else return x z 2 (mod N); end Another formulation: x y (mod N) = { (x y/2 ) 2 (mod N) if y is even x (x y/2 ) 2 (mod N) if y is odd

83 Arithmetic Algorithm What is the GCD of 42 and 96? Can you prove it?

84 Chapter 2. Divide-and-Conquer

85 An Example Foo(x); if x > 1 then Printf( Hello World ); Foo(x/4); Foo(x/4); Foo(x/4); end

86 Chapter 3. Decompositions of Graphs

87 Our Tools and Weapons

88 DFS (Explore) Our Tools and Weapons

89 Our Tools and Weapons DFS (Explore) DAG, Linearization

90 Our Tools and Weapons DFS (Explore) DAG, Linearization SCC

91 Exercise 1 Suppose a CS curriculum consists of n courses, all of them mandatory. The prerequisite graph G hasanodeforeachcourse,andanedgefromcourse vtocourse wifandonlyif visaprerequisite for w. Find an algorithm that works directly with this graph representation, and computes the minimum number of semesters necessary to complete the curriculum(assume that a student cantakeanynumberofcoursesinonesemester).therunningtimeofyouralgorithmshouldbe linear.

92 Exercise Giveanefficientalgorithmwhichtakesasinputadirectedgraph G = (V, E),anddetermines whetherornotthereisavertex s V fromwhichallotherverticesarereachable.

93 Exercise 3 A bipartite graph is a graph G = (V, E) whose vertices can be partitioned into two sets (V = V 1 V 2 and V 1 V 2 = ) such that there are no edges between vertices in the same set (for instance, if u, v V 1, then there is no edge between u and v). (a) Give a linear-time algorithm to determine whether an undirected graph is bipartite. (b) Prove that an undirected graph is bipartite if and only if it contains no cycles of odd length. (c) At most how many colors are needed to color in an undirected graph with exactly one odd length cycle?

94 Exercise 4 Prove that given a connected undirected graph G = (V, E), there exists a node v V, such that G is still connected after deleting v.

95 Exercise 5 Given a directed graph, work out an efficient algorithm, to find all cross edges in DFS searching.

96 Chapter 4. Path in Graphs

97 Our Tools and Weapons

98 BFS Our Tools and Weapons

99 Our Tools and Weapons BFS Shortest Path

100 Our Tools and Weapons BFS Shortest Path Shortest Path in DAG

101 Exercise 1 You are given a strongly connected directed graph G = (V, E) with positive edge weights along withaparticularnode v 0 V.Giveanefficientalgorithmforfindingshortestpathsbetweenall pairsofnodes,withtheonerestrictionthatthesepathsmustallpassthrough v 0.

102 Exercise 2 Give an efficient algorithm that takes as input a directed acyclic graph G = (V, E), and two vertices s, t V, and outputs the number of different directed paths from s to t in G.

103 Exercise 3 Give an efficient algorithm to solve: Input Given an undirected graph G = (V, E),every edge is assigned a weight l e > 0, and given a specific edge e E. Output the length of that shortest circle that contains e.

104 Exercise 4 Given an undirected graph with n nodes and the degree of each node is 1, there must exists a circle, and the number of nodes with degree 3 is less than 2 log 2 n

105 Chapter 5. Greedy Algorithms

106 Our Tools and Weapons

107 MST Our Tools and Weapons

108 Our Tools and Weapons MST Cut Theorem

109 Chapter 6. Dynamic Programming

110 Exercise Youaregoingonalongtrip. Youstartontheroadatmilepost0. Alongthewaythereare n hotels,atmileposts a 1 < a 2 < < a n,whereeach a iismeasuredfromthestartingpoint.the onlyplacesyouareallowedtostopareatthesehotels,butyoucanchoosewhichofthehotels youstopat.youmuststopatthefinalhotel(atdistance a n),whichisyourdestination. You dideallyliketotravel200milesaday,butthismaynotbepossible(dependingonthespacing ofthehotels).ifyoutravel xmilesduringaday,thepenaltyforthatdayis (200 x) 2.Youwant toplanyourtripsoastominimizethetotalpenalty thatis,thesum,overalltraveldays,ofthe daily penalties. Give an efficient algorithm that determines the optimal sequence of hotels at which to stop.

111 Exercise 2 Given two strings x = x 1 x 2... x n and y = y 1 y 2... y m, we wish to find the length of their longest common subsequence. Show how to do this in time O(mn).

112 Chapter 7. Linear Programming

113 An Example max x + 2y 1 2x + y 3 x + 3y 5 x, y 0