CS 781 Advanced Algorithms 1 Winter 2011

Transcription

1 CS 781 Advanced Algorithms 1 Winter 2011 Meeting Time: Thursday 5:00pm-7:45pm Room Zimmer 302 (also broadcast to Zimmer 410 and NG) Instructor: Prof. Fred Annexstein Office: 889 Rhodes (Office Hours: WTh 2:00-3:00) Phone: fred.annexstein@uc.edu Web: (will take tweeted questions) 1

2 General Course Information Grading : There will be 5-6 homework assignments-one every two weeks. There will be a midterm and final exam. Homework will count 35% and midterm 25% and final exam 40%. Description: -Review of analysis of sequential algorithms. -Three major design strategies: the greedy method, divide-andconquer, dynamic programming -Graph and network algorithms, Fast Fourier Transform -Introduction to NP-completeness and Optimization -Approximation and Randomized algorithms Textbook: Algorithms: Sequential, Parallel, and Distributed, by Kenneth A. Berman and Jerome L. Paul, Thompson Publishing. Powerpoint Slides: Available on BB. Some material taken with acknowledgements to Kevin Wayne, Jon Kleinberg, Eva Tardos. 2

3 Approximate Schedule Week 1. Introduction and Review of Basic Algorithm Analysis (Chapters 2-3). Jan 6. Week 2. Graphs and Graph Traversals (Chapters 11). Jan 13 Week 3. Greedy Algorithms and Minimum Spanning Trees (Chapter 7,12) Jan 20 Week 4. Data Clustering and Data Compression (Chapter 7.4, notes) Jan 27 Weeks 5. Divide and Conquer Algorithms and FFT (Chapter 8, 22) Feb 3. Midterm Exam Feb 10 3

4 Approximate Schedule Cont. Week 6. Dynamic Programming and Bellman-Ford (Chapter 9,12) Feb 10 Week 7. Matchings and Max Flow (Chapter 14) Feb 17 Week 8. NP-completeness (Chapter 26) Feb 24 Week 9. Local Search and Optimization (Notes) March 3 Week 10. Randomized Algorithms and Hashing (Chapter 24,17) March 10 Final Exam - March

5 A First Problem: College Football Team Rankings On Monday there will be NCAA College Football Championship Game played. How were the two teams chosen? 1. Problem Statement 2. Goal Objective 3. Representation of Input 4. Representation of Output 5

6 BCS Computer Rankings! Two Polls and Six different computer ranking algorithms are used: 1. Jeff Sagarin (Visit site) 2. Anderson & Hester (Visit site) 3. Richard Billingsley (Visit site) 4. Colley Matrix (Visit site) 5. Kenneth Massey (Visit site) 6. Dr. Peter Wolfe (Visit site) A team's highest and lowest computer ranking is discarded from figuring a team's computer poll average. Points are assigned in inverse order of ranking. The four remaining computer scores are averaged and the total calculated as a percentage of 100. Three components -- The Harris Interactive Poll, the USA Today Coaches Poll and the computer rankings are added together and averaged for a team's ranking in the BCS standings. The two teams with the highest average shall rank first and second and play in the BCS Championship. 6

7 A Real World Computer SNAFU On December 6, 2010 it was announced that an error had occurred in computer rankings. One game was omitted from the final Colley rankings. Mr. Colley discovered this error in a subsequent review and informed the BCS. The result of the error inverted the ranks of Boise State and LSU. The final revised 2010 rankings: Auburn 2. Oregon 3. TCU 4. Stanford 5. Wisconsin 6. Ohio State 7. Oklahoma 8. Arkansas 9. Michigan State 10. Boise State 11. LSU BCS Championship Game: Auburn and Oregon January 10 8:00PM ESPN Lessons Learned: Algorithms can be arcane and complex, small changes in data matters, verification is generally hard, need to make goals mathematically rigorous, verifiable, not subjective. Need to make algorithms comparable. What about scalability? Can we apply rank methods to larger classes, e.g., musicians, students, teachers, tweeps, etc. 7

8 Scalability and Polynomial-Time Brute force. For many non-trivial problems, there is a natural brute force search algorithm that checks every possible solution. Typically this takes N! or 2 N time for inputs of size N. Unacceptable in practice. For example, 120 Division 1A Football teams 120! > 10^197 possible rankings. Desirable scaling property. When the input size doubles, the algorithm should only slow down by some constant factor C. Def. An algorithm is poly-time if the above scaling property holds, that is, There exists constant d > 0 such that on every input of size N, its running time is bounded by d N d steps. 8

9 Worst-Case and Average-Case Analysis Worst case running time. Obtain bound on largest possible running time of algorithm on input of a given size N. Generally captures efficiency in practice. Draconian view, but hard to find effective alternative. Helps if we know what a typical input looks like. Average case running time. Obtain bound on running time of algorithm on random input as a function of input size N. Hard (or impossible) to accurately model real instances by random distributions. Algorithm tuned for a certain distribution may perform poorly on other inputs. Most often aimed at uniform distributions. 9

10 Worst-Case Polynomial-Time Def. An algorithm is efficient if its worst-case running time is polynomial. Justification: It really works in practice! Although N 20 is technically poly-time, it would be useless in practice. In practice, the poly-time algorithms that people develop almost always have low constants and low exponents. Breaking through the exponential barrier of brute force typically exposes some crucial structure of the problem. Exceptions (that proves the rule). Some poly-time algorithms do have high constants and/or exponents, and are useless in practice. Some exponential-time (or worse) algorithms are widely used because the worst-case instances seem to be rare. simplex method Unix grep 10

11 Why It Matters 11

12 Asymptotic Order of Growth Say it in Greek: Big-omicron, Big-omega, and Big-theta! Upper bounds. T(n) is O(f(n)) if there exist constants c > 0 and n 0 0 such that for all n n 0 we have T(n) c f(n). Lower bounds. T(n) is Ω(f(n)) if there exist constants c > 0 and n 0 0 such that for all n n 0 we have T(n) c f(n). Tight bounds. T(n) is Θ(f(n)) if T(n) is both O(f(n)) and Ω(f(n)). Ex: T(n) = 32n n T(n) is O(n 2 ), O(n 3 ), Ω(n 2 ), Ω(n), and Θ(n 2 ). T(n) is not O(n), Ω(n 3 ), Θ(n), or Θ(n 3 ). 12

13 Notation Slight abuse of notation. T(n) = O(f(n)). Asymmetric: f(n) = 5n 3 ; g(n) = 3n 2 f(n) = O(n 3 ) = g(n) but f(n) g(n). Better notation: T(n) O(f(n)). The following is a meaningless statement. Any comparison-based sorting algorithm requires at least O(n log n) comparisons. Statement doesn't "type-check." Use Ω for lower bounds. 13

14 Properties Transitivity. If f = O(g) and g = O(h) then f = O(h). If f = Ω(g) and g = Ω(h) then f = Ω(h). If f = Θ(g) and g = Θ(h) then f = Θ(h). Additivity. If f = O(h) and g = O(h) then f + g = O(h). If f = Ω(h) and g = Ω(h) then f + g = Ω(h). If f = Θ(h) and g = O(h) then f + g = Θ(h). 14

15 Asymptotic Bounds for Some Common Functions Polynomials. a 0 + a 1 n + + a d n d is Θ(n d ) if a d > 0. Polynomial time. Running time is O(n d ) for some constant d independent of the input size n. Logarithms. O(log a n) = O(log b n) for any constants a, b > 0. can avoid specifying the base Logarithms. For every x > 0, log n = O(n x ). log grows slower than every polynomial Exponentials. For every r > 1 and every d > 0, n d = O(r n ). every exponential grows faster than every polynomial 15

16 Linear Time: O(n) Linear time. Running time is at most a constant factor times the size of the input. Computing the maximum. Compute maximum of n numbers a 1,, a n. #Compute maximum of list a of n numbers def computemax(a): max=a[0] for i in range(len(a)): if a[i]> max: max = a[i] return max from random import randint print computemax([randint(1,1000) for i in range(100)]) >>997 16

17 Linear Time: O(n) Merge. Combine two sorted lists A = a 1,a 2,,a n with B = b 1,b 2,,b n into sorted whole. i = 1, j = 1 while (both lists are nonempty) { if (a i b j ) append a i to output list and increment i else(a i b j )append b j to output list and increment j } append remainder of nonempty list to output list Claim. Merging two lists of size n takes O(n) time. Pf. After each comparison, the length of output list increases by 1. 17

18 O(n log n) Time O(n log n) time. Arises often in divide-and-conquer algorithms. Sorting. Mergesort and heapsort are sorting algorithms that perform O(n log n) comparisons. Quicksort is O(n log n) in average case. Largest empty interval. Given n time-stamps x 1,, x n on which copies of a file arrive at a server, what is largest interval of time when no copies of the file arrive? Give an O(n log n) solution 18

19 Quadratic Time: O(n 2 ) Quadratic time. Enumerate all pairs of elements. Closest pair of points. Given a list of n points in the plane (x 1, y 1 ),, (x n, y n ), find the pair that is closest. O(n 2 ) solution. Try all pairs of points. def computeclosestpair(x,y): min=1<<32 for i in range(len(x)): for j in range(i+1,len(y)): distij= (x[i]-x[j])**2 + (y[i]-y[j])**2 if distij < min: min = distij return min from random import randint print computeclosestpair([randint(0,1000) for xi in range(100)], [randint(0,1000) for yi in range(100)]) Remark. Ω(n 2 ) seems inevitable, but is it? 19

20 Cubic Time: O(n 3 ) Cubic time. Enumerate all triples of elements. Problem of Set disjointness. Given n sets S 1,, S n each of which is a subset of 1, 2,, n, is there some pair of these which are disjoint? O(n 3 ) solution. For each pairs of sets, determine if they are disjoint. foreach set S i { foreach other set S j { foreach element p of S i { determine whether p also belongs to S j } if (no element of S i belongs to S j ) report that S i and S j are disjoint } } Remark: How would you make this more efficient? 20

21 Polynomial Time: O(n k ) Time Problem k-independent set. Given a graph, are there k nodes such that no two are joined by an edge? O(n k ) solution. Enumerate all subsets of k nodes. k is a constant foreach subset S of k nodes { check whether S in an independent set if (S is an independent set) report S is an independent set } } Check whether S is an independent set = O(k 2 ). Number of k element subsets = n O(k 2 n k / k!) = O(n k n (n 1) (n 2) (n k +1) = ). k k (k 1) (k 2) (2) (1) nk k! poly-time for k=17, but not practical 21

22 Exponential Time Independent set. Given a graph, what is maximum size of an independent set? O(n 2 2 n ) solution. Enumerate all subsets. S* φ foreach subset S of nodes { check whether S in an independent set if (S is largest independent set seen so far) update S* S } } 22

23 5 min Break time 23

24 Undirected Graphs Undirected graph. G = (V, E) V = nodes. E = edges between pairs of nodes. Captures pairwise relationship between objects. Graph size parameters: n = V, m = E. V = { 1, 2, 3, 4, 5, 6, 7, 8 } E = { 1-2, 1-3, 2-3, 2-4, 2-5, 3-5, 3-7, 3-8, 4-5, 5-6 } n = 8 m = 11 24

25 # An example graph in python V=range(1,9) E={ 1:[2,3], 2:[3,4,5], 3:[5,7,8], 4:[5], 5:[6] } print len(v), "number n of nodes" print sum([len(e[n]) for n in E]), "number m of edges print max([len(e[n]) for n in E]), "largest degree d" >>> 8 number n of nodes 10 number m of edges 3 largest degree d 25

26 Some Graph Applications Graph Nodes Edges transportation street intersections highways communication computers fiber optic cables World Wide Web web pages hyperlinks social people relationships food web species predator-prey software systems functions function calls scheduling tasks precedence constraints circuits gates wires 26

27 World Wide Web Web graph. Node: web page. Edge: hyperlink from one page to another. cnn.com netscape.com novell.com cnnsi.com timewarner.com hbo.com sorpranos.com 27

28 9-11 Terrorist Network Social network graph. Node: people. Edge: relationship between two people. Reference: Valdis Krebs, 28

29 Ecological Food Web Food web graph. Node = species. Edge = from prey to predator. Reference: 29

30 Graph Representation: Adjacency Matrix Adjacency matrix. n-by-n matrix with A uv = 1 if (u, v) is an edge. Two representations of each edge. Space proportional to n 2. Checking if (u, v) is an edge takes Θ(1) time. Identifying all edges takes Θ(n 2 ) time

31 Graph Representation: Adjacency List Adjacency list. Node indexed array of lists. Two representations of each edge. Space proportional to m + n. Checking if (u, v) is an edge takes O(deg(u)) time. Identifying all edges takes Θ(m + n) time. degree = number of neighbors of u

32 Paths and Connectivity Def. A path in an undirected graph G = (V, E) is a sequence P of nodes v 1, v 2,, v k-1, v k with the property that each consecutive pair v i, v i+1 is joined by an edge in E. Def. A path is simple if all nodes are distinct. Def. An undirected graph is connected if for every pair of nodes u and v, there is a path between u and v. 32

33 Cycles Def. A cycle is a path v 1, v 2,, v k-1, v k in which v 1 = v k, k > 2, and the first k-1 nodes are all distinct. cycle C =

34 Trees Def. An undirected graph is a tree if it is connected and does not contain a cycle. Theorem. Let G be an undirected graph on n nodes. Any two of the following statements imply the third. G is connected. G does not contain a cycle. G has n-1 edges. 34

35 Rooted Trees Rooted tree. Given a tree T, choose a root node r and orient each edge away from r. Importance. Models hierarchical structure. root r parent of v v child of v a tree the same tree, rooted at 1 35

36 Phylogeny Trees Phylogeny trees. Describe evolutionary history of species. 36

37 GUI Containment Hierarchy GUI containment hierarchy. Describe organization of GUI widgets. Reference: 37

38 Next time: Graph Traversals and Minimum Spanning Trees