Examples of P vs NP: More Problems

Transcription

1 Examples of P vs NP: More Problems COMP1600 / COMP6260 Dirk Pattinson Australian National University Semester 2, 2017

2 Catch Up / Drop in Lab When Fridays, Where N335, CSIT Building (bldg 108) Until the end of the semester This is not a tutorial, but we re happy to answer all your questions. No registration or anything required just drop by! 1 / 41

3 Hamiltonian Paths Given. An undirected Graph G. Decision Problem. is there a path in G that visits each vertex exactly once? Computation Problem. Compute a path in G that visits each vertex exactly once (if it exists) Your Gut Feeling. Easy (polynomial) or hard (NP or worse)? 2 / 41

4 Hamiltonian Paths: Naive Approach Naive Algorithm Given G = (V, E) n possibilities for 1st vertex, n 2 for 2nd, n 3 for third... overall: n! different paths to check Estimate on a machine that does 1,000,000 steps per second simplified: this does not include overhead for moving on Turing tapes size time Q. What is the unit of time in the right-hand column? 3 / 41

5 Hamiltonian Paths: Naive Approach Naive Algorithm Given G = (V, E) n possibilities for 1st vertex, n 2 for 2nd, n 3 for third... overall: n! different paths to check Estimate on a machine that does 1,000,000 steps per second simplified: this does not include overhead for moving on Turing tapes size time Q. What is the unit of time in the right-hand column? A. Millions of years... 3 / 41

6 NP-Approach to Hamiltonian Paths Recall. An NP-problem is a problem that has polytime verifiers for each instance, there exists a polysize certificate that can be checked in polynomial time Q. What counts as certificate for a Hamiltionian Path? 4 / 41

7 NP-Approach to Hamiltonian Paths Recall. An NP-problem is a problem that has polytime verifiers for each instance, there exists a polysize certificate that can be checked in polynomial time Q. What counts as certificate for a Hamiltionian Path? A. Of course the path itself! list of vertices v 1,..., v n Checking of a certificate v 1,..., v n each v i, v i+1 must be an edge there are no duplicates there are exactly n vertices 4 / 41

8 Certificates for Hamiltonian Paths Certificate given a graph G = (V, E) a list v 1,..., v n of vertices Certificate Checking. each two successive entries must be an edge: walk through all edges and compare polytime absence of duplicates: walk though list and check polynomial length of path = number of vertices: counting polytime Difficult Question. If we consider a path as a certificate, how large is this certificate? 5 / 41

9 Hamiltonian Paths: Certificate Size Adjacency Matrix Representation of a graph with n vertices n bitstrings of length n i-th bit in the j-th bitstring = 1: have vertex v i v j Size of Certificate for a graph with n vertices each vertex can be represented using log n bits need n vertices, so n log n bits overall so n 2 bits, i.e. polynomial in the size of the graph Corollary. Checking whether a Hamiltonian Path exists is in NP. 6 / 41

10 Hamiltonian Paths: Certificate Size List Representation of a graph with n vertices number of vertices (in binary): log n bits list of vertices, each vertex 2 log n bits Worst Case. The empty graph with n vertices coded by a binary representation of n log n bits Certificate Size. n log n bits not polynomial in the representation of the graph n log n = 2 k k where k = log n is the size of the graph Observation. coding is important depending on representation, have polynomial or exponential certificates Hamiltonian Path is in NP if an n-vertex graph is represented by n bits (at least) 7 / 41

11 Hamiltonian Path via Dynamic Programming Given. Graph G = (V, E) Idea. Let S V and v S. Then 1. there is a path that visits all vertices in S and ends in v iff 2. v is connected to a vertex w and there is a path that covers all vertices in S \ {v} and ends in w Dynamic Programming. for S V and v V, record existence of cover of S ending in v (yes/no) Start with smallest S/v pairs ({v}, v) successively enlarge S by one element, and use recurrence above Complexity Analysis. number of subset / element pairs: n 2 n in each step, check all neighbours of a vertex n steps Overall Complexity. O(n 2 2 n ) = O(2 n ) 8 / 41

12 Weighted Graphs Definition. A weighted graph is an undirected graph where every edge is (additionally) labelled with a non-negative integer. Formally: A weighted graph is a triple G = (V, E, l) where (V, E) is an undirected graph (with vertices V and edges E) l : E N is a labelling function that assigns a weight (or cost) to each edge. Example. Intuition. the vertices can be locations the edges indicate whether we can travel between two locations the labels indicate the cost of travelling between two locations 9 / 41

13 Travelling Between Two Nodes Q. Given two vertices v 1 and v 2, what is the cheapest way of getting from v 1 to v 2? In More detail. as a computation problem: given vertices v 1 and v 2, find a cheapest path that connects v 1 and v 2 as a decision problem: given graph G and k 0, is there a path that connects vertices v 1 and v 2 of total cost k? Q. Easy or hard? Solvable in polynomial time? 10 / 41

14 Dijkstra s Algorithm Main Idea. to find a shortest path between v 1 and v 2, need to consider intermediate nodes more general problem: shortest path between v 1 and any vertex v 2 a bit like breadth-first search Data Structures. Given a start vertex s cheapest[v]: For every vertex v, the price of the cheapest path from s to v explored[v]: a boolean marker whether v has been explored 11 / 41

15 Algorithm Detail for source vertex s Initialisation. set cheapest[s] = 0, cheapest[v] = undef for v s set explored[v] = false, all v V Iteration. 1. Select a vertex in v that is not explored and minimal: explored[v] = false cheapest[v] cheapest[w] for all w with explored[w] = false 2. for each vertex w that is adjacent to v: update, if the path s v u is cheaper, that is if cheapest[v] + cost(v, w) cheapest[w] then cheapest[w] = cheapest[v] + cost(v, w) 3. mark v as explored: explored[v] = true once a node v is marked explored, cheapest[v] is the price of the cheapest path from source to v 12 / 41

16 Dijkstra s Algorithm: Correctness Invariant. if E is the set of explored nodes, then for e E, cheapest[e] is the cost of cheapest path from source to e for u V \ E, cheapest[u] is the cost of the cheapest path to u that only visits explored nodes. True after Initialisation. trivial, as all nodes are marked as unexplored Invariant holds after every iteration. pick minimal and unexplored node u cheapest path from source to u cannot traverse unexplored nodes after (possible) update: cheapest is still minimal for paths through explored vertices At End of Iteration. cheapest[v] is cost of cheapest path from source to v Recognise the While-Rule in Hoare Logic? could formalise this in Hoare logic here: Hoare-Logic as informal guidance principle 13 / 41

17 Dijkstra s Algorithm: Complexity Worst Case Focus. need to explore all nodes In Every Iteration possibly update cheapest[v], for all vertices v Overall Complexity for a graph with n vertices run through n iterations, in each iteration: find minimal unexplored vertex n steps update cheapest[v] n steps so overall O(n 2 ) steps Low-Level Operations are harmless (polynomial) comparing two n-bit binary numbers marking / checking explored status 14 / 41

18 Shortest Paths Decision Problem. Given G, v 1, v 2 and k 0, is there a path of cost k from v 1 to v 2? run Dijkstra s algorithm and then check whether cheapest[v 2 ] k Computational Problem Given G, v 1 and v 2, find the shortest path from v 1 to v 2 Dijstra s algorithm only gives cost paths needs to be re-constructed idea: remember penultimate nodes 15 / 41

19 Computing Shortest Paths Initialisation. set cheapest[s] = 0, cheapest[v] = undef for v s set explored[v] = false, all v V set penultimate[v] = undef, all v V Iteration. 1. Select a vertex in v that is not explored and minimal: explored[v] = false cheapest[v] cheapest[w] for all w with explored[w] = false 2. for each vertex w that is adjacent to v: update, if the path s v u is cheaper, that is if cheapest[v] + cost(v, w) cheapest[w] then cheapest[w] = cheapest[v] + cost(v, w) and put penultimate[w] = v 3. mark v as explored: explored[v] = true once a node v is marked explored, cheapest[v] is the price of the cheapest path from source to v 16 / 41

20 Path Reconstruction Idea. penultimate[v] is the penultimate node of a cheapest path from source to v Reconstruction of a path from source to v initialisation: the last node is v iteration: path expansion on the left if partial path v1,..., v n is already constructed extend it to penultimate[v 1 ], v 1,..., v n termination: if the constructed path is of the form source,..., v n Complexity. reconstruction phase: at most n additional steps iteration phase: adds constant overhead so overall, still polynomial How About Coding Graphs as Strings? our analysis in terms of number of vertices bounded above by length of encoding 17 / 41

21 Travelling Salesman Problem Given. A weighted undirected graph vertices represent cities edges represent travel time Q. Find a path v 1 v 2 v n v 1 that begins and ends in the same city that visits each city exactly once for which the overal travel time is minimal. Q. Easy or hard? Solvable in Polytime? 18 / 41

22 Travelling Salesman: Naive Approach Naive Algorithm Given n cities, and their distances dist(c, d) initialise: construct set S consisting of all possible sequences of nodes sequences may not have repeated vertices sequences must contain all vertices of the graph computation phase: for each s = (c 1,..., c n ) S, compute the total distance i dist(c i, c j ) report the smalles such distance Q. What is the complexity of this algorithm? 19 / 41

23 Travelling Salesman: Naive Approach Counting Permutations of n vertices n possibilities for 1st city, n 2 for 2nd, n 3 for third... overall: n! different paths to check Estimate on a machine that does 1,000,000 steps per second simplified: this does not include overhead for moving on Turing tapes size time Q. What is the unit of time in the right-hand column? (we have seen this before) 20 / 41

24 Travelling Salesman as Decision Problem Travelling Salesman as Decision Problem. Given weighted graph G and k 0, is there a path that vists each vertex in G exactly once is of total cost k Guessing and Checking. Certificate for existence of path is path itself of polynomial size (if G is encoded reasonably, i.e at least one bit per vertex) Certificate Checking. As for Hamiltonian paths, plus check that the total cost of the path is k n 1 additions, one comparision polynomial so checking is polynomial, overall Corollary. Travellign Salesman as a decision problem is in NP. 21 / 41

25 The Knapsack Problem Given. A knapsack with maximal capacity C, and items with weights w 1,..., w n and values v 1,... v n What s the best way to fill the knapsack? sum of the weight of the items shouldn t exceed capacity sum of the values of the items should be maximal. Assumptions. all weights are strictly positive, i.e. w i 0 for all i = 1,..., n there is an unlimited supply of each item 22 / 41

26 Knapsack: Main Idea Construct a Table C m(0) m(1) m(2)... m(c) m(w) = value of the best knapsack with weight w m(0) = 0 (empty knapsack) m(w) = max{v i + m(w w i ) w i w} Solution. m(c) value of the best knapsack with capacity C Correctness Argument. the best knapsack with weight w must contain an item say item i removing this item, we get the best knapsack with weight w w i (otherwise, can have better knapsack with weight w) Solution. iteratively compute m(0), m(1),..., m(c) store already computed values in a table (don t recompute) 23 / 41

27 Knapsack: Complexity Analysis Complexity Analysis given capacity C and n items need to construct a table with C + 1 entries (starting at zero) for each > 0 entry for weight: need to check n items, compute maximum Overall complexity: n C Q. Does that mean that knapsack is linear or quadratic? 24 / 41

28 Knapsack: Encoding Recall. Complexity depends on encoding of input data usual encoding for numbers: as binary strings For Knapsack: Capacity: C as a binary integer (log C bits) number of items n as binary ingeger (log n bits) weights as n-element lists of binary integers values as n-element lists of binary integers Q. How large is C as a function of the length of the encoding of the knapsack problem in the worst case? 25 / 41

29 Knapsack: Encoding Recall. Complexity depends on encoding of input data usual encoding for numbers: as binary strings For Knapsack: Capacity: C as a binary integer (log C bits) number of items n as binary ingeger (log n bits) weights as n-element lists of binary integers values as n-element lists of binary integers Q. How large is C as a function of the length of the encoding of the knapsack problem in the worst case? A. In the worst case: one item, value 1, weight 1: 3 bits (plus separators) encoding of C has log C many bits, so C = 2 log C is exponential Overall Complexity. Exponential in the size of the problem if numbers are coded in binary only polynomial if C is coded in unary also called pseudo polytime 25 / 41

30 Summary of Problems Polytime Problems Given graph G, is G connected? Given weighted graph G, is there a path between vertices of cost k? Given propositional formula and truth-value assignment for the variables, does the formula evalute to true under this assignment? NP-Complete Problems Given Graph G, does it have a Hamiltonian path? Given Graph G, does it have a vertex cover of size k? Given graph G, does it have an independent set of size k? Given weighted graph G, does it have a Hamiltionian path of cost k? Given knapsack, can a value of k be achieved? On NP-Hardness. usually difficult to prove and omitted here strategy: by reduction (see e.g. vertex cover and independent set) 26 / 41

31 How About Route Planning? Route Planning as a graph problem. intersections of two roads are the nodes of the graph direct road connections are the vertices between intersections distance between intersections are the weights of the edges Google Maps (or similar) give starting point, end point, computes shortest path in seconds solves a travelling salesman problem over large graphs (millions of intersections) Q. Can we solve NP-complete problems in practice? 27 / 41

32 Route Planning in Practice Additional Information. can prune intersections by culling streets that are, say, 20km away from straight line start finish have additional information: often arterial roads connected with small roads either end not always the optimal route Problems In Practice. want/need to solve NP-complete problems (or worse) goal: best solution within given time constraints tradeoff computing time / optimality of the solution 28 / 41

33 Approximative Algorithms and Heuristics Heuristics. special problem solving strategies that work for only some instances problem instances of interest domain-dependent often: instances generated by humnans (e.g. road network) Approximative Algorithms. Goal: just a a good enough solution (e.g. for travelling salesman) strategy-based, e.g. greedy, evolutionary Example: Greedy Algorithms for travelling salesman start with a vertex, and go to nearest neighbour continue with nearest neighbour not already visited, etc. often gives good (within 25% of optimum) path Example: Iterative Strategies for travelling salesman start wiht a possibly non-optimal tour perturb the tour by removing a numner of edges replace the edges to get a shorter tour repeat until improvement is only marginal 29 / 41

34 Do not be afraid? NP-Complete Problems usually hard to solve in all cases often, just need solutions that are good enough or solutions for special cases Algorithmic Problems have appreciation of their complexity recognise hard problems choose best solutions, depending on context Engineering Context be aware of resources needed by software solution solutions for small sizes may not scale right tool for the job approach 30 / 41

35 Back to Computability Meta-Programming Use programs to to check properties of other programs here: use Turing machines to determine properties of other TMs Q. What type of properties of programs can we verify automatically? Decision Problem: given a TM, does the TM have this property? Negative Examples. Halting Problem Totality Problem: does TM M terminate on all inputs Q. Are there any positive examples? 31 / 41

36 Rice s Theorem Definition. A property P of languages of Turing machines is trivial, if either it holds for all languages or it holds for none. A property P is non-trivial, if it is not trivial. Examples of non-trivial properties of languages does a TM accept the string aab there is a TM that accepts aab, so it holds for some TMs there is a TM that doesn t accept aab, so it doesn t hold for all is the set of strings accepted by M finite? does M accept two different length strings?... Note. Properties of languages, i.e. the set of strings accepted by a TM. can reformulate in terms of L(M), sets of strings accepted by M 32 / 41

37 Rice s Theorem Theorem. Every non-trivial property of languages of Turing machines is undecidable. Detailed Meaning. Write L(M) for the language (accepted strings) of a TM. every language of a TM corresponds to a TM property of languages of TMs is of the form P = {C C a code of TM M and L(M) has property P} properties of languages are sets of TM codes the problem is C a code of a TM M s.t. L(M) has property P is undecidable if P is non-trivial. 33 / 41

38 Rice s Theorem: Non-Examples Caveat. Rice s Theorem is about properties of languages. It s not about properties of TMs. Properties of TMs Rice s Theorem doesn t apply does M ever write a 1 onto the tape? does M have exactly 17 states? Decidable Properties of TMs give algorithm M has 17 states (just check the encoding) on blank input, M terminates in 188 steps (just simulate M) Non-Decidable Properties of TMs give reduction there exists an input of length 20 on which M halts there is an input such that M writes a 1 onto the tape 34 / 41

39 Preliminaries: The Acceptance Problem Acceptance Problem. The problem {(M, w) M accepts w} is undecidable. Proof. By reduction to the halting problem. Let A be a TM that decides whether or not a TM M accepts a string w. Construction. Given input (M, w) to the halting problem. Result. change M to M which is like M, but all states are accepting thenm M accepts w if and only if M terminates on w run the TM A on the input (M, w) if A accepts (M, w) then M terminates on w if A does not accept (M, w) then M does not terminate on w Summary. If A would exist, we would be able to solve the halting problem, which is impossible. 35 / 41

40 Rice s Theorem: Proof Proof Strategy. By contradiction. Assume that a hypothetical TM M exists that decides whether a language has property P Create a contradiction by showing that then, we can also solve the acceptance problem (last slide) give input (M, w) to acceptance problem, construct TM M so that L(M ) has property P M accepts w as the acceptance problem is undecidable, M cannot exist. (We have seen this proof strategy before) 36 / 41

41 Proof Detail. Assumptions. let P be a non-trivial property of languages of TMs. assume first that / P and let M P be a TM such that L(M P ) P. Construction. Given input (M, w) for acceptance problem construct TM M w that behaves as follows on input x first run M on w if M accepts w, run M P on string x Result. M w accepts x iff M accepts w and M P accepts x so L(M w ) = if M does not accept w and L(M w ) = L(M P ) if M does accept w hence L(M w ) satisfies P iff M accepts w because L(MP ) P and / P Summary. If P were decidable, we could decide the acceptance problem. 37 / 41

42 Proof of Rice s Theorem, Continued Recall. have assumed that / P i.e. the empty language does not have property P Missing Case. P then the complement of P of P is a non-trivial property the complement P satisfies / P by first case, P is not decidable. Construction. Assume that P were decidable by TM M P swap accepting and non-accepting states to obtain M P accepts precisely the complement of P M P Summary. M P cannot exist, as the complement of P is not decidable. 38 / 41

43 Summary Bad News. there are lots of problems that are undecidable about TMs Rice s theorem gives plenty any property of languages But... specific instances may be decidable often useful to search for solutions Example. Provability in First-Order Logic not decidable, but there are plenty of implementations implementations may not terminate... goal: try and solve as many problems as possible problem of interest: usually human-generated 39 / 41

44 Course Summary Alignment you can already do programming (functional / imperative) Questions. What do programs really do? What is computation in general? What are the limits? Answers. use logic to formally describe systems functional programs: induction proofs imperative programs: Hoare logics computation in general: Turing machines 40 / 41

45 Exam: Nov 3, 2017 Good Luck! use the drop-in session use the study event use your tutorials 41 / 41