ABHELSINKI UNIVERSITY OF TECHNOLOGY

Transcription

1 Local Search Algorithms for Random Satisfiability Pekka Orponen (joint work with Sakari Seitz and Mikko Alava) Helsinki University of Technology Local Search Algorithms for Random Satisfiability 1/30

2 Outline Background WalkSAT and related algorithms Results and conjectures on WalkSAT Experiments on WalkSAT Record-to-Record Travel and variants Experiments on RRT Focused Metropolis Search Experiments on FMS Dynamics of FMS Analysis? Local Search Algorithms for Random Satisfiability 2/30

3 Background Denote N = number of variables, M = number of clauses, α = M/N. Satisfiability transition at α c (Mitchell et al. 1992,..., Braunstein et al. 2002). Good experiences with local search methods in the satisfiable region α < α c : e.g. GSAT (Selman et al. 1992), WalkSAT (Selman et al. 1996). Experiments 1996: N 2000 at α α c. Local Search Algorithms for Random Satisfiability 3/30

4 GSAT Selman et al Denote by E = E F (s) the number of unsatisfied clauses in formula F under truth assignment s. GSAT(F): s = initial truth assignment; while flips < max_flips do if s satisfies F then output s & halt, else: - find a variable x whose flipping causes largest decrease in E (if no decrease is possible, then smallest increase); - flip x. Local Search Algorithms for Random Satisfiability 4/30

5 NoisyGSAT GSAT augmented by a fraction p of random walk moves. NoisyGSAT(F,p): s = initial truth assignment; while flips < max_flips do if s satisfies F then output s & halt, else: - with probability p, pick a variable x uniformly at random and flip it; - with probability (1-p), do basic GSAT move: - find a variable x whose flipping causes largest decrease in E (if no decrease is possible, then smallest increase); - flip x. Local Search Algorithms for Random Satisfiability 5/30

6 WalkSAT NoisyGSAT focused on the unsatisfied clauses. WalkSAT(F,p): s = initial truth assignment; while flips < max_flips do if s satisfies F then output s & halt, else: - pick a random unsatisfied clause C in F; - if some variables in C can be flipped without breaking any presently satisfied clauses, then pick one such variable x at random; else: - with probability p, pick a variable x in C at random; - with probability (1-p), pick an x in C that breaks a minimal number of presently satisfied clauses; - flip x. Local Search Algorithms for Random Satisfiability 6/30

7 WalkSAT vs. NoisyGSAT The focusing seems to be important: in the (unsystematic) experiments in Selman et al. (1996), WalkSAT outperforms NoisyGSAT by several orders of magnitude. Local Search Algorithms for Random Satisfiability 7/30

8 Recent results and conjectures Barthel et al. (2003): numerical experiments with WalkSAT (mainly with p = 1, some also with p < 1) at N = 50,000, α = Observed transition in the dynamics at α dyn When α < α dyn, satisfying assignments are found in linear time per variable (i.e. in a total of cn flips ), when α > α dyn exponential time is required. Similar results obtained by Semerjian & Monasson (2003), though with smaller experiments (N = 500). Explanation: for α > α dyn the search equilibrates at a nonzero energy level, and can only escape to a ground state through a large enough random fluctuation. Conjecture: no local search algorithm works in linear time beyond the clustering transition at α s (Mézard, Monasson, Weigt et al.) Local Search Algorithms for Random Satisfiability 8/30

9 WalkSAT experiments (3-SAT) WalkSAT (N = 10^5) WalkSAT (N = 10^5) flips/n 100 flips/n p = 0.45 p = 0.55 p = 0.57 p = p = 0.45 p = 0.55 p = 0.57 p = alpha alpha Normalised solution times for WalkSAT, α = Left: complete data; right: medians and quartiles. Local Search Algorithms for Random Satisfiability 9/30

10 WalkSAT linear scaling WalkSAT (p = 0.55, alpha = 4.15) WalkSAT (p = 0.55, alpha = 4.20) cum.freq cum.freq N = 10^4 N = 3*10^4 N = 10^5 N = 3*10^5 N = 10^ N = 10^4 N = 3*10^ flips/n flips/n Cumulative solution time distributions for WalkSAT with p = Local Search Algorithms for Random Satisfiability 10/30

11 WalkSAT optimal noise level? WalkSAT (N = 10^5) flips/n alpha = 4.10 alpha = 4.12 alpha = alpha = 4.16 alpha = 4.18 alpha = 4.20 alpha = p Normalised solution times for WalkSAT with p = , α = Local Search Algorithms for Random Satisfiability 11/30

12 WalkSAT sensitivity to noise WalkSAT (p = 0.55, alpha = 4.20) WalkSAT (p = 0.57, alpha = 4.20) cum.freq cum.freq N = 10^4 N = 3*10^ N = 10^4 N = 3*10^4 N = 10^5 N = 3*10^5 N = 10^ flips/n flips/n Cumulative solution time distributions for WalkSAT at α = 4.20 with p = 0.55 and p = Local Search Algorithms for Random Satisfiability 12/30

13 Record-to-Record Travel (RRT) Very simple stochastic local optimisation algorithm introduced by Dueck (1993). Dueck claimed good results on solving 442-city and 532-city TSP s; after that little used. RRT(E,d): s = initial feasible solution; s* = s; E* = E(s); while moves < max_moves do if s is a global min. of E then output s & halt, else: pick a random neighbour s of s; if E(s ) <= E* + d then let s = s ; if E(s ) < E* then: s* = s ; E* = E(s ). Local Search Algorithms for Random Satisfiability 13/30

14 RRT in action (d = 2) Local Search Algorithms for Random Satisfiability 14/30

15 RRT in action (d = 2) Local Search Algorithms for Random Satisfiability 14/30

16 RRT in action (d = 2) Local Search Algorithms for Random Satisfiability 14/30

17 RRT in action (d = 2) Local Search Algorithms for Random Satisfiability 14/30

18 RRT in action (d = 2) Local Search Algorithms for Random Satisfiability 14/30

19 RRT in action (d = 2) Local Search Algorithms for Random Satisfiability 14/30

20 RRT in action (d = 2) Local Search Algorithms for Random Satisfiability 14/30

21 RRT in action (d = 2) Local Search Algorithms for Random Satisfiability 14/30

22 Focused RRT In applying RRT to SAT, E(s) = number of clauses unsatisfied by truth assignment s. Single-variable flip neighbourhoods. Focusing: flipped variables chosen from unsatisfied clauses. (Precisely: one unsatisfied clause is chosen at random, and from there a variable at random.) FRRT = focused RRT. Local Search Algorithms for Random Satisfiability 15/30

23 FRRT experiments (3-SAT) Focused Record-to-Record Travel (N = 10^5) Focused Record-to-Record Travel (N = 10^5) flips/n 100 flips/n d = 5 d = 7 d = 9 d = d = 5 d = 7 d = 9 d = alpha alpha Normalised solution times for FRRT, α = Left: complete data; right: medians and quartiles. Local Search Algorithms for Random Satisfiability 16/30

24 FRRT linear scaling Focused Record-to-Record Travel (alpha = 4.15) Focused Record-to-Record Travel (alpha = 4.20) cum.freq cum.freq N = 10^4 N = 3*10^4 N = 10^5 N = 3*10^5 N = 10^ N = 10^4 N = 3*10^4 N = 10^5 N = 3*10^5 N = 10^ flips/n flips/n Cumulative solution time distributions for FRRT with d = 9. Local Search Algorithms for Random Satisfiability 17/30

25 FRRT linear scaling (cont d) Focused Record-to-Record Travel (alpha = 4.15) Focused Record-to-Record Travel (alpha = 4.20) cum.freq cum.freq N = 10^4 N = 3*10^4 N = 10^5 N = 3*10^5 N = 10^ N = 10^4 N = 3*10^ flips/n flips/n Cumulative solution time distributions for FRRT with d = 7. Local Search Algorithms for Random Satisfiability 18/30

26 FRRT qualitative observations 10 4 x N flips d=5 d=6 d=7 d=8 d=9 WalkSAT fitted curves intersections a α For each fixed d there seems to be a transition value α d s.th. for α < α d the algorithm runs in linear time per variable, and for α > α d requires exponential time per variable. (Empirical estimates: α , α , α , α , α ) Local Search Algorithms for Random Satisfiability 19/30

27 FRRT qualitative observations cont d 10 4 x N flips d=5 d=6 d=7 d=8 d=9 WalkSAT fitted curves intersections a α An empirical fit of the transition points α d suggests that for large d they converge towards α Comparative experiments using WalkSAT with near optimal parameter settings (p = 0.55) yield estimate α dyn 4.19 for WalkSAT s transition point. Local Search Algorithms for Random Satisfiability 20/30

28 WalkSAT & FRRT on structured problems Jia, Moore & Selman (2004) tested WalkSAT and FRRT on highly structured glassy 3-SAT formulas. Here the number of variables is always of the form N = L L; values of L = 5,8,10,11,16 were tried out. At L = 16 WalkSAT no longer converged; FRRT did, but only for d = 5. 1e+09 WalkSAT and FRRT on NM glassy formulas (alpha = 4) 1e+08 1e+07 total flips 1e WalkSAT WalkSAT median FRRT FRRT median N Local Search Algorithms for Random Satisfiability 21/30

29 Focused Metropolis Search Arguably the most natural focused local search algorithm. Variable flip acceptance probabilities determined by a parameter η, 0 η 1. FMS(F,eta): s = initial truth assignment; while flips < max_flips do if s satisfies F then output s & halt, else: pick a random unsatisfied clause C in F; pick a variable x in C at random; let x = flip(x), s = s[x /x]; if E(s ) <= E(s) then flip x, else: flip x with prob. eta^(e(s )-E(s)). Local Search Algorithms for Random Satisfiability 22/30

30 FMS experiments (3-SAT) Focused Metropolis Search (N = 10^5) Focused Metropolis Search (N = 10^5) flips/n 100 flips/n eta = 0.2 eta = 0.3 eta = 0.36 eta = eta = 0.2 eta = 0.3 eta = 0.36 eta = alpha alpha Normalised solution times for FMS, α = Left: complete data; right: medians and quartiles. Local Search Algorithms for Random Satisfiability 23/30

31 FMS linear scaling Focused Metropolis Search (eta = 0.3, alpha = 4.00) Focused Metropolis Search (eta = 0.3, alpha = 4.10) cum.freq cum.freq N = 10^4 N = 3*10^4 N = 10^5 N = 3*10^5 N = 10^ N = 10^4 N = 3*10^4 N = 10^5 N = 3*10^5 N = 10^ flips/n flips/n Cumulative solution time distributions for FMS with η = 0.3. Local Search Algorithms for Random Satisfiability 24/30

32 FMS optimal acceptance ratio? Focused Metropolis Search (N = 10^5) flips/n alpha = 4.10 alpha = 4.12 alpha = alpha = 4.16 alpha = 4.18 alpha = alpha = eta Normalised solution times for FMS with η = , α = Local Search Algorithms for Random Satisfiability 25/30

33 FMS optimal acceptance ratio cont d Focused Metropolis Search (eta = 0.36, alpha = 4.20) cum.freq N = 10^4 N = 3*10^4 N = 10^5 N = 3*10^ flips/n Cumulative solution time distributions for FMS with η = 0.36, α = Local Search Algorithms for Random Satisfiability 26/30

34 Analysis? For FRRT: landscape structure? For FMS: contact processes? Local Search Algorithms for Random Satisfiability 27/30

35 Combinatorial landscapes Reidys & Stadler (SIAM Rev. 2002) f z b(x) x y barrier height from x to y: b(x,y) = min{max{f(z) f(x),0 z p} p an x-y path} barrier height of x Opt: b(x) = min{b(x,y) f(y) < f(x)} depth of a landscape: D f = max{b(x) x Opt} Local Search Algorithms for Random Satisfiability 28/30

36 Focused search as a contact process Local Search Algorithms for Random Satisfiability 29/30

37 Focused search as a contact process Local Search Algorithms for Random Satisfiability 29/30

38 Focused search as a contact process Local Search Algorithms for Random Satisfiability 29/30

39 Focused search as a contact process Local Search Algorithms for Random Satisfiability 29/30

40 References References [1] S. Seitz, P. Orponen: An efficient local search method for random 3-satisfiability. Proc. LICS 03 Workshop on Typical Case Complexity and Phase Transitions (Ottawa, Canada, June 2003). Elsevier Electronic Notes in Discrete Mathematics Vol. 16. [2] S. Seitz, M. Alava, P. Orponen: Threshold behaviour of WalkSAT and focused Metropolis search on random 3-satisfiability. Proc. 8th Intl. Conf. on Theory and Applications of Satisfiability Testing (St. Andrews, Scotland, June 2005). Springer-Verlag, Berlin, to appear. [3] S. Seitz, M. Alava, P. Orponen: Focused local search algorithms for random 3-satisfiability. To appear. Local Search Algorithms for Random Satisfiability 30/30