Integrating Probabilistic Reasoning with Constraint Satisfaction

Transcription

1 Integrating Probabilistic Reasoning with Constraint Satisfaction IJCAI Tutorial #7 Instructor: Eric I. Hsu July 17,

2 Bottom Line Up Front (BLUF) Thesis: The insights from Block A can be operationalized to make concrete contributions to the state of the art for constraint satisfaction. (For instance, what s the most likely setting for a variable if we are trying to satisfy a given problem.) Part I: The decision problem: SAT and CSP Part II: The optimization problem: (partial, weighted) MAXSAT 2

3 BLUF: Opportunities and Challenges Relative to standard CSP heuristics and techniques, methods resulting from probabilistic reasoning are: Extremely powerful (in finding solutions). Extremely expensive (in running time, in terms of engineering if not algorithmically). Sensitive to a different kind of problem structure. There remain a lot of other designs to consider. To this point, more formalization than experimentation. Approach generalizes a wide space of existing work. 3

4 Motivation Research today is driven by solving problems. 4

5 Overview of Block B (and C) 5

6 I. The Decision Problem: SAT and CSP List of Solutions Magic Viewer Constraint Program List of Solutions 6

7 Overview of Part I Calculating Surveys to Guide Search. EMBP (Expectation Maximization Belief Propagation). Review: The Algebraic Semiring as a Unifying Representation. Using Surveys within a Backtracking Solver. Empirical Results. 7

8 CALCULATING SURVEYS TO GUIDE SEARCH Motivation Representation Algorithm

9 Motivational Dream List of Solutions Magic Viewer Constraint Program List of Solutions 9

10 Instead: Sampling from the Solution Space List of Solutions Maximum Likelihood Estimator Constraint Program Survey : Probability for each variable to be fixed a certain way when sampling a solution at random. P(Θ v SAT) 10

11 Finding and Exploiting the Highly Constrained Variables Finding Sets of Highly Constrained Variables Guessing e.g. (Gomes et al., 98) Heuristics / Learning e.g. (Boussemart et al., 04) Surveys: Belief Propagation (BP) and Survey Propagation (SP) e.g. (Kask et al., 04; Braunstein et al., 05) 11

12 Finding and Exploiting the Highly Constrained Variables Exploiting Sets of Highly Constrained Variables within Backtracking Search Finding Sets of Highly Constrained Variables Guessing e.g. (Gomes et al., 98) Heuristics / Learning e.g. (Boussemart et al., 04) Surveys: Belief Propagation (BP) and Survey Propagation (SP) e.g. (Kask et al., 04; Braunstein et al., 05) Random Restarts Variable Ordering toward Solutions Multiple Options 12

13 Example: Quasigroup with Holes 13

14 Example: Boolean Satisfiability 14

15 Example: Boolean Satisfiability 15

16 Survey Propagation (for SAT) BP SP Θ v+ : variable is constrained to be positive. Θ v - : variable is constrained to be negative. Θ v* : variable is unconstrained. BP SP 16

17 Estimating Biases via Propagation C2 C3 C4 V2 C5 V3 C6 C7 Message-Passing Variable->Clause: Likely Bias Clause->Variable: Likely Need for Support C1 var appears positively in clause var appears negatively in clause Probabilities reflect status in absence of recipient. V1 17

18 Belief Propagation Update Rule Single update rule can represent entire messagepassing dynamics. Θ v : new positive bias for variable v Numerator: probability of positive bias Denominator: completed sample space 18

19 Belief Propagation Update Rule Single update rule can represent entire messagepassing dynamics. Θ v : new positive bias for variable v Numerator: probability of positive bias FORALL clauses c where v appears negatively: Denominator: completed sample space [ NOT all other variables fail to support c ] 19

20 Factor Graph Representation for QWH Variable node for each cell in the Latin Square. 1,1 1,2 1,3 1,4 1,5 2,1 2,2 2,3 2,4 2,5 3,1 3,2 3,3 3,4 3,5 4,1 4,2 4,3 4,4 4,5 5,1 5,2 5,3 5,4 5,5 Using Expectation Maximization to Find Likely Assignments for Solving CSP s 20

21 Factor Graph Representation for QWH Variable node for each cell in the Latin Square. 1,1 1,2 1,3 1,4 1,5 Constraint node for each row alldiff. Row 1 Row 2 2,1 3,1 2,2 3,2 2,3 3,3 2,4 3,4 2,5 3,5 Row 3 4,1 4,2 4,3 4,4 4,5 Row 4 5,1 5,2 5,3 5,4 5,5 Row 5 Using Expectation Maximization to Find Likely Assignments for Solving CSP s 21

22 Factor Graph Representation for QWH Variable node for each cell in the Latin Square. 1,1 Column 1 1,2 Column 2 Column 3 Column 4 Column 5 1,3 1,4 1,5 Constraint node for each row alldiff. Row 1 Row 2 2,1 3,1 2,2 3,2 2,3 3,3 2,4 3,4 2,5 3,5 Constraint node for each column alldiff. Row 3 4,1 4,2 4,3 4,4 4,5 Row 4 5,1 5,2 5,3 5,4 5,5 Row 5 Using Expectation Maximization to Find Likely Assignments for Solving CSP s 22

23 Message Passing in the Factor Graph Variable to Constraint: Current bias distribution. (i.e. Θ 2,4 (): the probability that variable 2,4 holds each possible value) Constraints re-weight possible extensions. 2,4 Column 4 1,4 3,4 Row 2 4,4 2,1 2,2 2,3 2,5 5,4 Using Expectation Maximization to Find Likely Assignments for Solving CSP s 23

24 Message Passing in the Factor Graph Constraint to Variable: Level of support for each possible value. (i.e. the probability that Row 2 realizes each possible permutation, or some approximation) Variables update biases. Row 2 2,4 Column 4 1,4 3,4 4,4 2,1 2,2 2,3 2,5 5,4 Using Expectation Maximization to Find Likely Assignments for Solving CSP s 24

25 Problems with regular BP/SP BP/SP might not converge. Only proper for problems whose constraint graphs are trees; non-trivial problems are full of loops Surveys are only approximations. Local maximum in likelihood Iteration dynamics are poorly understood. Characterizing non-convergence, likely setting 25

26 EXPECTATION MAXIMIZATION BELIEF PROPAGATION (EMBP) Expectation Maximization (EM) Interpreting the Algorithm Choosing a Consistency Condition

27 Expectation Maximization (EM) P(Y Θ) P(Y, Z Θ) Z=1 Z=2 Z=3 Problem: Find Θ to maximize P(Y, Z Θ). Y: observed data Θ: model parameters Z: unobserved data Can t marginalize on Z, or else it would be easy. 27

28 The EM Algorithm Solution: Artificial estimate Q(Z) to match P(Z Y, Θ) E-Step: Set Q(Z) to estimate P(Z Y, Θ) M-Step: Set Θ to maximize P(Y, Z Θ) Z=1 Z=2 Z=3 P(Y, Z Θ) Important: Use logs of all probabilities, to exploit Jensen s Inequality EM always converges. Q(Z) 28

29 Using EM to Compute Surveys for CP The basic intuition is to deceive EM. Observations Y: We saw that each of the constraints was satisfied i.e., We saw a solution to the CSP Hidden Variables Z: but we didn t get to see how exactly the constraints were satisfied by the variables in their domains. Parameters Θ: Please figure out how the variables were likely set (by hypothesizing likely Z s representing how the constraints were satisfied.) Parameters are just the variables biases. 29

30 The EMBP Update Rule 1 ( val) Q( Z) var Z:var val 30

31 Primal/Dual Semantics E-Step (Dual) Use the biases to update Q(Z). Inference variable settings determine the possible extensional values of the constraints. M-Step (Primal) Use Q(Z) to update the biases. Search constraint extensions filter the possible values of the variables. Optimal alternation between steps, to guaranteed convergence. 31

32 QWH: The Dual (Extensional) Problem EMBP-e (exact) Expensive: represent each of the d! possible permutations in the extension of a row/column constraint. Dynamic programming allows 2 d complexity. Necessary to represent global consistency EMBP-a (approximate) Inexact: a cell can hold a particular value if nobody else in its row/column does. (Doesn t represent relationship between other cells within the same constraint.) Linear time complexity on d. Represents arc consistency. 5 32

33 QWH Dynamics individual bias distribution Θ 2,4 () Θ 2,4 (1) =.081 Θ 2,4 (2) =.074 Θ 2,4 (3) =.653 Θ 2,4 (4) =.078 probability that cell 2,4 contains 5 artificial distribution over constraint extensions Q(Z) Θ 2,4 (5) = probability of an extension to the 2 nd row constraint where 5 is in 4 th position probability of extension to the 4 th column constraint where 5 is in 2 nd position 33

34 SAT: The Dual (Extensional) Problem Suppose clause c = x y z. Support profile represents which of the possible instantiations to x, y, and z was chosen to satisfy clause c. S X Y Z X valid BP/SP invalid SP only valid BP/SP SP only valid BP/SP SP only SP only C {1,4,6} Z Y 34

35 SAT Dynamics E-Step Q(S) Θ M-Step S X Y Z X valid BP/SP invalid SP only valid BP/SP SP only valid BP/SP SP only SP only C {1,4,6} Z Y 35

36 BP becomes EMBP: New Update Rule Almost the same as original BP equation! As if BP/SP accidentally produced this primal/dual form directly, but by declaring probabilities instead of sums. Using ratios of sums results in gentler steps. Result: Guarantee of Convergence to Local Maximum in Likelihood 36

37 Survey Techniques and Problem Topography Techniques can reach different local maxima in likelihood from the same random initialization. BP/EMBP both follow gradient, but non-convergent techniques like BP take bigger steps that can overshoot local maxima. Using Expectation Maximization to Find Likely Assignments for Solving CSP s 37

38 SAT Basic Building Block: Sole Support Variable v is the sole support of constraint c. C U V W X Y Z 38

39 EMBP-L / EMSP-L Local Approximation 39

40 EMBP-G / EMSP-G Globalized Approximation 40

41 THE ALGEBRAIC SEMIRING AS A UNIFYING REPRESENTATION

42 Different Problem Applications Problem Area (many) (many) Constraint Optimization [Bistarelli & Rossi, 2001] (many) (many) Statistical Algorithms [Aji &McEliece, 2000] Maximization Product MAP/MPE Maximization Summation (Weighted) MAXSAT Minimization Maximization Game Theory [von Neumann, 1928] Disjunction Conjunction AND/OR Trees, Compilation [Dechter & Mateescu, 2007] Project Join Database Query Optimization [Mishra & Koudas, 2009] 42

43 Primal / Dual Optimization Primal Problem Search (like DPLL) BP s clause-to-variable message EM s M-Step (MLE over Q(S)) Entropy Minimization Sum: disjunction, addition, etc. Dual Problem Inference (like DP) BP s variable-to-clause message EM s E-Step (Building Q(S)) Free energy minimization Product: conjunction, multiplication, etc. 43 Also: Computing Surveys vs. Local Search/MAXSAT: P(Θ SAT) P(SAT) = P(SAT Θ) P(Θ)

44 USING BIAS ESTIMATES WITHIN A SOLVER Frameworks Design Decisions

45 Non-Backtracking Search Survey Propagation Framework: Unit Decimation CP Instance New Instance New... Instance SOLUTION - OR - FAILURE Maximum Likelihood Estimator 45

46 Integrating with Backtracking Search Maximum Likelihood Estimator New Instance SAT Instance Survey Choose Variable(s), Value(s) Fix Variable(s) and Simplify Variable/Value-Ordering heuristic for backtracking search Pick a variable to instantiate next, by bias strength Set that variable according to its stronger bias Revert to standard search heuristic when bias levels off 46

47 Design Issues / Parameters Survey Choose Variable(s), Value(s) Decimation Block Size Branching Rule Selection Criterion New Instance Fix Variable(s) and Simplify Deactivation Threshold Working with nogoods 47

48 6. EMPIRICAL RESULTS Standalone Bias Estimation Non-Backtracking Search Backtracking Search

49 SAT Standalone Performance (Experiment conducted with Christian Muise) 49

50 SAT: Backbone Detection EMBP: True (Positive) Bias vs. Estimated Bias 50

51 SAT: Dynamics for SP SP: Bias Estimate for Selected Variables, over Solver Iterations 51

52 SAT: Dynamics for EMSP EMSP: Bias Estimate for Selected Variables, over Solver Iterations 52

53 SAT: Non-Backtracking Solver 53

54 SAT: Backtracking Solver 54

55 SAT: Larger / Industrial Problems Engineering: Memory management a key issue. Toughest type of problem set: SAT-Race. Comparison with default MiniSat (SAT-Race 2010): 131 problems were solved by both. Minisat Mean Runtime / STD: 111 seconds / 154 seconds VARSAT Mean Runtime / STD: 147 seconds / 202 seconds 9 problems solved by only Minisat. 5 problems solved by only using VARSAT. 55

56 Nonograms: EMBP-Gsup [LeBras, Zanarini, and Pesant 09] 56

57 QWH: EMBP-Gsup [LeBras, Zanarini, and Pesant 09] 57

58 Magic Square: EMBP-Gsup [LeBras, Zanarini, and Pesant 09] 58

59 Summary Formally, Applying EMBP with global constraints significantly improves BP/SP. Algorithmically, Provides very valuable information (for preventing/seeking conflicts.) Practically, Information is very expensive (in time/space.) 59

60 II. The Optimization Problem: MAXSAT f c (v 1,,v n ) = 60

61 Working with MAXSAT

62 Underlying Framework: MaxSAT f c (v 1,,v n ) = 1, if v 1, v n satisfies constraint c; 0 if it doesn t and c is a hard constraint; otherwise. P (v 1,,v n ) = (1/Z) f c (v 1,, v n ) c P (v 1 = + ) = (1/Z 1 ) f c (+,, v n ) V:v 1 = + c Encode near-uniform distribution over solutions to a COP. 62

63 Variable Ordering Results 63

64 Extensional Representation within a Factor Graph Function height: maximum achievable value for each function under any extension (i.e. any tuple of arguments). Problem height: sum of function heights. Can use as an upper bound on the score of any solution, for pruning during branch-and-bound-search.

65 MHET: Minimum Height Equivalent Transformation Equivalent Transformation: fixed format for mapping a given problem to one that assigns to same score for all configurations. We use edge potentials. MHET: within the space of equivalent problems that are representable under a chosen transformation scheme, find one with minimum height; this produces tighter bounds and thus encourages more frequent pruning. We use iterated node averaging with guaranteed convergence to a local minimum.

66 MHET Lower-Bounding Results 66

67 Integrating Probabilistic Reasoning with Constraint Satisfaction IJCAI Tutorial #7 Instructor: Eric I. Hsu July 17,

68 Lots of Room for Improvement Parameters and Design Algorithmic Shortcuts Implementational Shortcuts Interaction with Rest of System: Restarts, Branching, Learning 68

69 Open Challenges Other queries, like MAP. Configuring the Algorithms. Other Direction: Applying CP Concepts to ML. Local Search. SP for Constraints. Better Bias Estimators. 69

70 Future Work Formal: Exploiting Correspondences. Global constraints for probabilistic inference. Backdoor variables. Treatment of joker bias in SP model. Algorithmic: Local Search and Gibbs Sampling. Implementational: Winning SAT and MAXSAT contests. Approximate surveys and implementational shortcuts. Parameter tuning. 70

71 Conclusions Formally, Constraint Satisfaction and Probabilistic Inference are more closely related than even folklore would suggest. Optimization perspective extends connection beyond graph structure. Algorithmically, Applying EMBP with global constraints significantly improves BP/SP. Provides very valuable information (for preventing/seeking conflicts.) Practically, Information is very expensive (in time/space.) 71

72 Correspondence and Resources 72

73 Collaboration with Sheila McIlraith, Fahiem Bacchus, Chris Beck, Matthew Kitching, Christian Muise, and many others!

74 Map of Subjects 74