CMU 15-7/381 CSPs. Teachers: Ariel Procaccia Emma Brunskill (THIS TIME) With thanks to Ariel Procaccia and other prior instructions for slides

Size: px

Start display at page:

Download "CMU 15-7/381 CSPs. Teachers: Ariel Procaccia Emma Brunskill (THIS TIME) With thanks to Ariel Procaccia and other prior instructions for slides"

Meagan Mosley
5 years ago
Views:

1 CMU 15-7/381 CSPs Teachers: Ariel Prcaccia Emma Brunskill (THIS TIME) With thanks t Ariel Prcaccia and ther prir instructins fr slides

2 Class Scheduling Wes 4 mre required classes t graduate A: Algrithms B: Bayesian Learning C: Cmputer Prgramming D: Distributed Cmputing A few restrictins Algrithms must be taken same semester as distributed cmputing Cmputer prgramming is a prereq fr distributed cmputing and Bayesian learning, s it must be taken in an earlier semester Advanced algrithms and Bayesian Learning are always ffered at the same time, s they cannt be taken the same semester 3 semesters (semester 1,2,3) when can take classes 2

3 Cnstraint Satisfactin Prblems (CSPs) Variables: V = {V 1,..,V N } Dmain: Set f d pssible values fr each variable Cnstraints: C = {C 1,..,C K } A cnstraint cnsists f variable tuple list f pssible values fr tuple (ex.[(v 2,V 3 ),{(R,B),(R,G)]) Or functin that describes pssible values (ex. V 2 V 3 ) Allws useful general-purpse algrithms with mre pwer than standard search algrithms 3

4 Overview Real wrld CSPs Basic algrithms fr slving CSPs Pruning space thrugh prpagating infrmatin 4

5 Overview Real wrld CSPs Basic algrithms fr slving CSPs Pruning space thrugh prpagating infrmatin 5

6 Example: Map Clring Clr a map s that adjacent areas are different clrs 6

7 Map Clring Variables Dmain Cnstraints Slutins 7

8 Example: Suduk Variables: Dmain: Cnstraints: 8

9 Example: Suduk Variables: Each pen sqr Dmain: {1:9} Cnstraints: 9-way all diff cl 9-way all diff rw 9-way all diff bx 9

10 Scheduling (Imprtant Ex.) Many industries. Many multi-millin $ decisins. Used extensively fr space missin planning. Military uses. Peple really care abut imprving scheduling algrithms! Prblems with phenmenally huge state spaces. But fr which slutins are needed very quickly Many kinds f scheduling prblems e.g.: Jb shp: Discrete time; weird rdering f peratins pssible; set f separate jbs. Batch shp: Discrete r cntinuus time; restricted peratin f rdering; gruping is imprtant. 10

11 Jb Scheduling A set f N jbs, J 1,, J n. Each jb j is a seq f peratins O j 1,..., Oj Lj Each peratin may use resurce R, and has a specific duratin in time. A resurce must be used by a single peratin at a time. All jbs must be cmpleted by a due time. Prblem: assign a start time t each jb. 11

12 Exercise: Define CSP 4 mre required classes t graduate: A, B, C, D A must be taken same semester as D C is a prereq fr D and B s must take C earlier than D & B A & B are always ffered at the same time, s they cannt be taken the same semester 3 semesters (semester 1,2,3) when can take classes Variables: A,B,C,D Dmain: {1,2,3} Cnstraints: A B, A=D, C < B, C < D 12

13 Overview Real wrld CSPs Basic algrithms fr slving CSPs Pruning space thrugh prpagating infrmatin 13

14 Why nt just d basic search algrithms frm last time? 14

15 Backtracking Only cnsider a single variable at each pint Dn t care abut path! 15

16 Backtracking Only cnsider a single variable at each pint Dn t care abut path! Order f variable assignment desn t matter, s fix rdering 16

17 Backtracking Only cnsider a single variable at each pint Dn t care abut path! Order f variable assignment desn t matter, s fix rdering Only cnsider values which d nt cnflict with assignment made s far 17

18 Backtracking Only cnsider a single variable at each pint Dn t care abut path! Order f variable assignment desn t matter, s fix rdering Only cnsider values which d nt cnflict with assignment made s far Depth-first search fr CSPs with these tw imprvements is called backtracking search 18

19 Backtracking Functin Backtracking(csp) returns sln r fail Return Backtrack({},csp) Functin Backtrack(assignment,csp) returns sln r fail If assignment is cmplete, return assignment V i ß select_unassigned_var(csp) Fr each val in rder-dmain-values(var,csp,assign) If value is cnsistent with assignment Add [V i = val] t assignment Result ß Backtrack(assignment,csp) If Result fail, return result Remve [V i = val] frm assignments Return fail 19

20 Backtracking Example Variables: A,B,C,D Dmain: {1,2,3} Cnstraints: A B, A=D, C < B, C < D Variable rder:? Value rder:? 20

21 Backtracking Functin Backtracking(csp) returns sln r fail Return Backtrack({},csp) Functin Backtrack(assignment,csp) returns sln r fail If assignment is cmplete, return assignment V i ß select_unassigned_var(csp) Fr each val in rder-dmain-values(var,csp,assign) If value is cnsistent with assignment Add [V i = val] t assignment Result ß Backtrack(assignment,csp) If Result fail, return result Remve [V i = val] frm assignments Return fail 21

22 Think and discuss Des the value rder used affect hw lng backtracking takes t find a slutin? Des the value rder used affect the slutin fund by backtracking? 22

23 Example Variables: A,B,C,D Dmain: {1,2,3} Cnstraints: A B, A=D, C < B, C < D Variable rder: alphabetical Value rder: Descending (A=3) 23

24 Example Variables: A,B,C,D Dmain: {1,2,3} Cnstraints: A B, A=D, C < B, C < D Variable rder: alphabetical Value rder: Descending (A=3) (A=3, B=3) incnsistent with A B (A=3, B=2) (A=3, B=2, C=3) incnsistent with C < B (A=3, B=2, C=2) incnsistent with C < B (A=3, B=2, C=1) (A=3, B=2, C=1,D=3) VALID 24

25 Example Variables: A,B,C,D Dmain: {1,2,3} Cnstraints: A B, A=D, C < B, C < D Variable rder: alphabetical Value rder: Ascending (A=1) 25

26 Example Variables: A,B,C,D Dmain: {1,2,3} Cnstraints: A B, A=D, C < B, C < D Variable rder: alphabetical Value rder: Ascending (A=1) (A=1,B=1) incnsistent with A B (A=1,B=2) (A=1,B=2,C=1) (A=1,B=2,C=1,D=1) incnsistent with C < D (A=1,B=2,C=1,D=2) incnsistent with A=D (A=1,B=2,C=1,D=3) incnsistent with A=D 26

27 Example Variables: A,B,C,D Dmain: {1,2,3} Cnstraints: A B, A=D, C < B, C < D Variable rder: alphabetical Value rder: Ascending (A=1) (A=1,B=1) incnsistent with A B (A=1,B=2) (A=1,B=2,C=1) (A=1,B=2,C=1,D=1) incnsistent with C < D (A=1,B=2,C=1,D=2) incnsistent with A=D (A=1,B=2,C=1,D=3) incnsistent with A=D N valid assignment fr D, return result = fail Backtrack t (A=1,B=2,C=) Try (A=1,B=2,C=2) but incnsistent with C < B Try (A=1,B=2,C=3) but incnsistent with C < B N ther assignments fr C, return result= fail Backtrack t (A=1,B=) (A=1,B=3) (A=1,B=3,C=1) (A=1,B=3,C=1,D=1) incnsistent with C < D (A=1,B=3,C=1,D=2) incnsistent with A = D (A=1,B=3,C=1,D=3) incnsistent with A = D Return result = fail Backtrack t (A=1,B=3,C=) (A=1,B=3,C=2) incnsistent with C < B (A=1,B=3,C=3) incnsistent with C < B N remaining assignments fr C, return fail Backtrack t (A=1,B=) N remaining assignments fr B, return fail Backtrack t A (A=2) (A=2,B=1) (A=2,B=1,C=1) incnsistent with C < B (A=2,B=1,C=2) incnsistent with C < B (A=2,B=1,C=3) incnsistent with C < B N remaining assignments fr C, return fail Backtrack t (A=2,B=?) (A=2,B=2) incnsistent with A B (A=2,B=3) (A=2,B=3,C=1) (A=2,B=3,C=1,D=1) incnsistent with C < D (A=2,B=3,C=1,D=2) ALL VALID 27

28 Ordering Matters! Functin Backtracking(csp) returns sln r fail Return Backtrack({},csp) Functin Backtrack(assignment,csp) returns sln r fail If assignment is cmplete, return assignment V i ß select_unassigned_var(csp) Fr each val in rder-dmain-values(var,csp,assign) If value is cnsistent with assignment Add [V i = val] t assignment Result ß Backtrack(assignment,csp) If Result fail, return result Remve [V i = val] frm assignments Return fail 28

29 Min Remaining Values (MRV) Chse variable with minimum number f remaining values in its dmain Why min rather than max? 29

30 Min Remaining Values (MRV) Chse variable with minimum number f remaining values in its dmain Mst cnstrained variable Fail-fast rdering 30

31 Least Cnstraining Value Given chice f variable: Chse least cnstraining value Aka value that rules ut the least values in the remaining variables t be assigned May take sme cmputatin t find this Why least rather than mst? 31

32 Click! Cst f Backtracking? d values per variable n variables Pssible number f CSP assignments? A) O(d n ) B) O(n d ) C) O(nd) 32

33 Overview Real wrld CSPs Basic algrithms fr slving CSPs Pruning space thrugh prpagating infrmatin 33

34 Limitatins f Backtracking Functin Backtracking(csp) returns sln r fail Return Backtrack({},csp) Functin Backtrack(assignment,csp) returns sln r fail If assignment is cmplete, return assignment V i ß select_unassigned_var(csp) Fr each val in rder-dmain-values(var,csp,assign) If value is cnsistent with assignment Add [V i = val] t assignment Result ß Backtrack(assignment,csp) If Result fail, return result Remve [V i = val] frm assignments Return fail 34

35 Cnstraint Graphs Ndes are variables Arcs shw cnstraints 35

36 Prpagate Infrmatin If we chse a value fr ne variable, that affects its neighbrs And then ptentially thse neighbrs Prunes the space f slutins 36

37 Arc Cnsistency Definitin: An arc (cnnectin between tw variables X à Y in cnstraint graph) is cnsistent if: Fr every value culd assign t X There exists sme value f Y that culd be assigned withut vilating a cnstraint 37

38 AC-3 (Assume binary cnstraints) Input: CSP Output: CSP, pssible with reduced dmains fr variables, r incnsistent Lcal variables: stack, initially stack f all arcs (binary cnstraints in csp) While stack is nt empty (X i,x j ) = Remve-First(stack) [dmainx i, anychangetdmainx i ] = Revise(csp,X i,x j ) if anychangetdmainx i == true if size(dmainx i ) = 0, return incnsistent else fr each X k in Neighbrs(X i ) except X j add (X k,x i ) t stack Return csp Functin Revise(csp,X i,x j ) returns DmainXi and anychangetdmainx i anychangetdmainx i = false fr each x in Dmain(X i ) if n value y in Dmain(Xj) allws (x,y) t satisfy cnstraint between (X i,x j ) delete x frm Dmain(X i ) anychangetdmainx i = true Have t add in arc fr (X i,x j ) and (X j,x i ) fr i,j cnstraint 38

39 AC-3 (Assume binary cnstraints) Input: CSP Output: CSP, pssible with reduced dmains fr variables, r incnsistent Lcal variables: stack, initially stack f all arcs (binary cnstraints in csp) While stack is nt empty (X i,x j ) = Remve-First(stack) [dmainx i, anychangetdmainx i ] = Revise(csp,X i,x j ) if anychangetdmainx i == true if size(dmainx i ) = 0, return incnsistent else fr each X k in Neighbrs(X i ) except X j add (X k,x i ) t stack Return csp Functin Revise(csp,X i,x j ) returns DmainXi and anychangetdmainx i anychangetdmainx i = false fr each x in Dmain(X i ) if n value y in Dmain(Xj) allws (x,y) t satisfy cnstraint between (X i,x j ) delete x frm Dmain(X i ) anychangetdmainx i = true Have t add in arc fr (X i,x j ) and (X j,x i ) fr i,j cnstraint 39

40 AC-3 Cmputatinal Cmplexity? Input: CSP Output: CSP, pssible with reduced dmains fr variables, r incnsistent Lcal variables: stack, initially stack f all arcs (binary cnstraints in csp) While stack is nt empty (X i,x j ) = Remve-First(stack) [dmainx i, anychangetdmainx i ] = Revise(csp,X i,x j ) if anychangetdmainx i == true if size(dmainx i ) = 0, return incnsistent else fr each X k in Neighbrs(X i ) except X j Cmplexity f revise add (X k,x i ) t stack functin? D 2 Return csp Functin Revise(csp,X i,x j ) returns DmainXi and anychangetdmainx i anychangetdmainx i = false fr each x in Dmain(X i ) if n value y in Dmain(Xj) allws (x,y) t satisfy cnstraint between (X i,x j ) delete x frm Dmain(X i ) anychangetdmainx i = true Have t add in arc fr (X i,x j ) and (X j,x i ) fr i,j cnstraint D dmain values C binary cnstraints 40

41 AC-3 Cmputatinal Cmplexity? Input: CSP Output: CSP, pssible with reduced dmains fr variables, r incnsistent Lcal variables: stack, initially stack f all arcs (binary cnstraints in csp) While stack is nt empty (X i,x j ) = Remve-First(stack) [dmainx i, anychangetdmainx i ] = Revise(csp,X i,x j ) if anychangetdmainx i == true if size(dmainx i ) = 0, return incnsistent else fr each X k in Neighbrs(X i ) except X j add (X k,x i ) t stack Return csp Functin Revise(csp,X i,x j ) returns DmainXi and anychangetdmainx i anychangetdmainx i = false fr each x in Dmain(X i ) if n value y in Dmain(Xj) allws (x,y) t satisfy cnstraint between (X i,x j ) delete x frm Dmain(X i ) anychangetdmainx i = true Have t add in arc fr (X i,x j ) and (X j,x i ) fr i,j cnstraint D dmain values C binary cnstraints Cmplexity f revise functin? D 2 Number f times can put a cnstraint in stack? 41

42 AC-3 Cmputatinal Cmplexity? Input: CSP Output: CSP, pssible with reduced dmains fr variables, r incnsistent Lcal variables: stack, initially stack f all arcs (binary cnstraints in csp) While stack is nt empty (X i,x j ) = Remve-First(stack) [dmainx i, anychangetdmainx i ] = Revise(csp,X i,x j ) if anychangetdmainx i == true if size(dmainx i ) = 0, return incnsistent else fr each X k in Neighbrs(X i ) except X j add (X k,x i ) t stack Return csp Functin Revise(csp,X i,x j ) returns DmainXi and anychangetdmainx i anychangetdmainx i = false fr each x in Dmain(X i ) if n value y in Dmain(Xj) allws (x,y) t satisfy cnstraint between (X i,x j ) delete x frm Dmain(X i ) anychangetdmainx i = true Have t add in arc fr (X i,x j ) and (X j,x i ) fr i,j cnstraint D dmain values C binary cnstraints Cmplexity f revise functin? D 2 Number f times can put a cnstraint in stack? D Ttal: CD 3 42

43 AC-3 Example Variables: A,B,C,D Dmain: {1,2,3} Cnstraints: A B, C < B, C < D (subset f cnstraints frm befre) 44

44 AC-3 Example Variables: A,B,C,D Dmain: {1,2,3} Cnstraints: A B, C < B, C < D (subset f cnstraints frm befre) Cnstraints bth ways: A B, B A, C < B, B > C, C < D, D > C 45

45 AC-3 Example Variables: A,B,C,D Dmain: {1,2,3} Cnstraints: A B, C < B, C < D (subset f cnstraints frm befre) Cnstraints bth ways: A B, B A, C < B, B > C, C < D, D > C stack: AB, BA, BC, CB, CD, DC 46

46 AC-3 Example Variables: A,B,C,D Dmain: {1,2,3} Cnstraints: A B, C < B, C < D (subset f cnstraints frm befre) Cnstraints bth ways: A B, B A, C < B, B > C, C < D, D > C stack: AB, BA, BC, CB, CD, DC Pp AB: fr each x in Dmain(A) if n value y in Dmain(B) that allws (x,y) t satisfy cnstraint between (A,B), delete x frm Dmain(A) N change t dmain f A 47

47 AC-3 Example Variables: A,B,C,D Dmain: {1,2,3} Cnstraints: A B, C < B, C < D (subset f cnstraints frm befre) Cnstraints bth ways: A B, B A, C < B, B > C, C < D, D > C stack: AB, BA, BC, CB, CD, DC Pp AB stack: BA, BC, CB, CD, DC 48

48 AC-3 Example Variables: A,B,C,D Dmain: {1,2,3} Cnstraints: A B, C < B, C < D (subset f cnstraints frm befre) Cnstraints bth ways: A B, B A, C < B, B > C, C < D, D > C stack: AB, BA, BC, CB, CD, DC Pp AB stack: BA, BC, CB, CD, DC Pp BA fr each x in Dmain(B) if n value y in Dmain(A) that allws (x,y) t satisfy cnstraint between (B,A), delete x frm Dmain(B) N change t dmain f B 49

49 AC-3 Example Variables: A,B,C,D Dmain: {1,2,3} Cnstraints: A B, C < B, C < D (subset f cnstraints frm befre) Cnstraints bth ways: A B, B A, C < B, B > C, C < D, D > C stack: AB, BA, BC, CB, CD, DC stack: BA, BC, CB, CD, DC stack: BC, CB, CD, DC Pp BC fr each x in Dmain(B) if n value y in Dmain(C) that allws (x,y) t satisfy cnstraint between (B,C), delete x frm Dmain(B) If B is 1, cnstraint B >C cannt be satisfied. S delete 1 frm B s dmain, B={2,3} Als have t add neighbrs f B (except C) back t stack: AB stack: AB, CB, CD, DC 50

50 AC-3 Example Variables: A,B,C,D Dmain: {1,2,3} Cnstraints: A B, C < B, C < D stack: AB, BA, BC, CB, CD, DC A-D = {1,2,3} stack: BA, BC, CB, CD, DC A-D = {1,2,3} stack: BC, CB, CD, DC A-D = {1,2,3} stack: AB, CB, CD, DC B={2,3}, A/C/D = {1,2,3} Pp AB Fr every value f A is there a value f B such that A B? Yes, s n change 51

51 AC-3 Example Variables: A,B,C,D Dmain: {1,2,3} Cnstraints: A B, C < B, C < D stack: AB, BA, BC, CB, CD, DC, A-D = {1,2,3} stack: BA, BC, CB, CD, DC A-D = {1,2,3} stack: BC, CB, CD, DC A-D = {1,2,3} stack: AB, CB, CD, DC B={2,3}, A/C/D = {1,2,3} stack: CB, CD, DC B={2,3}, A/C/D = {1,2,3} Pp CB Fr every value f C is there a value f B such that C < B If C = 3, n value f B that fits S delete 3 frm C s dmain, C = {1,2} Als have t add neighbrs f C (except B) back t stack: n change because already in 52

52 AC-3 Example Variables: A,B,C,D Dmain: {1,2,3} Cnstraints: A B, C < B, C < D stack: AB, BA, BC, CB, CD, DC, A-D = {1,2,3} stack: BA, BC, CB, CD, DC A-D = {1,2,3} stack: BC, CB, CD, DC A-D = {1,2,3} stack: AB, CB, CD, DC B={2,3}, A/C/D = {1,2,3} stack: CB, CD, DC B={2,3}, A/C/D = {1,2,3} stack: CD, DC B={2,3}, C = {1,2} A,D = {1,2,3} Pp CD Fr every value f C, is there a value f D such that C < D? Yes, s n change 53

53 AC-3 Example Variables: A,B,C,D Dmain: {1,2,3} Cnstraints: A B, C < B, C < D stack: AB, BA, BC, CB, CD, DC A-D = {1,2,3} stack: BA, BC, CB, CD, DC A-D = {1,2,3} stack: BC, CB, CD, DC A-D = {1,2,3} stack: AB, CB, CD, DC B={2,3}, A/C/D = {1,2,3} stack: CB, CD, DC B={2,3}, A/C/D = {1,2,3} stack: CD, DC B={2,3}, C = {1,2} A,D = {1,2,3} stack: DC B={2,3}, C = {1,2} A,D = {1,2,3} Fr every value f D is there a value f C such that D > C? Nt if D = 1 S D = {2,3} 54

54 AC-3 Example Variables: A,B,C,D Dmain: {1,2,3} Cnstraints: A B, C < B, C < D stack: AB, BA, BC, CB, CD, DC A-D = {1,2,3} stack: BA, BC, CB, CD, DC A-D = {1,2,3} stack: BC, CB, CD, DC A-D = {1,2,3} stack: AB, CB, CD, DC B={2,3}, A/C/D = {1,2,3} stack: CB, CD, DC B={2,3}, A/C/D = {1,2,3} stack: CD, DC B={2,3}, C = {1,2} A,D = {1,2,3} stack: DC B={2,3}, C = {1,2} A,D = {1,2,3} A = {1,2,3} B={2,3}, C = {1,2} D = {2,3} 55

55 Frward Checking When assign a variable, make all f its neighbrs arc-cnsistent 56

56 Backtracking + Frward Checking Functin Backtrack(assignment,csp) returns sln r fail If assignment is cmplete, return assignment V i ß select_unassigned_var(csp) Fr each val in rder-dmain-values(var,csp,assign) If value is cnsistent with assignment Add [V i = val] t assignment Make dmains f all neighbrs f V i arc-cnsistent with [V i = val] Result ß Backtrack(assignment,csp) If Result fail, return result Remve [V i = val] frm assignments Return fail 57

57 Maintaining Arc Cnsistency Frward checking desn t ensure all arcs are cnsistent AC-3 detects failure faster than frward checking What s the dwnside? Cmputatin 58

58 Maintaining Arc Cnsistency (MAC) Functin Backtrack(assignment,csp) returns sln r fail If assignment is cmplete, return assignment V i ß select_unassigned_var(csp) Fr each val in rder-dmain-values(var,csp,assign) If value is cnsistent with assignment Add [V i = val] t assignment Run AC-3 t make all variables arc-cnsistent with [V i = val]. Initial stack is arcs (X j,v i ) f neighbrs f V i that are unassigned, but add ther arcs if these vars change dmains. Result ß Backtrack(assignment,csp) If Result fail, return result Remve [V i = val] frm assignments Return fail 59

59 Sufficient t Avid backtracking? If we maintain arc cnsistency, we will never have t backtrack while slving a CSP A) True B) False 60

60 AC-3 Limitatins After running AC-3 Can have ne slutin left Can have multiple slutins left Can have n slutins left (and nt knw it) 61

61 AC-3 Limitatins After running AC-3 Can have ne slutin left Can have multiple slutins left Can have n slutins left (and nt knw it) What went wrng here? 62

62 Cmplexity CSP in general are NP-hard Sme structured dmains are easier 63

63 Cnstraint Trees Cnstraint tree Any 2 variables in cnstraint graph cnnected by <= 1 path Can be slved in time linear in # f variables Figure frm Russell & Nrvig 64

64 Algrthm fr CSP Trees 1) Chse any var as rt and rder vars such that every var s parents in cnstraint graph precede it in rdering 2) Let Xi be the parent f Xj in the new rdering 3) Fr j=n:2, run arc cnsistency t arc (Xi,Xj) 4) Fr j=1:n, assign val fr Xj cnsistent w/val assigned fr Xi Figure frm Russell & Nrvig 65

65 Cmputatinal Cmplexity? 1) Chse any var as rt and rder vars such that every var s parents in cnstraint graph precede it in rdering 2) Let Xi be the parent f Xj in the new rdering 3) Fr j=n:2, run arc cnsistency t arc (Xi,Xj) 4) Fr j=1:n, assign val fr Xj cnsistent w/val assigned fr Xi Figure frm Russell & Nrvig 66

66 Summary Be able t define real wrld CSPs Understand basic algrithm (backtracking) Cmplexity relative t basic search algrithms Desn t require prblem specific heuristics Ideas shaping search (LCV, etc) Pruning space thrugh prpagating infrmatin Arc cnsistency Tradeffs: + reduces search space, - csts cmputatin Cmputatinal cmplexity and special cases (tree) Relevant reading: R&N Chapter 6 67

You need to be able to define the following terms and answer basic questions about them:

You need to be able to define the following terms and answer basic questions about them: CS440/ECE448 Fall 2016 Midterm Review Yu need t be able t define the fllwing terms and answer basic questins abut them: Intr t AI, agents and envirnments Pssible definitins f AI, prs and cns f each Turing