On Solving Soft Temporal Constraints using SAT Techniques

Transcription

1 On Solving Soft Temporal Constraints using SAT Techniques Hossein M. Sheini, Bart Peintner, Karem A. Sakallah, and Martha E. Pollack University of Michigan, Ann Arbor, MI 89, USA {hsheini, bpeintne, karem, Abstract. In this paper, we present an algorithm for finding utilitarian optimal solutions to Simple and Disjunctive Temporal Problems with Preferences (STPPs and DTPPs) based on Benders decomposition and adopting SAT techniques. In our approach, each temporal constraint is replaced by a Boolean indicator variable and the decomposed problem is solved by a tightly integrated STP solver and SAT solver. Several hybridization techniques that take advantage of each solver s strengths are introduced. Finally, empirical evidence is presented to demonstrate the effectiveness of our method compared to other algorithms. Introduction Temporal Constraint Satisfaction Problems (TCSPs) [] are a subclass of constraint satisfaction problems that model constraints over variables with temporal domains. Several TCSP subproblems have been used successfully as components in planning and scheduling applications such as NASA s Mars Rover [] and Autominder []. In planning and scheduling domains, the variables in TCSPs represent events to be scheduled or executed by an agent, and the constraints specify allowable times and temporal differences between those events. The main task for these systems, when given a TCSP, is to assign to all variables times that respect all constraints. Over the last decade, efficient algorithms have been developed for reasoning about TCSP subproblems, which include Simple Temporal Problems (STPs)[], Binary TCSPs (btcsps) [] and Disjunctive Temporal Problems (DTPs) [ 6]. The most expressive of these, the DTP, has received considerable attention due to its ability to express relationships found in temporal planning problems. Recent efforts have extended STPs and DTPs to include soft constraints, or preferences [7 ]. Preferences, which are represented as functions that map temporal assignments to a local preference value, denote how well an assignment satisfies the constraint to which it is attached. By aggregating the local preference values, the value of the entire set of assignments (a solution) can be quantified. Preferences change the problem to one of finding optimal solutions, i.e., solutions that maximize the aggregated preference value. Three different types of aggregation have been explored in the STP case, which result in three types of optimality: maximin optimality, where the lowest local preference value

2 is maximized; utilitarian optimality, where the sum of local preference values is maximized; and stratified egalitarian optimality, where the Pareto-optimal maximin solution is found. For the DTP case, only a solution for the maximin case has been published []. In this paper, we present a single solution to the problems of finding utilitarian-optimal solutions to both STPs with Preferences (STPPs) and DTPs with Preferences (DTPPs). To solve STPPs and DTPPs, we adapt a recent SAT-based algorithm [5] designed for solving Mixed Logical Linear Programming (MLLP) problems involving two types of constraints: logical constraints over Boolean variables, and Unit- Two-Variable-Per-inequality (UTVPI) integer constraints of the form ax by d, where a,b {,,}. The temporal constraints of the form x y d, where x and y represent times assigned to events, and d is a real bound on their temporal difference, can be safely solved using the same UTVPI solver of [5]. This algorithm is implemented in the Ario Satisfiability Modulo Theories (SMT) Solver [6]. To encode the STPPs and DTPPs as MLLPs, we adopt a recent strategy for approximating preference functions in an STPP or DTPP with a set of temporal difference constraints [] and show how to encode the relationships between the temporal constraints and the logical constraints. The result is an MLLP SAT problem with an objective function. We adapt the SATbased algorithm of Ario to reason about the objective function and exploit the structure in the STPPs and DTPPs. This paper presents the first algorithm (to our knowledge) for finding utilitarian-optimal solutions to DTPPs. While this is the main contribution of the paper, we emphasize that the DTPP is a restricted form of the problems solvable by our algorithm. Adding non-temporal logical constraints requires no modification. In fact, Ario is optimized to handle them, having been built on the recent advances in SAT solvers. In the future work section, we discuss other uses for the solver. Background A Simple Temporal Problem (STP) is a pair X,C, where X is a set of events, and C is a set of temporal constraints of the form: x y [c,d], x,y X and c,d R. Often, STP constraints are single-bounded, i.e. in the form x y d. The dual-bounded version can be represented as a pair of single-bounded constraints: x y d y x c. While there is no difference in the expressive power of STPs that use the single- or dual-bounded constraints, the dualbounded variety are more convenient when preferences are involved. An STP solution is an assignment of times to events that satisfies all constraints. An STP is consistent if at least one solution exists. Checking consistency requires O( X ) time, using an all-pairs shortest path algorithm. To extend an STP to an STPP, each constraint is assigned a preference function that maps a temporal difference between the constraint s events to a Tractable DTP subproblems have been identified [, ], as have tractable instances of soft constraint problems []. However, in this paper, we are interested in arbitrary DTPs with preferences a more expressive, but intractable class of problems.

3 5 - MC S-A 6 8 (a) 5 MC E-S (b) 5 MC E-A (c) Fig.. Example preference functions for the Mars Rover example. preference value, a quantitative measure of the difference s desirability. STPP constraints thus have the form: x y [a,b],f : t [a,b] {, R+}. As noted above, once preference functions are added, the challenge becomes one of finding an optimal solution. We evaluate a solution with an objective function that aggregates the values from each constraint s preference function. In this paper, we use the sum function, which results in utilitarian optimality. A DTP is also described as a pair X,C, only each constraint is a disjunction of the STP constraints just described. A DTP solution is an assignment of times to events that satisfy at least one disjunct of all constraints. A DTP is consistent if at least one solution exists. Checking the consistency of a DTP is NP-Hard, but recent techniques have substantially improved their average-case performance. We extend a DTP to a DTPP by adding a preference function to the disjuncts in each constraint. If multiple disjuncts in a constraint are satisfied by a solution, the constraint s local preference value is taken from the highest valued disjunct. Motivating Example We describe a very simple example of an STPP based on the Mars rover domain [9], taken directly from []. We then extend the example to the DTPP case and discuss why the DTPP case is so much more difficult. The daily plan for a Mars rover must satisfy a set of constraints that ensure the safety of the rover while maximizing the scientific value of the experiments it performs. Imagine a very simple scenario in which two events need to be optimally scheduled: the start- and end-time of a single experiment (events S and E). The experiment can begin immediately after the instrument it requires becomes available (event A, set to time ), but it is preferable that some time separates A and S to allow the instrument to cool. This preference is expressed as MC (Mars constraint ) in Figure (a); the horizontal axis represents the difference between events A and S, while the vertical axis represents the preference value assigned to each temporal difference. The scientific value of the experiment changes irregularly with time spent running it (E-S), but is mitigated by the instrument s significant power usage. The net value is expressed by the function in Figure (b). Finally, since other experiments can begin once the current experiment ends, it is preferred that the experiment finish as early as possible. Figure (c) expresses this relationship.

4 A legal solution in this example is one in which E < and E S <7; that is, one in which all preference functions map to a nonnegative value. For a solution {A =,S =,E = 6}, constraints MC, MC, and MC have respective preference values,, and, for a utilitarian value of 7. An optimal solution, with a utilitarian value, is {A =,S =,E = 5}. Notice that even with such a trivial example, it is not immediately apparent whether a given solution is optimal. In fact, unless all preference functions are convex, the problem of finding the optimal solution is NP-Hard. Extending to the DTPP case makes the problem even more difficult. Imagine that the number of experiments to schedule in our Mars rover example increases from one to three: the one originally described (which uses the same instrument as the previous experiment), and two others that use a second instrument. The three experiments can be executed in any order, but they cannot overlap. Therefore, three constraints are needed to express that each pair cannot overlap, i.e. DTPP constraints are needed to express sentiments such as Experiment can end before Experiment begins or start after Experiment ends and to designate which option is more preferable. Whereas the STPP case had only a single ordering possible for the events to be scheduled, the new situation allows six different orderings for the experiments. If we were scheduling four experiments, the six disjunctive constraints needed would allow different orderings. This exponential increase in possible orderings is intuitive evidence of why finding utilitarian optimal solutions to DTPPs is so much more difficult than the STPP case. Solving MLLPs Mixed Logical Linear Programming (MLLP) problems [7] consist of logical constraints over Boolean variables and temporal difference constraints over temporal variables, i.e. STP constraints. Disjunctions involving both Boolean variables and STP constraints allow the representation of conditional STP constraints. One common method for solving an MLLP problem is Benders Decomposition [8], which partitions the problem into a master subproblem containing only Boolean variables and logical formulas, and a subproblem containing the STP constraints. The two subproblems are linked using indicator variables, which are Boolean variables that represent whether a particular STP constraint is asserted. Every reference to an STP constraint in the original problem is replaced with the constraint s indicator variable, converting all disjunctions that involve temporal constraints into pure Boolean formulas. An MLLP solution is an assignment to all variables, Boolean and temporal, in which all constraints are satisfied. A simple MLLP containing Boolean variables, STP constraints, and logical constraints is listed below, along with its Benders decomposition:

5 MLLP Benders decomposition A (x y ) A B y z 5 B Boolean subproblem A A (y x 5) A A B B (x y ) B (y z 5) STP subproblem B (y x 5) In the most straight-forward application of Benders decomposition, the two subproblems are solved separately. First, a SAT solver finds a consistent set of assignments for all Boolean variables. Then, all STP constraints whose indicator variables are set to true are gathered and checked for consistency. If the set of STP constraints is consistent, then a solution is found and the algorithm exits. If inconsistent, the set of STP constraints causing the inconsistency is determined. Since the set of inconsistent STP constraints cannot participate in any solution, they represent a conflict, which can be encoded as a clause in the Boolean part (known as a Benders Cut). The conflict clause states that the combination of indicator variables associated with those STP constraints cannot all be true at once. The process continues, attempting to find an alternative solution to the Boolean subproblem augmented with the Benders Cut. Despite the fact that satisfiability checking is NP-complete, recent enhancements in modern backtrack SAT algorithms have made them very efficient and scalable. At a high-level, SAT solvers operate by assigning values to the Boolean variables and detecting possible implications and conflicts caused by the assigned values. The SAT problem is encoded in Conjunctive Normal Form, which is a conjunction of clauses. A clause is the disjunction of literals, and a literal is a Boolean variable or its negation. Upon assigning a literal, consequent implications are detected by adopting a two-watched-literal strategy where each clause is only propagated if one of its two watched literals is assigned to false [9]. In case of a conflict, where all literals in a clause are false, a new clause is learned to prevent the conflict from being encountered again. Conflict analysis is performed by backward traversal of the implication graph that can result in non-chronological backtracking. For more details, the reader is referred to [].. Integrated Solving A recent algorithm has improved performance by solving both subproblems together. The algorithm, found in [5] and implemented in the Ario SMT solver [6] (hereafter, the Ario algorithm), tightly integrates both solvers. In this algorithm, each time an assignment is made to a Boolean indicator variable during search, its corresponding STP constraint is activated and checked for consistency with other activated STP constraints. This is achieved using the incremental algorithm of Jaffar et. al. [] for solving problems involving UTVPI integer constraints, which are slightly more general than STP constraints. Each time a new STP constraint is added to the STP solver, it generates a set of new STP constraints that are implied by the addition. For instance, adding the STP constraint y z d to the set {x y d } will result in generating the implied

6 STP constraint x z d +d. If a newly generated STP constraint is equivalent to or stronger than the STP constraint implied by an indicator variable, the indicator variable is assigned to true. Any conflict detected in the activated STP constraints can be encoded as a clause on the Boolean side. However, given that the integrated solver checks the consistency of the temporal side as it makes assignments, such conflicts will often be found much sooner.. Encoding a DTP as an MLLP DTPs (and by extension STPs) can be directly encoded as MLLP problems. DTPs are simply MLLPs that do not include any Boolean variables. During Benders decomposition, each disjunct i of DTP constraint k, denoted C k,i, is given an indicator variable B k,i on the Boolean side, and the disjunctive constraint will have the Boolean form of i B k,i. The MLLP algorithm of Ario bears a strong resemblance to the meta-csp concept used by many DTP solvers. To create the meta-csp of a DTP, a CSP variable is created for each DTP constraint; the domain of the CSP variable includes one value for each disjunct in the DTP constraint. During search, assignments are made to the meta-csp variables. After a variable is assigned a value, the disjunct associated with that value (i.e. an STP constraint) is added to an incremental STP solver, much like the Jaffar solver mentioned above. This act resembles the process of assigning an indicator variable to true and adding the indicated STP constraint to the Jaffar solver. Given the strong similarity of the two methods, it is unsurprising that many techniques in one apply to the other. In fact, many of the efficiency techniques used to prune the search of Meta-CSPs have an analogue in the Ario algorithm. For example, the method of incorporating conflicts detected on the STP side into the Boolean side has a CSP analogue called no-good learning. A method called removal of subsumed variables in DTP solvers identifies disjuncts that are already satisfied (without being inserted into the STP solver) and assigns them to their meta-csp variables. This technique is an integral part of the Ario algorithm, which sets indicator variables to true when the STP constraints they imply are subsumed by the implied constraints discovered by the STP solver. These techniques help reduce the role of the relatively slow STP solver by shifting more work toward the efficient SAT solver. When describing our modifications to the Ario algorithm, we will show how to incorporate another technique found in DTP solvers, called semantic branching. 5 Solving DTPPs To extend Ario s MLLP algorithm into a DTPP solver, we address three issues: how to represent arbitrary preference functions in an MLLP; how to convert the solver from a satisfaction algorithm to an optimization algorithm; and how to improve performance by taking advantage of the structure of DTPPs.

7 5. Encoding preference functions as an MLLP In the preceding section, we described how STP constraints are linked to Boolean variables using the conditional form of B i x y d. We can use the same mechanism to reason about arbitrary preference functions. In other words, we can represent a DTPP with arbitrary preference functions as an MLLP. If the preference functions are restricted to be convex, they can be easily represented (or at least approximated) by a set of linear functions [9]. However, when arbitrary, non-convex preference functions exist (such as the one in Figure (b)), more complicated schemes are required. For our purposes, we adopt the scheme presented in [], which approximates arbitrary preference functions using a set of hard STP constraints, called a preference projection []. After creating the preference projections, we assign an indicator variable to each element of the preference projection (i.e. to each hard STP constraint) and show how to define Boolean formulas that will ensure that any solution to the MLLP is a solution to the DTPP and vice-versa. To obtain a preference projection for a DTPP constraint, we first discretize the range of its preference functions into a finite set of real values A, called a preference value set ({,,,,} in our Mars rover example). Then, we project each disjunct (STPP constraint) at each level l A into a set of hard STP constraints. In the end, each function will be represented as a set of hard STP constraints. Again, we refer to disjunct i of DTPP constraint k as C k,i. The projection of disjunct C k,i at preference level l is a list of STP constraints representing the intervals at which C k,i is satisfied at preference level l or higher. Figure (d)-(f) shows the preference projection for each constraint in our Mars rover example (Figure (a)-(c)). Each horizontal line segment in Figure (d)-(f) is an STP constraint in the preference projection. For example, the line segment at preference level in Figure (f) denotes that if E A 6, then the preference value for constraint MC will be or greater. Once we have a list of STP constraints for each disjunct of each DTPP constraint, we union the lists for all disjuncts that belong to the same DTPP constraint and level. Therefore, we end with a list of STP constraints for each level and each DTPP constraint. The following two definitions formalize the idea of a preference projection for STPP and DTPP constraints. Definition (STPP Preference Projection). (From []) Given an STPP constraint C k,i = x y [a,b],f, the preference projection at level l is P k,i [l] = {c,c,...,c n }, where c p = x y [a p,b p ], b p <a p+ for p < n and n p= [a p,b p ] = {t f(t) l}. The STPP preference projection is P k,i = l A P k,i [l]. That is, the preference projection at level l for an STPP constraint is the unique set of maximal intervals for which the constraint receives preference l. Although the method only approximates functions, the degree of approximation can be controlled by varying the number of preference levels.

8 5 MC S-A (a) 5 MC E-S (b) 5 MC E-A (c) MC S-A MC E-S MC E-A (d) (e) (f) Fig.. The preference projections for each constraint in the Mars rover example (see Section ). Each line segment at level l represents intervals for which the value of the preference function is l or greater. X marks ends of open intervals. Definition (DTPP Preference Projection). Given a DTPP constraint C k = C k, C k,... C k,n, the preference projection for C k at level l is P k [l] = n i= P k,i [l]. We refer to the set of all hard STP constraints projected from C k using preference value set A as the preference projection for C k, denoted P k = l A P k[l]. Thus, for each DTPP constraint C k and each level l, the list of dual-bounded STP constraints is denoted as P k [l]. Using the definitions, we can show that if an assignment of times to each temporal variable satisfies at least one STP constraint in P k [l], then DTPP constraint C k will be satisfied at level l or greater. If we find the maximum l for which some STP constraint in P k [l] is satisfied, we say that the assignment satisfies the DTPP constraint at level l. If we sum this value for all DTPP constraints in the problem, we have the utilitarian value of the assignment. Therefore, the utilitarian value of any assignment can be approximated by finding the highest valued satisfied STP constraint in each DTPP constraint s preference projection. We can define an indicator variable B k,l,p to imply the p th element of P k [l]. Just as in the DTP case, a solution exists when at least one B variable for each DTPP constraint is true and the STP constraints implied by the true B variables represent a consistent STP. In terms of indicator variables, the following Boolean constraint must be satisfied: B k,l,p. k l p When preferences are present, we are not content with finding any assignment to the indicator variables that satisfies the above Boolean constraint; we want the assignment that maximizes an objective function. To prepare for encoding the objective function, we factor the above constraint by introducing an intermediate Subject to the approximation error of the discretization process.

9 variable for each DTPP constraint and level: B k,l = p B k,l,p. In the next section, we show how to tie these intermediate variables to the objective function. As an example, consider the preference projection for M C in Figure (e). There are 7 STP constraints in the projection: at level, B,, ; at level, B,, ; at level, B,, and B,, ; at level, B,, and B,, ; and at level, B,,. The Boolean constraints in the first column below represent this preference projection: B, = B,, M, = B, B, = B,, B,, M, = B, B, B, = B,, B,, M, = B, B, B, = B,, M, = B, B, B, = B,, M, = B, B, To ensure every DTPP constraint is satisfied, a final Boolean equivalent to the unfactored Boolean constraint above is required: B k,l. k l 5. Finding the optimal solution to the MLLP The optimal solution to a DTPP is an assignment of times to temporal variables that produces the highest utilitarian value. The utilitarian value is calculated by finding a true indicator variable from each DTPP constraint and summing their l values. If more than one indicator variable is true for a given DTPP constraint, the one with the maximum l value is used. To find the maximum l value for each DTPP constraint k, we add Boolean constraints of the form M k,l = B k,l B k,l+, where the variable M k,l becomes true if l is the highest level satisfied for constraint k. The second column in the table above shows the M variables and constraints for our example. The objective function for the optimization problem is therefore a sum consisting of one term for each level of each DTPP constraint: l M k,l. Thus, the objective function for the MLLP is the following: max l M k,l. k l We perform the optimization by solving a sequence of satisfaction problems. We encode the objective function as a special linear constraint: l M k,l k l lowerbound, where lowerbound is a constant that we increase after each problem in the sequence is solved. The search starts by solving the satisfaction problem in which lowerbound is set to. If the solver finds a solution, and the objective evaluation of that solution is some value v, then we update the lowerbound value in the special constraint to v +, and re-solve the problem. At each iteration the lower bound is updated and the search terminates when the problem becomes unsatisfiable. The last satisfiable lowerbound then will be the maximum value of the objective function. For each DTPP constraint s top-level M variable, the included Boolean constraint is M k,l = B k,l.

10 5. Exploiting the DTPP structure Thus far, we have described a method for finding the optimal solution to a DTPP by encoding the STP constraints in the preference projection as an MLLP and by solving a sequence of MLLPs with a special constraint encoding the objective function. We now discuss how to take advantage of the structure of the DTPP and its objective function to improve performance. Semantic Branching One method used by meta-csp solvers that did not already exist in our MLLP solver is semantic branching. In semantic branching, as soon as the solver finds that enforcing an STP constraint cannot lead to any solution, it infers that the negation of that STP constraint should hold in any solution. Enforcing the negation of the STP constraint further prunes the search, possibly detecting dead ends in the search sooner than otherwise possible. In our method, each time an indicator variable for an STP constraint is set to false, the temporal solver can enforce the negation of the STP constraint. A similar method was also successfully used in another SAT-based DTP solver [6]. Relationships among STP constraints STP constraints projected from a single DTPP disjunct are closely related. Note that each interval of a preference projection constraint is a subset of an interval in every level below it. Therefore, if a set of assignments satisfies a projected STP constraint at level l, then at least one STP constraint is satisfied at all lower levels. Conversely, if no STP constraint at level l is satisfied, then no STP constraint at any higher level can be satisfied. We encode this relationship using the Boolean formulas B (k,l) B (k,l+). Efficient processing of objective function Each time a Boolean variable is assigned or unassigned, the value of the objective function may change. To avoid constantly updating the objective value, our algorithm instead monitors an upper bound for the objective function, which changes much less often. To calculate the upper bound, we first locate the highest non-false M variable for each DTPP constraint, i.e., the non-false M variable with the highest l value. We refer to this set as the watched M variables. Then, we sum the l value of all watched variables to produce the upper bound. The upper bound only needs to be updated when one of the watched M variables is assigned to false. When the upper bound becomes equal to the lowerbound value in the objective function, all the corresponding watched variables should be implied to true. On the other hand, if the upper bound becomes less than lowerbound, a conflict is detected. The reason for the conflict is the set of M variables whose assignment to false resulted in the drop of the watched-sum below its lowerbound. The conflict set is passed to the SAT Solver as a new clause, and backtracking occurs. This method incorporates standard branch-and-bound concepts. Pruning using slack We use the difference between the upper and lower bounds to help prune the search. We use this difference, called the slack, to

11 prune disjuncts that are projected from the lower preference levels. Consider our Mars Rover example and the case where the lowerbound = 8 and the watched variables are M,, M, and M,. The upper bound in this case is (++), resulting in a slack of. When choosing an indicator variable to assert in constraint, all indicator variables at level or lower can be ignored because they would lower the upper bound by an amount greater than the available slack; therefore, we can safely assign false to the variables M, and M,. In general, any M k,l variable will be assigned to false if it is detected that assigning M k,l to true would prevent the sum in the objective function from reaching the lower bound. In short, if for some DTPP constraint k, the watched element of the objective function is M k,l, then all variables M k,l in which l < (l slack), should be assigned to false. When the lowest M variables are implied to false, it effectively raises the minimum level at which the DTPP constraint must be satisfied. In the above case, the minimum level was raised to. To encode this explicitly, we simply set B, to true. The same effect can be produced by adding a set of Boolean constraints of the form: l a= M (k,a) B (k,l+) to the problem. 6 Experimental Results Our algorithm for optimally solving the DTPPs is incorporated into the SMTsolver Ario [6]. Its STP algorithm is a specialization of its UTVPI engine fully explained in [5]. All experiments were conducted on a Pentium-IV 8MHz machine with GB of RAM running Linux. 6. Generating random DTPPs For the tests described in this section, we generated random DTPPs using a set of parameters E,C,D,D +,L,R,R +,S, where E is the number of events, C is the number of constraints, D and D + are the minimum and maximum bounds on any constraint, and the remaining elements define the preference functions. We first generate a set of events {x,x,...,x E }. Then, we generate a set of C -disjunct DTP constraints (without preference functions) by creating one disjunct at a time. For each disjunct of each constraint, we randomly choose a pair of events, and then randomly choose an upper and lower bound for the temporal difference between the events from the interval [D,D + ]. Once the hard DTP constraints are formed, we create a preference function for each disjunct. Instead of creating the functions directly, we create the constraints of its function s preference projection. The lowest level of the preference projection is formed using the bounds that were chosen for the underlying STP constraint. To form a new constraint at the next level, we first calculate the width of the new constraint s interval by multiplying the width of the previous constraint s interval by a reduction factor, chosen from the interval [R,R + ] [,]. An interval of the newly calculated width is created and placed randomly within the original interval. With probability S, the new interval is split into two parts,

12 Time (sec.) Time (sec.).. AVERAGE MEDIAN Number of Constraints (a).. AVERAGE MEDIAN Number of Preference Levels (b) Fig.. Running times for trials solving problems with different numbers of constraints and preference levels. (a) number of preference levels fixed at 5. (b) number of constraints fixed to 5. (Timeout set to s) and each interval is randomly placed. When S is, the interval never splits, and the result is a preference function that is semi-convex 5, as in Figure (d). When S is greater than, functions such as those in Figure (e) are created. We continue creating new preference levels until the calculated interval width for a new level is (occurs often because we round to the nearest integer) or we hit the maximum number of preference levels defined by L. 6. Experiment : Varying problem size Given that no other algorithm exists for finding utilitarian-optimal solutions to DTPPs, our first set of experiments evaluates Ario in terms of the different parameters in the DTPP problem, generated as previously explained. For each problem, we fixed the following parameters: {D = 5,D + =,R =.5,R + =.9,S = }. In the first run, we fixed the number of preference levels (L) to 5 and varied the number of -disjunct constraints (C) from to 7. In the second run, we fixed C to 5 and varied L from to 9. In both runs, the number of events (E) was set to 5C (constant constraint density). Figure shows the average and median running times for trials of each problem size. As expected, the running time increases exponentially with problem size. However, it is much more sensitive to an increase in the number of preference levels than the number of constraints. Fortunately, the number of preference levels is controllable; we choose this number when we create the preference projection. Thus, the knowledge engineer can trade off accuracy (number of preference levels) for speed. The number of constraints is problem dependent, so practically it is more important that we do well in this dimension. 5 A funtion f is semi-convex if the set {X : f(x) Y } forms an interval for all Y [7].

13 AVERAGE MEDIAN % UNSAT % 9% 8% 7% Time (sec.) 6% 5% % % Time (sec.). % %. GAPS MIP ARIO Density % Number of Constraints Fig.. Median and average running times of ARIO for DTPPs with varying density. The dotted line denotes the percentage of problems with no solution. Fig. 5. Median running times of the ARIO, GAPS and MIP methods on STPP problems with 5 preference levels. (Timeout set at 6s) In a third run, we held the problem size constant (C = 5, L = 5) and varied the constraint density by varying E from 8 to. The result, shown in Figure, qualitatively resembles the result of many CSP density tests: for over- or underconstrained problems, finding the optimal solution is fairly easy, while problems in the middle range are much more difficult. Note that the previous two runs in this experiment used a density of.5 (E = 5C), which is in the difficult range. 6. Experiment : Performance on STPPs We compared Ario s performance on STPPs to two methods for solving STPPs:. Simplex-based MIP method using the solver XPRESS-MP []. In this method, the disjunction between the temporal constraints associated with each preference level was removed using the Big-M method. Specifically, each temporal constraint of the form d x y d in the preference projections was replaced by the following two constraints: x y d + M( B k,l,p ) x y d M( B k,l,p ) where M is the Big-M parameter and B k,l,p is a binary variable representing the projected constraint. Since each constraint k should be individually satisfied, a constraint of the form l p B k,l,p = is also entered for each constraint k. The objective function is encoded as below: max l B k,l,p k l p. GAPS[]. GAPS was the first algorithm specifically-designed for solving STPPs with unrestricted preference functions. Although GAPS was designed to maximize anytime performance, it is also complete, which allows us to

14 compare against it. GAPS executes a greedy search in the space defined by the STPP preference projection. Each point in the search space is a component STP, which is an STP formed by selecting a single STP constraint from each preference projection. It starts with the component STP formed by selecting the STP constraint at the lowest preference level of each projection, and incrementally improves it by replacing one STP constraint in the component STP with another from a higher preference level. Once the greedy search finds a component STP that cannot be improved, it is finished with the first iteration. The first greedy solution is used to partition the problem into several smaller STPPs, each of which can be searched using the same greedy search. Each partition prunes much of the search space and prevents previously visited parts of the space from being revisited. In this test, we used to the same fixed parameters as the first run in Experiment, except we increased S to.5, resulting in non-convex preference functions. Figure 5 shows that Ario outperforms both MIP and GAPS methods for the cases in which the number of preference levels is 5. 7 Conclusions and Future Work In this paper, we presented an algorithm for finding utilitarian optimal solutions to STPPs and DTPPs based on Benders decomposition and adopting SAT techniques. In our approach, we adopt the technique of representing each preference function as a set of hard STP constraints called a preference projection. Each temporal constraint in the preference projection is replaced by a Boolean indicator variable and the decomposed problem is solved by a tightly integrated STP solver and a SAT solver. We showed empirically that our method finds an optimal solution to STPPs much faster than the most recent algorithm, and showed how the running time of our algorithm on DTPPs varied with problem size. As we mentioned in the introduction, STPPs and DTPPs only take advantage of part of the expressive power of Ario. Many real world problems contain not only temporal constraints, but logical constraints as well, which Ario can represent and reason about with no modification. Allowing logical constraints enables the representation of optional constraints, conditional constraints, and temporal constraints with variable bounds. Using such devices, we can extend our Mars rover example to include optional experiments, experiments that can only take place if some environmental condition holds, and experiments whose duration depends on the amount of sunlight available or amount of power remaining. References. Dechter, R., Meiri, I., Pearl, J.: Temporal constraint networks. Artificial Intelligence 9 (99) Muscettola, N., Nayak, P.P., Pell, B., Williams, B.C.: Remote agent: To boldly go where no AI system has gone before. Artificial Intelligence (998) 5 7

15 . Pollack, M.E., Brown, L., Colbry, D., McCarthy, C.E., Orosz, C., Peintner, B., Ramakrishnan, S., Tsamardinos, I.: Autominder: An intelligent cognitive orthotic system for people with memory impairment. Robotics and Autonomous Systems (-) () 7 8. Stergiou, K., Koubarakis, M.: Backtracking algorithms for disjunctions of temporal constraints. Artificial Intelligence () Tsamardinos, I., Pollack, M.E.: Efficient solution techniques for Disjunctive Temporal Reasoning Problems. Artificial Intelligence 5(-) () 9 6. Armando, A., Castellini, C., Giunchiglia, E., Maratea, M.: A SAT-based decision procedure for the boolean combination of difference constraints. In: 7th International Conference on Theory and Applications of Satisfiability Testing. () 7. Khatib, L., Morris, P., Morris, R., Rossi, F.: Temporal constraint reasoning with preferences. In: 7th International Joint Conference on Artificial Intelligence. () 7 8. Khatib, L., Morris, P., Morris, R., Venable, K.B.: Tractable pareto optimal optimization of temporal preferences. In: 8th International Joint Conference on Artificial Intelligence. () Morris, P., Morris, R., Khatib, L., Ramakrishnan, S., Bachmann, A.: Strategies for global optimization of temporal preferences. In: Tenth International Conference on Principles and Practice of Constraint Programming. () 8. Peintner, B., Pollack, M.E.: Low-cost addition of preferences to DTPs and TCSPs. In: 9th National Conference on Artificial Intelligence. () Peintner, B., Pollack, M.E.: Anytime, complete algorithm for finding utilitarian optimal solutions to STPPs. In: th National Conference on Artificial Intelligence. (5) 8. Koubarakis, M.: Tractable disjunctions of linear constraints: basic results and applications to temporal reasoning. Theoretical Computer Science 66 () 9. Jonsson, P., Bāckstrōm, C.: A unifying approach to temporal constraint reasoning. Artificial Intelligence (998) 55. Cohen, D.A., Cooper, M.C., Jeavons, P., Krokhin, A.A.: A maximal tractable class of soft constraints. Journal Artificial Intelligence Research () 5. Sheini, H.M., Sakallah, K.A.: A SAT-based decision procedure for Mixed Logical/Integer Linear Problems. In: Integration of AI and OR Techniques in Constraint Programming for Combinatorial Optimization Problems. (5) 5 6. (Ario SMT Solver) 7. Hooker, J.N., Ottosson, G., Thorsteinsson, E.S., Kim, H.J.: On integrating constraint propagation and linear programming for combinatorial optimization. In: 6th National Conference on Artificial Intelligence. (999) 6 8. Hooker, J.N.: Logic-Based Methods for Optimization: Combining Optimization and Constraint Satisfaction. Wiley, New York, NY () 9. Moskewicz, M.W., Madigan, C.F., Zhao, Y., Zhang, L., Malik, S.: Chaff: engineering an efficient SAT solver. In: 8th Design Automation Conference. () Marques-Silva, J.P., Sakallah, K.A.: GRASP: A search algorithm for propositional satisfiability. IEEE Trans. Comput. 8 (999) Jaffar, J., Maher, M.J., Stuckey, P.J., Yap, R.H.C.: Beyond finite domains. In: Proceedings of the Second International Workshop on Principles and Practice of Constraint Programming, Springer-Verlag (99) Dash Inc.: (XPRESS-MP version 5.5.)