A Conservative Scheme for Parallel Interval Narrowing

Size: px

Start display at page:

Download "A Conservative Scheme for Parallel Interval Narrowing"

Tracy McCarthy
6 years ago
Views:

1 Letters, 74(3-4): , Elsevier Science. A Conservative Scheme for Parallel Interval Narrowing Laurent Granvilliers a and Gaétan Hains b a IRIN, Université de Nantes, B.P , F Nantes Cedex 3, France b LIFO, Université d Orléans, B.P. 6759, F Orléans Cedex 2, France Abstract An efficient parallel interval narrowing algorithm for solving numerical problems is designed, implemented and tested. Differences with the corresponding sequential algorithm are clearly stated. The algorithm s performance is analyzed in the Bulk-Synchronous Parallel (BSP) cost model which suggests speed-ups on highbandwidth architectures. Experimental results on a massively parallel machine Cray T3E-1200 validate the model and show the parallel algorithm s efficiency as well as its limitations. Key words: parallel algorithms, numerical algorithms, interval narrowing, constraint propagation, BSP model. 1 Introduction Parallel processing of numerical problems via interval constraints has been proposed as a general framework for high-performance numerical computation in [5]. Its two potential advantages over classical methods are 1. the guarantee of numerically correct answers through interval arithmetic and IEEE standard arithmetic and 2. the use of only two algorithms, search and constraint propagation by interval contractions. This simple logical structure offers much scope for optimizing and/or parallelizing the two core algorithms. The problem of parallel search is in itself interesting and difficult but purely combinatorial. Nevertheless, whatever the complexity of the search problems faced by numerical solvers they must optimize constraint propagation. Efficient constraint propagation is therefore critical for the success of parallel interval constraint solving. Although general algorithms are known for it [10], it Preprint submitted to Elsevier Preprint 26 May 2000

2 has been observed that the classical notion of parallel speed-up is not a correct measure of success for such algorithms. It is a non-trivial problem to schedule interval contractions in parallel while retaining a rate of convergence comparable to the best sequential algorithms [4]. This is due to a parallel decoupling phenomenon: convergence may be faster when two interval contractions are applied in sequence than in parallel. In this paper we demonstrate for the first time that important speed-ups are possible for constraint propagation by parallelizing the purely asynchronous, selection part of the best known sequential heuristics. Analysis in the BSP cost model predicted speed-up for high-bandwidth architectures and our tests have confirmed this on T3E and Sun bi-processor systems. Our conservative algorithm is thus a proof of feasibility for parallel interval constraint solving and its limitations define precise requirements for future algorithms. The rest of the article is structured as follows. The next section recalls the definition of a constraint satisfaction problem on numerical intervals and its sequential solution by interval narrowing. The parallel algorithm is then described with its theoretical performance analysis. Finally, experimental results are shown to confirm the possibility of speed-ups and highlight the current algorithm s limitations. 2 Interval narrowing A floating-point interval, called thereafter interval, is a connected set of reals whose lowest upper bound and greatest lower bound are floating-point numbers. The notation [a.. b] is used as shorthand for the set {x R a x b}. The symbol I denotes the set of intervals. In the following, a constraint c is an atomic formula built from a real-based structure and a set of real-valued variables. It defines a relation ρ c on reals. A constraint satisfaction problem, denoted thereafter CSP, is a tuple X, C, I made of a set of real-valued variables X = {x 1,...,x n }, a set of constraints C built from X and a vector of intervals I = (I 1,..., I n ) where I k is the domain of x k for every k {1,..., n}. Interval narrowing algorithms narrow the variables domains while preserving the CSP s solution set, i.e. the set of tuples of I 1... I n satisfying all the constraints. As described in [1], they implement strategies for applying narrowing operators (elementary solvers) associated with constraints. A narrowing operator for a constraint c is a contracting, monotonic, idempotent function nar : I n I n such that ρ c I 1 I n nar (I) for every I I n (correctness property). It is said to depend on the variables contained in c. 2

3 Table 1 Interval narrowing algorithm. IN (in Nars : set of narrowing operators; inout I : vector of intervals) begin queue Nars repeat Choose nar in queue % selection J nar (I) % contraction/narrowing if J I then % propagation queue queue {nar Nars i {1,...,n} : J i I i nar depends on x i } % nar has to be reinvoked since the modification of a domain % it depends on may invalidate the fixed-point property % of the narrowing operators in Nars \ queue I J endif queue queue \ {nar} until queue = or I 1 I n = end % nar is removed from queue since it is idempotent % by definition The work described here uses algorithms that compute a fixed-point of the input narrowing operators 1, taking advantage of the confluence property: given a CSP their output is necessarily the greatest common fixed-point of the narrowing operators included in the Cartesian product of initial domains whatever the chosen strategy (order of application of narrowing operators). The quality of such algorithms is measured by their speed of convergence on various benchmark CSP. The sequential algorithm we parallelize is called IN (for interval narrowing) and is presented in Table 1. It is an immediate adaptation of AlgorithmAC3 [9] and of the filtering algorithms used in interval constraint-based systems like CLP(BNR) or Prolog IV. It maintains a queue that satisfies the following invariant: the domains are necessarily a fixed-point of every narrowing operator not in that queue. If the queue becomes empty, a global fixed-point is reached and if the product of domains ever becomes empty, the CSP s inconsistency has been established. 3 Parallel interval narrowing Table 2 presents the parallel interval narrowing (PIN) algorithm. It realizes a variant of algorithm IN using the following selection-contraction heuristic. 1 The implementation of narrowing operators is beyond the scope of this paper, see [1]. 3

4 First, all operators are applied independently. While doing so, every operator s rate of contraction is computed. A reduced queue is then built with the best operators for which a fixed-point is not yet reached. An operator is part of the reduced queue if its rate of contraction is maximal for some interval. Finally, Algorithm IN is applied to reach a fixed-point for the reduced queue and the whole procedure is repeated. The idea of selection is inspired by the seminal work of Lhomme et al. [8] on the acceleration of sequential interval narrowing algorithms. It produces very fast sequential convergence [2] and its selection phase is trivial to parallelize. The algorithm presented in [8] performs its selection based on cycles detected in the inter-operator dependencies for IN s strategy. It selects, for a given variable and a given cycle, the most contracting operator in that cycle. By comparison, PIN s selection is more optimal because it acts on the set of all operators. PIN also eliminates the sequential (reference) strategy for cycle detection. Although PIN uses IN as a procedure, we will not evaluate its parallel speedup by comparison with IN but with 1-processor PIN. The comparison between 1-processor PIN and IN has been studied in [3] where the former is found to be often much faster and sometimes marginally slower, for an equivalent precision. This experimental result is reinforced by the following general argument. The overall number of operator contractions for IN is O(n 2 md) where n is the number of variables, m is the number of constraints and d is the number of floats in the largest domain. Indeed, removal of one value from a domain can add (nm 1) operators to the queue. In the worst case, this is repeated for every of the nd values in the CSP. A similar argument gives an O(n 2 md) for PIN. These observations have led us to design PIN as alternating between a purely parallel selection (search) phase and a purely sequential propagation phase. The parallel selection phase is well-balanced if the narrowing operators computations times are quite comparable. That is the case for the narrowing operators implementing hull consistency [5], but this property is not guaranteed in general. For instance, box consistency is implemented by a binary search whose length strongly depends on the domains [3]. Finally, there are problems where communication costs dominate and PIN slows down with the number of processors. For this reason, we have used the BSP cost model to estimate PIN s parallelism/communication tradeoff before testing the algorithm. The BSP execution model [12] represents a parallel computation on p processors as an alternating sequence of computations supersteps (p asynchronous computations) and communications supersteps (data exchanges between processors) with global synchronization. The BSP cost model estimates execution times as follows. A computation superstep takes as long as its longest sequential process, a global synchronization takes a fixed, system-dependent 4

5 Table 2 Parallel interval narrowing algorithm. PIN (in Nars : set of narrowing operators; in p : number of processors; inout I : vector of intervals) % the processors are numbered from 0 to p 1 begin IDvar {1,...,n} repeat % 1. Creation of the global queue on all processors % identifiers of modified domains queue {nar Nars i IDvar and nar depends on x i } L size (queue)/p % block size for local queues % 2. Asynchronous parallel selection of narrowing operators in parallel for pid = 0 to p 1 do localqueue queue[(pid L).. min((pid + 1) L 1,size (queue))] for all nar localqueue do % J: vector of vectors of intervals: Nars [1.. n] Jnar nar (I) % contraction % W: vector of vectors of reals: Nars [1.. n] % Estimation of the quality of nar: % For Jnar,i = [c,d] [a,b] = I i define % d(jnar,i, I i ) = (c a) + (b d) with roundings towards + Wnar distance (I,Jnar) = n i=1 d(j nar,i, I i ) endfor endparfor % 3. Parallel computation of contracted domains and best operator for each. % Multiple fold operations: final result (reducedqueue,k) on processor 0. % K: vector of intervals: [1.. n] K i nar queue J nar,i reducedqueue n {nar nar(i) i I i Wnar,i = max m Nars (W m,i )} i=1 % 4. Sequential computation of a fixed-point of the selected operators where pid = 0 do if K I then IN (reducedqueue, K) IDvar {i {1,...,n} K i I i } % new modified domains I K else IDvar % fixed-point reached endif done until IDvar = or I 1 I n = end 5

6 time L and a communication superstep is completed in time proportional to a factor h: the maximal number of bytes sent or received by a processor. A system-dependent constant g is multiplied by h to obtain the estimated communication time. We now outline the analysis of PIN s execution time. The enumeration below refer to numbered parts of the main repeat loop in Table 2. The goal of this analysis is to determine whether PIN executed on p 2 processors can provide any speed-up w.r.t a sequential execution of PIN. For this purpose, it is possible to ignore the sequential propagation time of step 4 and consider only steps 1,2 and Creation of a copy of the global queue by every processor in time O(Ng+ L) where N is the total number of operators. 2. Selection is a pure computation superstep which completes in time proportional to t o N/p where t o is the number of Flops necessary for an operator s application. Its value depends on the CSP and was taken into account by our numerical estimates. 3. Step 3 is made entirely of standard parallel fold operations and complete in time σ/p+(log k p)(l+(k 1)g) where σ is proportional to the number of operators per variable and k 2, the arity of the reduction algorithm is to be selected as a function of (g, p, L). Numerical evaluation of the above estimates for published (g, p, L) parameters 2 has led us to conclude that speed-up forpin required the best possible g values. We have used a Sun bi-processor and a T3E architecture whose values for this parameter (measured with BSP-Lib s bsp probe) are indeed excellent: between 2.4 and Experimental results Algorithm PIN was implemented in C and BSPlib v1.4 [6] and ported on two architectures supporting IEEE binary floating-point arithmetic [7]: a massivelyparallel system Cray T3E-1200 with 256 nodes (450MHz and 128 MB per n- ode) and a Sun UltraSparc workstation with 2 nodes (166MHz per node with a 128MB shared memory). The results for the Sun system are consistent with those for the T3E system and we will only mention the latter. We report the results obtained for the dense system of nonlinear equations Moré-Cosnard [11] and the sparse system Broyden-Banded [2] well-known in 2 J.M.D.Hill, BSP cost parameters for the Oxford BSP toolset, Sept

7 Table 3 Experimental results on the Cray T3E (figures in seconds). n IN p= Seq Moré-Cosnard Moré-Cosnard Moré-Cosnard Broyden-Banded Broyden-Banded Broyden-Banded the Interval Analysis community. Both are square and scalable (of parameterized size), and they permit illustrating the extreme behaviours of Algorithms IN and PIN. Moré-Cosnard s equations are the following: x k + 1[(1 t 2 k) k j=1 t j (x j + t j + 1) 3 + t mj=k+1 k (1 t j )(x j + t j + 1) 2 ] = 0 1 k m t j = jh h = 1/(m + 1) x k [ 4..5] Broyden-Banded benchmark is expressed as follows: x k (2 + 5x 2 k ) + 1 j J k x j (1 + x j ) = 0 1 k m J k = {j j k & max(1, k 5) j min(m, k + 1)} x k [ ] Table 3 illustrate the total solving times on the Cray T3E of both Algorithms IN (column labelled by IN) and PIN. Figures in columns labelled by p = 1, 2,..., 64 are the solving times of PIN on the p-processors machine. The last column corresponds to the solving time thresholds of PIN, i.e. the computation times of the sequential process IN, that is independent from the number of processors. Both n-dimensional Moré-Cosnard and Broyden-Banded problems are tested; let remark that the number of narrowing operators for Moré- Cosnard is n 2, and about 7n for Broyden-Banded. The conclusions are the following: The selection phase is efficiently parallelized since its acceleration is almost linear according to the number of processors. Acceleration of solving depends on the amount of time spent during selection. Solving time is bounded by the computation time of Algorithm IN in PIN (column Seq). For instance, this time is about 7s and 3s respectively 7

8 for both Moré-Cosnard and Broyden-Banded problems with 40 variables. When the difference between selection and sequential solving is huge then total acceleration is small and even negative in the case of Broyden-banded: the communication times are greater than the gain due to parallel selection. The density of a system (the ratio between the number of constraints and the number of constraint-variable pairs) determines the number of narrowing operators generated and plays an important role in the distribution of the selection phase and in the global parallel speed-up. Algorithm PIN executed on one processor is more efficient than the pure sequential algorithm IN which is due to the selection phase of PIN. For example, the selection for Moré-Cosnard dismisses m 2 m narrowing operators from the queue, and about 7m m for Broyden-Banded since most variables appear in 7 constraints. In other words, the fixed-point is reached for both problems using only one narrowing operator per variable. The above conclusions are also valid for a diverse set of benchmarks used in [2] to demonstrate the capabilities of the selection-contraction strategy. Among them, 80 out of 100 exhibit important speed-ups. Moreover PIN is particularly effective for discovering fixed-points since in such a case only the parallel selection is executed, which process is well-balanced for the implemented narrowing operators computing box consistency [3], and fold operations have no datas to reduce. This case often happens in practice where constraint propagation alternates with a search process splitting the domains. 5 Conclusion We have studied theoretically and experimentally the possibility for speedups on systems of numerical interval constraints. The speed-ups observed for Algorithm PIN are relative to the best known sequential heuristics and are thus the lowest possible solution times for the given constraint systems. PIN s limitations follow from its centralized communication structure: the need for a high network bandwidth and an absolute lower-bound on solving time due to the sequential propagation part of the algorithm. Better parallel algorithms for CSP should make optimal use of distributed constraint propagation to improve on PIN s rate of convergence. Acknowledgments This work was supported by the Franco-Russian Liapunov project on real constraint solving and applications headed by Frédéric Benhamou and Tat yana Mikhajlovna Yakhno, and the LOCO project between INRIA Rocquencourt and University of Orléans. The authors thank the Central Institute for Applied Mathematics (ZAM) of Jülich, Germany for 8

9 making possible the experiments on the Cray T3E. G. Hains thanks Maarten van Emden for sharing many ideas related to parallel constraint solving. References [1] F. Benhamou. Heterogeneous Constraint Solving. In Proc. of Int. Conf. on Algebraic and Logic Programming, volume 1139 of LNCS, Aachen, Germany, [2] L. Granvilliers. Consistances locales et transformations symboliques de contraintes d intervalles. PhD thesis, Université d Orléans, France, [3] L. Granvilliers, F. Goualard, and F. Benhamou. Box Consistency through Weak Box Consistency. In Proc. of Int. Conf. on Tools with Artificial Intelligence, Chicago IL, USA, IEEE Computer Society. [4] L. Granvilliers, G. Hains, Q. Miller, and N. Romero. A System for the Highlevel Parallelization and Cooperation of Constraint Solvers. In Proc. of Int. Conf. on Parallel and Distributed Computing and Systems, Las Vegas, USA, [5] G. Hains and M. H. van Emden. Towards high-quality, high-speed numerical computation. In Proc. of Pacific Rim Conf. on Communications, Computers and Signal Processing, University of Victoria, B.C., Canada, IEEE. [6] J. M. D. Hill, B. McColl, D. C. Stefanescu, M. W. Goudreau, K. Lang, S. B. Rao, T. Suel, T. Tsantilas, and R. Bisseling. BSPlib: The BSP Programming Library. Technical report, Oxford University Computing Laboratory, [7] IEEE. IEEE Standard for Binary Floating-Point Arithmetic. Technical Report IEEE Std , IEEE, Reaffirmed [8] O. Lhomme, A. Gotlieb, and M. Rueher. Dynamic Optimization of Interval Narrowing Algorithms. Journal of Logic Programming, 37(1 2): , [9] A. Mackworth. Consistency in Networks of Relations. Artificial Intelligence, 8(1):99 118, [10] E. Monfroy and J-H. Réty. Chaotic Iteration for Distributed Constraint Propagation. In Proc. of ACM Symp. on Applied Computing, San Antonio, USA, [11] J. J. Moré and M. Y. Cosnard. Numerical Solution of Nonlinear Equations. ACM Transactions on Mathematical Software, 5:64 85, [12] L. Valiant. A Bridging Model for Parallel Computation. Communications of the ACM, pages ,

Consistency Techniques over Real Numbers

Consistency Techniques over Real Numbers Laurent Granvilliers granvilliers@irin.univ-nantes.fr University of Nantes Laurent Granvilliers, Dagstuhl Seminar 2003 p. 1/20 Reduction and Propagation Compute