Efficient Data Movement on the Grid

Transcription

1 Efficient Data Movement on the Grid Michal Zerola for the STAR collaboration Jérôme Lauret, Roman Barták and Michal Šumbera NPI, AS CR CFRJS - FJFI, February 13, 2009 Michal Zerola (STAR) Efficient Data Movement on the Grid February 13, / 26

2 Outline of the problem Challenges How to transfer a set of files for a physicist to his site in a shortest time possible for fast analysis turn around? How to achieve controlled planning of data-movement between sites on a grid (not necessarily following Tier architecture)? Ultimately: How to couple SE and CE aspects for distributing data on the grid for further processing? We concentrate on the first (most practical and of immediate need) question. On the grid: files are often replicated - available at multiple sites links diverse in bandwidth or QoS File A origin? File B origin Destination We will present a Constraint Programming approach for solving this problem. Michal Zerola (STAR) Efficient Data Movement on the Grid February 13, / 26

3 Scope of the architecture Michal Zerola (STAR) Efficient Data Movement on the Grid February 13, / 26

4 Introduction to Constraint Programming A Constraint Satisfaction Problem (CSP) consists of: a set of variables X = {x 1,...,x n }, for each variable x i, a finite set D i of possible values (its domain), and a set of constraints restricting the values that the variables can simultaneously take. A solution to a CSP is an assignment of a value from its domain to every variable, in such a way that every constraint is satisfied. We may want to find: any solution all solutions an optimal solution (defined by some objective function) Michal Zerola (STAR) Efficient Data Movement on the Grid February 13, / 26

5 Example - Sudoku logic-based number-placement puzzle the objective is to fill a 9 9 grid so that each column, each row, and each of the nine 3 3 boxes contains the digits from 1 to 9 only one time each How to model Sudoku as a CSP? variable with a domain equal to {1,..., 9} for each single cell constraint of inequality between all pairs of variables in each row, column, or 3 3 cell Michal Zerola (STAR) Efficient Data Movement on the Grid February 13, / 26

6 From Toy problem to the real life Notation: E dest N D- set of demands IN(n) OUT(n) orig(d) Michal Zerola (STAR) Efficient Data Movement on the Grid February 13, / 26

7 From Toy problem to the real life Notation: E dest N D- set of demands IN(n) OUT(n) orig(d) Michal Zerola (STAR) Efficient Data Movement on the Grid February 13, / 26

8 From Toy problem to the real life Notation: E dest N D- set of demands IN(n) OUT(n) orig(d) Michal Zerola (STAR) Efficient Data Movement on the Grid February 13, / 26

9 From Toy problem to the real life Notation: E dest N D- set of demands IN(n) OUT(n) orig(d) Michal Zerola (STAR) Efficient Data Movement on the Grid February 13, / 26

10 From Toy problem to the real life Notation: E dest N D- set of demands IN(n) OUT(n) orig(d) Michal Zerola (STAR) Efficient Data Movement on the Grid February 13, / 26

11 Solving process Composed of two iterative stages: Planning stage - generate a valid transfer path for each requested file from one of its origins to the destination node. link-based approach path-based approach Scheduling stage - allocate transfers to particular times, respecting link capacity constraints Michal Zerola (STAR) Efficient Data Movement on the Grid February 13, / 26

12 Planning stage - link-based approach X de decision {0,1} variable for each demand d and link e of the network orig(d) d D : d D : X e OUT(n n orig(d)) e OUT(dest(d)) X de = 0, X de = 1, X e IN(n n orig(d)) e IN(dest(d)) X de = 1 X de = 0 n d D, n / orig(d) {dest(d)} : X X X X de 1, X de 1, X de = X e OUT(n) e IN(n) e OUT(n) e IN(n) X de Michal Zerola (STAR) Efficient Data Movement on the Grid February 13, / 26

13 Planning stage - path-based approach X de variables are non-decisional P dp decision {0,1} variable for each demand d and path p of the network leading to dest d D : P dp = 1, P dp = 0 p n orig(d) OP(n) p/ n orig(d) OP(n) The assignment of X variables is provided automatically via constraint propagation by a constraint defined as: d D, e E : X de = p e p P dp, Michal Zerola (STAR) Efficient Data Movement on the Grid February 13, / 26

14 Scheduling stage For each link e of the graph that will be used by at least one file demand, we introduce a unique unary resource R e T d1,e 1 R e1 T d1,e 1 T d1,e 3 T d2,e 3 R e2 T d2,e 2 T d2,e 2 R e3 T d1,e 3 T d2,e 3 Figure: Example of assigning tasks (file transfers over particular links) into unary resources (links). In this case the transfer paths of demands d 1 and d 2 share link e 3, and consequently resource R e3 as well. Implementation provided with Edge Finding, Not-First/Not-Last, and Detectable Precedences filtering rules based on Theta-Lambda tree structures Michal Zerola (STAR) Efficient Data Movement on the Grid February 13, / 26

15 Complexity of the problem solving the second (scheduling) stage is strongly NP-hard we use the well-known three field classification α β γ for scheduling problems and demonstrate a possible polynomial-time reduction from J3 p ij = 1 C max, a 3-machine unit-time Job-shop scheduling problem (JSSP). The transformation of the JSSP instance is following: JSSP instance Data Transfer instance M 1 M 2 e 1 e 2 dest M 3 J =< A 3, A 1, A 2 > e 3 Path defined by job J Figure: The job J =< A 3, A 1, A 2 > defines a transfer path to the dest vertex using links e 3, e 1, e 2, with order given by precedences of actions. Connections between links (and dest vertex) are achieved by dummy edges (dashed lines) which have slowdown equal to 0, thus not affecting allocation times on real edges. Michal Zerola (STAR) Efficient Data Movement on the Grid February 13, / 26

16 Search procedure Algorithm 1 Pseudocode for a search procedure. makespan sup plan Planner.getFirstPlan() while plan!= null do schedule Scheduler.getSchedule(plan, makespan) {B-a-B on makespan} if schedule.getmakespan() < makespan then makespan schedule.getmakespan() {better schedule found} end if Planner.getNextPlan(makespan) {next feasible plan with cut constraint} end while the actual makespan is effectively used also for pruning the search space during the first (planning) stage: e E : d D X de slowdown(e) < makespan Michal Zerola (STAR) Efficient Data Movement on the Grid February 13, / 26

17 Symmetry breaking Planning stage: orig(d 1 ) l 1 l 2 l 3 l 4 orig(d 2 ) X d1,l 1 = 1 X d2,l 3 = 1 l 1 l 3 Figure: In the configuration shown, demands d 1 and d 2 have an overlapping sets of origins. The outgoing links from this intersection are l 1, l 2, l 3, and l 4. If demand d 1 is assigned to link l 1 and demand d 2 to link l 3, relation l 1 l 3 must hold. Scheduling stage: among the corresponding tasks assigned to each unary resource representing the link from the common path, we introduce additional precedences fixing their order Michal Zerola (STAR) Efficient Data Movement on the Grid February 13, / 26

18 Implied constraints of X variables let S 1 and S 2 be two sets of the decision X variables, let S 2 be a subset of S 1, e.g. S 2 S 1, if there exists a constraint stating X S 1 X = 1 and similarly for the second set X S 2 X = 1, we can reduce domains of variables in S 1 \ S 2 to value 0 (depicted in Fig. 4). 0/1 0/1 0/1 0/1 0/1 0/1 0 0 = 1 = 1 Figure: A configuration depicting a state of decision variables for some demand that is going to be transferred to the bottom destination site. If the origin of demand consists of two sites, symbolized as upper leftmost vertices, solver would still try an assignment for two rightmost links. By the filtering procedure we can reduce their values as can be seen on the right graph. Michal Zerola (STAR) Efficient Data Movement on the Grid February 13, / 26

19 Search heuristics variable and value selection heuristics determine the shape of the search tree clever branching strategy is a key ingredient of any constraint approach Planning stage: dom strategy (smallest domain first) FastestLink (the selected X dej corresponds to the fastest link (j = arg min i=1,...,m slowdown(e i )) FastestPath (analogous to FastestLink) Scheduling stage: SetTimes (is based on determining Pareto-optimal trade-offs between makespan and resource peak capacity) SumHeights (texture-based heuristic, using ordering tasks on unary resources.) Michal Zerola (STAR) Efficient Data Movement on the Grid February 13, / 26

20 Comparative studies (1) implementation of the solver is built on Choco - Java based library ( CSP is compared to the Peer-2-Peer model realistic-like network structure among 5 sites together with file distribution used CPU timelimit of 30 seconds Michal Zerola (STAR) Efficient Data Movement on the Grid February 13, / 26

21 Comparative studies (2) Files P2P Link based Path based dom ր dom ց fast ց fast ց dom ր dom ց fast ց fast ց ST ST ST SH ST ST ST SH ? ? ? ? ? ? ? Table: Comparison of makespans generated by several combination of heuristics for link-based and path-based model. Legend is following: ր - Increasing value selection, ց - Decreasing value selection, ST - SetTimes heuristic, SH - SumHeight heuristic. Michal Zerola (STAR) Efficient Data Movement on the Grid February 13, / 26

22 Comparative studies (3) Files number of requested files to transfer RunTime time in seconds taken by solver to generate result % in 1 st number of percents of overall time spent in the planning phase % in 2 nd number of percents of overall time spent in the scheduling phase # of 2 nd calls number of scheduling calls BST time in seconds when the best solution was found P2P-M time in general units to execute plan from P2P simulator Makespan time in general units to execute plan from CSP solver Files RunTime % in 1 st % in 2 nd #of 2 nd calls BST P2P-M Makespan % 30% % 23% % 9% % 2% % 2% % 2% % 20% % 5% % 6% % 8% % 11% Table: Results for link-based approach with FastestLink selection in Decreasing order for planning phase and SetTimes heuristic for scheduling phase. Michal Zerola (STAR) Efficient Data Movement on the Grid February 13, / 26

23 Imposing time variables into planning phase non-decision positive integer variables P de representing possible start times of transfer for demand d over edge e let dur de be the constant duration of transfer of d over edge e d D n N : X de (P de + dur de ) X de P de e IN(n) e OUT(n) Constraints do not restrict the domains of P de until the values X de are known. To estimate better te lower bound for each P de : min (P df + dur df ) P de, f IN(start) where start be the start vertex of e not containing demand d (start / orig(d)). Michal Zerola (STAR) Efficient Data Movement on the Grid February 13, / 26

24 Advantage of time variables P de able to break cycles along the valid paths improved cutting constraint for makespan: e E : min (P de) + X de dur de + SP e < makespan, d D d D where SP e stands for the value of the shortest path from the ending site of e to dest. improved heuristic, called MinPath - instantiate first variable X de such that the following value is minimal : inf P de + dur de + SP e, where inf P de means the smallest value in the current domain of P de. Michal Zerola (STAR) Efficient Data Movement on the Grid February 13, / 26

25 Convergence of the heuristics Heuristics performance for 50 files Heuristics performance for 150 files Makespan (units) FastestLink MinPath Peer-2-Peer Makespan (units) FastestLink MinPath Peer-2-Peer Time (sec) Time (sec) Figure: Convergence of makespan during the search process for FastestLink and MinPath. Michal Zerola (STAR) Efficient Data Movement on the Grid February 13, / 26

26 References Efficient scheduling of data transfers and job allocations Doktorandske dny 2008 (sbornik workshopu doktorandu FJFI oboru Matematicke inzenyrstvi). CVUT, Praha, 2008, pp (ISBN ) Using constraint programing to resolve the multi-source / multi-site data movement paradigm on the Grid - ACAT 2008, 10pp, Proceedings of Science (SISSA) Using Constraint Programming to Plan Efficient Data Movement on the Grid zer0/papers/cpaior 2009/ Submitted to CP-AI-OR 2009, 15pp (Springer Lecture Notes in Computer Science) Planning Heuristics for Efficient Data Movement on the Grid zer0/papers/mista 2009/ Submitted to MISTA 2009, 4pp (SPLNCS, Journal of Scheduling)... to appear in CHEP 2009 Michal Zerola (STAR) Efficient Data Movement on the Grid February 13, / 26