Optimal Multi-Robot Path Planning on Graphs: Complete Algorithms and Effective Heuristics

Transcription

1 Optimal Multi-Robot Path Planning on Graph: Complete Algorithm and Effective Heuritic Jingjin Yu Steven M. LaValle Abtract arxiv: v [c.ro] Jul 05 We tudy the problem of optimal multi-robot path planning on graph (MPP) over four ditinct minimization objective: the makepan (lat arrival time), the maximum (ingle-robot traveled) ditance, the total arrival time, and the total ditance. In a related paper Yu and LaValle (05), we how that thee objective are ditinct and NP-hard to optimize. In thi work, we focu on efficiently algorithmic olution for olving thee optimal MPP problem. Toward thi goal, we firt etablih a one-to-one olution mapping between MPP and network-flow. Baed on thi equivalence and integer linear programming (ILP), we deign novel and complete algorithm for optimizing over each of the four objective. In particular, our exact algorithm for computing optimal makepan olution i a firt uch that i capable of olving extremely challenging problem with robot-vertex ratio a high a 00%. Then, we further improve the computational performance of thee exact algorithm through the introduction of principled heuritic, at the expene of ome optimality lo. The combination of ILP model baed algorithm and the heuritic prove to be highly effective, allowing the computation of.x-optimal olution for problem containing hundred of robot, denely populated in the environment, often in jut econd. I. INTRODUCTION In thi paper, we tudy the problem of optimal multi-robot path planning on graph (MPP), focuing on the deign of complete algorithm and effective heuritic. In an MPP intance, the robot are uniquely labeled (i.e., ditinguihable) and are confined to an arbitrary connected graph. The robot may move from a vertex to an adjacent one in one time tep in the abence of colliion, which may occur when two robot imultaneouly move to the ame vertex or along the ame edge in different direction. A ditinguihing feature of our MPP formulation i that we allow robot on fully occupied cycle to rotate ynchronouly. Such a formulation, more appropriate for multi-robot application, ha not been widely tudied (except, e.g., Standley (00); Standley and Korf (0)). Over the baic MPP formulation, we look at four commonly tudied minimization objective: the makepan (lat arrival time), the maximum (ingle-robot traveled) ditance, the total arrival time, and the total ditance. We note that thee global objective have direct relevance toward real world multi-robot application uch a autonomou warehoue ytem Wurman et al. (008). For example, minimizing makepan i equivalent to minimizing the tak completion time, wherea minimizing total ditance i applicable to minimizing the fuel conumption of the entire fleet of robot. In a related paper Yu and LaValle (05), we how that thee objective are all ditinct and NP-hard to optimize, uggeting that effort on olving optimal MPP hould be directed at finding effective, near-optimal algorithm. In thi work, we make an attempt toward thi goal and propoe a novel yet general framework for olving MPP optimally. By examining pace and time dimenion jointly, we oberve a one-to-one mapping between a olution for an MPP intance and that for a multi-commodity flow problem derived from the MPP problem. Baed on the equivalence relationhip, we can then tranlate the MPP problem into an integer linear programming (ILP) model, which can be ubequently olved uing an ILP olver. The generality of ILP allow the encoding of all four objective to yield complete algorithm for optimizing thee objective. From here, we take a further tep and introduce everal heuritic to boot the algorithmic performance at a light lo of olution optimality. Our method i epecially effective in computing near-optimal minimum makepan olution, capable of computing.x-optimal olution for hundred of robot, denely populated on the underlying graph, often in jut econd. Related work. Multi-robot path planning problem, in it many formulation, have been actively tudied for decade Erdmann and Lozano-Pérez (986); Zelinky (99); LaValle and Hutchinon (998); Silver (005); Kloder and Hutchinon (006); Ryan (008); Janen and Sturtevant (008); Surynek (009); Luna and Bekri (0); Standley and Korf (0); van den Berg and Overmar (005); van den Berg et al. (009); Solovey and Halperin (0); Yu and LaValle (03a); Turpin et al. (04); Solovey et al. (05). A a univeral ubroutine, colliion-free path planning for multiple robot find application in tak panning aembly Halperin et al. (000); Nnaji (99), evacuation Rodriguez and Amato (00), formation control Balch and Arkin (998); Poduri and Sukhatme (004); Shucker et al. (007); Smith et al. (009); Tanner et al. (004), localization Fox et al. (000), micro droplet manipulation Ding et al. (00); Griffith and Akella (005), object tranportation Matarić et al. (995); Ru et al. (995), earch and recue Jenning et al. (997), and o on. We refer the reader to Choet et al. (005); Latombe (99); LaValle (006) and the reference therein for a more comprehenive review on the general ubject of multi-robot path and motion planning. Jingjin Yu i with the Computer Science and Artificial Intelligence Lab, Maachuett Intitute of Technology, Cambridge, MA 039, USA. jingjin@cail.mit.edu. Steven M. LaValle i with the Department of Computer Science, Univerity of Illinoi at Urbana-Champaign, Urbana, IL 680 USA. lavalle@illinoi.edu.

2 The algorithmic tudy of graph-baed multi-robot path planning problem, the focu of thi work, can be traced to a leat 879 Story (879), in which Story make the obervation that the feaibility of the 5-puzzle Loyd (959) i decided by the parity of the game. The 5-puzzle i a retricted MPP intance moving 5 labeled game piece on a 4 4 grid, from ome initial configuration to ome goal configuration. The retriction i that only a ingle game piece near the only empty vertex may move to the empty vertex in a tep multiple mobile robot could move imultaneouly. A generalization of the 5-puzzle i introduced in Wilon (974), extending the problem from 5 game piece on a 4 4 grid to n labeled pebble on an n-vertex, -connected graph. It i hown, together with an implied algorithm, that an intance i alway feaible if the graph i non-bipartite. When the graph i bipartite (uch a the 5-puzzle), all pebble configuration are plit into two group of equal ize uch that any two configuration in the ame group form a feaible intance. A further generalization i introduced in Kornhauer et al. (984), allowing p < n pebble on a graph with n vertice. For thi problem, an O(n 3 ) algorithm i provided to olve an intance or decide that the intance i infeaible. A computer game and multi-robot ytem gain popularity, concurrent movement are introduced and pebble are replaced with robot (or agent). On the feaibility ide, the MPP problem tudied in thi paper i hown to be olvable alo in O(n 3 ) time Yu and Ru (04). To ditinguih the formulation, we denote the formulation that doe not allow cyclic rotation of robot along fully occupied cycle a cycle-free MPP. Until recently, the majority of algorithmic tudy on MPP i on the cyclefree cae. Since the problem i hown to be tractable Kornhauer et al. (984), mot algorithmic tudy of cycle-free MPP put ome emphai on optimality. Through the clever ue of primitive operation, algorithm from Ryan (008); Surynek (009); Luna and Bekri (0); Sajid et al. (0); de Wilde et al. (04) could quickly olve difficult problem with ome form of completene guarantee. Thee algorithm do not have optimality guarantee but the produced olution are often of much better quality than the O(n 3 ) bound given by Kornhauer et al. (984). For more dicuion and reference on ub-optimal method, ee Sajid et al. (0); de Wilde et al. (04). Puhing more toward the optimality ide, mot algorithmic reult explore way to limit the exponential earch pace growth induced by multiple robot. One of the firt uch algorithm, Local Repair A* (LRA*) Zelinky (99), plan robot path imultaneouly and perform local repair when conflict arie. Focuing on fixing the (locality) hortcoming of LRA*, Windowed Hierarchical Cooperative A* (WHCA*) propoe uing a pace-time window to allow more choice for reolving local conflict while imultaneouly limiting the earch pace ize Silver (005). Iterative deepening technique i hown to be applicable to MPP problem in Sharon et al. (0). A technique called ub-dimenional expanion i hown to perform well in complex environment Wagner and Choet (0); the robot denity i however relatively low (04 cell per robot according to the paper). In Standley (00); Standley and Korf (0), intead of agnotic diection of an intance, the natural idea of independence detection (ID) i explored to only conider multiple robot jointly (the ource of exponential earch pace growth) a neceary. With operator decompoition (OD) that treat each legal move a an operator, the author produced algorithm (ID,OD+ID, and related variant) that prove to be quite effective in computing total time- or ditance-optimal olution. We point out that ID and OD+ID have upport for handling cycle (i.e., they apply to MPP intead of cycle-free MPP). Algorithm deigned for minimizing makepan have alo been attempted, e.g., Surynek (0), but the olution quality degrade rapidly a the robot-vertex ratio increae. Many approache have alo been propoed for olving multi-robot path planning problem in the continuou domain. A repreentative method called velocity-obtacle Kant and Zucker (986); van den Berg et al. (008, 0) explicitly examine velocity-time pace for coordinating robot motion. In Griffith and Akella (005), mixed integer programming (MIP) model are employed to encode the robot interaction. A method baed on the pace-time perpective, imilar to our, i explored in I. Karamouza (0). In Peagood et al. (008), an A*-baed earch i performed over a dicrete roadmap abtracted from the continuou environment. In Solovey et al. (04), dicrete-rrt (d-rrt) i propoed for the efficient earch of multi-robot roadmap. Algorithm for dicrete MPP, cycle-free or not, have alo helped olving continuou problem Solovey and Halperin (0); Krontiri et al. (0). Contribution. We tudy the optimal MPP formulation allowing up to n robot on a n-vertex connected graph, which we believe i better uited for multi-robot application. The formulation i not widely tudied, perhap due to the inherent difficulty in handling cyclic rotation of robot. Beide the novelty of the problem, thi work bring everal algorithmic contribution. Firt, baed on the equivalence relationhip between MPP and network flow, we etablih a general and novel olution framework allowing the compact encoding of optimal MPP problem uing ILP model. We how that the framework readily produce complete algorithm for minimizing the makepan (lat arrival time), the maximum (ingle-robot traveled) ditance, the total arrival time, and the total ditance, which are perhap four mot common global objective for MPP. The reulting algorithm, in particular the one for computing minimum makepan, are highly effective in computing challenging problem intance with robot-vertex ratio up to 00%. Second, we introduce everal principled heuritic, in particular a k-way plit heuritic that divide an MPP intance over the time horizon, to give the exact algorithm a izable performance boot at the expene of ome olution optimality lo. With thee heuritic, we are able to extend our algorithm to tackle problem with everal hundred robot that are extremely denely populated, while at the ame time maintain.x olution optimality. Lat but not leat, our ucceful exploit of ILP to attack optimal MPP how that integer linear programming method i competitive with direct earch method in thi problem domain, epecially when the number of robot become large. Thi i urpriing becaue ILP olver are not deigned pecifically for MPP.

3 3 The ret of the paper i organized a follow. In Section II, we define the optimal MPP problem and provide background material on network flow. We then etablih the equivalence relationhip between MPP and multi-commodity flow in Section III. We derive complete algorithm in Section IV baed on the equivalence and continue to decribe the performance booting heuritic in Section V. We evaluate the algorithm in Section VI and conclude in Section VII. Thi paper i partly baed on Yu and LaValle (03a,b,c). In comparion to Yu and LaValle (03a,b), beide demontrating ignificantly improved computational performance due to the addition of the k-way plit heuritic, we have ubtantially extended the generality of our ILP-baed algorithmic framework, which now upport all common global time- and ditance-baed objective. II. PRELIMINARIES We now define MPP and the optimality objective tudied in thi paper. Following the problem tatement, we gave a brief review on network flow. A. Multi-Robot Path Planning on Graph Let G = (V, E) be a connected, undirected, imple graph, with V = {v i } being the vertex et and E = {(v i, v j )} the edge et. Let R = {r,..., r n } be a et of n robot. The robot move at dicrete time tep (i.e., at t = 0,,...). At time tep t = 0, each robot occupie a ditinct vertex of G. In general, at any time tep t = 0,,..., the robot aume a configuration that i an injective map from R to V. The tart (initial) and goal configuration of the robot are denoted a x I and x G, repectively. Fig. (a) how a poible configuration of 9 robot on a 3 3 grid graph. Fig. (b) how a poible goal configuration, in which the robot are ordered baed on row-major ordering (a) (b) Fig.. a) A 9-puzzle problem. b) The deired goal configuration. During a dicrete time tep, each robot may either remain tationary or move to an adjacent vertex. To formally decribe a plan, let a cheduled path be a map p i : Z + V, in which Z + := N {0}. A cheduled path p i i feaible if it atifie the following propertie: ) p i (0) = x I (r i ). ) For each i, there exit a mallet t i Z + uch that p i (t i ) = x G (r i ). 3) For any t t i, p i (t) x G (r i ). 4) For any 0 t < t i, (p i (t), p i (t + )) E or p i (t) = p i (t + ) (if p i (t) = p i (t + ), robot r i tay at vertex p i (t) between the time tep t and t + ). We ay that two path p i, p j are in colliion if there exit k Z + uch that p i (t) = p j (t) (meet colliion) or (p i (t), p i (t + )) = (p j (t + ), p j (t)) (head-on colliion). A an illutration, Fig. how the feaible and infeaible move for two robot during a ingle time tep 3. The multi-robot path planning on graph (MPP) problem i defined a follow. (a) (b) (c) Fig.. Some feaible and infeaible move for two robot. a) A feaible ynchronou move. b) An infeaible ynchronou move in which two robot collide head-on. c) An infeaible ynchronou move in which two robot meet at a vertex. Problem (Multi-robot Path Planning on Graph) Given a 4-tuple (G, R, x I, x G ), find a et of path P = {p,..., p n } uch that p i are feaible path for repective robot r i and no two path p i, p j are in colliion. For example, Fig. (a) and Fig. (b) define an MPP problem on the 3 3 grid. We call thi particular problem the 9-puzzle problem (a variant of the 5-puzzle Ratner and Warmuth (990)), which readily generalize to N -puzzle. Remark. With a few exception (e.g., Standley and Korf (0)), mot exiting tudie on dicrete multi-robot path planning problem require empty vertice a wap pace. In thee formulation, in a time tep, a non-interecting chain of robot may move imultaneouly only if the head of the chain i moving into a previouly unoccupied vertex. In contrat, our MPP Yu and LaValle (03c) i a preliminary poter preentation. In thi paper, we generally ue haded dic to mark tart location of robot and dic without hading for goal location. 3 We aume that the graph G only allow meet or head-on colliion. The aumption i mild. For example, a (arbitrary dimenional) grid with unit edge ditance i uch a graph for robot of with radii of no more than /4.

4 4 formulation allow ynchronized rotation of robot along fully occupied cycle (ee, e.g., Fig. and 3). Thi implie that even when the number of robot equal the number of vertice, robot can till move on dijoint cycle. We note that MPP can be olved in polynomial time with feaibility tet taking only linear time Yu and Ru (04). B. Optimal Formulation Let P = {p,..., p n } be an arbitrary feaible olution to ome fixed MPP intance. For a path p i P, let len(p i ) denote the length of the path p i, which i increaed by one each time when the robot r i pae an edge. A robot, following a path p i, may viit the ame edge multiple time. Recall that t i denote the arrival time of robot r i. In the tudy of optimal MPP formulation, we examine four common objective with two focuing on time optimality and two focuing on ditance optimality. Below, each objective i formally defined. Objective (Makepan) Compute a path et P that minimize max i n t i. Objective (Maximum Ditance) Compute a path et P that minimize max i n len(p i ). Objective 3 (Total Arrival Time) Compute a path et P that minimize n i= t i. Objective 4 (Total Ditance) Compute a path et P that minimize n i= len(p i). The intuitive meaning of thee objective i clear. Fig. 3 illutrate the four-tep minimum makepan olution to the 9-puzzle problem from Fig.. The olution i optimal a it take at leat four tep for robot 9 to move to it goal. Fig. 3. A 4-tep olution from our algorithm. The directed edge how the moving direction of the robot at the tail of the edge. C. Network Flow Review A network N = (G, c, c, S) conit of a directed graph G = (V, E) with c, c : E Z + a the map pecifying capacitie and cot over edge, repectively, and S V a the et of ource and ink. Let S = S + S, with S + denoting the et of ource vertice, S denoting the et of ink vertice, and S + S =. For a vertex v V, let δ + (v) (rep., δ (v)) denote the et of edge of G going to (rep., leaving) v. A feaible (tatic) S +, S -flow on thi network N i a map f : E Z + that atifie edge capacity contraint, the flow conervation contraint at non terminal vertice, v V \S, e E, f(e) c (e), () e δ + (v) and the flow conervation contraint at terminal vertice, F (f) = ( v S + e δ (v) ( v S e δ + (v) = f(e) f(e) f(e) e δ (v) e δ + (v) e δ (v) f(e) = 0, () The quantity F (f) i called the value of the flow f. The claic (ingle-commodity) maximum flow problem ak the following quetion: Given a network N, what i the maximum F (f) that can be puhed through the network? The minimum cot maximum flow problem further require the flow to have a minimum total cot among all maximum flow. That i, we want to find a flow among all maximum flow that alo minimize the quantity c (e) f(e). (4) e E The network flow formulation decribed o far only conider a ingle commodity, correponding to all robot being interchangeable. For general MPP formulation, the robot are ditinct and mut be treated a different commoditie. Such f(e)) f(e)). (3)

5 5 problem can be captured with the Multi-commodity flow or imply multiflow. Intead of having a ingle flow function f, we have a flow function f i for each commodity i. The contraint (), (), and 3 become i, e E, f i (e) c (e), (5) i, v V \S, i, e δ + (v) ( v S + e δ (v) ( v S e δ + (v) = i f i (e) f i (e) f i (e) e δ (v) e δ + (v) e δ (v) f i (e) = 0, (6) f i (e)) f i (e)). Maximum flow and minimum cot flow problem may alo be poed under a multiflow etup; we omit the detail. Our review on network flow only touche apect pertinent to the current work; for a thorough coverage on the ubject of network flow, ee Aronon (989); Ahuja et al. (993) and the reference therein. Note that the multiflow model tated here i ometime alo referred to a integer multiflow becaue f i mut have integer value. (7) III. FROM MULTI-ROBOT PATH PLANNING TO MULTIFLOW - - u(t ) 0 u(t+) A cloe algorithmic connection exit between optimal MPP and network flow problem. Maximum (ingle commodity) flow problem generally admit efficient (low-degree polynomial time) algorithmic olution Ahuja et al. (993), wherea maximum multiflow i a well-known NP-hard problem, difficult to even approximate Andrew and Zhang (005). Mirroring the diparity between ingle- and multi-commodity flow, in the domain of MPP problem, if there i a ingle group of interchangeable robot (here, for a group, it doe not matter which robot goe to which goal a long a all goal location aigned to the group are occupied by robot from the ame group), then many optimal formulation admit polynomial time algorithm Yu and LaValle (03a). However, a oon a a ingle group of robot plit into two or more group, finding optimal path for thee robot become intractable Yu and LaValle (05). The apparent imilarity between optimal MPP and multiflow i perhap bet explained through a graph-baed reduction from MPP problem to network flow problem. The reduction will alo form the bai of our algorithmic olution. To decribe the reduction, we ue a an example the undirected graph G in Fig. 4(a), with tart vertice { + i }, i =, and goal vertice { i }, i =,. An intance of MPP i given by (G, {r, r }, x I : r i + i, x G : r i i ). We will reduce the problem to a network flow problem N = (G, c, c, S). The reduction proceed by contructing a network that i a time- + + v(t ) 0 v(t+) (a) (b) Fig. 4. a) A imple G. b) A merge-plit gadget for plitting an undirected edge through time tep, for enforcing the head-on colliion contraint. expanded verion of the graph G, which then allow the explicit conideration of the interaction among the robot over pace and time. To carry out thi expanion, a time horizon mut firt be decided. For different optimality objective, the expanion time horizon, ome natural number T, i generally different; for now we aume that T i fixed. To begin building the network, we create T + copie of G vertice, with indice 0,,,..., a hown in Fig. 5. For each vertex v G, denote thee copie v(0) = v(0), v(), v(), v(),..., v(t ). For each edge (u, v) G and time tep t, t +, 0 t < T, the merge-plit gadget hown in Fig. 4(b) i added between u(t), v(t) and u(t + ), v(t + ) (arrow from the gadget are omitted from Fig. 5 ince they are mall). For the gadget, we aign unit capacity to all edge, unit cot to the horizontal middle edge, and zero cot to the other four edge. The merge-plit gadget enure that two robot cannot travel in oppoite direction on an edge in the ame time tep, which prevent head-on colliion between two robot. To finih the contruction of Fig. 5, for each vertex v G, we add one edge between every two ucceive copie (i.e., we add the edge (v(0), v()), (v(), v() ),..., (v(t ), v(t ) )). Thee correpond to the green and blue edge in Fig. 5. For all green edge, we aign them unit capacity and cot; for all blue edge, we aign them unit capacity and zero cot. The green edge allow

6 6 e e Fig. 5. The time-expanded network with an expanion time horizon of T = over the bae graph Fig. 4(a). robot to tay at a vertex during a time tep wherea blue edge enure that each vertex hold at mot one robot, enforcing the meet colliion contraint. Fig. 5 (with the exception of edge e and e, which will become relevant hortly), the time-expanded network, i the deired G. For the et S = S + S, we may imply let S + = {v(0) : v { + i }} and S = {v(t ) : v { i }}. N = (G, c, c, S) i now complete; we have reduced MPP to an integer multiflow problem on N with each robot from R a a ingle type of commodity. Theorem Let (G, R, x I, x G ) be an MPP intance. There i a bijection between it olution et (with a maximum number of time tep up to T ) and the integer maximum multiflow olution of flow value n on the time-expanded network N contructed from (G, R, x I, x G ) with T time tep. PROOF. Injectivity. Aume that P = {p,..., p n } (n i the number of robot) i a olution to an intance of MPP. For each p i and every time tep t = 0,..., T, we mark the copy of p i (t) and p i (t) (recall that p i (t) correpond to a vertex of G) at time tep t in the time-expanded graph G. Connecting thee vertice of G equentially (there i a unique way to do thi) yield one unit of flow f i on N (after connecting to appropriate ource and ink vertice in S +, S, which i trivial). It i traightforward to ee that if two path p i, p j are not in colliion, then the correponding flow f i, f j on N are vertex dijoint path and therefore do not violate any flow contraint. Since any two path in P are not in colliion, the correponding et of flow {f,..., f n } i feaible and maximal on N. Surjectivity. Aume that {f,..., f n } i an integer maximum multiflow on the network N with f i = for all i. Firt we etablih that any pair of flow f i, f j are vertex dijoint. To ee thi, we note that f i, f j (both are unit flow) cannot hare the ame ource or ink vertice due to the unit capacity tructure of N enforced by the blue edge. If f i, f j hare ome non-ink vertex v at time tep t > 0, both flow then mut pa through the ame blue edge (ee Fig. 4(b)) with v being either the head or tail vertex, which i not poible. Thu, f i, f j are vertex dijoint on N. We can readily convert each flow f i to a correponding path p i (after deleting extra ource vertex, ink vertice, vertice in the middle of the gadget, and tail vertice of blue edge) with the guarantee that no p i, p j will collide due to a meet colliion. By contruction of N, the gadget we ued enure that a head-on colliion i alo impoible. The et {p,..., p n } i then a olution to MPP. Remark. A multiflow problem can be reduced to an MPP problem (on a directed graph) a well. In particular, if all edge in the network have unit capacity and there i a ingle unit of each type of commodity, then a multiflow problem i a multi-robot path planning problem, often known a the edge dijoint path problem Roberton and Seymour (995). IV. COMPLETE, INTEGER LINEAR PROGRAMMING-BASED ALGORITHMS FOR OPTIMAL MPP PROBLEMS Becaue optimizing MPP olution over Objective -4 are computationally intractable, reducing MPP to multiflow problem doe not make thee optimal MPP problem any eaier. However, with a network flow formulation (ee Secion II-C), it become poible to etablih integer linear programming (ILP) model for optimal MPP formulation. Thee ILP model can then be olved with powerful linear programming package. In comparion to A -baed algorithm augmented with heuritic which often target important but a limit et of problem tructure, ILP-baed algorithm propoed here are agnotic to pecific problem tructure. A uch, ILP-baed algorithm appear more capable at attacking a wider range of MPP problem and in particular difficult MPP intance in which the robot-vertex ratio i high. In thi ection, we build ILP model for each of Objective -4. A. Minimizing the Makepan A minimum makepan olution to an MPP intance I = (G, R, x I, x G ) can be computed uing a maximum multiflow formulation. Fixing a time pan T, let N = (G, c, c, S) be the time-expanded network for I, a et of n loopback edge are

7 7 added to G by connecting each pair of correponding tart and goal vertice in S, from the goal to the tart. We ue e j to denote edge of G and let the n loopback edge take the firt n indice, with e j, j n being the edge connecting the goal vertex of r j to the tart vertex of r j. For example, for the G in Fig. 5, e and e are the loopback edge for r and r, repectively. The lookback edge have unit capacitie and zero cot. Next. for each edge e j G (including the lookback edge), create n binary variable x,j,..., x n,j correponding to the flow through that edge, one for each robot. The i, x i,j = if and only if robot r i pae through e j in G. The variable x i,j mut atify two edge capacity contraint and one flow conervation contraint, n e j, x i,j, (8) The objective function i i= i, j n, i j, x i,j = 0, v G and i n, x i,j = max e j δ + (v) i n e j δ (v) x i,j. (9) x i,i. (0) For each fixed T, a olution to the above ILP model with an objective equaling n mean that a feaible olution to MPP i found. We are to find the minimal T that yield uch a feaible olution. To do thi, tart with T being the maximum over all robot the hortet poible path length for each robot, ignoring all other robot. An ILP model for thi T i then built and teted for a feaible olution. If the model i not feaible, T i increaed and the procedure repeated. The firt feaible T i the optimal T. Once a feaible model with the minimum expanion time horizon T i found, the robot path can be extracted baed on the proof of Theorem. The algorithm i in fact complete: Since the problem i dicrete, there i only a finite number of poible tate. Therefore, for ome ufficiently large T, there mut either be a feaible olution or we can pronounce that none can exit (T = O( V 3 ) i big enough Yu and Ru (04)). Denoting the reulting algorithm a MINMAKESPAN we have hown the following. Propoition Algorithm MINMAKESPAN i a complete algorithm for finding minimum makepan olution for MPP. B. Minimizing the Maximum Single-Robot Traveled Ditance For minimizing the maximum ditance traveled by any ingle robot, the network and variable creation remain the ame a the minimum makepan etup; contraint (8) and (9) remain unchanged. Becaue we want to end all robot to their goal, a maximum flow may be forced through the additional contraint i n, x i,i =. () To encode the min-max objective function, we introduce an additional integer variable x max and add the contraint c (e j ) x i,j x max () e j G,j>n for all i n. For a fixed i, the left ide of () repreent the ditance traveled by robot r i. The objective function i then imply min x max. (3) Denoting the algorithm a MINMAXDIST, we have Propoition 3 Algorithm MINMAXDIST ditance traveled by a ingle robot. i a complete algorithm for finding olution to MPP that minimize the maximum Remark. Auming that we have a minimum makepan olution, we can better bound the time horizon T needed for MINMAXDIST. Let t min be the minimum makepan computed by MINMAKESPAN, etting T = nt min i then ufficient for finding the min x max. To ee that thi i true, we firt note that min x max t min becaue the minimum makepan olution require robot to to ynchronize their move, which may force ome robot to travel unnecearily. Then, the n robot cannot move a total ditance of more than n min x max nt min becaue no robot may travel more than min x max edge and therefore cannot ue more than n min x max time in total.

8 8 C. Minimizing the Total Arrival Time The ILP model for minimizing the total arrival time i more involved in the way the objective function i contructed. Firt, the network and all variable from the minimum makepan ILP-model are inherited. We alo inherit contraint (8), (9), and (). To repreent the objective function, for each time tep t T and each v := x G (r i ), i n, we create a binary variable yi t. Then, we give new indicie to ome variable that are already created. Recall that for each edge e j = (v(t), v(t) ) G (e.g., the four extra bold blue edge in Fig. 5), a variable x i,j i created. There are a total of nt uch variable. Here, we give thee variable a econd index x t i. That i, xt i i the binary variable indicating whether edge (v(t), v(t) ) G, v := x G (r i ), i ued by robot r i. Given a network with a fixed T, if contraint (8), (9), and () can be atified, then there i a feaible olution to the original MPP problem. In thi cae, x T i for i n. We let yi T = x T i. Then, each yt i, t < T i defined recurively over x t i and yt+ i a yi t y t+ i + x t i, yi t y t+ i, yi t x t i. (4) The contraint (4) effectively perform the logical and operation over x t i and yt+ i and tore the reult in yi t. In the end, the mallet t for which yi t = i the time robot r i reache it goal (and top). Therefore, for each i n, T t= yt i i the number of time tep from the time r i arrive at it goal until time T. T T t= yt i i then the time pent by r i. To minimize the total arrival time, the objective function can be expreed a nt yi. t (5) i n, t T Denoting the reulting algorithm (with a ufficiently large T ) a MINTOTALTIME, we have Propoition 4 Algorithm MINTOTALTIME i a complete algorithm for finding minimum total arrival time olution for MPP. D. Minimizing the Total Ditance From the ILP model for minimizing the maximum ditance, we only need to change the objective function for computing a minimum total ditance olution. We do not need the variable x max and imply update the objective function to min c (e j ) x i,j. (6) e j G,j>n, i n Denoting the algorithm a MINTOTALDIST, we have Propoition 5 Algorithm MINTOTALDIST i a complete algorithm for finding minimum total ditance olution for MPP. Again, T = nt min i ufficient for building a network that contain a minimum total ditance olution, if one exit. V. HEURISTICS FOR EFFECTIVE COMPUTATION OF NEAR-OPTIMAL SOLUTIONS In Section IV, in contructing the ILP model for Objective -4, our goal i to how the univeral applicability of the network flow model (e.g., Fig. 5) toward many different optimal MPP formulation. The reulting ILP-baed algorithm are complete and alway produce the optimal olution in principle. A will be hown in Section VI, uch algorithm perform very well in handling relatively mall but extremely challenging problem. Neverthele, a the problem ize grow (i.e., a the graph G and the number of robot n get larger), the computation time needed by ILP olver grow rapidly. From a practical point of view, it may be far more deirable to quickly compute a good quality but ub-optimal olution than waiting forever for the optimal olution. In thi ection, we introduce everal heuritic to accomplih jut that, with a particular focu on computing olution with minimum makepan. A. Building More Compact Model To get the bet performance out of a olver, it i beneficial to have a lean model (i.e., fewer column and row). So far, our focu ha been to provide a general network flow baed framework o that the ILP model can be eaily built. When it come to tranlating the model to an ILP olver, they can be further implified. The heuritic dicued in thi ubection aim at making the repreentation of the contraint (8) and (9) more compact in the reulting ILP model. A uch, they apply to all the optimality objective.

9 9 e e Fig. 6. A more compact repreentation of the network flow graph from Fig. 5. u(t ) u(t+) v(t ) v(t+) Fig. 7. The implified merge-plit gadget for enforcing the head-on colliion contraint. Better encoding of the colliion contraint: Recall that in building the network flow model (e.g., Fig. 5), we ued a mergeplit gadget (Fig. 4(b)) for enforcing the head-on colliion contraint and extra time tep (e.g., the blue edge in Fig. 5) for avoiding meet colliion. When we tranlate thi into linear contraint, thee tructure can be implified to yield the more compact tructure illutrated in Fig. 6. In the newer tructure, each merge-plit gadget now ha two edge intead of five. Alo, the blue edge are removed. The updated gadget for an edge (u, v) E between time tep t and t + i hown in Fig. 7 (note that due to the removal of the blue edge, vertice uch a v(t) i no longer needed). That i, intead of five, only two variable are needed for each robot. Denoting thee binary variable a x i,(u(t),v(t+)) and x i,(v(t),u(t+)) for a robot r i, the head-on colliion contraint for a ingle gadget can be readily encoded a n n x i,(u(t),v(t+)) + x i,(v(t),u(t+)). (7) i= i= Then, to enforce the meet contraint, for example at a vertex v(t), we imply require that at mot one outgoing edge from v(t) may be ued, i.e., x i,j. (8) e j δ + (v(t)), i n Overall, the newer ILP model i roughly half of the ize of the original model. Reachability analyi: In the time-expanded graph, there are redundant binary (edge) variable that can never be true becaue ome edge are never reachable. For example, in Fig. 6, at t = 0, the only outgoing edge that can poible be ued are thoe originate from + and +. The ret can be afely removed. In general, for each robot r i, baed on it reachability from it tart vertex and to it goal vertex, a izable number of binary variable x i,j can be deleted. B. Divide-and-Conquer Over Time Domain In evaluating the ILP model-baed algorithm for optimal makepan computation, we oberve that the ILP olver running time appear to grow exponentially a the ize of the model grow. Thi prevent the algorithm from performing well over intance with more than a few ten of robot. The obervation, while hampering the effectivene of the exact algorithm, turn out to offer a ueful inight toward a highly effective heuritic. We find that, when the overall ize of the ILP model i relatively mall and the robot-vertex ratio i not approaching, even when there are a large number of robot, the model uually doe not preent much challenge for ILP olver. To apply the ILP model-baed method to more challenging problem (e.g., olving problem with hundred of robot quickly), we imply limit the ize of the individual ILP model fed to the olver. One way to achieve thi i through divide-and-conquer over the time domain. We ue a imple example (ee Fig. 8) to illutrate the idea.

10 0 Fig. 8. (a) a) A imple two-robot problem. b) The time-divided intance. (b) In Fig. 8(a), we have a imple planning problem for two robot on a 3 3 grid. To execute the heuritic, we firt compute a hortet path for every pair of tart and goal location. In thi cae, we get the orange and green path for robot and, repectively. Then, if we decide to plit the problem into two maller problem, for each of the path, it i plit into two (generally) equal length piece and the middle node i et a the intermediate goal. In our example, we may do thi for robot by etting the intermediate goal location at (, ) from the top-left corner (the brown dic labeled in Fig. 8(b)). For robot, becaue the middle location coincide with that of robot, we pick an alternative location that i not already occupied a the intermediate goal for robot, in thi cae (, ) from the top-left corner. The intermediate goal for the firt intance will alo erve a the tart location of the econd intance. Thi yield two child intance with both requiring a time expanion with tep each, effectively making the individual ILP model roughly half the ize of the original one, which require a time expanion with 4 tep. In general, we may divide a problem into arbitrarily many maller intance in the time domain. If a problem i divided in thi manner to k ub problem, we call the reulting heuritic a k-way plit. Becaue the diviion i over time, there i no interaction between the individual, maller intance. Once we obtain the olution for each child intance, the olution can be glued together by concatenating the reult. In practice, thi imple heuritic dramatically improve algorithm performance without heavy negative impact on path optimality in term of makepan; we oberve a conitent peedup in computational experiment. Remark. The k-way plit heuritic, by deign, i particularly uitable for the makepan objective. Thi i the cae due to the additive nature of the makepan objective over the plit ub-problem. Beide makepan, the heuritic alo applie to ditance objective (i.e., Objective and 4) quite well, a long a the time horizon required for finding ditance optimal olution doe not differ greatly from the time horizon required for minimum makepan olution. The heuritic doe not directly apply to Objective 3 becaue total time i not additive over the plit ub-problem. A an example, uppoe that a -way plit i carried out with each ub-problem having a time horizon of T/. If a robot r i doe not move in the olution to the firt ub-problem (i.e., 0 t T/), it contribute 0 to the total ditance. However, if r i move even a ingle tep in the olution to the econd ub-problem (i.e., T/ t T ), then r i will contribute at leat T/ to the total arrival time. Neverthele, k-way plit i till helpful in thi cae a we may ue it to quickly compute an initial T for performing the time expanion. VI. EXPERIMENTAL EVALUATION In thi ection, we evaluate the performance of our optimal and near-optimal MPP algorithm with an emphai on MINMAKESPAN. Our performance evaluation cover a broad pectrum of typical problem etting. For each etting, we puh the limit on the robot-vertex ratio to a high a 00%. To the bet of our knowledge, the majority of the etting with high robot-vertex ratio have never been attempted with much ucce prior to our tudy. When applicable, we alo compare our reult with the tate-of-the-art found in the literature. In particular, we have examined OD+ID, ID, WHCA*, and COBOPT Surynek (0), among other. OD+ID and ID upport cycle wherea WHCA*appear to be deigned for cycle-free MPP a it could not olve any 9-puzzle. The problem definition for COBOPT ugget it olve MPP but it employ a cycle-free ubroutine for finding feaible olution. OD+ID, ID, and WHCA* are deigned for optimizing total time and total ditance optimal objective, and do not naturally extend to makepan computation. However, the aociated makepan produced by thee algorithm are uually of good quality. Among thee three, our experiment how that ID-baed anytime algorithm i the mot veratile due to it IDA*-like incremental tructure. On the other hand, OD+ID and WHCA* do not cale well when the robot-vertex ratio goe beyond 0-0%, depending on the particular problem etting. 4 COBOPT i deigned for makepan computation. We implemented all algorithm (MINMAKESPAN, MINMAXDIST, MINTOTALTIME, and MINTOTALDIST) in the Java programming language. We take advantage of multi-core CPU when the k-way plit heuritic i being ued. Alo, Gurobi Gurobi Optimization (04), the ILP olver ued in our implementation, can engage multiple core automatically for hard problem. We ran all our tet on a MacBook Pro laptop computer (Intel Core i7-4850hq, 6GB memory). We thank Trevor Standley for haring the C code implementing OD+ID, ID, and WHCA*, among other. We modified (the original code 4 Some of thee algorithm were evaluated without requiring all robot reach their goal. For example, in the WHCA* work Silver (005), if an intance with n robot i olved for p < n robot, the problem i counted a partially olved. We require each intance to be fully olved to be counted a a ucce.

11 upport only 3 3 grid) and compiled the code a a 64-bit window executable under MSVC 00 with all peed optimization flag turned on. The comparion to COBOPT ue the reult provided in Surynek (0), which cover only 8 8 and 6 6 grid. A. Performance of MINMAKESPAN and k-way Split Heuritic We begin our experimental evaluation focuing on the MINMAKESPAN algorithm and the k-way plit heuritic. For thi purpoe, we ue a the baed graph a 4 8 grid with varying number of vertice (0-5%) removed to imulate obtacle. The connectivity of the graph i alway maintained. We note that with 5% vertice removed, the graph i already parely connected at ome place (ee e.g., Fig. 9), making olving thee problem optimally a challenging tak. In preenting the Fig. 9. A typical 4 8 grid intance with 5% vertice removed to imulate obtacle and 50 tart and goal location. Note that the connectivity of the graph i low at ome area. For example, the lower right corner blob i only ingly-connected to the ret of the graph. Some (blue) tart location may overlap with ome (red) goal location. computational reult, each data point in the figure i an average over 0 equentially, randomly generated intance. For each obtacle percentage, we tart from the lowet number of robot (uually 0 or 0) and allocate a maximum of 600 econd for each problem intance. If an intance take more than 600 econd to produce a reult, the intance i failed and we move to the next obtacle percentage. Thi alo mean that a data point i given only if each of the 0 intance i completed within 600 econd. Over the ame et of problem intance, the MINMAKESPAN algorithm i executed in the exact manner (which produce optimal makepan olution) and with the k-way plit heuritic. The exact makepan computation reult i ummarized in Fig. 0. For all obtacle etting, the MINMAKESPAN algorithm compute optimal makepan olution conitently for up to 00 robot with an average computation time of no more than 00 econd. notably, for 50 robot and obtacle up to 0%, the MINMAKESPAN algorithm i able to complete in about 5 econd in all cae. From the top plot of Fig. 0, we oberve that for each fixed obtacle percentage, the computation time appear to grow exponentially with repect to the number of robot. The computational difficulty of a particular problem intance alo heavily depend on the actual optimal makepan. For example, a problem intance in the cae of 5% and 0 robot ha a particularly long makepan (ee the bottom plot of Fig. 0), reulting an unexpected jump of the computation time. Wherea the exact MINMAKESPAN algorithm i reaonably efficient, the k-way plit heuritic bring a ignificant performance boot, allowing a much higher robot-vertex denity in general. In our tet, we are often able to double or triple the upported robot-vertex denity. For the 4 8 grid, we evaluated the k-way plit heuritic for k up to 6. The 4-way plit performance i illutrated in Fig.. In the figure, we meaure optimality uing a conervatively etimated optimality ratio. To obtain thi, we divide the objective value returned by the optimizer over a conervative etimate (a lower bound). For makepan, thi lower bound etimate i obtained by firt computing the hortet path for each robot ignoring all other robot. The minimum makepan etimate i obtained by taking the maximum length over all thee hortet path. Clearly, the optimality ratio obtained thi way i an underetimate. We make three comment over Fig.. Firt, from the top plot, we oberve that our method i highly effective in term of computation time, capable of computing minimum makepan olution for up to 80 robot, which tranlate to a maximum robot-vertex ratio of 44%. The majority of the cae are olved in under 0 econd. Even when there are 5% obtacle, we could olve the problem conitently for up to 60 robot in about 40 econd. Second, we again oberve an exponential relationhip between computation time and the number of robot. Third, all computed olution are very cloe to being optimal, with all but one cae having an optimality ratio of below.. The average minimum makepan for thee intance i about 35.

12 Comp. Time (Second) % Ob 5% Ob 0% Ob 5% Ob 0% Ob 5% Ob Number of Robot 36 Optimal Makepan % Ob 5% Ob 8 0% Ob 5% Ob 6 0% Ob 5% Ob Number of Robot Fig. 0. [top] Average computation time of the exact MINMAKESPAN algorithm over intance on a 4 8 grid with randomly placed obtacle and tart/goal location. [bottom] The (average) optimal makepan. Comp. Time (Second) % Ob 5% Ob 0% Ob 5% Ob 0% Ob 5% Ob Number of Robot.3 Optimality Ratio % Ob 5% Ob 0% Ob 5% Ob 0% Ob 5% Ob Number of Robot Fig.. [top] Average computation time of MINMAKESPAN algorithm (with 4-way plit heuritic) over intance on a 4 8 grid with randomly placed obtacle and tart/goal location. [bottom] The achieved (conervatively etimated) optimality ratio. The ret of the k-way plit evaluation i preented in Fig., in which the computation time and optimality ratio are hown ide by ide without the axi label. We alo omit the key of the plot, which i the ame a thoe from Fig. 0, repreenting different obtacle percentage. In conjunction with Fig. 0 and Fig., a k increae, we oberve a general trend of reduced computation time at the expene of ome optimality lo. With 6-plit, MINMAKESPAN can olve problem with 300 robot, correponding to a robot-vertex ratio of 69%. For comparion, we ran ID-baed anytime algorithm over the ame et of intance with a 600-econd time limit (we alo attempted OD+ID and WHCA*, which do not go pat 40 robot under the ame etup). The reult i plotted in Fig. 3. ID actually perform quite well for up to 60 robot, which can be attributed to it A* root with minimum overhead a compared to our method. However, the performance of ID degrade fater it doe not cale well beyond 00 robot in our tet. MINMAKESPAN, with -way plit, readily outperform ID when there are 40 or more robot. Conceivably, it may be poible to combine ID and k-way plit to make it run fater. However, adding k-way plit to ID will inevitably make the overall makepan more ub-optimal.

13 Fig.. Performance of the MINMAKESPAN algorithm with k-way plit for k = (top row), 8 (middle row), and 6 (bottom row). The computation time and optimality ratio plot for each k are hown ide-by-ide. The x-axi for all plot repreent the number of robot. The y-axi for the plot on the left repreent the computation time in econd. The y-axi for the plot on the right repreent the optimality ratio. The key for all plot are the obtacle percentage, identical to that of Fig Fig. 3. Performance of the ID-baed anytime algorithm over the ame et of problem intance. The plot etup i the ame a that from Fig.. B. Minimum Makepan Solution to N -puzzle Next, we evaluate the efficiency of the algorithm MINMAKESPAN for finding minimum makepan olution to the N -puzzle for n = 3, 4, 5, and 6. Thee problem have a robot-vertex ratio of 00%, making them highly contrained and extremely challenging. Note that an N -puzzle intance i alway olvable for n 3 (ee the Appendix); thi mean that all tate are connected in the tate (earch) pace. We ran Algorithm MINMAKESPAN on 00 randomly generated N -puzzle intance for n = 3, 4, 5. For the 9-puzzle, computation on all intance completed uccefully with an average computation time of 0.46 econd per intance. To compare the computational reult, we implemented a (optimal) BFS algorithm. The BFS algorithm i heavily optimized: For example, cycle of the grid are precomputed and hard coded to ave computation time. Since the tate pace of the 9-puzzle i mall, the BFS algorithm i capable of optimally olving the ame et of 9-puzzle intance with an average computation time of about.08 econd per intance. Once we move to the 6-puzzle, the power of general ILP olver become evident. MINMAKESPAN olved all 00 randomly generated 6-puzzle intance with an average computation time of 4. econd. On the other hand, the BFS algorithm with a priority queue that worked for the 9-puzzle ran out of memory after a few minute. A our reult how that an optimal olution for the 6-puzzle generally require 6 time tep, it eem natural to alo try bidirectional earch, which cut down the