No.6 Distributing and Scheduling Divisible Task 789 ssor Connection Graph and denote it by PCG = (P; L; S; C; A). Each node P i in P represents a proc

Transcription

1 Vol.17 No.6 J. Comput. Sci. & Technol. Nov Distributing and Scheduling Divisible Task on Parallel Communicating Processors LI Guodong (ΞΛ ) and ZHANG Defu (Π ) State Key Lab for Novel Software Technology, Nanjing University, Nanjing , P.R. China Department of Computer Science and Technology, Nanjing University, Nanjing , P.R. China dong@dislab.nju.edu.cn Received January 15, 2001; revised January 10, Abstract In this paper we propose a novel scheme for scheduling divisible task on parallel processors connected by system interconnection network with arbitrary topology. The divisible task is a computation that can be divided into arbitrary independent subtasks solved in parallel. Our model takes into consideration communication initial time and communication delays between processors. Moreover, by constructing the corresponding Network Spanning Tree (NST) for a network, our scheme can be applied to all kinds of network topologies. We present the concept of Balanced Task Distribution Tree and use it to design the Equation Set Creation Algorithm in which the set of linear equations is created by traversing the NST in post-order. After solving the created equations, we get the optimal task assignment scheme. Experiments confirm the applicability of our scheme in real-life situations. Keywords equation 1 Introduction divisible task, distributed processing, scheduling, network topology, linear Recently the model of scheduling divisible task has been brought under investigation [1;2]. Divisible task refers to the task that can be divided into parts of arbitrary size and each part requires exactly the same type of processing. Such a task may be partitioned among processors in an arbitrary way, whereby each part is processed independently and separately. Many important applications can be regarded as divisible tasks. The examples include processing vector, searching records in database, processing long linear data files and so on. This leads to the design of a tractable, continuous model of computation that may be analyzed and solved with standard methods of linear algebra and recursive analysis. During the last few years, many computation models including shared memory models and distributed memory models have been suggested, e.g., log P addresses two aspects of inter-processor communication: communication overheads on processors (Parameter o") and communication delay (Parameter L") [3]. A network's topology can be the bus, star, ring, tree, array, cube or irregular structures. Studies on scheduling divisible task for linear [2;4], tree [5;6], bus [6;7], star [2], two-dimensional mesh [8;9], three-dimensional mesh [10] and cube [11] have been investigated in the past years. However, there has been no efficient model on this problem, which can accommodate all kinds of networks and consider both communication setup time and communication delay. In this paper we propose a novel scheme for scheduling divisible task on parallel processors. The structure of this paper is as follows. Section 2 contains a brief description of our architecture and mathematical models as well as some basic concepts. Section 3 describes the Equation Set Creation Algorithm along with the way of constructing Network Spanning Tree. Section 4 and Section 5 respectively show the results of case analysis and simulation. Finally, we give the conclusion. 2 Model, Concepts and Notations A distributed computer system consists of n processors connected with a system interconnection network whose topology can be of the structure of any kind. Each processor may have one or many neighbors to which it can send data or message directly. We describe this system with a graph Proce-

2 No.6 Distributing and Scheduling Divisible Task 789 ssor Connection Graph and denote it by PCG = (P; L; S; C; A). Each node P i in P represents a processor, each arc L ij in L represents a communication link connecting P i and P j directly, S ij in S denotes the communication startup time required to initiate communication link L ij, C ij (e.g., seconds/byte) denotes the transfer rate of link L ij and A i (e.g., seconds/byte) denotes the data's processing rate on P i. Initially, the whole volume W of data to be processed resides on the first processor called originator. The originator processes locally W 1 units of data and sends the rest W W 1 units of data to its idle neighbors; after receiving data P i intercepts W i units of data and sends the rest to its still idle neighbors. If a process has m idle neighbors, it will divide the whole sending data into m parts (but not necessarily m parts) and send them to those neighbors separately. The whole task processing finishes when all subtasks have been processed by their separate processors. We consider a distribution scheme aiming to minimize the completion time of the whole task [1;2]. 2.1 Network Spanning Tree We use a PCG to model the distributed system whose interconnection type can be of the structure of any kind (e.g., star, cube). Our equation creation algorithm is based on the tree structure derived from PCG. Initially from the originator, each processor processes its own part of the task and sends the rest to its idle neighbors until no idle neighbor exists. The data transferring paths of the subtasks construct a tree Network Spanning Tree (NST): the originator is the tree's root, its unvisited neighbors are the first layer nodes of the tree, the first layer nodes' unvisited neighbors are the second layer nodes:::, the nodes (processors) without unvisited neighbors are leaves. From PCG = (P; L; S; C; A), we derive NST as (P; L 0 ;S 0 ;C 0 ;A). L 0 consists of all the links through which task loads have passed. The S ij in S 0 and the C ij in C 0 are selected from S and C by this law: S ij in S and C ij in C are added into S 0 and C 0 respectively if and only if there is a corresponding L 0 ij in L0. This means L 0 L, S 0 S and C 0 C: In Section 3 we discuss the algorithm used to derive the NST from a PCG. Fig.1(a) presents an example of constructing the NST from a cube, (b) shows another example dealing with an irregular structure. In these two examples the breath-first search method is used. Fig.1. Constructing NST from PCGs. (a) A cube PCG. (b) Irregular structure PCG. 2.2 Task Distribution Tree To an NST = (P; L 0 ;S 0 ;C 0 ;A), we add in our tree model a parameter W i which refers to the volume of subtask scheduling to P i. This new Task Distribution Tree is denoted as TDT = (P; L 0 ;S 0 ;C 0 ;A;W), whose P; L 0 ;S 0 ;C 0, A are just the same as those of NST, and W represents the task distribution on processors through interconnection network. TDT is used to model the task division and execution in the parallel system. Definition 1. Balanced Task Distribution Tree (BTDT) is a kind of TDT in which each P i finishes processing its local load W i at the same time. When there are no results from processors needed to return to the originator, the completion time (TC) of subtask i refers to the moment when P i finishes processing W i ; otherwise, the completion time is the processing time plus the delay time of returning results to the originator. Theorem 1. A TDT is a BTDT iff: (1) Each node's completion time is equal to that of its nearest right sibling (if this sibling exists); (2) Each non-leaf node's completion time is equal to that of its first

3 790 LI Guodong, ZHANG Defu Vol.17 left child. Proof. In a BTDT, each node's TC is the same, so Conditions (1) and (2) are both satisfied. As to a TDT satisfying Conditions (1) and (2), we consider it as a BTDT by mathematical induction: BASIC STEP: Obviously a TDT whose height is one (this tree has only one node) is a BTDT. INDUCTION STEP: We must show that if the subtrees of a TDT with h+1 height are all BTDT, this TDT is BTDT too. Assume that all nodes whose height is less than or equal to h constitute a BTDT, the TC of each node of this TDT, denoted as TC h, is the same. As to a node n c whose height is h + 1, its parent is denoted as n p, Condition (1) indicates that the TC of each n p 's child (including n c ) is the same, e.g., TC h+1. Through Condition (2) we get TC h+1 = TC h (non-leaf node n p 's completion time is equal to that of its first left child). Because n c is selected arbitrarily, we can conclude that the TDT whose height is h +1is a BTDT. By the principle of mathematical induction, a TDT satisfying Condition (1) and Condition (2) is a BTDT. 2 In Section 3 our algorithm creates the equation set based on Theorem 1. Similarly, we can prove that totally n 1 equations (n is the number of nodes in TDT) are created. In BASIC STEP in which TDT has only one node, no linear equations need to be created because W = W 1. In INDUCTION STEP, if a h +1 height node is the first left child of an h height node, an equation is added according to Condition (1), otherwise, its nearest left sibling has at least one right sibling, and according to Condition (2) an equation is added. In short, adding x nodes in the TDT will bring x new equations into the equation set, thus, eventually the equation set includes n 1 equations established according to Theorem 1, and the last equation to be added is W = W 1 + W W n. Our paper also uses an extended Gantt Chart called Task Distribution Pattern to describe a BTDT. Fig.3 shows the Task Distribution Pattern Fig.2. A derived TDT. of the BTDT depicted in Fig.2. Fig.3. Task distribution pattern of the TDT indicated in Fig.2. In Fig.3 we assume that each processor can process local data and send the rest data in parallel, so immediately after the reception of the load the processor starts both processing its share and sending data to its idle neighbor. If simultaneous communication and computation is not possible, the equations reflecting the time relation of subtasks should be modified, which is shown in Section 3. 3 Algorithms 3.1 Constructing Network Spanning Tree As to linear, bus, star and ring networks, we just use the breath-first search method to create the Network Spanning Tree (NST). In a linear system the first processor is the originator; in bus and ring systems any processor can be the originator; in a star system the central processor is the originator. In tree- and irregular-structure networks, the breath-first search method is applicable. These cases are discussed in Section 4 too. As to mesh and hypercube, we can also construct the NST by the breath-first search method. However, more efficient methods are available. The solution to the problem is tightly connected with the data scattering method. In the circuit-switched routing mode communication delay is virtually

4 No.6 Distributing and Scheduling Divisible Task 791 independent of the covered distance. This property allows each active processor to transfer chunks of load far away to a set of carefully selected processors, instead of sending data to its immediate neighbors [10]. In Section 4 we discuss this in detail. 3.2 Linear Equations In a BTDT = (P; L 0 ;S 0 ;C 0 ;A;W), assume that the total number of processors in P is n, so the whole task is divided into n subtasks, which means kw k = kp k = n. Since each subtask should be finished at the same time, we get n 1 equations TC 1 = TC 2 = = TC n ; moreover, the volume of the whole task W = W 1 + W W n, which is the n-th equation. By solving these equations, each W i is derived and the division and scheduling scheme is found. Our BTDT model considers the communication startup time and delay time. The processing time of W i on P i is A i W i. P i sends the rest data to its neighbors. As to a P i 's neighbor P j, P i sends W ij data to it. Here W ij = k2ds [j] W k where DS[j] = fkj Node k is the descendant of node j in the TDTg [fjg The set DS[j] consists of all descendant nodes of node j. It is formed dynamically while traversing the TDT in post-order. The transferring of W ij data over link L ij lasts S ij +W ij C ij. Using these parameters we formulate n linear equations. For the NST depicted in Fig.1(b), whose corresponding TDT and task distribution pattern are showed in Fig.2 and Fig.3 respectively, the generated equation set is: 8>< >: W 1 A 1 = S 12 +(W 2 + W 4 + W 5 + W 6 + W 7 )C 12 + W 2 A 2 W 2 A 2 = S 13 +(W 3 + W 7 )C 13 + W 3 A 3 W 2 A 2 = S 24 + W 4 C 24 + W 4 A 4 W 3 A 3 = S 37 + W 7 C 37 + W 7 A 7 W 4 A 4 = S 25 + W 5 C 25 + W 5 A 5 W 5 A 5 = S 26 +(W 6 + W 8 )C 26 + W 6 A 6 W 6 A 6 = S 68 + W 8 C 68 + W 8 A 8 W = W 1 + W W 8 We call the set consisting of the upper equations LES (Linear Equation Set). Our scheme uses BTDT model and Theorem 1 to create the LES as indicated in the next section. 3.3 Equation Set Creation Algorithm The TDT is traversed in post-order to create the LES. As to a non-leaf node, according to Condition (2) of Theorem 1, an equation representing the time relation of this node and its most left child is created and added into the LES. If this node has right sibling, according to Condition (1) of Theorem 1, another equation reflecting the time relation of this node and its most right sibling is created too. In Algorithm 1 function Children (Node i) returns the set of Node i's children. The function of procedure Add Equation (a: integer; b: integer) is to add an equation into the LES. If each processor processes data and sends data in parallel, which means the computation and communication is overlapped, the new equation is W a A a = S cb + C cb j2ds(b) (1) (2) (3) (4) (5) (6) (7) (8) W j + W b A b (9) where c = parent(b), if Node a is Node b's parent then c = a, in such a case the equation is created in terms of Condition (2) of Theorem 1. The following table shows the steps of deriving the LES from the NST depicted in Fig.2. The subtask pair indicates the two subtasks corresponding to the OldNode and NewNode, and the equation generated at each step is shown in the Eq." column.

5 792 LI Guodong, ZHANG Defu Vol.17 Algorithm 1. CREATE EQUATION SET Input: An NST = (P; L 0 ;S 0 ;C 0 ;A) and Task W Output: The LES corresponding to the NST Variable: OldNode, NewNode: integer; DS: array(n) of set of integer ftraverse the TDT recursively in post-order, begin with visit (root) to create the whole LESg Visit (CurNode: integer) Begin If Node CurNode is leaf then return; OldNode := maxfjjchildren(curnode)g; ffind the most right child of Current Node, the child's number is ordered from left to rightg visit (OldNode); DS[CurNode] := DS[OldNode]; for i := 2 to i := kchildren(curnode)k do begin NewNode := OldNode 1; visit (NewNode); fvisit the most right unvisited sibling of the new nodeg DS[CurNode] := DS[CurNode] [ DS[NewNode]; Add Equation (NewNode, OldNode); fadd an equation according to Condition (1) of Theorem 1g OldNode := NewNode; end for DS [CurNode] := DS[CurNode] [fcurnodeg Add Equation (CurNode, OldNode); fadd equation according to Condition (2) of Theorem 1g End g Fig.4. Algorithm 1. Table 1. The Steps of Creating LES Step NewNode OldNode CurNode Subtask Pair Descendant Set DS Eq (W 3 ;W 7 ) DS[3] = f7; 3g (4) (W 6 ;W 8 ) DS[6] = f8; 6g (7) (W 5 ;W 6 ) DS[2] = f8; 6; 5g (6) (W 4 ;W 5 ) DS[2] = f8; 6; 5; 4g (5) (W 2 ;W 4 ) DS[2] = f8; 6; 5; 4; 2g (3) (W 2 ;W 3 ) DS[1] = f7; 3; 8; 6; 5; 4; 2g (2) (W 1 ;W 2 ) DS[1] = f7; 3; 8; 6; 5; 4; 2; 1g (1) If each processor firstly sends data except its own subtask to its neighbors and then begins to process its local data, which means that overlapping computation and communication on the same processor is not possible, the Task Distribution Pattern of the BTDT depicted in Fig.2 is altered as in Fig.5. Fig.5. Task distribution pattern of the TDT indicated in Fig.2. (9) should be modified as S ai + C ai W j + W aa a = S cb + C cb W j + S bi + C bi i2children(a) j2ds(i) j2ds(b) i2children(b) j2ds(i) W j + W b A b where function children(a) returns the set of Node a's children. If the equation is created according to Condition (2) of Theorem 1, then c = a, the equation can be transformed to W a A a = S bi + C bi W j S ai + C ai W j + W b A b i2children(b) j2ds(i) i2children(a) fbg j2ds(i)

6 No.6 Distributing and Scheduling Divisible Task Solution to the Equations In the TDT, there are some special nodes appearing in the left side of the two equations in LES, whereas other nodes only appear once. Thus, we introduce the concept of overlapping nodes. Definition 2. Overlapping node is the node whose degree is more than 2 and has at least one right sibling. Theorem 2. Provided that there are m overlapping nodes in a TDT, if a feasible solution to the relative LES exists (W i 0), the LES can be solved in O((m + 1)n) time; when no feasible solutions exist, the maximum usable set of nodes can be found in O((m +1)n log n) time. Proof. If node i is an overlapping node, there are two equations about W i in the LES, and if there are m overlapping nodes, obviously O(mn) time is needed to transform the matrix to an uppertriangular matrix. If there exists a feasible solution which can be solved in O(n) time, thereby, in total the original matrix can be solved in O((m + 1)n) time. If a feasible solution does not exist (a negative value W i is possible), the biggest number of phases for which a feasible solution exists can be found by binary search over n in O(log n) trials, the optimal distribution of the load can be found in O((m +1)n log n) time. A special kind of TDT is that in this tree all non-leaf nodes have and only have one child. It is obvious that the number of overlapping nodes in a TDT is 0, in other words, each node appears in the left of only one equation in LES. This means that the coefficient matrix is an upper-triangular matrix. The equation system from which we find a distribution of the load will be: A i W i = S i;i+1 + C i;i+1 W = n i=1 W i n j=i+1 W j + A i+1 W i+1 ; i = 1; 2;:::;n 1 These equations can be solved in linear time. In the above equations, each variable W i depends on W i+1, W i+2 ;:::;W n (for i = 1; 2;:::;n 1). W n 1 can be represented as a function of W n ; W n 2 can be represented as a function of W n 1 and W n and further as a function of W n ; :::;W i can be represented as a linear function of W i+1 ;:::;W n, and further as a linear function of W n. Thus, W i = k i W n + l i, for i = 1;:::;n 1, represents W i as a function of W n. The k i, l i coefficients can be calculated by the following recursive equations: K n = 1; l n = 0; K n 1 = C n 1;n + A n 1 A n 1 ; L n 1 = S n 1;n A n 1 k i i;i+1p = C n K j=i+1 j + A i+1 K i+1 ; l i = S i;i+1 + C i;i+1p n l j=i+1 j + A i+1l i+1 ; i = 1;:::;n 2 A i A P i (10) (K i=1 iw n + L i ) we obtain W n = W n L i=1 i P n K. i=1 i From W = P n i=1 W i = P n Then W i (for i = 1;:::;n 1) can be calculated by equation W i = k i W n + l i. The time complexity of solving the whole equation set is O(n). 4 Case Analysis and Related Work 4.1 Linear and Ring In a linear array processor i is connected with processor i + 1 by link i (i = 1;:::;n 1), to this topology a linear NST is derived, in which each non-leaf node has only one child. The LES is as follows: W i A i = S i +(W i+1 + W i W n )C i + W i+1 A i+1 ; for i = 1;:::;n 1 W = W 1 + W W n This LES can be solved using (2). It can be solved in O(n) time. When no feasible solution exists

7 794 LI Guodong, ZHANG Defu Vol.17 the originator and some closest processors are able to process the whole load before the last processor receives any data. In such a case the maximum usable set of processors can be found in O(n log n) time [2;4]. Until now we have not found any published papers focusing on this ring topology. The case of a unidirectional ring is just the same as that of linear network. In the bi-directional ring case there is only one overlapping node in it, whose LES can be solved in O(2n) = O(n) time. When no feasible solution exists, the maximum usable set of processors can be found in O(2n log n) = O(n log n) time. 4.2 Bus and Star For the star, all the load resides initially on Processor 1 in the center of the star. Due to the fact that for the star only Processor 1 (the originator) has connection with other processors and that for the bus there is only one bus which cannot be used simultaneously, both networks can be analyzed in the same way. Similar to other schemes [2;6;7], the LES of both cases is as follows: W i A i = S i+1 + W i+1 (C i+1 + A i+1 ) W = W 1 + W W n This LES can be solved using (2), the time complexity of solving this LES is the same as that of the linear network case. 4.3 Mesh, Torus and Hybercube The case of two-dimensional mesh has been discussed by some researchers [8;9;12]. We consider the model of machine characterized by: square 2D-torus topology of the interconnection network, 4-port communications, circuit- or packet-switched communication mode, overlapping of communications by computations, identical processors (processing rate A, startup time S, communication rate C is identical). The Peters-syska's broadcasting algorithm [8] is optimal for a square 2D-torus with 5 2k processors and circuit-switched communication mode. The idea of this algorithm is to send the load far away so that each processor can reach four new processors in each phase. The algorithm is composed of a series of steps consisting in two communication phases as indicated in Fig.6(a). A knight" move and a cross" move are deployed alternatively. For a 2D-torus with 5 2k processors k steps and 2k phases are needed to activate all processors. Fig.6. Deriving NST from a 5 5 mesh PCG. We use Peters-syska's algorithm to construct the NST, which is shown in Fig.6(b), the knight" move and the cross" move are applied alternately to generate new layer nodes of the NST. Under the assumption of overlapping of communications by computations and identical processors (the transfer rate of each communication link is identical in 2D-torus, so does the processing rate of each processor), the NST of (b) can be transformed into a simple tree indicated in (c). It is easy to explain because each node in the same layer has the same beginning time and finishing time, so the most left node of one layer (e.g., [3; 3]; [5; 2]; [1; 2]) can represent all other nodes in the same layer. Thus, the equation set from the simple NST for a 2D-grid with 5 2k processors is W N i A = S + C(W N i+1 +4 i j=2 W N i+j 5 j 2 )+AW N i+1 ; i = 1;:::;N

8 No.6 Distributing and Scheduling Divisible Task = W 0 +4 N i=1 5 i 1 W i This LES can be solved using (10). Moreover, as to the case without overlapping of communications by computations and non-identical processors, our approach can still be applicable, and this approach can be easily extended to larger mesh and three-dimensional mesh. The case of three-dimensional mesh and hypercube is much similar to that of mesh, only the types of moves and the way of selecting processors in successive moves are different. The maximum number of usable layers and distribution of the load among them can be found in O(d logd) time, where d = log 2 n. A load distribution algorithm SCATTER(p) can be used to activate the processors [10;11], and the NST can be constructed using these scattering ways and get the same distribution results. 4.4 Tree and Irregular Structures The cases of trees [6;13] (no startup time), pipelined data distribution [14], variable link and/or PE speed [15] and so on have been investigated respectively, and the approaches used are quite different from ours. In our scheme, for the case of tree, the NST is the exact copy of the original tree. Dealing with the case of irregular structure, we just use the breath-first search to derive the NST from a PCG, e.g., the example in Fig.1. The LES can be established by applying Algorithm 1, and the time complexity of solving the LES is indicated in Theorem Advantages and Extension Compared with the existing schemes [2;4 11], our model can deal with all kinds of networks. Moreover, the efficiency of applying our scheme to a special network is not lower than any of those schemes, e.g., to linear, bus, star, mesh and hybercube network. To schemes [10 12] assuming overlap of communications by computations and identical processor and communication link, the results can be not only applied to not-simultaneous cases, but also hard to extend to the system consisting of processors with diverse capacities. Similar drawbacks exist in many systems [2;4;5;13]. To those tasks that are not arbitrarily divisible, we still use the same method and calculate a lower bound on the schedule length. We may round the calculated W i up to the nearest unit of granularity, which may result in the deviation of the processing time from the value obtained from the equations. 5 Experimental Results The divisible task in our experiment is searching records in database. The experimental environment is a parallel system consisting of SGI Indy workstations (processors). We test the cases of unidirectional ring and irregular structure network. In each case, the data's processing rate A i is gotten by testing the rate of finding records in database on P i repetitively; the communication startup time S ij and the transfer rate C ij of each link are tested while transferring the record through that link. Fig.7. Speed-up of two different sizes of task. Fig.8. Relative difference between test results and theoretical results. Fig.7 shows the speed-up of ring systems. The increase of the number of processors used will cause the increase of speed-up, thus verifying the efficiency of our scheme. While the number of processors increases, the improvement of speed-up becomes more insignificant. This is caused by the increasing communication overheads while adding processors into the system. Furthermore, for small

9 796 LI Guodong, ZHANG Defu Vol.17 computational task (A = 1G) only small numbers of processors can take part in computation and only a limited speed-up can be sustained. The bigger the task, the greater the speed-up can be achieved. Fig.8 shows the relative difference between experimental ones and theoretical results in irregular structure system indicated in Fig.2, where eight processors are used and connected. The difference is greater than that in ring system because its routing is more time-consuming and parameter S ij does not reflect the exact communication startup time, and so does C ij. In this case we can modify S ij and C ij to get better results. 6 Conclusion In this paper we propose a scheme to schedule divisible task on parallel processors connected by any kind of interconnection network. We present the concept of Network Spanning Tree as well as Balanced Task Distribution Tree and use them to design the Equations Set Creation Algorithm, in which the set of linear equations reflecting the time relations of subtasks is created while traversing the NST in post-order. Using unified models our scheme can accommodate many famous distribution schemes focusing on specific networks. References [1] Blazewicz J, Drozdowski M, Markiewicz M. Divisible task scheduling Concept and verification. Parallel Computing, 1999, 25: [2] Drozdowski M. Selected Problems of Scheduling Tasks in Multiprocessor Computer Systems. Poznan University of Technology Press, [3] Culler D E et al. LogP : A practical model of parallel computation. Communications of the ACM, 1996, 39(11): [4] Mani V, Ghose D. Distributed computation in linear networks: Closed-form solutions. IEEE Transactions on Aerospace and Electronic Systems, 1994, 30(2): [5] Bharadwaj V, Ghose D, Mani V. Optimal sequencing and arrangement in distributed single-level tree networks with communication delays. IEEE Transactions on Parallel and Distributed Systems, 1994, 5(9): [6] Bataineh S, Hsiung T, Robertazzi T G. Closed form solutions for bus and tree networks of processors. IEEE Transactions on Computers, 1994, 43(10): [7] Blazewicz J, Drozdowski M. Scheduling divisible jobs with communication startup costs. Discrete Applied Mathematics, 1997, 76(1-3). [8] Park J L, Choi H. Circuit-switched broadcasting in torus mesh networks. IEEE Transactions on Parallel and Distributed Systems, 1996, 7(2): [9] Blazewicz J, Drozdowski M. Performance limits of two-dimensional network of load-sharing processors. Foundations of Computing and Decision Sciences, 1996, 21(1): [10] Drozdowski M, Glazek W. Scheduling divisible loads in a three-dimensional mesh of processors. Parallel Computing, 1999, 25: [11] Blazewicz J, Drozdowski M. Scheduling divisible jobs on hypercubes. Parallel Computing, 1995, 21: [12] Blazewicz J, Drozdowski M. Scheduling a divisible task in a two-dimensional toroidal mesh. Discrete Applied Mathematics, 1999, 94: [13] Cheng Y C, Robertazzi T G. Distributed computation for a tree network with communication delays. IEEE Transactions on Aerospace and Electronic Systems, 1990, 26(3): [14] Bharadwaj V, Ghose D, Mani V. Multi-installment techniques in tree networks. IEEE Transactions on Aerospace and Electronic Systems, 1995, 31(2): [15] Sohn J, Robertazzi T G. A multi-job load sharing strategy for divisible jobs on bus networks. CEAS Technical Report 697, University of Stony Brook, LI Guodong received the B.S. degree in computer science from Zhongshan University, China, in 1997, and the M.S. degree in computer science from Nanjing University, China, in Currently, he is a teaching assistant and research assistant in the Department of Computer Science and Technology, Nanjing University. His research interests include parallel and distributed computing, graph theory. He is a member of IEEE and the IEEE Computer Society. ZHANG Defu received his B.S. degree in mathematics from Nanjing University, China, in He is a professor in the Department of Computer Science and Technology, Nanjing University. His main research areas include parallel and distributed systems, parallel numerical algorithms. Professor Zhang has coauthored more than 90 papers. He has served on the program committees of several conferences, organized several workshops, and guest-edited special issues.