An efficient multilevel master-slave model for distributed parallel computation

Size: px

Start display at page:

Download "An efficient multilevel master-slave model for distributed parallel computation"

Martha McGee
6 years ago
Views:

An efficient multilevel master-slave model for distributed parallel computation Hsin-Chu Chen,W Alvin Lim,<*) and Nazir A. Warsi^ W Army Center of Research in Information Sciences, Dept.

1 An efficient multilevel master-slave model for distributed parallel computation Hsin-Chu Chen,W Alvin Lim,<*) and Nazir A. Warsi^ W Army Center of Research in Information Sciences, Dept. of Computer and Information Sciences, Clark Atlanta University, 223 James P. Brawley Dr. SW, Atlanta, GA hchen@diamond.cau.edu (2) Department of Computer Science and Engineering, Auburn University, 107 Dunstan Hall, Auburn, AL lim@eng.auburn.edu Abstract The master-slave (MS) parallel computing model is one of the most widely used model in a networked computing environment due to its ease of implementation. This model, however, suffers from the disadvantages of the sequential generation of slave processes and heavy communication overheads imposed on the master processor. To overcome this problem, we present in this paper an efficient multilevel master-slave (MMS) scheme which is especially useful for solving decomposable large-scale problems such as structure mechanics or dynamics problems with rotational symmetry, on networked workstations. Our MMS model implements the MS model at multiple levels and generates processes using a special class of tree structures, allowing parallel creation of slave processes. It also improves performance in the distribution of initial data and merging of computed results to and from slave processes. We shall describe the generation of processes using different MMS structures to generate a prescribed number of processes and to broadcast global data to all processes. We then present the implementation of the optimal MMS model via PVM on a networked computer system consisting of workstations for a plate-bending problem that is discretized using the finite strip method The performance of our numerical experiments employing this MMS model is reported to demonstrate its efficiency. This scheme can be applied equally well to other types of problems that can be decomposed using the Fourier decomposition or circular decomposition, no matter whether the physical problem is discretized by the boundary element method or finite element method.

440 Boundary Element Technology 1 Introduction The master-slave (MS) parallel programming model [Geis93, GeSS87] is currently one of the most widely used model in networked computing environment in

2 440 Boundary Element Technology 1 Introduction The master-slave (MS) parallel programming model [Geis93, GeSS87] is currently one of the most widely used model in networked computing environment in which a collection of heterogeneous or homogeneous computers are connected by a network to serve as a large parallel virtual machine. In this model the initiating process, referred to as the master process, is responsible for spawning (generating) all other processes, referred to as slave processes, to perform tasks assigned to them. Although the tasks assigned to the slaves processes can be different, a typical application of this model is that all slaves perform a set of similar tasks (with different set of data). In such cases, usually only two programs are necessary no matter how many slaves there are. One of the two programs is for the master and the other is for all the slaves. The interprocess communications are usually handled through some form of message passing between the master and the slaves. The advantage of the MS model for parallel computation is simplicity in terms of implementation. It suffers, however, from the disadvantage of heavy communication overheads since all slaves communicate only with the master. There is no direct communications among slaves. This disadvantage is especially apparent when the master needs to collect data from all slaves in order to perform an accumulated sum of them. To overcome the drawbacks of this MS model, an approach that implements the MS model at multiple levels as a complete tree structure is presented in [ChBy95]. In this paper, we develop a more efficient approach for the multilevel master-slave (MMS) scheme, which is useful for solving large-scale master-slave problems, e.g., plate-bending problems arising in the area of structural analysis or groundwater modeling that can be handled by the finite strip method [PuSc90, Cheu68] or heat conduction and wave problems that are solved by a fast Poisson solver [Stra86, pp.457] 2 Multilevel Master-Slave Trees The MMS scheme has been demonstrated to be more efficient than the MS model for the problem presented in [ChBy95] and seems to have a great potential for other applications. In this paper, we further investigate the potential of this approach and present our new results. In particular, we present and analyze a special class of unbalanced m-ary tree structure for the MMS approach. It should be mentioned that the generation of our unbalanced tree structure is not arbitrary. Instead, it is generated based on certain rules to be described in this section. Since there can be many master processes in an MMS model, we refer the process at the root of the tree to as the initial master process. We assume that each process can directly generate at most ra slaves, and any single master process cannot simultaneously generate more than one process

3 Boundary Element Technology 441 Figure 1: Spatial representation of an unbalanced tree at any given time which is the case for a machine with only one processing unit. All distinct processes are assumed to execute on different processors, one process per processor. Note also that processes are created as time advances. Therefore, they are dynamic although processors are static. The time spent in generating a new process is assumed to be fixed and taken to be unity in this paper for convenience. We shall refer processes generated at time t = k, k 0, 1, 2,, to as step-k processes. Now, we are in a position to present our proposed unbalanced m-ary tree models for the MMS scheme. To begin with, wefirstconsider the special case when m 2 and then generalize it to the general case of an arbitrary m, m > 1. Note that when TTi 1, the resulting configuration is simply a pipeline of processes, a degenerate tree with no branches, which is of no interest to us since the generation of processes in this case becomes strictly sequential. Due to a close relation between the Fibonacci sequence and the number of processes generated at each time step, this class of unbalanced m-ary tress will be referred to as Fibonacci m-ary trees in this paper. 2.1 Fibonacci Binary Tree When m = 2, each process can generate at most two processes and the resulting configuration is usually an unbalanced binary tree for time t = k > 1. Shown in Figure 1 is the spatial representation of such an example generated in five time steps where the number next to each of the arrows represents the step number at which the process pointed to by the arrow is generated and the depth denotes the level number. A temporal representation of this tree is shown in Figure 2 where the depth denotes the step number instead of the level. It is not necessary to display the step numbers as shown on the links in this representation. We show them in this figure only for the sake of emphasis. We now use this example to explain the generation procedure. First, the initial master process (A) must enroll itself to the networked computing system and is considered as having been generated at time t 0 (step 0). This process then spends one time unit (from time t = 0 to t = I) to generate process B (step 1). Recall that A cannot generate B and C

4 442 Boundary Element Technology Figure 2: Temporal representation of an unbalanced tree at the same time by our assumption. Once B have been generated, both A and B are ready for generating their own slave, C and D, respectively. The generation of C and D can be completed in one time unit since the generation of C by A is entirely independent of the generation of D by B, by our assumption that different processes execute on different processors. This is done at step 2. Now, process A has already generated two slaves, B and C, and, therefore, stops generating new slaves. Accordingly, only three new slaves are generated at step 3. They are processes E, F, and G generated by B, C, and D, respectively. By the same argument, it is not difficult to see that the five leaves in this figure are the only processes that can be generated at the next step. Similar to processes B and C, these five processes can be generated in parallel since their immediate masters are all distinct. In general, the generation of processes continues until a prescribed number of time steps (or a fixed number of processes) has been reached. Let fn be the number of processes generated according to this procedure at time step n (from time t = n- l t o t = n) and Fn be the total number of processes in the tree at time t = n. It is easy to see that fn follows the interesting Fibonacci sequence: 1, 1, 2, 3, 5, 8,..., i.e., with the initial conditions A = A-i+A_2, n = l, 2,, (1) /o = 1 and /_i = 0. (2) The explicit expression for /n, given (2), is well-known [Tuck84, pp.282]: f _ 1 [,1 + An+l (1-An+ll '"-Vl[^2^ ^ 2 > [ As mentioned earlier, we shall refer this type of unbalanced binary trees to as Fibonacci binary trees due to this recurrence relation, in order to distinguish them from other unbalanced binary trees that do not satisfy this recurrence relation. Since Fn is simply the sum of all processes generated

5 Boundary Element Technology 443 Figure 3: A Fibonacci tree of depth 4 with ra = oo (temporal representation). from step 0 up to step n, we have Fn ]Cfc=o f* which follows the following recurrence relation F^i=2^-F^2, ^ = 0, 1,... (3) with the initial conditions FQ = 1 and F_i = F_2 = 0. This relation enables us to determine iteratively the maximum number of steps required to generate a prescribed number of processes for our MMS scheme. To close this subsection, we stress that all the processes generated at a given time step can be accomplished in parallel since their immediate masters are all distinct, a clear advantage of this approach over the traditional MS model, which spawns processes sequentially. For example, to spawn 11 processes (excluding the initial master), this MMS model takes only four time steps as clearly seen in Figure 2, instead of 11 steps as required by the MS model. 2.2 Minimum-generation-time Tree Model (ra = oo) Obviously, as ra increases, each process is allowed to have more slaves. In the extreme case of m oo, there is no constraint on the number of slaves a process can generate. Given a total number of processes to be generated, this extreme case takes the smallest number of steps (the smallest amount of time) to generate a Fibonacci tree, possibly incomplete. This model is, thus, referred to as the minimum-generation-time (MGT) tree model. Figure 3 shows an example of the Fibonacci tree with ra = oo, generated using 5 time steps, A: = 0, 1,, 4. It deserves mentioning that every complete Fibonacci ra-ary tree with TV < ra is a MGT tree. From our generation rules, it is not difficult to see that 2*~* additional slaves can be generated at time step A:, k > 1, making the total number of processes in the tree equal to 2* in the MGT tree model. This appears, in some sense, to be similar to the creation of hypercube nodes [Leig92, 392]. However, the configuration of this model is different from the hypercube structure, as seen in Figure 3. The recurrence relation of / for MGT can

6 444 Boundary Element Technology be expressed as n-l /n = /* = Z>' n = l, 2,.-. (4) k<n k=0 with /o = 1 since every process generated before time t = n can generate an additional slave at t n for all n > 0 and fk = 0 for any A: < 0. It is trivial to see that Fn+i=2Fn, n = 0, 1, (5) with FQ = 1. The MGT model has some very desirable features. Not only does it spend the least time in generating a Fibonacci tree when the total number of processes is prescribed, but it requires the least time (among all possible ra) to flood data from the root to all other processes and to accumulate results (the reduction operation) computed by each process back to the root, which occurs very often in scientific computations. For each of the flood and reduction operations, it takes only log2(fw) steps to accomplish the task where FN is the total number of processes in the tree, assuming FN is a power of 2. 3 Applications The MMS programming paradigm presented in this paper not only can save time spent in spawning processes in a distributed computing environment, as compared with the standard MS model, but is suitable for distributing global data to all processes and for performing reduction operations, e.g., the accumulated sum of a series of numerical values (scalars or vectors). This programming paradigm is useful for solving discretized physical problems (or partial differential equations) that use some form of Fourier decomposition techniques, such as those involved in the fast Poisson solver, fast biharmonic solver, circular decomposition method, and finite strip method. In each of these schemes, three stages are involved. First, a single problem is decomposed into a number of smaller and independent subproblems. In this stage global data (data required by all processes), normally read from an input file or generated by the initial master processes, need to be broadcast to all processes and process-specific data must be distributed to each involved process, except those that can be generated by the involved process itself. In the second stage, the subproblems are solved in parallel, one problem on one processor in principle, without the need of interprocess communications. Once the solutions to all subproblems are obtained, they are combined/accumulated to yield the final result. Parallel reduction operations are involved in this final stage. In the following, we present an application of our proposed MGT MMS model to the finite strip method for solving a special class of plate-bending problems on a networked system consisting of SUN4 workstations. Before

7 Boundary Element Technology ft Figure 4: A rectangular plate discretized into (n 1) strips presenting the physical problem for our experiments, we briefly describe this approach. The finite strip method [Cheu68] is a highly parallel approach with large-grain parallelism. It is a special type of finite element analysis that approximates the true solution of the displacementfieldusing a combination of continuous harmonic functions that satisfy the boundary conditions in one direction and piecewise interpolation polynomials in the other. The method decomposes a single problem into m subproblems if m harmonic functions are employed in the approximation. In other words, each harmonic term will result in a linear subsystem to solve and the three stages mentioned above are all involved in the analysis. With the MGT model, runtime reconfiguration is not necessary in this approach. 3.1 Problem Statement The physical problem we consider for our experiments is the static analysis of a 48 ft by 32 ft rectangular Mindlin plate (0.5 ft thick), simply supported on all four edges. The plate is subject to a 2Q-kip concentrated load acting downward in the z-direction at (z,y) = (12ft,8ft], as shown in Figure 4. The material of the plate is assumed to be isotropic with a Young's modulus equal to kips/ft* and a Poisson ratio of This problem is taken from [Chen94]. The mathematical modeling of the Mindlin plate problems and the formulation of the algebraic equations using thefinitestrip method can be found in [Mind51, BeHi76] or in [Chen94]. We discretize the plate into 64, 128,..., 2048 strips. Eight harmonic (Fourier) terms are employed to approximate the true solution (displacementfield)for each discretization. 3.2 Implementations and Experiments Our experiments consist of three different implementations: one for sequential execution and two for parallel execution. The two implementations for parallel execution are the standard MS model and our new MGT MMS model, as shown in Figures 5 and 6, respectively. We use the software package PVM3 (Parallel Virtual Machine, Version 3.3.9) [Geis93] to implement interprocess communications. In the standard MS model, we only

8 446 Boundary Element Technology Figure 5: MS model with eight processes (temporal representation) Figure 6: MGT MMS model with eight processes (temporal representation) need two computer programs: one for the master and the other for all the slaves. In the MGT model, we employ four computer programs, which differ mainly in the way the interprocess communication is handled. All processes generated at the same time step share the same code. This, of course, is not a requirement. There are a great number of varieties. For example, one may develop a separate program for a process, or combine the two programs for the intermediate processes into a single program as an alternative. Extensive code sharing among processes, however, usually adds a certain degree of difficulty to the development of the code and may not always be preferable. Our preliminary experimental results are shown in Tables 1, where the CPU time includes both the user and system CPU time in seconds spent in the entire analysis, including the generation of the strip stiffness matrices, assemblage of the linear subsystems, solution to each subsystem, and the calculation of displacements. It should be noted that each timing result represents the best timing observed in a series of six consecutive executions of the same code on the same data. The speedups, defined to be the ratio of the time spent in the sequential code run on one workstation to that spent in the parallel code executed on the networked system configured with eight workstations, for each individual discretization are shown in Table 2. As seen from these two tables, both the MS and MGT (MMS) models yield pretty good performance, as compared with the sequential execution. It is also clear that the MGT model improves the performance obtained by the standard MS scheme, as expected. The improvement is more than 13% in all cases. This is mainly due to the better exploitation of parallelism by the

9 Boundary Element Technology 447 Table 1: Performance of MS and MMS models (CPU time in seconds) No. of strips Sequential CPU time in seconds Traditional MS MGT (MMS) Table 2: Performance of MS and MMS models (individual speedup) No. of strips Sequential Individual speedup Traditional MS MGT (MMS) MGT than by the MS model. 4 Conclusions The master-slave parallel programming model has been widely used in a networked computing environment. This model, however, suffers from the disadvantages that the master alone is responsible for spawning all the slave processes and the heavy communication overhead and delays when all slaves attempt to simultaneously communicate with the single master. In this paper we have presented a more efficient approach for the multilevel masterslave (MMS) model that implements the MS model at multiple levels. The MMS model is useful for large-scale master-slave problems, because it not only allows for parallel creation of slave processes but has the ability to improve performance in flooding global data to and merging results from slave processes. To demonstrate the effectiveness of our proposed approach, we have presented an application of the MGT model to the finite strip method for

10 448 Boundary Element Technology solving a particular class of plate-bending problems that arise in the area of structural analysis. The results of our experiments show that the MGT model improves the performance of parallel computation. 5 Acknowledgment This work was supported in part by the Army Research Laboratory under Grant No. DAAL D065 and in part by the National Science Foundation under grant CCR References [BeHi76] P.R. Benson and E. Hinton, A thick finite strip solution for static, free vibration and stability problems, Int. J. for Numer. Meth. in Eng., 10 (1976), pp [Chen94] H.-C. Chen, Increasing parallelism in the finite strip formulation: static analysis, International Journal on Neural, Parallel & Scientific Computations, Vol. 2, No. 3 (1994), pp [ChBy95] H.-C. Chen and V. Byreddy, Solving plate bending problems using finite strips on network workstations, submitted to Computers & Structures. [Cheu68] Y.K. Cheung, The finite strip method in the analysis of elastic plates with two opposite simply supported ends, Proc. Inst. Civ. Eng., 40(1968), pp [EGSM94] G. Eisenhauer, W. Gu, K. Schwan, and N. Mallavarupu, Falcon - Toward Interactive Parallel Programs: The On-line Steering of a Molecular Dynamics Application, Proceedings of High-Performance Distributed Computing, August [Geis93] A. Geist, et al, PVM3 User's Guide and Reference Manual, ORNL/TM-12187, Oak Ridge National Laboratory. [GeSS87] E.F. Gehringer, D.P. Siewiorek, Z. Segall, Parallel Processing: The CM Experience, Digital Press, MA, [Leig92] F.T. Leighton, Introduction to Parallel Algorithms and Architectures: Arrays. Trees.Hypercubes, Morgan Kaufmann, CA, [Mind51] R.D. Mindlin, Influence of rotatory inertia and shear on flexural motions of isotropic, elastic plates, J. of Applied Mechanics, 18 (1951), pp [PuSc90] J. A. Puckett and R. J. Schmidt, Finite strip method for groundwater modeling in a parallel computing environment, Eng. Comput., 7 (1990), pp [Stra86] G. Strang, Introduction to Applied Mathematics, Wellesley-Cambridge, MA, [Tuck84] A. Tucher, Applied Combinatorics (2nd ed.), John Wiley & Sons, NY, 1984.

Second-order shape optimization of a steel bridge

Computer Aided Optimum Design of Structures 67 Second-order shape optimization of a steel bridge A.F.M. Azevedo, A. Adao da Fonseca Faculty of Engineering, University of Porto, Porto, Portugal Email: alvaro@fe.up.pt,