HEURISTIC METHOD OF DYNAMIC STRESS ANALYSIS IN MULTIBODY SIMULATION USING HPC

Size: px

Start display at page:

Download "HEURISTIC METHOD OF DYNAMIC STRESS ANALYSIS IN MULTIBODY SIMULATION USING HPC"

Margaret Booth
5 years ago
Views:

1 11th World Congress on Computatonal Mechancs (WCCM XI) 5th European Conference on Computatonal Mechancs (ECCM V) 6th European Conference on Computatonal Flud Dynamcs (ECFD VI) E. Oñate, J. Olver and A. Huerta (Eds) HEURISTIC METHOD OF DYNAMIC STRESS ANALYSIS IN MULTIBODY SIMULATION USING HPC V.V. GETMANSKIY*, A.S. GOROBTSOV*, T.D. ISMAILOV* AND A.E. ANDREEV* * Volgograd State Techncal Unversty, Faculty Electroncs and Computer Scence, Volgograd, Lenn avenue, 28, 45, Russa, e-mal: vm@vstu.ru Key Words: Dynamc stress analyss, Multbody smulaton, Ambguous constrants, Dscrete elements, Doman decomposton, Parallel computng. Abstract. A parallel method of stress-stran smulaton of sngle body n multbody dynamc model s proposed n ths paper. Method s based on dscrete elements approxmaton of deformable body and doman decomposton. Multthreaded mplementaton usng POSIX threads lbrary wth custom synchronzaton prmtves gves a relatvely good speedup of parallel computng on NUMA archtecture n comparson wth OpenMP mplementaton. Smulaton of 13M elements mesh s performed on up to 32 cores. The results and effcency estmaton of concurrent smulaton are presented. 1 INTRODUCTION Actual task n vehcle desgn s a stress-stran analyss of vehcle parts wth maxmal dynamc loads. Tradtonal stress estmaton approaches are based on FEM methods whch compute only quas-statc stress dstrbuton and gve poor results n nonlnear dynamc cases. For example a lower arm model n truck suspenson system s consdered. It s connected to vehcle frame and wheel hub. Reactons from jonts to these bodes are used for determnng the stress dstrbuton. The reactons are obtaned from multbody smulaton of full vehcle. Weak couplng of multbody solver and stress solver s used n proposed method [1]. It uses separate solvers wth transferrng data durng smulaton and assgnng correspondng data to coupled parameters [2, 3]. Coupled parameters are reacton forces whch are transferred from multbody model to stress submodel at each tme step. Multbody solver (MBS) s based on augmented Lagrangan formulaton wth natural coordnates [4, 5]. Parallelzaton of stress solver usng OpenMP s lmted by shared memory systems and loses n performance n case of NUMA archtecture. Iteratve procedure allows loops parallelng whch go along wth threads reallocaton on each teraton. Overheads caused by threads reallocaton are analysed for example n work [6]. Small amount of tme for parallel blocks n comparson wth total smulaton tme does not allow usng OpenMP effcently. As a soluton to the problem stated above t was developed a parallel stress-stran smulaton method based on doman decomposton and custom threads synchronzaton routnes.

2 2 DISCRETE ELEMENTS METHOD FOR DEFORMABLE BODY REPRESENTATION A lower arm n vehcle suspenson system has large dynamc loads when the vehcle runs on random road profle. Some consderaton for lower arm desgn usng FEM smulaton s presented n work [7]. However quas-statc approach lacks n case of dynamc loads. In proposed method deformable body s approxmated by rgd dscrete elements. Computatonal doman s regular orthogonal mesh based on CAD geometry of the part. Each mesh node s a rgd dscrete element wth 6 degrees of freedoms. Each element s connected to adjacent elements by flexble jont. Regular dscrete elements structure greatly smplfes the constructon of dfferental equaton system n natural coordnates. The nerta matrx s dagonal and all elements are descrbed by smlar ordnary dfferental equatons to wthn number of lnks to adjacent elements. Flexble body has a rgd counterpart n multbody model whch s called reference body. The system of equatons for flexble body has a followng form n coordnate system of reference body: y, y, Mat sy y M y = q t,, (1) where y s a vector of dscrete elements coordnates, M s a matrx of nerta for set of dscrete elements, a s a vector constructed of reference body acceleraton vector components, s y, y s a stablzng component. Reference body acceleraton s obtaned from multbody model. The functon qy,y,t descrbes the forces between rgd elements. Stress values are computed based on the values of the forces on each teraton. Random perturbng effect makes flexble body model unstable. Addtonal stablzng component s y, y s ntroduced n Equaton (1) to constran the poston of dscrete elements. Rght-hand sdes of the equaton are completed wth nerta forces takng nto account moton of the reference body n multbody model. Fgure 1: Flexble body to rgd body connecton Flexble body s connected to multbody model wth jonts. To complete the equaton t s necessary to add reacton forces n flexble jonts to subset of Equatons (1) corresponded to 2

3 sold elements on contact surfaces. There are two knds of jonts shown n Fgure 1: sprngdamper force f 1 = f and stff jont wth reacton force f2 r x, ry, rz. The equaton of moton for boundary dscrete element has a followng form n case of unform dstrbuton of reacton force along boundary: M y = f( t) F q ( y, y, t) M a ( t) s ( y, y), F (2) where F s a set of node ndces of boundary wth appled reacton f. So the reference body data obtaned from multbody solver s enough to calculate forces of nerta and reactons n jonts. Equaton (1) s solved usng Runge-Kutta 4-th order method (RK4). The data of reference body s transferred to stress solver on sync tme steps. Numercal ntegraton tme steps of multbody solver ( tman ) and stress solver ( tsub ) are dfferent n general case. For smplfcaton t s posed that tme steps are dvsble. If tsub tman, then tman N tsub, where N s an nteger multpler. There s general tme counter wth mnmal tme step t mn t sub, tman and couplng tme counter wth maxmal tme step t max t t. Solver algorthm s presented below. sub, man The followng seral algorthm s mplemented and used for smulaton: t, t, t mn t, t t max t, t sub t,, man sub man Whle t T If t tman Then MBS teraton solvng for current tme t tman tman tman If t t Then Rght-hand sde ate for stress-stran solver t t t If t tsub Then Stress-stran solver teraton for tme t sub tsub tsub tsub Wrtng smulaton results t t t Each teraton of stress solver gves computed postons of dscrete elements. They are used to obtan dsplacements. Stresses n deformable body n each dscrete element can be calculated usng formula sub man 3

4 , (3) where j are stress tensor components, 2 E 1, E 2 1 j are stran tensor components, 2,, E s an elastcty modulus. Equvalent tensle stresses are calculated usng von Mses crteron: jj 6 j.5. (4) 1 j1 3 PARALLEL STRESS-STRAIN SOLVER 3.1 Parallel algorthm Overlappng doman decomposton s used for parallel algorthm. The postons of nner nodes of each subdoman are solved correctly n sngle teratons. The postons of outer nodes whch overlap are solved ncorrectly but they have correspondng nner nodes n adjacent subdoman. It s requred to exchange data from nner elements of adjacent subdomans to outer elements to correct the postons. Each subdoman terates ndependently except data exchange steps. The equaton order reducton s used whle ntegratng Equaton (1). In ths case the equaton s wrtten n smplfed form as where x s a new vector of unknowns, T x q ~ t, x, (5) x y y, q ~ s a new rght-hand sde functon. For ntegraton tme step of h RK4 formulas for ODE system are where k hq ~ 1 t, x k hq ~ t h 2, x k ~ 3 hqt h 2, x k ~ 2 k h q t h x k. 2 k 1 4, 3 x 1 x 1 6 k1 2k 2 2k 3 k 4, (6) 2 2 4

5 Calculaton of each k requres computng rght-hand sdes of Equaton (5). It s necessary to exchange data after each rght-hand sde calculaton to take nto account adjacent subdomans. So t s necessary 4 barrer synchronzatons after each of k computng. Parallel algorthm mples multbody solver and each subdoman solvers n separate threads. It s stated below. Parallel algorthm t, t, sub t, t mn t t, t max t t man sub, man MBS thread Whle t T If t tman Then MBS teraton solvng for current tme t tman tman tman If t t Then Sendng reactons to submodels Barrer Wrtng smulaton results t t t Submodel thread Whle t T If t t Then Barrer Rght-hand sde ate for stress-stran solver t t t If t tsub Then Stress-stran solver teraton for tme t sub For 1 To 4 Do k calculatng Data exchange Barrer y t calculatng tsub tsub tsub Wrtng smulaton results t t t 3.2 Synchronzaton sub, Data exchange s organzed usng sync table. Sync has a structure of adjacency matrx for subdomans and contans node number from whch data s send and to whch t s receved. man 5

6 The rows of sync table for sendng data contan ndces of nodes from whch data s send from subdoman wth number correspond to row number. The rows of sync table for recevng data contan ndces of nodes to whch data s receved to subdoman wth number correspond to row number. At exchange step data s packed to buffer before send usng addresses from -th row of sync table for -th parallel process and unpacked to addresses n subdoman solver memory after receve. 3.3 Doman decomposton Important characterstc of developed method s load balancng and mnmzaton of data exchange. Both of these tasks are solved by doman decomposton methods. Two methods were researched: recursve coordnate bsecton and multlevel methods for graph parttonng descrbed n work [8]. Frst geometrcal method s suted for convex geometry. Orthogonalty of the mesh gves a plane sectons wth mnmal number of cut lnks n case of same amount of nodes n cut boundary. But ths method does not gve mnmal number of total cut lnks. Method does not guarantee the connectvty of subdoman n concave cases. Some modfcatons for connectvty also don t gve a soluton n general case. Decomposton to overlappng subdomans makes the mathematcal formulaton more smple and easy to mplement for parallel computng. The same approach s used n parallel mplementaton of FEM smulaton [9]. Dscrete elements approach also can be used n fatgue analyss whch s usually accompaned wth doman decomposton [1]. Second method of multlevel graph parttonng solves NP task of edge cut mnmzaton wth relatve amount of nodes n each part restrctons. Shortly algorthm conssts of 3 steps: Source graph s compressed by combnng vertces untl several hundred vertces remans. Random matchng s used or heavy edge matchng n case of weghted edges s used. Compressed graph s decomposed by some decomposton methods, for example recursve bsecton. Compressed graph s unpacked to graph wth ntal amount of nodes wth conservng of decomposton. Kernghan-Ln heurstc s used usually for refnng the parts durng unpack [11]. Fgure 2: Suspenson lower arm model before (a) and after (b) decomposton 6

7 So a multlevel algorthm whch s mplemented n METIS lbrary s used for decomposton. Decomposton results for lower arm model of 13M elements are shown n Fgure 2. Imbalance and redundancy factors are used for qualty estmaton of decomposton. Redundancy R n and mbalance D n for decomposton of model of N nodes to n submodels are calculated as follows: R n n 1 N N, D n N N P max, (7) N where P requred relatve amount of nodes n submodel,. Redundancy and mbalance n worst case of 32 subdomans are 32 3% 4 MULTITHREADED IMPLEMENTATION N number of nodes n submodel R, D 4 32 %. For parallelzaton t was used pthreads lbrary. Parallel mplementaton of the program uses 2 custom synchronzaton prmtves: barrer for specfed number of threads; sgnal wthout watng. Several multthreaded synchronzaton strateges can be found n work [12]. 4.1 Barrer Ths synchronzaton prmtve ensures that the code whch s stuated n the program after the method Wat wll not be executed by the current thread untl the number of watng threads n ths method does not reach N. Such mplementaton allows the reuse of the barrer requred for the correct work of the cyclc part of the program. The barrer prevents premature executon of program code located after a synchronzaton pont provdng equal performance of multple threads under unbalanced loadng. Wat method algorthm Lock mutex Increase counter n : n n 1 If n N Then Wat for condton sgnalzng and unlock mutex Else Reset counter n : n Sgnalze condton to watng threads Unlock mutex 4.2 Sgnal wthout watng Ths synchronzaton prmtve extends the functonalty of a conventonal varable (condton) for use n the cyclc part of the program. Sgnal wthout watng ensures that the thread that calls method Wat wll wat untl the number of calls of ths method by that thread 7

8 does not match the number of calls of method Sgnal by the sgnallng thread. Sgnal wthout watng prevents the executon of the program code located after the synchronzaton pont by the watng threads untl recevng a sgnal from the sgnallng thread. Wat method algorthm Lock mutex Get dentfer of the current thread If the map counters does not contan key Add par, to the map counters Fnd the counter n w n the map counters by the key Increase the counter n w : n n 1 w w If n w 1 Increase the counter n : n n 1 If nw ns If the flag f s not set: f false Set the flag f : f true Wat for condton sgnalzng and unlock mutex Else If n N Reset the counter n : n Decrease the counter n s : n s n s 1 Decrease all counters n the map counters Unlock mutex Sgnal method algorthm Lock mutex Increase the counter n s : n s n s 1 If the flag f s not set: f true Reset the flag f : f false Sgnalze condton to watng threads Else If n N Reset the counter n : n Decrease the counter n s : n s n s 1 Decrease all counters n the map counters Unlock mutex 8

5 RESULTS Meshes wth dfferent number of nodes are used for smulaton. It s necessary to use hgh detaled meshes of several mllon elements for detaled geometry descrpton by orthogonal mesh.

9 5 RESULTS Meshes wth dfferent number of nodes are used for smulaton. It s necessary to use hgh detaled meshes of several mllon elements for detaled geometry descrpton by orthogonal mesh. Smulaton was performed on Core-7 92 platform and on 2-socket Opteron x16 cores platform. The frst platform has turbo-boost and hyper-threadng capabltes. The second platform has NUMA archtecture. Fgure 3: Speedup of multthreaded smulaton on platform wth sngle 4-core CPU Intel Core 2,67GHz (a) and two 16-core CPUs AMD Opteron 2.1GHz (b) Speedup results are presented n Fgure 3. Effcency for 2 4 cores s almost perfect on NUMA platform. It s decreases to 75% on 8 cores and to 25 3% on 32 cores. Effcency decreases faster to 8% on 4 cores of Core-7 CPU, and to 44% usng 8 threads (wth hyperthreadng). Fgure 4: Suspenson lower arm stress-stran smulaton results The obtaned speedup s well correlated wth the research on custom OpenMP mplementaton for NUMA. 9

10 6 CONCLUSION The proposed method gves qute good speedup on shared memory platform wth NUMA. Effcency decreases wth ncreasng of number of threads but the speedup of 1 s acheved on 32 cores. A custom thread handlng sgnfcantly reduces overheads on NUMA-archtecture n comparson wth OpenMP. Parallel stress-stran smulaton method s also applcable to dstrbuted memory system such as clusters and can be mplemented usng MPI. REFERENCES [1] V.V. Getmansky, A.S. Gorobtsov, E.S. Sergeev, T.D. Ismalov, O.V. Shapovalov, Concurrent smulaton of multbody systems coupled wth stress-stran and heat transfer solvers, Journal of Computatonal Scence, 3(6), , 212. [2] M. Arnold, Numercal methods for smulaton n appled dynamcs, M. Arnold, W. Schehlen, (Edtors), Smulaton Technques for Appled Dynamcs, , Sprnger Venna, 28. [3] R. Kübler, W. Schehlen, Two Methods of Smulator Couplng, Mathematcal and Computer Modellng of Dynamcal Systems, 6(2), , 2. [4] E. Bayo, J. Garca de Jalon, M.A. Serna, A modfed Lagrangan formulaton for the dynamc analyss of constraned mechancal systems, Comput. Methods Appl. Mech. Eng., 71(2), , [5] M. Valasek, Z. Ska, O. Vaculn, Multbody formalsm for real-tme applcaton usng natural coordnates and modfed state space, Multbody System Dynamcs, 17, , 27. [6] P. Carrbault, M. Pérache, H. Jourdren, Enablng low-overhead hybrd MPI/OpenMP parallelsm wth MPC, Proc. of the 6th nternatonal conference on Beyond Loop Level Parallelsm n OpenMP accelerators, Taskng and more, 1-14, Sprnger-Verlag, Berln, Hedelberg, 21. [7] M.M. Rahman, M.M. Noor, K. Kadrgama, M.A. Maleque, R.A. Bakar, Modelng, analyss and fatgue lfe predcton of lower suspenson arm, Advanced Materals Research, 264, , 211. [8] G. Karyps, V. Kumar, Multlevel algorthms for mult-constrant graph parttonng, Proc. of the 1998 ACM/IEEE conference on Supercomputng IEEE Computer Socety, 1-13, Washngton, DC, USA, [9] J. Zhang, L. Zhang, H. Jang, An Implementaton Method of Parallel Fnte Element Computaton Based on Overlappng Doman Decomposton, Proceedngs of the Second nternatonal conference on Hgh Performance Computng and Applcatons, , 21. [1] L.C. L, C.A. Tang, G. L, S.Y. Wang, Z.Z. Lang, Y.B. Zhang, Numercal Smulaton of 3D Hydraulc Fracturng Based on an Improved Flow-Stress-Damage Model and a Parallel FEM Technque, Rock Mechancs and Rock Engneerng, 45(5), , 212. [11] B.W. Kernghan, S. Ln, An Effcent Heurstc Procedure for Parttonng Graphs, Bell Sys. Tech. J., 49(2), , 197. [12] A.B. Downey, The lttle book of semaphores 2 nd edton, CreateSpace, 29. 1

Parallelism for Nested Loops with Non-uniform and Flow Dependences

Parallelism for Nested Loops with Non-uniform and Flow Dependences Parallelsm for Nested Loops wth Non-unform and Flow Dependences Sam-Jn Jeong Dept. of Informaton & Communcaton Engneerng, Cheonan Unversty, 5, Anseo-dong, Cheonan, Chungnam, 330-80, Korea. seong@cheonan.ac.kr