Efficient Parallel Multi-Copy Simulation of the Hybrid Atomistic-Continuum Simulation Based on Geometric Coupling

Size: px

Start display at page:

Download "Efficient Parallel Multi-Copy Simulation of the Hybrid Atomistic-Continuum Simulation Based on Geometric Coupling"

Naomi Sherman
5 years ago
Views:

1 , pp Efficient Parallel Multi-Copy Simulation of the Hybrid Atomistic-Continuum Simulation Based on Geometric Coupling Qian Wang 1,2, Xin-Hai Xu 1,2,*, Chao Li 1,2, Juan Chen 1,2, 1 College of Computer, National University of Defense Technology, Changsha, China 2 State Key Laboratory of High Performance Computing, National University of Defense Technology, Changsha, China wangqiannudt@gmail.com, {xuxinhai, dirk911, juanchen}@nudt.edu.cn Abstract. The hybrid atomistic-continuum (HAC) simulation based on geometric coupling should properly deal with the sampling technique and noise in the atomistic simulation. Based on the need of ensemble simulation, we design and implement the associated parallel supporting mechanism of our hybrid atomistic-continuum framework based on the open source code OpenFOAM and LAMMPS, propose the Multi-Copy advancing model and decision-making mechanism for the Multi-Copy simulation. The results of the benchmark cases indicate that our HAC framework can functionality support the Multi-Copy simulation. Our decision-making mechanism can sufficiently utilize the computational resources and perform efficient coupling simulation. Keywords: multi copies, decision-making mechanism, efficient parallel simulation 1 Introduction Recently much attention has been focused on the hybrid atomistic-continuum (HAC) simulation based on the geometric coupling which is an approach capable of accurately describing the flow characteristics at micro- and nano- scales[1-3]. The HAC method based on geometric coupling splits the simulation domain into the continuum region, the atomistic region and the overlap region, uses certain coupling procedures to exchange data between two simulation methods[4]. In molecular simulation (MD) methods, the connection to macroscopic observable fields, such as velocity, density and temperature, is achieved through averaging appropriate microscopic properties[5]. This is the kernel procedure when coupling the particle fields to continuum that requires both averaging of finite numbers of particles in a sequence of molecular realizations as well as the generation of molecular realizations with relatively small numbers of particles. In the previous research, researchers usually run the realizations of the simulation cases several times to reduce * Corresponding author: State Key Laboratory of High Performance Computing, National University of Defense Technology, Changsha, Hunan, China xuxinhai@nudt.edu.cn ISSN: ASTL Copyright 2016 SERSC

2 the noise and smooth the simulation data[4]. The existing hybrid coupling framework are hardly supported no multi copy synchronize simulation of the MD part. In order to realize multiple realizations of the particle simulation, the framework must provide the supporting mechanism. Furthermore, the framework should have the ability to highly efficient parallel simulation of the multi copy mode. The framework should have the decision-making mechanism to fully use the computational resources. In this paper, based on our previous research[6], we further modify our parallel supporting mechanism of the HAC framework based on the open source code OpenFOAM[7] and LAMMPS[8], propose the Multi-Copy advancing model and fully utilize the computational resources. The main contributions of this paper are summarized as follows: 1. We propose the Multi-Copy advancing model and decision-making mechanism for our hybrid atomistic-continuum coupling framework. 2. We implement the supporting parallel mechanism in our hybrid atomisticcontinuum coupling framework. 3. We verify the efficiency of the proposed Multi-Copy advancing model through benchmark experiments on our hybrid framework. The results indicate that our decision-making mechanism can sufficiently utilize the computational resources and perform highly efficient coupling simulation. 2 Background Our hybrid atomistic-continuum coupling framework is designed based on the OpenFOAM[7] and LAMMPS[8]. The coupling method of the framework is geometric coupling. For the two-dimensional domain, the computational domain was split into an atomistic region and a continuum region, using an overlap region to alleviate dramatic density oscillation and couple the results of these two regions. The schematic diagram of the domain decomposition based HAC method is depicted in Fig. 1. The detailed coupling procedures can referenced in [6]. z y x continuum region overlap region y=l y=b y=a atomistic region y=0 Fig.1. Decomposition of the simulation domain, a 2D diagram. The decomposition of the overlap region is emerged in the above two methods. In order to match the cell in the continuum region, the overlap region in the MD simulation is divided into bins. Due to we will deal with multi copies of the atomistic region and the overlap region, there usually exists no pure CFD process in our simulation. The main coupling procedures in the HAC method is to exchange data between CFD and MD. On the MD part, we need to sample and average particles properties to transform Copyright 2016 SERSC 385

3 the CFD part. For steady flow, these may be sequential samples taken in time; for transient flows these are samples from an ensemble of realizations[9]. Because of the smallest module of CFD simulation is mesh cell, we choose the sample bins the same volume as the cell. 3 Efficient parallel Multi-Copy advancing model Under a given configuration of the MD simulation domain, the maximum degree of parallelism is should be a given number for the volume of sub domain cannot smaller than a certain value. For a proper volume of the sub domain, the side length of it should larger than the cutoff distance, better twice the cutoff distance. Therefore, we define the following maximum degree of parallelism for a given scale domain, max p = lx ú (1) 2r ú ë c û ly ú 2r ú ë c û lz ú 2r ú ë c û We use the parameter, num_couple, to define the number of coupled particle region. We also define the parameter, m, which is the number of copies of the particle region. Finally, we use the parameter, n, to define different execution mode for our advancing model, i.e, multiple partition. The diagram of the advancing model is depicted in Fig. 2. The total degree of parallelism is P processors. Table 1. Parameters of Multi-Copy advancing model Parameter Number of coupled regions Number of copies Number of partition Degree of parallelism Symbol num_couple m n P CFD MD1 MDi MDm MD1 MD1 MDm/n CFD MDi CFD MDi MD(i+m/n-1) MDm MD(m-m/n+1) MDm Fig. 2. Multi-Copy advancing model. We propose three kinds of advancing model for our HAC framework. The detailed illustration is listed as follows: Sequential type: each copy of the particle realization is executed one by one, which each copy will use the total processors P. 386 Copyright 2016 SERSC

4 Concurrent type: all the copies are executed simultaneously; each copy will use P/m processors. Half-concurrent type: all the m copies are split into n partitions. In each partition, the m/n copies are executed in sequential manner. Each copy uses P/n processors. We need a decision-making mechanism to decide which type of advancing model to be used to fully utilize the computational resources and efficiently simulate. Using the given configuration of the simulation domain, we can figure out how to advance the multi copies simulation. Algorithm 1 The algorithm for decision-making mechanism Input Simulation domain (lx,ly,lz), computational resources P, copy number m, num_couple Output Partition number n 1: Calculate maxp using (lx,ly,lz) 2: Calculate concurrent_n = P/ num_couple / maxp 3: if concurrent_n < m then 4: Partition n = ceil(m/concurrent_n) 5: else if concurrent_n > m then 6: Partition n = m 7: else if concurrent_n = 1 then 8: Partition n = 1 9: end if Due to the need of ensemble simulation, we add parallelism mechanism for the Multi- Copy execution. The parallelism mechanism of the HAC framework need to additional the master processes and slave processes. The master processes are those processes in copy 0. Other processes are the slave processes. The master processes are in charge of exchanging data with the continuum region, mostly they are on the same process. 4 Experiments and results In this section, we use the benchmark case, i.e., the channel flow past the rough wall to verify our decision-making mechanism and the Multi-Copy advancing model. The test case is the channel flow past the rough wall as shown in Fig. 3. The simulation domain is x y = σ 2, while the z direction of the MD domain is 10σ. The height of the atomistic region is 15σ and the height of the overlap region is 12σ. The moving and still wall are no slip boundary condition. The length of cell and bin are the same with 2σ. The rough bottom wall consists two layers of FCC (111) solid particles and the height of it is about 5σ. The cutoff radius is chosen 2.2σ for time sake. The velocity of the top wall is U w = 3.0 σ τ. The time step of MD is 0.005τ. The total number of particle is Copyright 2016 SERSC 387

5 Fig.3. The simulation domain of the benchmark case. We want to run an 8 copies coupling simulation. The simulation platform is the same as in [10, 11]. In order to test the efficiency of our model, we use 96 degree of parallelism. Under above configuration of the simulation domain, the max p is 48 calculated by the Equation (1). The concurrent_n is 2 for this configuration. Under the instruction of Algorithm 1, we should use the half-concurrent advancing type. We also take the sequential type and concurrent type as comparison to show the efficiency. The input parameters are listed in Table 2. The simulation results of the communication time occupied the total simulation time and t the total simulation time are shown in Fig. 4 and Fig. 5. The total simulation time is used the sequential type as the base and depicted in relative value. Table 2. Input parameters of different advancing model No. Number of partition Processor of copy Advancing type sequential half-concurrent half-concurrent concurrent Fig. 4. The ratio of the communication time to the total simulation time. Fig. 5. The relative total simulation time for different advancing model From Fig. 4 and Fig. 5, we can see that half-concurrent type performs better than the sequential type. The decision-making mechanism can calculate the proper advancing model for the Multi-Copy simulation. When the computational domain increase, the communication overhead increases following the scale. From the above results, we can 388 Copyright 2016 SERSC

6 figure out that our decision-making mechanism can choose the proper advancing type for Multi-Copy simulation and fully utilized the computational resources. If the computational resources are sufficient, we can directly use the concurrent type, while user can use the decision-making mechanism to perform efficient coupling simulation. 5 Conclusion In this paper, we propose the Multi-Copy advancing model and decision-making mechanism, implement the supporting parallel mechanism in our HAC framework. We verify the efficiency of the proposed Multi-Copy advancing model through benchmark experiments on our hybrid framework. The results indicate that our decision-making mechanism can sufficiently utilize the computational resources and perform highly efficient coupling simulation. References 1. O'Connell, S.T., Thompson, P.A.: Molecular dynamics-continuum hybrid computations: A tool for studying complex fluid flows. Phys Rev E Stat Phys Plasmas Fluids Relat Interdiscip Topics 52, R5792-R5795 (1995) 2. Squires, T.M., Quake, S.R.: Microfluidics: Fluid physics at the nanoliter scale. Rev Mod Phys 77, (2005) 3. Flekkøy, E., Wagner, G., Feder, J.: Hybrid model for combined particle and continuum dynamics. EPL (Europhysics Letters) 52, 271 (2000) 4. Nie, X.B., Chen, S.Y., E, W.N., Robbins, M.O.: A continuum and molecular dynamics hybrid method for micro- and nano-fluid flow. Journal of Fluid Mechanics 500, (2004) 5. Landau, L.D., Lifshitz, E.: Statistical physics, part I. Course of theoretical physics 5, 468 (1980) 6. Wang, Q., Xu, X.-H., Li, C., Ji, H.-Y., Ren, X.-G., Yang, X.-J.: Coupling Strategies Investigation of Hybrid Atomistic-Continuum Method Based on State Variable Coupling. (2016) Plimpton, S.: Fast Parallel Algorithms for Short-Range Molecular Dynamics. Journal of Computational Physics 117, 1-19 (1995) 9. Hadjiconstantinou, N.G., Garcia, A.L., Bazant, M.Z., He, G.: Statistical error in particle simulations of hydrodynamic phenomena. Journal of Computational Physics 187, (2003) 10. Yang, W.-J., Yi, W., Ren, X.-G., Xu, L.-Y., Xu, X.-H., Yuan, X.-F.: Toward large scale parallel computer simulation of viscoelastic fluid flow: A study of benchmark flow problems. Journal of Non-Newtonian Fluid Mechanics 222, (2015) 11. Xu, X.-H., Guo, X.-W., Cao, Y., Ren, X.-G., Chen, J., Yang, X.-J.: Multi-scale simulation of non-equilibrium phase transitions under shear flow in dilute polymer solutions. RSC Advances 5, (2015) Copyright 2016 SERSC 389

Meshing of flow and heat transfer problems

Meshing of flow and heat transfer problems Luyao Zou a, Zhe Li b, Qiqi Fu c and Lujie Sun d School of, Shandong University of science and technology, Shandong 266590, China. a zouluyaoxf@163.com, b 1214164853@qq.com,