repex Documentation Release 0.2 Antons Treikalis

Transcription

1 repex Documentation Release 0.2 Antons Treikalis October 13, 2015

2

3 Contents 1 Introduction What is RepEx? What can I do with it? Why should I use it? Installation 5 3 Getting Started Invoking RepEx T-REMD example (peptide ala10) with Amber kernel One-dimensional REMD simulations US-REMD example using Alanine Dipeptide system with Amber kernel Multi-dimensional REMD simulations TUU-REMD example (alanine dipeptide) with Amber kernel Replica Exchange Patterns Synchronous Replica Exchange Pattern Asynchronous Replica Exchange Pattern Flexible execution modes Execution Strategy S Execution Strategy S Execution Strategy S Tutorial Running on Stampede Running on Archer T-REMD example (peptide ala10) with Amber kernel US-REMD example using Alanine Dipeptide system with Amber kernel TUU-REMD example (alanine dipeptide) with Amber kernel Frequently Asked Questions Where are.mdout files? Where are.mdinfo files? How can I obtain information about accepted exchanges? How can I obtain information about attempted exchanges? i

4 10 Indices and tables 33 ii

5 Contents: Contents 1

6 2 Contents

7 CHAPTER 1 Introduction 1.1 What is RepEx? RepEx is a new Replica-Exchange Molecular Dynamics (REMD) simulations package written in Python programming language. RepEx supports Amber [1] and NAMD [2] as Molecular Dynamics application kernels and can be easily modified to support any conventional MD package. The main motivation behind RepEx is to enable efficient and scalable multidimensional REMD simulations on HPC systems, while separating execution details from simulation setup, specific to a given MD package. RepEx provides several Execution Patterns designed to meet the needs of it s users. RepEx relies on a concept of Pilot-Job to run RE simulations on HPC clusters. Namely, RepEx is using Radical Pilot Pilot System for execution of it s workloads. RepEx effectively takes advantage of a task-level-parallelism concept to run REMD simulations. RepEx is modular, object-oriented code, which is designed to facilitate development of extension modules by it s users. [1] - [2] What can I do with it? Currently are supported the following one-dimentional REMD simulations: Temperature-Exchange (T-REMD), Umbrella Sampling (US-REMD) and Salt Concentration (S-REMD). It is possible to combine supported one-dimensional cases into multi-dimentional cases with arbitrary ordering and number of dimensions. This level of flexibility is not attainable by conventional MD software packages. RepEx easily can be used as a testing platform for new or unexplored REMD algorithms. Due to relative simplicity of the code, development time is significantly reduced, enabling scientists to focus on their experiments and not on a software engineering task at hand. 1.3 Why should I use it? While many MD software packages provide implementations of REMD algorithms, a number of implementation challenges exist. Despite the fact that REMD algorithms are very well suited for parallelization, implementing dynamic pairwise communication between replicas is non-trivial. This results in REMD implementations being limited in terms of number of parameters being exchanged and being rigid in terms of synchronization mechanisms. The above challenges together with the limitations arising from design specifics contribute to scalability barriers in some MD software packages. For many scientific problems, simulations with number of replicas at the order of thousands would substantially improve sampling quality. Main distinguishing features of RepEx are: 3

8 low barrier for implementation of new REMD algorithms facilitated by separation of simulaiton execuiton details from implementation specific to current MD package functionality to run multi-dimentional REMD simulations with arbitrary ordering of dimensions 4 Chapter 1. Introduction

9 CHAPTER 2 Installation This page describes the requirements and procedure to be followed to install the RepEx package. Note: Pre-requisites.The following are the minimal requirements to install the RepEx package. python >= 2.7 virtualenv >= 1.11 pip >= 1.5 Password-less ssh login to target cluster The easiest way to install RepEx is to create virtualenv. This way, RepEx and its dependencies can easily be installed in user-space without clashing with potentially incompatible system-wide packages. Tip: If the virtualenv command is not available, try the following set of commands: wget --no-check-certificate tar xzf virtualenv-1.11.tar.gz python virtualenv-1.11/virtualenv.py --system-site-packages $HOME/repex-env/ source $HOME/repex-env/bin/activate Step 1 : Create and activate virtualenv: virtualenv $HOME/repex-env/ source $HOME/repex-env/bin/activate Step 2 : Install RepEx: git clone cd radical.repex python setup.py install Now you should be able to print the installed version of RepEx: repex-version Installation is complete! 5

10 6 Chapter 2. Installation

11 CHAPTER 3 Getting Started In this section we will briefly describe how RepEx can be invoked, how input and resource configuration files should be used. We will also introduce two concepts, central to RepEx - Replica Exchange Patterns and Execution Strategies. 3.1 Invoking RepEx To run RepEx users need to use a command line tool corresponding to MD package kernel they intend to use. For example, if user wants to use Amber as MD kernel, she would use repex-amber command line tool. In addition to specifying an appropriate command line tool, user need to specify a resource configuration file and REMD simulation input file. The resulting invocation of RepEx should be: repex-amber --input= tsu_remd_ace_ala_nme.json --rconfig= stampede.json where: --input= - specifies the REMD simulation input file --rconfig= - specifies resource configuration file Both REMD simulation input file and resource configuration file must conform to JSON format Resource configuration file In resource configuration file must be provided the following parameters: resource - this is the name of the target machine. Currently supported machines are: local.localhost - your local system xsede.stampede - Stampede supercomputer at TACC xsede.supermic - SuperMIC supercomputer at LSU xsede.comet - Comet supercomputer at SDSC xsede.gordon - Gordon supercomputer at SDSC epsrc.archer - Archer supercomputer at EPCC ncsa.bw_orte - Blue Waters supercomputer at NCSA username - your username on the target machine project - your allocation on specified machine cores - number of cores you would like to allocate 7

12 runtime - for how long you would like to allocate cores on target machine (in minutes). In addition are provided the following optional parameters: queue - specifies which queue to use for job submission. Values are machine specific. cleanup - specifies if files on remote machine must be deleted. Possible values are: True or False Example resource configuration file for Stampede supercomputer might look like this: { "target": { "resource" : "stampede.tacc.utexas.edu", "username" : "octocat", "project" : "TG-XYZ123456", "queue" : "development", "runtime" : "30", "cleanup" : "False", "cores" : "16" REMD input file for Amber kernel For use with Amber kernel, in REMD simulation input file must be provided the following parameters: re_pattern - this parameter specifies Replica Exchange Pattern to use, options are: S - synchronous and A - asynchronous exchange - this parameter specifies type of REMD simulation, for 1D simulation options are: T-REMD, S-REMD and US-REMD number_of_cycles - number of cycles for a given simulation number_of_replicas - number of replicas to use input_folder - path to folder which contains simulation input files input_file_basename - base name of generated input/output files amber_input - name of input file template amber_parameters - name of parameters file amber_coordinates - name of coordinates file replica_mpi - specifies if sander or sander.mpi is used for MD-step. Options are: True or False replica_cores - number of cores to use for MD-step for each replica, if replica_mpi is False this parameters must be equal to 1 steps_per_cycle - number of simulation time-steps download_mdinfo - specifies if Amber.mdinfo files must be downloaded. Options are: True or False. If this parameter is ommited, value defaults to True download_mdout - specifies if Amber.mdout files must be downloaded. Options are: True or False. If this parameter is ommited, value defaults to True Optional parameters are specific to each simulation type. Example REMD simulation input file for T-REMD simulation might look like this: 8 Chapter 3. Getting Started

13 { "remd.input": { "re_pattern": "S", "exchange": "T-REMD", "number_of_cycles": "4", "number_of_replicas": "16", "input_folder": "t_remd_inputs", "input_file_basename": "ace_ala_nme_remd", "amber_input": "ace_ala_nme.mdin", "amber_parameters": "ace_ala_nme.parm7", "amber_coordinates": "ace_ala_nme.inpcrd", "replica_mpi": "False", "replica_cores": "1", "min_temperature": "300", "max_temperature": "600", "steps_per_cycle": "1000", "download_mdinfo": "True", "download_mdout" : "True", 3.2 T-REMD example (peptide ala10) with Amber kernel We will take a look at Temperature-Exchange REMD example using peptide ala10 system with Amber simulations kernel. To run this example locally you must have Amber installed on your system. If you don t have Amber installed please download it from: and install it using instructions at: This guide assumes that you have already cloned RepEx repository during the installation. If you haven t, please do: git clone and cd into repex examples directory where input files recide: cd radical.repex/examples/amber Amongst other things in this directory are present: t_remd_inputs - input files for T-REMD simulations t_remd_ala10.json - REMD input file for Temperature-Exchnage example using peptide ala10 system local.json - resource configuration file to run on local system (your laptop) Run locally To run this example locally you need to make appropriate changes to local.json resouce configuration file. You need to open this file in your favorite text editor (vim in this case): vim local.json By default this file looks like this: { "target": { "resource": "local.localhost", "username" : "octocat", 3.2. T-REMD example (peptide ala10) with Amber kernel 9

14 "runtime" : "30", "cleanup" : "False", "cores" : "4" You need to modify only two parameters in this file: username - this should be your username on your laptop cores - change this parameter to number of cores supported by your laptop Next you need to verify if parameters specified in t_remd_ala10.json REMD input file satisfy your requirements. By default t_remd_ala10.json file looks like this: { "remd.input": { "re_pattern": "S", "exchange": "T-REMD", "number_of_cycles": "4", "number_of_replicas": "8", "input_folder": "t_remd_inputs", "input_file_basename": "ala10_remd", "amber_input": "ala10.mdin", "amber_parameters": "ala10.prmtop", "amber_coordinates": "ala10_minimized.inpcrd", "replica_mpi": "False", "replica_cores": "1", "exchange_mpi": "False", "min_temperature": "300", "max_temperature": "600", "steps_per_cycle": "4000", "exchange_mpi": "False", "download_mdinfo": "True", "download_mdout" : "True" In comparison with general REMD input file format discussed above this input file contains some additional parameters: min_temperature - minimal temperature value to be assigned to replicas max_temperature - maximal temperature value to be assigned to replicas (we use geometrical progression for temperature assignment) exchange_mpi - specifies if exchange step should use MPI interface. Options are: True or False To run this example, all you need to do is to specify path to sander executable on your laptop. To do that please add amber_path parameter under remd.input. For example: "amber_path": "/home/octocat/amber/amber14/bin/sander" To get notified about important events during the simulation please specify in terminal: export RADICAL_REPEX_VERBOSE=info Now you can run this simulation by: repex-amber --input= t_remd_ala10.json --rconfig= local.json 10 Chapter 3. Getting Started

15 3.2.2 Verify output If simulation has successfully finished, last three lines of terminal log should be similar to: 2015:10:11 18:49: MainThread radical.repex.amber : [INFO ] Simulation successfully fi 2015:10:11 18:49: MainThread radical.repex.amber : [INFO ] Please check output files 2015:10:11 18:49: MainThread radical.repex.amber : [INFO ] Closing session. You should see nine new directories in your current path: eight replica_x directories one shared_files directory If you want to check which replicas exchanged configurations during each cycle you can cd into shared_files directory and check each of four pairs_for_exchange_x.dat files. In these files are recorded indexes of replicas exchanging configurations during each cycle. If you want to check.mdinfo or.mdout files for some replica, you can find those files in corresponding replica_x directory. File format is ala10_remd_i_c.mdinfo where: i is index of replica c is current cycle 3.2. T-REMD example (peptide ala10) with Amber kernel 11

16 12 Chapter 3. Getting Started

17 CHAPTER 4 One-dimensional REMD simulations In addition to T-REMD simulations, RepEx also supports Umbrella Sampling (biasing potentials) and Salt Concentration (ionic strength) one-dimensional REMD simulations with Amber kernel. In this section we will take a look at Umbrella Sampling - US-REMD example. 4.1 US-REMD example using Alanine Dipeptide system with Amber kernel For the example we will use Alanine Dipeptide (Ace-Ala-Nme) system. To run this example locally you must have Amber installed on your system. If you don t have Amber installed please download it from: and install it using instructions at: This guide assumes that you have already run example in getting-started section and are currently in amber directory, if not please cd into this directory from repex root directory: cd examples/amber Amongst other things in this directory are present: us_remd_inputs - input files for US-REMD simulations us_remd_ace_ala_nme.json - REMD input file for Umbrella Sampling REMD example using Alanine Dipeptide system local.json - resource configuration file to run on local system (your laptop) Run locally To run this example locally you need to make appropriate changes to local.json resouce configuration file. We assume that you have already done this in getting started section. Next you need to verify if parameters specified in us_remd_ace_ala_nme.json REMD input file satisfy your requirements. By default us_remd_ace_ala_nme.json file looks like this: { "remd.input": { "re_pattern": "S", "exchange": "US-REMD", "number_of_cycles": "4", "number_of_replicas": "8", "input_folder": "us_remd_inputs", 13

18 "input_file_basename": "ace_ala_nme_remd", "amber_input": "ace_ala_nme.mdin", "amber_parameters": "ace_ala_nme.parm7", "amber_coordinates_folder": "ace_ala_nme_coors", "same_coordinates": "True", "us_template": "ace_ala_nme_us.rst", "replica_mpi": "False", "replica_cores": "1", "us_start_param": "120", "us_end_param": "160", "init_temperature": "300.0", "steps_per_cycle": "2000", "exchange_mpi": "False", "download_mdinfo": "True", "download_mdout" : "True" In comparison with general REMD input file format discussed in getting-started section this input file contains some additional parameters: same_coordinates - specifies if each replica should use an individual coordinates file. Options are: True or False. If True is selected, in amber_coordinates_folder must be provided coordinate files for each replica. Format of coordinates file is: filename.inpcrd.x.y, where filename can be any valid python string, inpcrd is required file extension, x is index of replica in 1st dimension and y is index of replica in second dimension. For one-dimensional REMD, y = 0 must be provided us_template - name of Restraints template file us_start_param - starting value of Umbrella interval us_end_param - ending value of Umbrella interval init_temperature - initial temperature to use exchange_mpi - specifies if exchange step should use MPI interface. Options are: True or False To run this example, all you need to do is to specify path to sander executable on your laptop. To do that please add amber_path parameter under remd.input. For example: "amber_path": "/home/octocat/amber/amber14/bin/sander" To get notified about important events during the simulation please specify in terminal: export RADICAL_REPEX_VERBOSE=info Now you can run this simulation by: repex-amber --input= us_remd_ace_ala_nme.json --rconfig= local.json Verify output If simulation has successfully finished, last three lines of terminal log should be similar to: 14 Chapter 4. One-dimensional REMD simulations

19 2015:10:11 18:49: MainThread radical.repex.amber : [INFO ] Simulation successfully fi 2015:10:11 18:49: MainThread radical.repex.amber : [INFO ] Please check output files 2015:10:11 18:49: MainThread radical.repex.amber : [INFO ] Closing session. You should see nine new directories in your current path: eight replica_x directories one shared_files directory If you want to check which replicas exchanged configurations during each cycle you can cd into shared_files directory and check each of four pairs_for_exchange_x.dat files. In these files are recorded indexes of replicas exchanging configurations during each cycle. If you want to check.mdinfo or.mdout files for some replica, you can find those files in corresponding replica_x directory. File format is ala10_remd_i_c.mdinfo where: i is index of replica c is current cycle 4.1. US-REMD example using Alanine Dipeptide system with Amber kernel 15

20 16 Chapter 4. One-dimensional REMD simulations

21 CHAPTER 5 Multi-dimensional REMD simulations In addition to one-dimensional REMD simulations, RepEx also supports multi-dimensional REMD simulations. With Amber Kernel currently supported are two three-dimensional usecases: TSU-REMD with one Temperature, one Salt Concentraiton and one Umbrella restraint dimension TUU-REMD with one Temperature dimension and two Umbrella restraint dimensions 5.1 TUU-REMD example (alanine dipeptide) with Amber kernel For the example we will use Alanine Dipeptide (Ace-Ala-Nme) system. To run this example locally you must have Amber installed on your system. If you don t have Amber installed please download it from: and install it using instructions at: This guide assumes that you have already run example in getting-started section and are currently in amber directory, if not please cd into this directory from repex root directory: cd examples/amber Amongst other things in this directory are present: tuu_remd_inputs - input files for TUU-REMD simulations tuu_remd_ace_ala_nme.json - REMD input file for TUU-REMD usecase using Alanine Dipeptide system local.json - resource configuration file to run on local system (your laptop) Run locally To run this example locally you need to make appropriate changes to local.json resouce configuration file. We assume that you have already done this in getting started section. Next you need to verify if parameters specified in tuu_remd_ace_ala_nme.json REMD input file satisfy your requirements. By default tuu_remd_ace_ala_nme.json file looks like this: { "remd.input": { "re_pattern": "S", "exchange": "TUU-REMD", "number_of_cycles": "4", "input_folder": "tuu_remd_inputs", 17

22 "input_file_basename": "ace_ala_nme_remd", "amber_input": "ace_ala_nme.mdin", "amber_parameters": "ace_ala_nme.parm7", "amber_coordinates_folder": "ace_ala_nme_coors", "us_template": "ace_ala_nme_us.rst", "replica_mpi": "False", "replica_cores": "1", "steps_per_cycle": "6000", "dim.input": { "umbrella_sampling_1": { "number_of_replicas": "2", "us_start_param": "0", "us_end_param": "360", "temperature_2": { "number_of_replicas": "2", "min_temperature": "300", "max_temperature": "600", "umbrella_sampling_3": { "number_of_replicas": "2", "us_start_param": "0", "us_end_param": "360" In comparison to REMD simulaiton input files used previously, this file has the following additional parameters: dim.input - under this key must be specified parameters and names of individual dimensions for all multidimensional REMD simulations. umbrella_sampling_1 - indicates that first dimension is Umbrella potential temperature_2 - indicates that second dimension is Temperature umbrella_sampling_1 - indicates that third dimension is Umbrella potential number_of_replicas - indicates number of replicas in this dimension To run this example, all you need to do is to specify path to sander executable on your laptop. To do that please add amber_path parameter under remd.input. For example: "amber_path": "/home/octocat/amber/amber14/bin/sander" To get notified about important events during the simulation please specify in terminal: export RADICAL_REPEX_VERBOSE=info Now you can run this simulation by: repex-amber --input= tuu_remd_ace_ala_nme.json --rconfig= local.json Verify output If simulation has successfully finished, last three lines of terminal log should be similar to: 18 Chapter 5. Multi-dimensional REMD simulations

23 2015:10:11 18:49: MainThread radical.repex.amber : [INFO ] Simulation successfully fi 2015:10:11 18:49: MainThread radical.repex.amber : [INFO ] Please check output files 2015:10:11 18:49: MainThread radical.repex.amber : [INFO ] Closing session. You should see nine new directories in your current path: eight replica_x directories one shared_files directory If you want to check which replicas exchanged configurations during each cycle you can cd into shared_files directory and check each of four pairs_for_exchange_x.dat files. In these files are recorded indexes of replicas exchanging configurations during each cycle. If you want to check.mdinfo or.mdout files for some replica, you can find those files in corresponding replica_x directory. File format is ala10_remd_i_c.mdinfo where: i is index of replica c is current cycle 5.1. TUU-REMD example (alanine dipeptide) with Amber kernel 19

24 20 Chapter 5. Multi-dimensional REMD simulations

25 CHAPTER 6 Replica Exchange Patterns One of the distinctive features that RepEx provides to its users, is ability to select a Replica Exchange Pattern. Replica Exchange Patterns differ in synchronization modes between MD and Exchange steps. We define two types of Replica Exchange Patterns: 1. Synchronous Replica Exchange Pattern 2. Asynchronous Replica Exchange Pattern 6.1 Synchronous Replica Exchange Pattern Synchronous Pattern, corresponds to conventional way of running REMD simulations, where all replicas propagate MD for a fixed period of simulation time (e.g. 2 ps) and execution time for replicas is not fixed - all replicas must finish MD-step before Exchange-step takes place. When all replicas have finished MD-step, the Exchange-step is performed. 6.2 Asynchronous Replica Exchange Pattern Contrary to Synchronous Pattern, Asynchronous Pattern does not have a global synchronization barrier - while some replicas are performing an MD-step others might be performing an Exchange-step amongst a subset of replicas. In current implementation of Asynchronous Pattern, MD-step is defined as a fixed period of simulation time (e.g. 2 ps), but execution time for MD-step is fixed (e.g. 30 secs). Then predefined execution time elapses, Exchange-step is performed amongst replicas which have finished MD-step. In this pattern there is no synchronization between MD and Exchange-step, thus this pattern can be referred to as asynchronous. 21

26 22 Chapter 6. Replica Exchange Patterns

27 CHAPTER 7 Flexible execution modes REMD simulation corresponding to any of the two Replica Exchange Patterns can be executed in multiple ways. Execution Strategies specify simulation execution details and in particular the resource management details. These strategies differ in: 1. MD simulation time definition: fixed period of simulation time (e.g. 2 ps) for all replicas or fixed period of wall clock time (e.g. 2 minutes) for all replicas, meaning that after this time interval elapses all running replicas will be stopped, regardless of how much simulation time was obtained. 2. task submission modes (bulk submission vs sequential submission) 3. task execution modes on remote HPC system (order and level of concurrency) 4. number of Pilots used for a given simulation 5. number of target resources used concurrently for a given simulation Next we will introduce three Execution Strategies which can be used with Synchronous Replica Exchange Pattern. 7.1 Execution Strategy S1 Synchronous Replica Exchange simulations, may be executed using Execution strategy S1. This strategy differs from a conventional one in number of allocated cores on a target resource (bullet point 3.). In this case number of cores is 1/2 of the number of replicas. As a result of this, only a half of replicas can propogate MD or Exchange-step concurrently. In this execution strategy MD simulation time is defined as a fixed period of simulation time (e.g. 2 ps) for all replicas, meaning that replicas which will finish simulation earlier will have to wait for other replicas before exchange-step may take place. This strategy demonstrates advantage of using a task-level parallelism based approach. Many MD packages are lacking the capability to use less cores than replicas. 23

28 7.2 Execution Strategy S2 Execution Strategy S2 differs from Strategy S1 in MD simulation time definition. Here MD is specified as a fixed period of wall clock time (e.g. 2 minutes) for all replicas. Replicas which will not finish MD-step within this time interval, will be stopped. In addition, Strategy S2 differs from Strategy S1 in the number of allocated cores. Here number of cores equals to the number of replicas. 7.3 Execution Strategy S3 Last Execution strategy we will discuss in this section is Execution Strategy S3. In this strategy all replicas are run concurrently for a presumably indefinite period. At predefined intervals exchanges are performed amongst all (or a subset) of replicas on resource using data from checkpoint files. Any replicas that accept the exchange are reset and then restarted. Since only a small fraction of replicas will actually accept this exchange (10-30%) the amount of time discarded by the exchange is assumed to be minimal. Differences of this strategy from a conventional one can be attributed to bullet point Chapter 7. Flexible execution modes

29 CHAPTER 8 Tutorial In this tutorial we will run several 1D-REMD and 3D-REMD examples on Stampede and Archer supercomputers. This guide assumes that you have already installed RepEx and cloned RepEx repository during the installation. If you haven t installed RepEx, please follow the steps in Installation section of this user guide. If you can t find location of radical.repex directory, please clone repository again: git clone and cd into Amber examples directory where input files recide: cd radical.repex/examples/amber To run examples of this tutorial you will need to modify two resource configuration files - stampede.json and archer.json. Once you have these two files properly configured you can use them for all examples of this tutorial. 8.1 Running on Stampede To run on Stampede you need to make appropriate changes to stampede.json resouce configuration file. Open this file in your favorite text editor (vim in this case): vim stampede.json By default this file looks like this: { "target": { "resource": "xsede.stampede", "username" : "octocat", "project" : "bigthings", "runtime" : "30", "cleanup" : "False", "cores" : "16" You need to modify two parameters in this file: username - this should be your username on Stampede project - this should be your allocation on Stampede 25

30 8.2 Running on Archer To run on Archer you need to make appropriate changes to archer.json resouce configuration file. Open this file in your favorite text editor (vim in this case): vim archer.json By default this file looks like this: { "target": { "resource": "epsrc.archer", "username" : "octocat", "project" : "bigthings", "runtime" : "40", "cleanup" : "False", "cores" : "24" You need to modify two parameters in this file: username - this should be your username on Archer project - this should be your allocation on Archer At this point you are done with resource configuration files and are ready to run simulations. 8.3 T-REMD example (peptide ala10) with Amber kernel First, we will take a look at Temperature-Exchange REMD example using peptide ala10 system with Amber simulations kernel. You need to verify if parameters specified in t_remd_ala10.json REMD input file satisfy your requirements. By default t_remd_ala10.json file looks like this: { "remd.input": { "re_pattern": "S", "exchange": "T-REMD", "number_of_cycles": "4", "number_of_replicas": "8", "input_folder": "t_remd_inputs", "input_file_basename": "ala10_remd", "amber_input": "ala10.mdin", "amber_parameters": "ala10.prmtop", "amber_coordinates": "ala10_minimized.inpcrd", "replica_mpi": "False", "replica_cores": "1", "exchange_mpi": "False", "min_temperature": "300", "max_temperature": "600", "steps_per_cycle": "4000", "exchange_mpi": "False", "download_mdinfo": "True", "download_mdout" : "True" 26 Chapter 8. Tutorial

31 In comparison with general REMD input file format discussed above this input file contains some additional parameters: min_temperature - minimal temperature value to be assigned to replicas max_temperature - maximal temperature value to be assigned to replicas (we use geometrical progression for temperature assignment) exchange_mpi - specifies if exchange step should use MPI interface. Options are: True or False Since we are using a supercomputer to run REMD simulation we increase the nuber of replicas to use. Please set "number_of_replicas" to "16". To get notified about important events during the simulation please specify in terminal: export RADICAL_REPEX_VERBOSE=info Now you are ready to run this simulation. If you want to run on Stampede run in terminal: repex-amber --input= t_remd_ala10.json --rconfig= stampede.json If you want to run on Archer run in terminal: repex-amber --input= t_remd_ala10.json --rconfig= archer.json Verify output If simulation has successfully finished, last three lines of terminal log should be similar to: 2015:10:11 18:49: MainThread radical.repex.amber : [INFO ] Simulation successfully fi 2015:10:11 18:49: MainThread radical.repex.amber : [INFO ] Please check output files 2015:10:11 18:49: MainThread radical.repex.amber : [INFO ] Closing session. You should see 17 new directories in your current path: sixteen replica_x directories one shared_files directory If you want to check which replicas exchanged configurations during each cycle you can cd into shared_files directory and check each of four pairs_for_exchange_x.dat files. In these files are recorded indexes of replicas exchanging configurations during each cycle. If you want to check.mdinfo or.mdout files for some replica, you can find those files in corresponding replica_x directory. File format is ala10_remd_i_c.mdinfo where: i is index of replica c is current cycle Simulation output can similarly be verified for all other examples of this tutorial. 8.4 US-REMD example using Alanine Dipeptide system with Amber kernel For the example we will use Alanine Dipeptide (Ace-Ala-Nme) system. In examples/amber directory are present: us_remd_inputs - input files for US-REMD simulations us_remd_ace_ala_nme.json - REMD input file for Umbrella Sampling REMD example using Alanine Dipeptide system 8.4. US-REMD example using Alanine Dipeptide system with Amber kernel 27

32 To run this example you need to verify if parameters specified in us_remd_ace_ala_nme.json REMD input file satisfy your requirements. By default us_remd_ace_ala_nme.json file looks like this: { "remd.input": { "re_pattern": "S", "exchange": "US-REMD", "number_of_cycles": "4", "number_of_replicas": "8", "input_folder": "us_remd_inputs", "input_file_basename": "ace_ala_nme_remd", "amber_input": "ace_ala_nme.mdin", "amber_parameters": "ace_ala_nme.parm7", "amber_coordinates_folder": "ace_ala_nme_coors", "same_coordinates": "True", "us_template": "ace_ala_nme_us.rst", "replica_mpi": "False", "replica_cores": "1", "us_start_param": "120", "us_end_param": "160", "init_temperature": "300.0", "steps_per_cycle": "2000", "exchange_mpi": "False", "download_mdinfo": "True", "download_mdout" : "True" In comparison with general REMD input file format discussed in getting-started section this input file contains some additional parameters: same_coordinates - specifies if each replica should use an individual coordinates file. Options are: True or False. If True is selected, in amber_coordinates_folder must be provided coordinate files for each replica. Format of coordinates file is: filename.inpcrd.x.y, where filename can be any valid python string, inpcrd is required file extension, x is index of replica in 1st dimension and y is index of replica in second dimension. For one-dimensional REMD, y = 0 must be provided us_template - name of Restraints template file us_start_param - starting value of Umbrella interval us_end_param - ending value of Umbrella interval init_temperature - initial temperature to use exchange_mpi - specifies if exchange step should use MPI interface. Options are: True or False Since we are using a supercomputer to run REMD simulation we increase the nuber of replicas to use. Please set "number_of_replicas" to "16". Now you are ready to run this simulation. If you want to run on Stampede run in terminal: repex-amber --input= us_remd_ace_ala_nme.json --rconfig= stampede.json If you want to run on Archer run in terminal: repex-amber --input= us_remd_ace_ala_nme.json --rconfig= archer.json Output verification can be done similarly as for T-REMD example. 28 Chapter 8. Tutorial

33 8.5 TUU-REMD example (alanine dipeptide) with Amber kernel For the example we also will use Alanine Dipeptide (Ace-Ala-Nme) system. In examples/amber directory are present: tuu_remd_inputs - input files for TUU-REMD simulations tuu_remd_ace_ala_nme.json - REMD input file for TUU-REMD usecase using Alanine Dipeptide system To run this example you need to verify if parameters specified in tuu_remd_ace_ala_nme.json REMD input file satisfy your requirements. By default tuu_remd_ace_ala_nme.json file looks like this: { "input.md": { "re_pattern": "S", "exchange": "TUU-REMD", "number_of_cycles": "4", "input_folder": "tuu_remd_inputs", "input_file_basename": "ace_ala_nme_remd", "amber_input": "ace_ala_nme.mdin", "amber_parameters": "ace_ala_nme.parm7", "amber_coordinates_folder": "ace_ala_nme_coors", "us_template": "ace_ala_nme_us.rst", "replica_mpi": "False", "replica_cores": "1", "steps_per_cycle": "6000", "input.dim": { "umbrella_sampling_1": { "number_of_replicas": "4", "us_start_param": "0", "us_end_param": "360", "temperature_2": { "number_of_replicas": "4", "min_temperature": "300", "max_temperature": "600", "umbrella_sampling_3": { "number_of_replicas": "4", "us_start_param": "0", "us_end_param": "360" In comparison to general REMD simulaiton input file, this file has the following additional parameters: input.dim - under this key must be specified parameters and names of individual dimensions for all multidimensional REMD simulations. umbrella_sampling_1 - indicates that first dimension is Umbrella potential temperature_2 - indicates that second dimension is Temperature umbrella_sampling_1 - indicates that third dimension is Umbrella potential number_of_replicas - indicates number of replicas in this dimension Now you are ready to run this simulation. If you want to run on Stampede run in terminal: 8.5. TUU-REMD example (alanine dipeptide) with Amber kernel 29

34 repex-amber --input= tuu_remd_ace_ala_nme.json --rconfig= stampede.json If you want to run on Archer run in terminal: repex-amber --input= tuu_remd_ace_ala_nme.json --rconfig= archer.json Output verification can be done similarly as for T-REMD example. 30 Chapter 8. Tutorial

35 CHAPTER 9 Frequently Asked Questions 9.1 Where are.mdout files? todo 9.2 Where are.mdinfo files? Amber.mdinfo files by default are residing in respective replica directories on target cluster. 9.3 How can I obtain information about accepted exchanges? todo 9.4 How can I obtain information about attempted exchanges? todo 31

36 32 Chapter 9. Frequently Asked Questions

37 CHAPTER 10 Indices and tables genindex modindex search 33