Accelerating Science with High Throughput Computing (HTC)

Size: px

Start display at page:

Download "Accelerating Science with High Throughput Computing (HTC)"

Pearl Williams
5 years ago
Views:

1 Accelerating Science with High Throughput Computing (HTC) Miron Livny Morgridge Institute Of Research Center for High Throughput Computing Computer Sciences Department University of Wisconsin-Madison

2 The process is made practical using Condor high-throughput computing (Thain et al. 2005), which enables O(10**2) update evaluations of the expensive simulations to be executed concurrently across workstations that would otherwise sit idle. This work was carried out at the University of Southampton condor pool, which is composed of approximately 1100 laboratory workstations and office PCs running various versions of the WINDOWS operating system.

3 GPro installation on Computational Nodes for HPC For High Performance Computing (HPC) it is required to install GPro_HPC_3.3.3.exe. Please read the updated installation instruction in the GPro User Manual. The recommended Condor version to use is v Condor downloads are available from the home page of the University of Wisconsin: downloads/

4 Genotype to Phenotype The current run will identify regions of the plant chromosomes that govern the growth and behavior of a seedling root that has been contrived to react to an external situation. The raw data from 1293 trials, one root per trial, images every 2 minutes for 8 hours. The total number of images is 311,613 (~265 GB), which were processed by Condor. The run will map the tip angle at each time points as a separate DAG. To assess the satistical significance, the data must be shuffled and rerun. Each of the DAGs runs 25,000 permutations of the genotype data. The run may take 60 days using ~ 14,000 CPU hours per day.

6 High Throughput Computing We first introduced the distinction between High Performance Computing (HPC) and High Throughput Computing (HTC) in a seminar at the NASA Goddard Flight Center in July of 1996 and a month later at the European Laboratory for Particle Physics (CERN). In June of 1997 HPCWire published an interview on High Throughput Computing.

7 Why HTC? For many experimental scientists, scientific progress and quality of research are strongly linked to computing throughput. In other words, they are less concerned about instantaneous computing power. Instead, what matters to them is the amount of computing they can harness over a month or a year --- they measure computing power in units of scenarios per day, wind patterns per week, instructions sets per month, or crystal configurations per year.

8 High Throughput Computing is a activity FLOPY (60*60*24*7*52)*FLOPS

9 Addison Snell, VP/GM, High Productivity Computing: Predictions for the New HPC Actionable Market Intelligence for High Productivity Computing October 2007

10 Use of Condor by the LIGO Scientific Collaboration Condor handles 10 s of millions of jobs per year running on the LDG, and up to 500k jobs per DAG. Condor standard universe check pointing widely used, saving us from having to manage this. At Caltech, 30 million jobs processed using 22.8 million CPU hrs. on 1324 CPUs in last 30 months. For example, to search 1 yr. of data for GWs from the inspiral of binary neutron star and black hole systems takes ~2 million jobs, and months to run on several thousand ~2.6 GHz nodes.

11 From: Stuart Anderson Date: February 28, :51:32 PM EST To: Condor-LIGO mailing list Subject: [CondorLIGO] Largest LIGO workflow Pete, Here are some numbers you ask about for LIGO's use of DAGs to manage large data analysis tasks broken down by the largest number of jobs managed in different categories: 1) DAG Instance--one condor_dagman process: 196,862. 2) DAG Workflow--launched from a single condor_submit_dag but may include multiple automatic sub- or spliced DAGs: 1,120,659. 3) DAG Analysis--multiple instances of condor_submit_dag to analyze a common dataset with results combined into a single coherent scientific result: 6,200,000. 4) DAG Total--sum over all instances of condor dagman run: O (100M). P.S. These are lower bounds as I did not perform an exhaustive survey/ search, but they are probably close. Thanks.

12 Grid Laboratory of Wisconsin (GLOW) HTC at the campus level Usage 04/04-04/10 114M Hours

13 Open Science Grid (OSG) HTC at the National Level

14 The Qiang Cui (QC) Group We develop and apply theoretical tools (electronic structure, nuclear dynamics and statistical mechanics) to biophysical problems (enzyme catalysis, bioenergy transduction and biomaterials etc.). Started to use the High Through Parallel Computing (HTPC) provided by CHTC in 05/09 and expended to OSG in 11/09

15 QC Group Comsumption

16 Impact of these cycles 1. Iron-Catalyzed Oxidation Intermediates Captured in A DNA Repair Monooxygenase, C. Yi, G. Jia, G. Hou, Q. Dai, G. Zheng, X. Jian, C. G. Yang, Q. Cui, and C. He, Science, Submitted 2. Disruption and formation of surface salt bridges are coupled to DNA binding in integration host factor (IHF): a computational analysis, L. Ma, M. T. Record, Jr., N. Sundlass, R. T. Raines and Q. Cui, J. Mol. Biol, Submitted 3. An implicit solvent model for SCC-DFTB with Charge-Dependent Radii, G. Hou, X. Zhu and Q. Cui, J. Chem. Theo. Comp., Submitted 4. Sequence-dependent interaction of $\beta$-peptides with membranes, J. Mondal, X. Zhu, Q. Cui and A. Yethiraj, J. Am. Chem. Soc., Submitted 5. A new coarse-grained model for water: The importance of electrostatic interactions, Z. Wu, Q. Cui and A. Yethiraj, J. Phys. Chem. Submitted 6. How does bone sialoprotein promote the nucleation of hydroxyapatite? A molecular dynamics study using model peptides of different conformations, Y. Yang, Q. Cui, and N. Sahai, {\it Langmuir}, Submitted 7. Preferential interactions between small solutes and the protein backbone: A computational analysis, L. Ma, L. Pegram, M. T. Record, Jr., Q. Cui, Biochem., 49, (2010) 8. Establishing effective simulation protocols for $\beta$- and $\alpha/\beta$-peptides. III. Molecular Mechanical (MM) model for a non-cyclic $\beta$-residue, X. Zhu, P. K\"onig, M. Hoffman, A. Yethiraj and Q. Cui, J. Comp. Chem., In press (DOI: /jcc.21493) 9. Curvature Generation and Pressure Profile in Membrane with lysolipids: Insights from coarse-grained simulations, J. Yoo and Q. Cui, Biophys. J. 97, (2009)

17 And what about managing energy?

18 From Forbes Magazine Open Source Energy Savings Dan Woods, Software for spreading work over huge collections of computers can be used to cut power costs. Condor supports all the operating systems a typical company or research institution would have and is rock solid in terms of stability and functions for its intended purpose, which is carving up work and sending it out to any number of computers for processing. 18

(HT)Condor - Past and Future

(HT)Condor - Past and Future Miron Livny John P. Morgridge Professor of Computer Science Wisconsin Institutes for Discovery University of Wisconsin-Madison חי has the value of 18 חי means alive Europe