Grid Computing in Numerical Relativity and Astrophysics
|
|
- Amberlynn Greene
- 6 years ago
- Views:
Transcription
1 Grid Computing in Numerical Relativity and Astrophysics Gabrielle Allen: Depts Computer Science & Physics Center for Computation & Technology (CCT) Louisiana State University Challenge Problems Cosmology Black Hole and Neutron Star Models Supernovae Astronomical Databases Gravitational Wave Data Analysis Drive HEC & Grids 1
2 Gravitational Wave Physics Analysis & Insight Observations Models Complex Simulations 2
3 Computational Science Needs Requires incredible mix of technologies & expertise! Many scientific/engineering components Physics, astrophysics, CFD, engineering,... Many numerical algorithm components Finite difference? Finite volume? Finite elements? Elliptic equations: multigrid, Krylov subspace,... Mesh refinement Many different computational components Parallelism (HPF, MPI, PVM,???) Multipatch Architecture (MPP, DSM, Vector, PC Clusters, FPGA,???) I/O (generate TBs/simulation, checkpointing ) Visualization of all that comes out! New technologies Grid computing Steering, data archives Such work cuts across many disciplines, areas of CS Cactus Code Freely available, modular, portable and manageable environment for collaboratively developing parallel, high-performance multidimensional simulations Developed for Numerical Relativity, but now general framework for parallel computing (CFD, astrophysics, climate modeling, chemical eng, quantum gravity, ) Finite difference, adaptive mesh refinement (Carpet, Samrai, Grace), adding FE/FV, multipatch Active user and developer communities, main development now at LSU and AEI. Open source, documentation, etc 3
4 Cactus Einstein Cactus modules (thorns) for numerical relativity. Many additional thorns available from other groups (AEI, CCT, ) Agree on some basic principles (e.g. names of variables) and then can share evolution, analysis etc. Can choose whether or not to use e.g. gauge choice, macros, masks, matter coupling, conformal factor Over 100 relativity papers & 30 student theses: production research code Evolve ADM EvolSimple InitialData IDAnalyticBH IDAxiBrillBH IDBrillData IDLinearWaves IDSimple SpaceMask ADMMacros ADMBase Analysis ADMAnalysis ADMConstraints AHFinder Extract PsiKadelia TimeGeodesic Gauge Conditions CoordGauge Maximal ADMCoupling StaticConformal Grand Challenge Collaborations NASA Neutron Star Grand Challenge 5 US sites 3 years Colliding neutron star problem EU Astrophysics Network 10 EU sites 3 years Continuing these problems NSF Black Hole Grand Challenge 8 US Institutions 5 years Attack colliding black hole problem Examples of Future of Science & Engineering Require Large Scale Simulations, beyond reach of any machine Require Large Geo-distributed Cross-Disciplinary Collaborations Require Grid Technologies, but not yet using them! 4
5 New Paradigm: Grid Computing Computational resources across the world Compute servers (double each 18 months) File servers Networks (double each 9 months) Playstations, cell phones etc Grid computing integrates communities and resources How to take advantage of this for scientific simulations? Harness multiple sites and devices Models with new level of complexity and scale, interacting with data New possibilities for collaboration and advanced scenarios NLR and Louisiana Optical Network (LONI) State initiative ($40M) to support research: 40 Gbps optical network Connects 7 sites Grid resources (IBM P5) at sites LIGO/CAMD New possibilities: Dynamical provisioning and scheduling of network bandwidth Network dependent scenarios EnLIGHTened Computing (NSF) 5
6 Current Grid Application Types Community Driven Distributed communities share resources Video Conferencing Virtual Collaborative Environments Data Driven Remote access of huge data, data mining Eg. Gravitational wave analysis, particle physics, astronomy Process/Simulation Driven Demanding Simulations of Science and Engineering Task farming, resource brokering, distributed computations, workflow Remote visualization, steering and interaction, etc Typical scenario: Find remote resources (task farm, distribute) Launch jobs (static) Visualize, collect results Prototypes and demos: need to move to: Fault tolerance Robustness Scaling Easy to use Complete solutions New Paradigms for Dynamic Grids Addressing large, complex, multidisciplinary problems with collaborative teams of varied researchers... Code/User/Infrastructure should be aware of environment Discover and monitor resources available NOW What is my allocation on these resources? What is bandwidth/latency Code/User/Infrastructure should make decisions Slow part of simulation can run independently spawn it off! New powerful resources just became available migrate there! Machine went down reconfigure and recover! Need more memory (or less!), get by adding (dropping) machines! Dynamically provision and use new high end resources and networks 6
7 Future Dynamic Grid Computing We see something, but too weak. Please simulate to enhance signal! S S 1 S 2 P 1 P 2 S 1 S 2 P 1 P 2 Future Dynamic Grid Computing Free CPUs!! RZG Queue time over, find new machine Add more resources SDSC LRZ Archive data Clone job with steered parameter Calculate/Output Invariants SDSC Further Calculations Found a black hole, Load new component Find best resources Look for horizon Calculate/Output Grav. Waves AEI Archive to LIGO experiment NCSA 7
8 New Grid Scenarios Intelligent Parameter Surveys, speculative computing, monte carlo Dynamic Staging: move to faster/cheaper/bigger machine Multiple Universe: create clone to investigate steered parameter Automatic Component Loading: needs of process change, discover/load/execute new calc. component on approp.machine Automatic Convergence Testing Look Ahead: spawn off and run coarser resolution to predict likely future Spawn Independent/Asynchronous Tasks: send to cheaper machine, main simulation carries on Routine Profiling: best machine/queue, choose resolution parameters based on queue Dynamic Load Balancing: inhomogeneous loads, multiple grids Inject dynamically acquired data But Need Grid Apps and Programming Tools Need application programming tools for Grid environments Frameworks for developing Grid applications Toolkits providing Grid functionality Grid debuggers and profilers Robust, dependable, flexible Grid tools Challenging CS problems: Missing or immature grid services Changing environment Different and evolving interfaces to the grid Interfaces are not simple for scientific application developers Application developers need easy, robust and dependable tools 8
9 GridLab Project EU 5th Framework ($7M) Partners in Europe and US PSNC (Poland), AEI & ZIB (Germany), VU (Netherlands), MASARYK (Czech), SZTAKI (Hungary), ISUFI (Italy), Cardiff (UK), NTUA (Greece), Chicago, ISI & Wisconsin (US), Sun, Compaq/HP, LSU Application and test bed oriented (Cactus + Triana) Numerical relativity Dynamic use of grids Main goal: develop application programming environment for Grid Abstract programming interface between applications and Grid services Designed for applications (move file, run remote task, migrate, write to remote file) Led to GGF Simple API for Grid Applications Grid Application Toolkit (GAT) Main result from GridLab project 9
10 Distributed Computation Harnessing Multiple Computers Why do this? Capacity: computers can t keep up with needs Throughput: combine resources Issues Bandwidth (increasing faster than CPU) Latency Communication needs, Topology Communication/computation Techniques to be developed Overlapping communication/computation Extra ghost zones to reduce latency Compression Algorithms to do this for scientist Dynamic Adaptive Distributed Computation 17 SDSC IBM SP 1024 procs 5x12x17 = OC-12 line (But only 2.5MB/sec) NCSA Origin Array x12x(4+2+2) =480 GigE:100MB/sec Cactus + MPICH-G2 Communications dynamically adapt to application and environment Any Cactus application Scaling: 15% -> 85% Gordon Bell Prize (With U. Chicago/Northern, Supercomputing 2001, Denver) 10
11 Remote Viz & Steering HTTP Any Viz Client: LCA Vision, OpenDX Streaming HDF5 Autodownsample Changing steerable parameters Parameters Physics, algorithms Performance Cactus Worm (SC2000) Cactus simulation starts, launched from portal Migrates itself to another site Grid technologies Registers new location User tracks/steers, using HTTP, streaming data, etc Continues around Europe 11
12 Task Spawning (SC2001) Cactus Spawner thorn automatically prepares analysis tasks for spawning Grid technologies find resources, manage tasks, collect data Intelligence to decide when to spawn SC2001: resources of GGTC testbed. Main Cactus BH simulation starts here Appropriate analysis tasks spawned automatically to free resources worldwide User only has to invoke Cactus Spawner thorn Global Grid Testbed Collaboration Supercomputing 2001 Cactus black hole simulations spawned apparent horizon finding tasks across the grid. Prizes for most heterogeneous and most distributed testbed 5 continents and over 14 countries. Around 70 machines, processors Many hardware types, including PS2, IA32, IA64, MIPS, Many OSs, including Linux, Irix, AIX, OSF, True64, Solaris, Hitachi Many organizations: DOE, NSF, MPG, universities, vendors All ran same Grid infrastructure, and used for different applications 12
13 Black Hole Task Farming (SC2002) Black hole server controls tasks and steers main job Main Cactus BH Simulation started in California Error measure returned Dozens of low resolution jobs test corotation parameter Huge job generates remote data visualized in Baltimore Job Migration GridLab demonstration SC
14 Notification and Information Replica Catalog SMS Server GridSphere Portal The Grid IM Server Mail Server User details, notification prefs and simulation information Grid-enabled Gravitational Physics Adaptive, intelligent simulation codes able to adapt to environment Simulation data stored across geographically distributed spaces Organization, access, mining issues Analysis of federated data sets by virtual organizations Data analysis of LIGO, GEO, LISA signals Interacting with simulation data Managing parameter space/signal analysis Now working on domain specific information and knowledge based services: Gravitational physics description language Schema for describing, searching, encoding simulation results Automated logging of simulations: reproducibility Notification and data sharing services to enable collaboration Relativity services Remote servers running e.g. waveform extraction, horizon finding etc. Connection to publications and information Automated analysis 14
15 Credits This talk describes work carried out over a number of years by physicists, computer scientists, mathematicians etc by the joint AEI-LSU numerical relativity groups and colleagues. 15
PHYSICS TOMORROW. The Grid from an Application Viewpoint. Ed Seidel
The Grid from an Application Viewpoint PHYSICS TOMORROW TODAY Ed Seidel Max-Planck-Institut für Gravitationsphysik (Albert Einstein Institute) + Lots of colleagues eseidel@aei.mpg.de Co-Chair, GGF Applications
More informationHigh Performance and Grid Computing Applications with the Cactus Framework. HPCC Program Grand Challenges (1995)
High Performance and Grid Computing Applications with the Cactus Framework Gabrielle Allen Department of Computer Science Center for Computation & Technology Louisiana State University HPCC Program Grand
More informationThe Cactus Framework: Design, Applications and Future Directions. Cactus Code
The Cactus Framework: Design, Applications and Future Directions Gabrielle Allen gallen@cct.lsu.edu Center for Computation & Technology Departments of Computer Science & Physics Louisiana State University
More informationFourteen years of Cactus Community
Fourteen years of Cactus Community Frank Löffler Center for Computation and Technology Louisiana State University, Baton Rouge, LA September 6th 2012 Outline Motivation scenario from Astrophysics Cactus
More informationWhat is Cactus? Cactus is a framework for developing portable, modular applications
What is Cactus? Cactus is a framework for developing portable, modular applications What is Cactus? Cactus is a framework for developing portable, modular applications focusing, although not exclusively,
More informationThe Cactus Framework. Erik Schnetter September 2006
The Cactus Framework Erik Schnetter September 2006 Outline History The Cactus User Community Cactus Usage Patterns Bird s eye view Cactus is a freely available, portable, and manageable environment for
More informationCactus Framework: Scaling and Lessons Learnt
Cactus Framework: Scaling and Lessons Learnt Gabrielle Allen, Erik Schnetter, Jian Tao Center for Computation & Technology Departments of Computer Science & Physics Louisiana State University Also: Christian
More informationThe Cactus Framework and Toolkit: Design and Applications
The Cactus Framework and Toolkit: Design and Applications Tom Goodale 1, Gabrielle Allen 1, Gerd Lanfermann 1, Joan Massó 2, Thomas Radke 1, Edward Seidel 1, and John Shalf 3 1 Max-Planck-Institut für
More informationChapter 4:- Introduction to Grid and its Evolution. Prepared By:- NITIN PANDYA Assistant Professor SVBIT.
Chapter 4:- Introduction to Grid and its Evolution Prepared By:- Assistant Professor SVBIT. Overview Background: What is the Grid? Related technologies Grid applications Communities Grid Tools Case Studies
More informationCactus Tutorial. Introduction to Cactus. Yaakoub El Khamra. Cactus Developer, Frameworks Group CCT 27 March, 2007
Cactus Tutorial Introduction to Cactus Yaakoub El Khamra Cactus Developer, Frameworks Group CCT 27 March, 2007 Agenda Introduction to Cactus What is Cactus Flesh and thorns Cactus Computational Toolkit
More informationIntegration of Trilinos Into The Cactus Code Framework
Integration of Trilinos Into The Cactus Code Framework Josh Abadie Research programmer Center for Computation & Technology Louisiana State University Summary Motivation Objectives The Cactus Code Trilinos
More informationGrid Computing Fall 2005 Lecture 2: About Grid Computing. Gabrielle Allen
Grid Computing 7700 Fall 2005 Lecture 2: About Grid Computing Gabrielle Allen allen@bit.csc.lsu.edu http://www.cct.lsu.edu/~gallen/teaching Quick Test What reason does Foster (2002) give that the Web is
More informationIntroduction to Grid Computing
Milestone 2 Include the names of the papers You only have a page be selective about what you include Be specific; summarize the authors contributions, not just what the paper is about. You might be able
More informationCactus: Current Status and Future Plans
Cactus: Current Status and Future Plans Tom Goodale goodale@aei-potsdam.mpg.de Penn State Numrel Lunch February 27 h 2003 What Is Cactus Cactus is a framework for developing portable, modular applications,
More informationDatura The new HPC-Plant at Albert Einstein Institute
Datura The new HPC-Plant at Albert Einstein Institute Nico Budewitz Max Planck Institue for Gravitational Physics, Germany Cluster Day, 2011 Outline 1 History HPC-Plants at AEI -2009 Peyote, Lagavulin,
More informationDay 2 August 06, 2004 (Friday)
An Overview of Grid Computing Day 2 August 06, 2004 (Friday) By CDAC Experts Contact :vcvrao@cdacindia.com; betatest@cdacindia.com URL : http://www.cs.umn.edu/~vcvrao 1 Betatesting Group,NPSF, C-DAC,Pune
More informationOutline. Definition of a Distributed System Goals of a Distributed System Types of Distributed Systems
Distributed Systems Outline Definition of a Distributed System Goals of a Distributed System Types of Distributed Systems What Is A Distributed System? A collection of independent computers that appears
More informationIntroduction. Kelly Davis. MPI-AEI. Author s name
Introduction Kelly Davis kdavis@aei.mpg.de MPI-AEI Table of Contents Introduction Why GAT? Installation Hello Cruel World GAT Object Model File Management FileStream Management LogicalFile Management Advert
More informationDeliverable D8.6 - Data Management Integration
GridLab - A Grid Application Toolkit and Testbed Deliverable D8.6 - Data Management Integration Author(s): Document Filename: Work package: Partner(s): Lead Partner: Config ID: Document classification:
More informationHungarian Supercomputing Grid 1
Hungarian Supercomputing Grid 1 Péter Kacsuk MTA SZTAKI Victor Hugo u. 18-22, Budapest, HUNGARY www.lpds.sztaki.hu E-mail: kacsuk@sztaki.hu Abstract. The main objective of the paper is to describe the
More informationGrid Programming Models: Current Tools, Issues and Directions. Computer Systems Research Department The Aerospace Corporation, P.O.
Grid Programming Models: Current Tools, Issues and Directions Craig Lee Computer Systems Research Department The Aerospace Corporation, P.O. Box 92957 El Segundo, CA USA lee@aero.org Domenico Talia DEIS
More informationGrid Scheduling Architectures with Globus
Grid Scheduling Architectures with Workshop on Scheduling WS 07 Cetraro, Italy July 28, 2007 Ignacio Martin Llorente Distributed Systems Architecture Group Universidad Complutense de Madrid 1/38 Contents
More informationAn Experience in Accessing Grid Computing from Mobile Device with GridLab Mobile Services
An Experience in Accessing Grid Computing from Mobile Device with GridLab Mobile Services Riri Fitri Sari, Rene Paulus Department of Electrical Engineering, Faculty of Engineering University of Indonesia
More informationModerator: Edward Seidel, Director, Center for Computation & Technology, Louisiana State University
Ask A Grid Expert Panel & Interaction Thursday January 6, 2005, 4:00-5:30PM Moderator: Edward Seidel, Director, Center for Computation & Technology, Louisiana State University Panelists: Gabrielle Allen,
More informationParallelism. Wolfgang Kastaun. May 9, 2008
Parallelism Wolfgang Kastaun May 9, 2008 Outline Parallel computing Frameworks MPI and the batch system Running MPI code at TAT The CACTUS framework Overview Mesh refinement Writing Cactus modules Links
More informationAstroGrid-D. Advanced Prototype Implementation of Monitoring & Steering Methods. Documentation and Test Report 1. Deliverable D6.5
AstroGrid-D Deliverable D6.5 Advanced Prototype Implementation of Monitoring & Steering Methods Documentation and Test Report 1 Deliverable D6.5 Authors Thomas Radke (AEI) Editors Thomas Radke (AEI) Date
More informationIST GridLab - A Grid Application Toolkit and Testbed. Result Evaluation. Jason Maassen, Rob V. van Nieuwpoort, Andre Merzky, Thilo Kielmann
GridLab - A Grid Application Toolkit and Testbed Result Evaluation Author(s): Document Filename: Work package: Partner(s): Lead Partner: Config ID: Document classification: Jason Maassen, Rob V. van Nieuwpoort,
More informationScaling a Global File System to the Greatest Possible Extent, Performance, Capacity, and Number of Users
Scaling a Global File System to the Greatest Possible Extent, Performance, Capacity, and Number of Users Phil Andrews, Bryan Banister, Patricia Kovatch, Chris Jordan San Diego Supercomputer Center University
More informationAndrea Sciabà CERN, Switzerland
Frascati Physics Series Vol. VVVVVV (xxxx), pp. 000-000 XX Conference Location, Date-start - Date-end, Year THE LHC COMPUTING GRID Andrea Sciabà CERN, Switzerland Abstract The LHC experiments will start
More informationCactus-Simulationen on the Grid
------------------------------- Cactus-Simulationen on the Grid ------------------------------- Contact Person: Thomas Radke, AEI tradke@aei.mpg.de, 0331 / 567 7626 -------------------- 1. Scenario Overview
More informationProject Vision and Mission
Solving End to End connectivity with GMPLS Radek Krzywania, PSNC, Poland radek.krzywania@man.poznan.pl April 4th 2008, Munich NGN meeting Phopshorus project European and Global alliance of partners to
More informationErik Schnetter Rochester, August Friday, August 27, 2010
Erik Schnetter Rochester, August 2010 1 Goal: have state-of-the-art set of tools for NR available as open source Organised by Einstein Consortium, open to everyone See http://einsteintoolkit.org 2 Guiding
More informationGiovanni Lamanna LAPP - Laboratoire d'annecy-le-vieux de Physique des Particules, Université de Savoie, CNRS/IN2P3, Annecy-le-Vieux, France
Giovanni Lamanna LAPP - Laboratoire d'annecy-le-vieux de Physique des Particules, Université de Savoie, CNRS/IN2P3, Annecy-le-Vieux, France ERF, Big data & Open data Brussels, 7-8 May 2014 EU-T0, Data
More informationSolving End-to-End connectivity with GMPLS
Solving End-to-End connectivity with GMPLS Radek Krzywania, PSNC, Poland radek.krzywania@man.poznan.pl April 4 th 2008, Munich NGN meeting Phopshorus project European and Global alliance of partners to
More informationMoving e-infrastructure into a new era the FP7 challenge
GARR Conference 18 May 2006 Moving e-infrastructure into a new era the FP7 challenge Mário Campolargo European Commission - DG INFSO Head of Unit Research Infrastructures Example of e-science challenges
More informationUpdate on EZ-Grid. Priya Raghunath University of Houston. PI : Dr Barbara Chapman
Update on EZ-Grid Priya Raghunath University of Houston PI : Dr Barbara Chapman chapman@cs.uh.edu Outline Campus Grid at the University of Houston (UH) Functionality of EZ-Grid Design and Implementation
More informationGrid Computing. MCSN - N. Tonellotto - Distributed Enabling Platforms
Grid Computing 1 Resource sharing Elements of Grid Computing - Computers, data, storage, sensors, networks, - Sharing always conditional: issues of trust, policy, negotiation, payment, Coordinated problem
More informationThe Grid: Feng Shui for the Terminally Rectilinear
The Grid: Feng Shui for the Terminally Rectilinear Martha Stewart Introduction While the rapid evolution of The Internet continues to define a new medium for the sharing and management of information,
More informationLeveraging Software-Defined Storage to Meet Today and Tomorrow s Infrastructure Demands
Leveraging Software-Defined Storage to Meet Today and Tomorrow s Infrastructure Demands Unleash Your Data Center s Hidden Power September 16, 2014 Molly Rector CMO, EVP Product Management & WW Marketing
More informationParallel Programming with MPI on Clusters
Parallel Programming with MPI on Clusters Rusty Lusk Mathematics and Computer Science Division Argonne National Laboratory (The rest of our group: Bill Gropp, Rob Ross, David Ashton, Brian Toonen, Anthony
More informationHeterogeneous Multi-Computer System A New Platform for Multi-Paradigm Scientific Simulation
Heterogeneous Multi-Computer System A New Platform for Multi-Paradigm Scientific Simulation Taisuke Boku, Hajime Susa, Masayuki Umemura, Akira Ukawa Center for Computational Physics, University of Tsukuba
More informationIsolation Forest for Anomaly Detection
Isolation Forest for Anomaly Detection Sahand Hariri PhD Student, MechSE UIUC Matias Carrasco Kind Senior Research Scientist, NCSA LSST Workshop 2018, June 21, NCSA, UIUC Overview Goal: Build a resilient
More informationExtreme I/O Scaling with HDF5
Extreme I/O Scaling with HDF5 Quincey Koziol Director of Core Software Development and HPC The HDF Group koziol@hdfgroup.org July 15, 2012 XSEDE 12 - Extreme Scaling Workshop 1 Outline Brief overview of
More informationIntroduction to GT3. Introduction to GT3. What is a Grid? A Story of Evolution. The Globus Project
Introduction to GT3 The Globus Project Argonne National Laboratory USC Information Sciences Institute Copyright (C) 2003 University of Chicago and The University of Southern California. All Rights Reserved.
More informationOverview of HPC at LONI
Overview of HPC at LONI Le Yan HPC Consultant User Services @ LONI What Is HPC High performance computing is to use supercomputers to solve problems computationally The most powerful supercomputer today
More informationGrid Challenges and Experience
Grid Challenges and Experience Heinz Stockinger Outreach & Education Manager EU DataGrid project CERN (European Organization for Nuclear Research) Grid Technology Workshop, Islamabad, Pakistan, 20 October
More informationHPC learning using Cloud infrastructure
HPC learning using Cloud infrastructure Florin MANAILA IT Architect florin.manaila@ro.ibm.com Cluj-Napoca 16 March, 2010 Agenda 1. Leveraging Cloud model 2. HPC on Cloud 3. Recent projects - FutureGRID
More informationConference The Data Challenges of the LHC. Reda Tafirout, TRIUMF
Conference 2017 The Data Challenges of the LHC Reda Tafirout, TRIUMF Outline LHC Science goals, tools and data Worldwide LHC Computing Grid Collaboration & Scale Key challenges Networking ATLAS experiment
More informationDistributed ASCI Supercomputer DAS-1 DAS-2 DAS-3 DAS-4 DAS-5
Distributed ASCI Supercomputer DAS-1 DAS-2 DAS-3 DAS-4 DAS-5 Paper IEEE Computer (May 2016) What is DAS? Distributed common infrastructure for Dutch Computer Science Distributed: multiple (4-6) clusters
More informationGrid Programming: Concepts and Challenges. Michael Rokitka CSE510B 10/2007
Grid Programming: Concepts and Challenges Michael Rokitka SUNY@Buffalo CSE510B 10/2007 Issues Due to Heterogeneous Hardware level Environment Different architectures, chipsets, execution speeds Software
More informationMPI versions. MPI History
MPI versions MPI History Standardization started (1992) MPI-1 completed (1.0) (May 1994) Clarifications (1.1) (June 1995) MPI-2 (started: 1995, finished: 1997) MPI-2 book 1999 MPICH 1.2.4 partial implemention
More informationAn Overview of Computational Science (Based on CSEP)
An Overview of Computational Science (Based on CSEP) Craig C. Douglas January 20-22, 2004 CS 521, Spring 2004 What Is Computational Science? There is no uniformly accepted definition!!! Ken Wilson s definition,
More informationBeyond Petascale. Roger Haskin Manager, Parallel File Systems IBM Almaden Research Center
Beyond Petascale Roger Haskin Manager, Parallel File Systems IBM Almaden Research Center GPFS Research and Development! GPFS product originated at IBM Almaden Research Laboratory! Research continues to
More informationEnzo-P / Cello. Scalable Adaptive Mesh Refinement for Astrophysics and Cosmology. San Diego Supercomputer Center. Department of Physics and Astronomy
Enzo-P / Cello Scalable Adaptive Mesh Refinement for Astrophysics and Cosmology James Bordner 1 Michael L. Norman 1 Brian O Shea 2 1 University of California, San Diego San Diego Supercomputer Center 2
More informationHigh Performance Computing from an EU perspective
High Performance Computing from an EU perspective DEISA PRACE Symposium 2010 Barcelona, 10 May 2010 Kostas Glinos European Commission - DG INFSO Head of Unit GÉANT & e-infrastructures 1 "The views expressed
More informationDRIVER Step One towards a Pan-European Digital Repository Infrastructure
DRIVER Step One towards a Pan-European Digital Repository Infrastructure Norbert Lossau Bielefeld University, Germany Scientific coordinator of the Project DRIVER, Funded by the European Commission Consultation
More informationMicrosoft SharePoint Server 2013 Plan, Configure & Manage
Microsoft SharePoint Server 2013 Plan, Configure & Manage Course 20331-20332B 5 Days Instructor-led, Hands on Course Information This five day instructor-led course omits the overlap and redundancy that
More informationThe FELIX project Sustainability. Bartosz Belter, PSNC
The FELIX project Sustainability Bartosz Belter, PSNC FIRE Board Meeting, Brussels.BE October 2, 2015 What is FELIX A federation is a union comprising a number of partially self-governing regions united
More informationHDFS Architecture. Gregory Kesden, CSE-291 (Storage Systems) Fall 2017
HDFS Architecture Gregory Kesden, CSE-291 (Storage Systems) Fall 2017 Based Upon: http://hadoop.apache.org/docs/r3.0.0-alpha1/hadoopproject-dist/hadoop-hdfs/hdfsdesign.html Assumptions At scale, hardware
More informationShaking-and-Baking on a Grid
Shaking-and-Baking on a Grid Russ Miller & Mark Green Center for Computational Research, SUNY-Buffalo Hauptman-Woodward Medical Inst NSF ITR ACI-02-04918 University at Buffalo The State University of New
More informationCompilers and Run-Time Systems for High-Performance Computing
Compilers and Run-Time Systems for High-Performance Computing Blurring the Distinction between Compile-Time and Run-Time Ken Kennedy Rice University http://www.cs.rice.edu/~ken/presentations/compilerruntime.pdf
More informationHigh Performance Computing
High Performance Computing Dheeraj Bhardwaj Department of Computer Science & Engineering Indian Institute of Technology, Delhi 110 016 India http://www.cse.iitd.ac.in/~dheerajb 1 Introduction Traditional
More informationA Scalable Adaptive Mesh Refinement Framework For Parallel Astrophysics Applications
A Scalable Adaptive Mesh Refinement Framework For Parallel Astrophysics Applications James Bordner, Michael L. Norman San Diego Supercomputer Center University of California, San Diego 15th SIAM Conference
More informationDistributed Computing: PVM, MPI, and MOSIX. Multiple Processor Systems. Dr. Shaaban. Judd E.N. Jenne
Distributed Computing: PVM, MPI, and MOSIX Multiple Processor Systems Dr. Shaaban Judd E.N. Jenne May 21, 1999 Abstract: Distributed computing is emerging as the preferred means of supporting parallel
More informationIOS: A Middleware for Decentralized Distributed Computing
IOS: A Middleware for Decentralized Distributed Computing Boleslaw Szymanski Kaoutar El Maghraoui, Carlos Varela Department of Computer Science Rensselaer Polytechnic Institute http://www.cs.rpi.edu/wwc
More informationIan Foster, CS554: Data-Intensive Computing
The advent of computation can be compared, in terms of the breadth and depth of its impact on research and scholarship, to the invention of writing and the development of modern mathematics. Ian Foster,
More informationCloud Computing Paradigms for Pleasingly Parallel Biomedical Applications
Cloud Computing Paradigms for Pleasingly Parallel Biomedical Applications Thilina Gunarathne, Tak-Lon Wu Judy Qiu, Geoffrey Fox School of Informatics, Pervasive Technology Institute Indiana University
More informationTITLE: PRE-REQUISITE THEORY. 1. Introduction to Hadoop. 2. Cluster. Implement sort algorithm and run it using HADOOP
TITLE: Implement sort algorithm and run it using HADOOP PRE-REQUISITE Preliminary knowledge of clusters and overview of Hadoop and its basic functionality. THEORY 1. Introduction to Hadoop The Apache Hadoop
More informationThe LHC Computing Grid
The LHC Computing Grid Gergely Debreczeni (CERN IT/Grid Deployment Group) The data factory of LHC 40 million collisions in each second After on-line triggers and selections, only 100 3-4 MB/event requires
More informationImproved Infrastructure Accessibility and Control with LSF for LS-DYNA
4 th European LS-DYNA Users Conference LS-DYNA Environment I Improved Infrastructure Accessibility and Control with LSF for LS-DYNA Author: Bernhard Schott Christof Westhues Platform Computing GmbH, Ratingen,
More informationMitigating Risk of Data Loss in Preservation Environments
Storage Resource Broker Mitigating Risk of Data Loss in Preservation Environments Reagan W. Moore San Diego Supercomputer Center Joseph JaJa University of Maryland Robert Chadduck National Archives and
More informationFrom Physics Model to Results: An Optimizing Framework for Cross-Architecture Code Generation
From Physics Model to Results: An Optimizing Framework for Cross-Architecture Code Generation Erik Schnetter, Perimeter Institute with M. Blazewicz, I. Hinder, D. Koppelman, S. Brandt, M. Ciznicki, M.
More informationCustomer Success Story Los Alamos National Laboratory
Customer Success Story Los Alamos National Laboratory Panasas High Performance Storage Powers the First Petaflop Supercomputer at Los Alamos National Laboratory Case Study June 2010 Highlights First Petaflop
More informationHigh Performance Computing Course Notes Grid Computing I
High Performance Computing Course Notes 2008-2009 2009 Grid Computing I Resource Demands Even as computer power, data storage, and communication continue to improve exponentially, resource capacities are
More informationWorldwide Production Distributed Data Management at the LHC. Brian Bockelman MSST 2010, 4 May 2010
Worldwide Production Distributed Data Management at the LHC Brian Bockelman MSST 2010, 4 May 2010 At the LHC http://op-webtools.web.cern.ch/opwebtools/vistar/vistars.php?usr=lhc1 Gratuitous detector pictures:
More informationHOW SDN AND NFV EXTEND THE ADOPTION AND CAPABILITIES OF SELF-ORGANIZING NETWORKS (SON)
WHITE PAPER HOW SDN AND NFV EXTEND THE ADOPTION AND CAPABILITIES OF SELF-ORGANIZING NETWORKS (SON) WHAT WILL YOU LEARN? n SON in the radio access network n Adoption of SON solutions typical use cases n
More informationMOHA: Many-Task Computing Framework on Hadoop
Apache: Big Data North America 2017 @ Miami MOHA: Many-Task Computing Framework on Hadoop Soonwook Hwang Korea Institute of Science and Technology Information May 18, 2017 Table of Contents Introduction
More informationBİL 542 Parallel Computing
BİL 542 Parallel Computing 1 Chapter 1 Parallel Programming 2 Why Use Parallel Computing? Main Reasons: Save time and/or money: In theory, throwing more resources at a task will shorten its time to completion,
More informationGrid Computing Systems: A Survey and Taxonomy
Grid Computing Systems: A Survey and Taxonomy Material for this lecture from: A Survey and Taxonomy of Resource Management Systems for Grid Computing Systems, K. Krauter, R. Buyya, M. Maheswaran, CS Technical
More informationMPI History. MPI versions MPI-2 MPICH2
MPI versions MPI History Standardization started (1992) MPI-1 completed (1.0) (May 1994) Clarifications (1.1) (June 1995) MPI-2 (started: 1995, finished: 1997) MPI-2 book 1999 MPICH 1.2.4 partial implemention
More informationWhatÕs New in the Message-Passing Toolkit
WhatÕs New in the Message-Passing Toolkit Karl Feind, Message-passing Toolkit Engineering Team, SGI ABSTRACT: SGI message-passing software has been enhanced in the past year to support larger Origin 2
More informationApplications and Tools for High-Performance- Computing in Wide-Area Networks
Applications and Tools for High-Performance- Computing in Wide-Area Networks Dr.Alfred Geiger High-Performance Computing-Center Stuttgart (HLRS) geiger@hlrs.de Distributed Computing Scenarios Tools for
More informationNUIT Tech Talk Topics in Research Computing: XSEDE and Northwestern University Campus Champions
NUIT Tech Talk Topics in Research Computing: XSEDE and Northwestern University Campus Champions Pradeep Sivakumar pradeep-sivakumar@northwestern.edu Contents What is XSEDE? Introduction Who uses XSEDE?
More informationReal Parallel Computers
Real Parallel Computers Modular data centers Background Information Recent trends in the marketplace of high performance computing Strohmaier, Dongarra, Meuer, Simon Parallel Computing 2005 Short history
More informationNetworking European Digital Repositories
Networking European Digital Repositories What to Network? Researchers generate knowledge This is going to become an amazing paper I only hope I will be able to access it Knowledge is wrapped in publications
More informationGridSphere s Grid Portlets
COMPUTATIONAL METHODS IN SCIENCE AND TECHNOLOGY 12(1), 89-97 (2006) GridSphere s Grid Portlets Michael Russell 1, Jason Novotny 2, Oliver Wehrens 3 1 Max-Planck-Institut für Gravitationsphysik, Albert-Einstein-Institut,
More informationScaling Internet TV Content Delivery ALEX GUTARIN DIRECTOR OF ENGINEERING, NETFLIX
Scaling Internet TV Content Delivery ALEX GUTARIN DIRECTOR OF ENGINEERING, NETFLIX Inventing Internet TV Available in more than 190 countries 104+ million subscribers Lots of Streaming == Lots of Traffic
More informationIntroduction to Data Management for Ocean Science Research
Introduction to Data Management for Ocean Science Research Cyndy Chandler Biological and Chemical Oceanography Data Management Office 12 November 2009 Ocean Acidification Short Course Woods Hole, MA USA
More informationNetworking European Digital Repositories
Networking European Digital Repositories What to Network? Researchers generate knowledge This is going to become an amazing paper I only hope I will be able to access it Knowledge is wrapped in publications
More informationApplication Projects in VIOLA
Application Projects in VIOLA The integration of applications with high communication demands is a crucial element of the vertically integrated approach of VIOLA. These demands include guaranteed high
More informationStorage and I/O requirements of the LHC experiments
Storage and I/O requirements of the LHC experiments Sverre Jarp CERN openlab, IT Dept where the Web was born 22 June 2006 OpenFabrics Workshop, Paris 1 Briefly about CERN 22 June 2006 OpenFabrics Workshop,
More informationApplication Example Running on Top of GPI-Space Integrating D/C
Application Example Running on Top of GPI-Space Integrating D/C Tiberiu Rotaru Fraunhofer ITWM This project is funded from the European Union s Horizon 2020 Research and Innovation programme under Grant
More informationN. Marusov, I. Semenov
GRID TECHNOLOGY FOR CONTROLLED FUSION: CONCEPTION OF THE UNIFIED CYBERSPACE AND ITER DATA MANAGEMENT N. Marusov, I. Semenov Project Center ITER (ITER Russian Domestic Agency N.Marusov@ITERRF.RU) Challenges
More informationAdvanced School in High Performance and GRID Computing November Introduction to Grid computing.
1967-14 Advanced School in High Performance and GRID Computing 3-14 November 2008 Introduction to Grid computing. TAFFONI Giuliano Osservatorio Astronomico di Trieste/INAF Via G.B. Tiepolo 11 34131 Trieste
More informationEnzo-P / Cello. Formation of the First Galaxies. San Diego Supercomputer Center. Department of Physics and Astronomy
Enzo-P / Cello Formation of the First Galaxies James Bordner 1 Michael L. Norman 1 Brian O Shea 2 1 University of California, San Diego San Diego Supercomputer Center 2 Michigan State University Department
More informationThe Virtual Observatory and the IVOA
The Virtual Observatory and the IVOA The Virtual Observatory Emergence of the Virtual Observatory concept by 2000 Concerns about the data avalanche, with in mind in particular very large surveys such as
More informationGRIDS INTRODUCTION TO GRID INFRASTRUCTURES. Fabrizio Gagliardi
GRIDS INTRODUCTION TO GRID INFRASTRUCTURES Fabrizio Gagliardi Dr. Fabrizio Gagliardi is the leader of the EU DataGrid project and designated director of the proposed EGEE (Enabling Grids for E-science
More informationLHC and LSST Use Cases
LHC and LSST Use Cases Depots Network 0 100 200 300 A B C Paul Sheldon & Alan Tackett Vanderbilt University LHC Data Movement and Placement n Model must evolve n Was: Hierarchical, strategic pre- placement
More informationIntroducing Overdecomposition to Existing Applications: PlasComCM and AMPI
Introducing Overdecomposition to Existing Applications: PlasComCM and AMPI Sam White Parallel Programming Lab UIUC 1 Introduction How to enable Overdecomposition, Asynchrony, and Migratability in existing
More informationDataONE: Open Persistent Access to Earth Observational Data
Open Persistent Access to al Robert J. Sandusky, UIC University of Illinois at Chicago The Net Partners Update: ONE and the Conservancy December 14, 2009 Outline NSF s Net Program ONE Introduction Motivating
More information