Distributed Monte Carlo Production for

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Distributed Monte Carlo Production for"

Transcription

1 Distributed Monte Carlo Production for Joel Snow Langston University DOE Review March 2011

2 Outline Introduction FNAL SAM SAMGrid Interoperability with OSG and LCG Production System Production Results LUHEP Computing Summary DOE Review March 2011 Joel Snow Langston University 2

3 Introduction Covers my tenure as MC production coordinator Simulation data (MC) crucial to physics analysis Tevatron luminosity and hence raw data volume is at record levels Challenge for analysts and production Personnel & computing resources migrating to LHC experiments DZero strategy Increase automation Leverage resources and support DOE Review March 2011 Joel Snow Langston University 3

4 Evolution Mature experiment, but nimble history of adopting innovative technologies distributed data handling - SAM early adopter of the grid for production - SAMGrid significant investment in these technologies Grid technology allows opportunistic usage DZero can mix traditional dedicated and opportunistic resources Grid interoperability Leverages resources and support, reduces personnel needs per CPU hour DOE Review March 2011 Joel Snow Langston University 4

5 Sequential data Access via Metadata Fermilab system first used by DZero SAM distributed data handling system predates grid Set of servers working together to store and retrieve files and metadata Permanent storage and local disk caches Database tracks location, metadata of files, job processing history Delivers files to jobs (using GridFTP over WAN), provides job submission capabilities DOE Review March 2011 Joel Snow Langston University 5

6 SAMGrid Fermilab developed grid first used by DZero for global MC production in 2004 SAMGrid = SAM + Job and Information Management (JIM) components Provides the user with transparent remote job submission, data processing and status monitoring. VDT based (Globus + Condor) Logically consists of Multiple execution sites Resource selector Multiple Job Submission (Scheduler) sites Multiple Clients (User Interface) to Submission site. DOE Review March 2011 Joel Snow Langston University 6

7 SAMGrid Interoperability As Open Science Grid (OSG) and LHC Computing Grid (LCG) became operational it was desirable to leverage these resources for DZero FNAL and DZero developed and deployed SAMGrid interoperability with both LCG and OSG resources Execution site acts as a Forwarding node packages SAMGrid jobs for OSG/LCG job submission via Condor-G DOE Review March 2011 Joel Snow Langston University 7

8 Consolidation, Automation, Exploitation SAMGrid sites require operational manpower and expert support People power and FNAL support migrating to LHC experiments Increase automation - Automc Reduce number of SAMGrid sites, increase use of OSG and LCG comes with support provides opportunistic job slots DOE Review March 2011 Joel Snow Langston University 8

9 MC production gets work from the SAM Request System Physics groups' MC requests are parametrized and prioritized as a Python object Production System DOE Review March 2011 Joel Snow Langston University 9

10 Automatic Monte Carlo Request Processing Developed Automc System in use at FNAL Handles official DZero MC production at all but 2 sites From approved request to final data storage Easy to use minimizes manpower needs Site independent deploy for any grid site (SAMGrid, OSG, LCG) capable of managing many sites Handle recovery of common failures Integrates with existing MC request priority protocol DOE Review March 2011 Joel Snow Langston University 10

11 AutoMC Monitoring Running at FNAL & managing production at 39 sites DOE Review March 2011 Joel Snow Langston University 11

12 Production System Resources MC production uses a variety of dedicated and opportunistic resources on 4 continents Non-grid site at ccin2p3 Lyon (FR) very productive, flexible Native Samgrid sites: FZU (CZ), GridKa (DE), LUHEP (US), USTC (CN) LCG resources: CE's, SE's, and Samgrid-LCG infrastructure in FR, UK, NL OSG resources: OSG resources: CE's, SE's, and Samgrid-OSG infrastructure in US DOE Review March 2011 Joel Snow Langston University 12

13 MC Production Results Looking back at the last 30 days Averaging 5.8M events per day and totaling 172.8M events in 30 days DOE Review March 2011 Joel Snow Langston University 13

14 MC Production Results Looking back at the last year (2010/02/ /02/14) cumulative since September Averaging 49M events per week and totaling 2.6B events in a year DOE Review March 2011 Joel Snow Langston University 14

15 MC Production Results Looking back at the last year by production segment 52 week averages per week (2010/02/ /02/14) Non-grid: 19.8M, OSG: 11.4M, Samgrid: 12.6M, LCG: 4.9M DOE Review March 2011 Joel Snow Langston University 15

16 MC Production Results Looking back at the last year by production segment Cumulative since September, 2005 Production Last Year By Segment Nongrid OSG Samgrid LCG 52 week totals (2010/02/ /02/14) Non-grid: 1041M, OSG: 596M, Samgrid: 658M, LCG: 257M 40.8% 23.3% 25.8% 10.1% DOE Review March 2011 Joel Snow Langston University 16

17 MC Production Geographic Events Last Year: Distribution 1.1% 22.5% Europe 1925M N. America 574M 0.9% Asia 29M S. America 24M 75.4% (2010/02/ /02/14) Europe S. America N. America Asia DOE Review March 2011 Joel Snow Langston University 17

18 MC Production Results Looking back at the last 5.5 years (2005/09/ /02/14) cumulative since September Averaging 19.2M events per week and totaling 2.82B events DOE Review March 2011 Joel Snow Langston University 18

19 MC Production Results Looking back at the last 5.5 years by production segment 5.5 year averages per week (2005/09/ /02/14) Non-grid: 8.0M, OSG: 4.8M, Samgrid: 5.3M, LCG: 1.1M DOE Review March 2011 Joel Snow Langston University 19

20 MC Production Results Looking back at the last 5.5 years by production segment Cumulative since September, 2005 Production Last Year By Segment Nongrid OSG Samgrid LCG 5.5 year totals (2005/09/ /02/14) Non-grid: 2.26B, OSG: 1.37B, Samgrid: 1.51B, LCG: 306M 41.5% 25.2% 27.7% 5.6% DOE Review March 2011 Joel Snow Langston University 20

21 Production Results Last 7 Years DZero MC Production in Millions of Events per year ending 12/26 DZero MC Production in Millions of Events LCG OSG SAMGrid Non-Grid Year Total Non-Grid SAMGrid OSG LCG DOE Review March 2011 Joel Snow Langston University 21

22 Production Results Last 7 Years DZero MC Production in Terabytes of Data per year ending 12/26 DZero MC Production in Terabytes of Data LCG OSG SAMGrid Non-Grid Year Total Non-Grid SAMGrid OSG LCG DOE Review March 2011 Joel Snow Langston University 22

23 OU DZero MC Production 2005/09/ /02/14 OUHEP produced 306 M events and 28.4 TB data Last year OUHEP produced 139 M events and 14.0 TB data 2010/02/ /02/14 Cumulative since Sept DOE Review March 2011 Joel Snow Langston University 23

24 LU DZero MC Production 2005/09/ /02/14 LUHEP produced 15.5 M events and 1.36 TB data Last year LUHEP produced 4.6 M events and 450 GB data 2010/02/ /02/14 Cumulative since Sept DOE Review March 2011 Joel Snow Langston University 24

25 LUHEP Computing 2 grid enabled clusters both producing DØ MC Old Samgrid cluster- 12 job slots New OSG cluster - 12 job slots with small associated SE used as DØ cache DOE Review March 2011 Joel Snow Langston University 25

26 Condor Q's at LUHEP SAMGrid Last Year OSG DOE Review March 2011 Joel Snow Langston University 26

27 Summary DZero 's early deployment of grid technology and automation has dramatically increased MC production First deployment SAM distributed data handling system Early SAMGrid deployment Use of OSG and LCG resources through interoperability with SAMGrid First opportunistic usage of OSG Storage Elements Automated MC production system Anticipate adequate MC through the last analysis DOE Review March 2011 Joel Snow Langston University 27

Fermilab FERMILAB-Conf-03/207-E August 2003

Fermilab FERMILAB-Conf-03/207-E August 2003 Fermilab FERMILAB-Conf-03/207-E August 2003 CHEP 2003, UCSD, La Jolla, California, March 24 28,2003 1 DØ Data Handling Operational Experience K. Yip Brookhaven National Laboratory, Upton, NY 11973 A. Baranovski,

More information

Grid Computing Activities at KIT

Grid Computing Activities at KIT Grid Computing Activities at KIT Meeting between NCP and KIT, 21.09.2015 Manuel Giffels Karlsruhe Institute of Technology Institute of Experimental Nuclear Physics & Steinbuch Center for Computing Courtesy

More information

High Throughput WAN Data Transfer with Hadoop-based Storage

High Throughput WAN Data Transfer with Hadoop-based Storage High Throughput WAN Data Transfer with Hadoop-based Storage A Amin 2, B Bockelman 4, J Letts 1, T Levshina 3, T Martin 1, H Pi 1, I Sfiligoi 1, M Thomas 2, F Wuerthwein 1 1 University of California, San

More information

The LHC Computing Grid

The LHC Computing Grid The LHC Computing Grid Visit of Finnish IT Centre for Science CSC Board Members Finland Tuesday 19 th May 2009 Frédéric Hemmer IT Department Head The LHC and Detectors Outline Computing Challenges Current

More information

UK Tier-2 site evolution for ATLAS. Alastair Dewhurst

UK Tier-2 site evolution for ATLAS. Alastair Dewhurst UK Tier-2 site evolution for ATLAS Alastair Dewhurst Introduction My understanding is that GridPP funding is only part of the story when it comes to paying for a Tier 2 site. Each site is unique. Aim to

More information

Outline. ASP 2012 Grid School

Outline. ASP 2012 Grid School Distributed Storage Rob Quick Indiana University Slides courtesy of Derek Weitzel University of Nebraska Lincoln Outline Storage Patterns in Grid Applications Storage

More information

The Grid: Processing the Data from the World s Largest Scientific Machine

The Grid: Processing the Data from the World s Largest Scientific Machine The Grid: Processing the Data from the World s Largest Scientific Machine 10th Topical Seminar On Innovative Particle and Radiation Detectors Siena, 1-5 October 2006 Patricia Méndez Lorenzo (IT-PSS/ED),

More information

Was ist dran an einer spezialisierten Data Warehousing platform?

Was ist dran an einer spezialisierten Data Warehousing platform? Was ist dran an einer spezialisierten Data Warehousing platform? Hermann Bär Oracle USA Redwood Shores, CA Schlüsselworte Data warehousing, Exadata, specialized hardware proprietary hardware Introduction

More information

Installation of CMSSW in the Grid DESY Computing Seminar May 17th, 2010 Wolf Behrenhoff, Christoph Wissing

Installation of CMSSW in the Grid DESY Computing Seminar May 17th, 2010 Wolf Behrenhoff, Christoph Wissing Installation of CMSSW in the Grid DESY Computing Seminar May 17th, 2010 Wolf Behrenhoff, Christoph Wissing Wolf Behrenhoff, Christoph Wissing DESY Computing Seminar May 17th, 2010 Page 1 Installation of

More information

Geographical failover for the EGEE-WLCG Grid collaboration tools. CHEP 2007 Victoria, Canada, 2-7 September. Enabling Grids for E-sciencE

Geographical failover for the EGEE-WLCG Grid collaboration tools. CHEP 2007 Victoria, Canada, 2-7 September. Enabling Grids for E-sciencE Geographical failover for the EGEE-WLCG Grid collaboration tools CHEP 2007 Victoria, Canada, 2-7 September Alessandro Cavalli, Alfredo Pagano (INFN/CNAF, Bologna, Italy) Cyril L'Orphelin, Gilles Mathieu,

More information

Workload Management. Stefano Lacaprara. CMS Physics Week, FNAL, 12/16 April Department of Physics INFN and University of Padova

Workload Management. Stefano Lacaprara. CMS Physics Week, FNAL, 12/16 April Department of Physics INFN and University of Padova Workload Management Stefano Lacaprara Department of Physics INFN and University of Padova CMS Physics Week, FNAL, 12/16 April 2005 Outline 1 Workload Management: the CMS way General Architecture Present

More information

Conference The Data Challenges of the LHC. Reda Tafirout, TRIUMF

Conference The Data Challenges of the LHC. Reda Tafirout, TRIUMF Conference 2017 The Data Challenges of the LHC Reda Tafirout, TRIUMF Outline LHC Science goals, tools and data Worldwide LHC Computing Grid Collaboration & Scale Key challenges Networking ATLAS experiment

More information

Data Transfers Between LHC Grid Sites Dorian Kcira

Data Transfers Between LHC Grid Sites Dorian Kcira Data Transfers Between LHC Grid Sites Dorian Kcira dkcira@caltech.edu Caltech High Energy Physics Group hep.caltech.edu/cms CERN Site: LHC and the Experiments Large Hadron Collider 27 km circumference

More information

AMAZON S3 FOR SCIENCE GRIDS: A VIABLE SOLUTION?

AMAZON S3 FOR SCIENCE GRIDS: A VIABLE SOLUTION? AMAZON S3 FOR SCIENCE GRIDS: A VIABLE SOLUTION? Mayur Palankar and Adriana Iamnitchi University of South Florida Matei Ripeanu University of British Columbia Simson Garfinkel Harvard University Amazon

More information

Deploying virtualisation in a production grid

Deploying virtualisation in a production grid Deploying virtualisation in a production grid Stephen Childs Trinity College Dublin & Grid-Ireland TERENA NRENs and Grids workshop 2 nd September 2008 www.eu-egee.org EGEE and glite are registered trademarks

More information

Computing for LHC in Germany

Computing for LHC in Germany 1 Computing for LHC in Germany Günter Quast Universität Karlsruhe (TH) Meeting with RECFA Berlin, October 5th 2007 WLCG Tier1 & Tier2 Additional resources for data analysis - HGF ''Physics at the Terascale''

More information

Overview of ATLAS PanDA Workload Management

Overview of ATLAS PanDA Workload Management Overview of ATLAS PanDA Workload Management T. Maeno 1, K. De 2, T. Wenaus 1, P. Nilsson 2, G. A. Stewart 3, R. Walker 4, A. Stradling 2, J. Caballero 1, M. Potekhin 1, D. Smith 5, for The ATLAS Collaboration

More information

Computing. DOE Program Review SLAC. Rainer Bartoldus. Breakout Session 3 June BaBar Deputy Computing Coordinator

Computing. DOE Program Review SLAC. Rainer Bartoldus. Breakout Session 3 June BaBar Deputy Computing Coordinator Computing DOE Program Review SLAC Breakout Session 3 June 2004 Rainer Bartoldus BaBar Deputy Computing Coordinator 1 Outline The New Computing Model (CM2) New Kanga/ROOT event store, new Analysis Model,

More information

Improving Enterprise Search at Microsoft

Improving Enterprise Search at Microsoft Improving Enterprise Search at Microsoft How FAST Search Server 2010 for SharePoint Powers Worldwide Intranet Search at Microsoft Technical White Paper Published: February 2012 The following content may

More information

GRIDS INTRODUCTION TO GRID INFRASTRUCTURES. Fabrizio Gagliardi

GRIDS INTRODUCTION TO GRID INFRASTRUCTURES. Fabrizio Gagliardi GRIDS INTRODUCTION TO GRID INFRASTRUCTURES Fabrizio Gagliardi Dr. Fabrizio Gagliardi is the leader of the EU DataGrid project and designated director of the proposed EGEE (Enabling Grids for E-science

More information

Batch Services at CERN: Status and Future Evolution

Batch Services at CERN: Status and Future Evolution Batch Services at CERN: Status and Future Evolution Helge Meinhard, CERN-IT Platform and Engineering Services Group Leader HTCondor Week 20 May 2015 20-May-2015 CERN batch status and evolution - Helge

More information

Monitoring System for the GRID Monte Carlo Mass Production in the H1 Experiment at DESY

Monitoring System for the GRID Monte Carlo Mass Production in the H1 Experiment at DESY Journal of Physics: Conference Series OPEN ACCESS Monitoring System for the GRID Monte Carlo Mass Production in the H1 Experiment at DESY To cite this article: Elena Bystritskaya et al 2014 J. Phys.: Conf.

More information

EGEE and Interoperation

EGEE and Interoperation EGEE and Interoperation Laurence Field CERN-IT-GD ISGC 2008 www.eu-egee.org EGEE and glite are registered trademarks Overview The grid problem definition GLite and EGEE The interoperability problem The

More information

The ATLAS Production System

The ATLAS Production System The ATLAS MC and Data Rodney Walker Ludwig Maximilians Universität Munich 2nd Feb, 2009 / DESY Computing Seminar Outline 1 Monte Carlo Production Data 2 3 MC Production Data MC Production Data Group and

More information

The LHC Computing Grid

The LHC Computing Grid The LHC Computing Grid Gergely Debreczeni (CERN IT/Grid Deployment Group) The data factory of LHC 40 million collisions in each second After on-line triggers and selections, only 100 3-4 MB/event requires

More information

ELFms industrialisation plans

ELFms industrialisation plans ELFms industrialisation plans CERN openlab workshop 13 June 2005 German Cancio CERN IT/FIO http://cern.ch/elfms ELFms industrialisation plans, 13/6/05 Outline Background What is ELFms Collaboration with

More information

IEPSAS-Kosice: experiences in running LCG site

IEPSAS-Kosice: experiences in running LCG site IEPSAS-Kosice: experiences in running LCG site Marian Babik 1, Dusan Bruncko 2, Tomas Daranyi 1, Ladislav Hluchy 1 and Pavol Strizenec 2 1 Department of Parallel and Distributed Computing, Institute of

More information

einfrastructures Concertation Event

einfrastructures Concertation Event einfrastructures Concertation Event Steve Crumb, Executive Director December 5, 2007 OGF Vision & Mission Our Vision: The Open Grid Forum accelerates grid adoption to enable scientific discovery and business

More information

SLATE. Services Layer at the Edge. First Meeting of the National Research Platform Montana State University August 7-8, 2017

SLATE. Services Layer at the Edge. First Meeting of the National Research Platform Montana State University August 7-8, 2017 SLATE Services Layer at the Edge Rob Gardner University of Chicago Shawn McKee University of Michigan Joe Breen University of Utah First Meeting of the National Research Platform Montana State University

More information

Data Staging: Moving large amounts of data around, and moving it close to compute resources

Data Staging: Moving large amounts of data around, and moving it close to compute resources Data Staging: Moving large amounts of data around, and moving it close to compute resources PRACE advanced training course on Data Staging and Data Movement Helsinki, September 10 th 2013 Claudio Cacciari

More information

ISTITUTO NAZIONALE DI FISICA NUCLEARE

ISTITUTO NAZIONALE DI FISICA NUCLEARE ISTITUTO NAZIONALE DI FISICA NUCLEARE Sezione di Perugia INFN/TC-05/10 July 4, 2005 DESIGN, IMPLEMENTATION AND CONFIGURATION OF A GRID SITE WITH A PRIVATE NETWORK ARCHITECTURE Leonello Servoli 1,2!, Mirko

More information

The LCG 3D Project. Maria Girone, CERN. The 23rd Open Grid Forum - OGF23 4th June 2008, Barcelona. CERN IT Department CH-1211 Genève 23 Switzerland

The LCG 3D Project. Maria Girone, CERN. The 23rd Open Grid Forum - OGF23 4th June 2008, Barcelona. CERN IT Department CH-1211 Genève 23 Switzerland The LCG 3D Project Maria Girone, CERN The rd Open Grid Forum - OGF 4th June 2008, Barcelona Outline Introduction The Distributed Database (3D) Project Streams Replication Technology and Performance Availability

More information

Evolution of the ATLAS PanDA Workload Management System for Exascale Computational Science

Evolution of the ATLAS PanDA Workload Management System for Exascale Computational Science Evolution of the ATLAS PanDA Workload Management System for Exascale Computational Science T. Maeno, K. De, A. Klimentov, P. Nilsson, D. Oleynik, S. Panitkin, A. Petrosyan, J. Schovancova, A. Vaniachine,

More information

ATLAS Distributed Computing Experience and Performance During the LHC Run-2

ATLAS Distributed Computing Experience and Performance During the LHC Run-2 ATLAS Distributed Computing Experience and Performance During the LHC Run-2 A Filipčič 1 for the ATLAS Collaboration 1 Jozef Stefan Institute, Jamova 39, 1000 Ljubljana, Slovenia E-mail: andrej.filipcic@ijs.si

More information

30 Nov Dec Advanced School in High Performance and GRID Computing Concepts and Applications, ICTP, Trieste, Italy

30 Nov Dec Advanced School in High Performance and GRID Computing Concepts and Applications, ICTP, Trieste, Italy Advanced School in High Performance and GRID Computing Concepts and Applications, ICTP, Trieste, Italy Why the Grid? Science is becoming increasingly digital and needs to deal with increasing amounts of

More information

LCG MCDB Knowledge Base of Monte Carlo Simulated Events

LCG MCDB Knowledge Base of Monte Carlo Simulated Events LCG MCDB Knowledge Base of Monte Carlo Simulated Events ACAT 2007, Amsterdam Sergey Belov, JINR, Dubna belov@jinr.ru http://mcdb.cern.ch LCG MCDB group: S. Belov, JINR L. Dudko, SINP MSU W. Pokorski, CERN

More information

Grid Interoperation and Regional Collaboration

Grid Interoperation and Regional Collaboration Grid Interoperation and Regional Collaboration Eric Yen ASGC Academia Sinica Taiwan 23 Jan. 2006 Dreams of Grid Computing Global collaboration across administrative domains by sharing of people, resources,

More information

802.11ac FREQUENTLY ASKED QUESTIONS. May 2012

802.11ac FREQUENTLY ASKED QUESTIONS. May 2012 802.11ac FREQUENTLY ASKED QUESTIONS May 2012 Table of Contents General Questions:... 3 1. What is 802.11ac?... 3 2. When will 802.11ac be ratified into a standard?... 3 5. Will 802.11ac come out before

More information

A User-level Secure Grid File System

A User-level Secure Grid File System A User-level Secure Grid File System Ming Zhao, Renato J. Figueiredo Advanced Computing and Information Systems (ACIS) Electrical and Computer Engineering University of Florida {ming, renato}@acis.ufl.edu

More information

FLASHARRAY//M Business and IT Transformation in 3U

FLASHARRAY//M Business and IT Transformation in 3U FLASHARRAY//M Business and IT Transformation in 3U TRANSFORM IT Who knew that moving to all-flash storage could help reduce the cost of IT? FlashArray//m makes server and workload investments more productive,

More information

ARC integration for CMS

ARC integration for CMS ARC integration for CMS ARC integration for CMS Erik Edelmann 2, Laurence Field 3, Jaime Frey 4, Michael Grønager 2, Kalle Happonen 1, Daniel Johansson 2, Josva Kleist 2, Jukka Klem 1, Jesper Koivumäki

More information

AMGA metadata catalogue system

AMGA metadata catalogue system AMGA metadata catalogue system Hurng-Chun Lee ACGrid School, Hanoi, Vietnam www.eu-egee.org EGEE and glite are registered trademarks Outline AMGA overview AMGA Background and Motivation for AMGA Interface,

More information

Data Management for the World s Largest Machine

Data Management for the World s Largest Machine Data Management for the World s Largest Machine Sigve Haug 1, Farid Ould-Saada 2, Katarina Pajchel 2, and Alexander L. Read 2 1 Laboratory for High Energy Physics, University of Bern, Sidlerstrasse 5,

More information

Surveillance Dell EMC Storage with Verint Nextiva

Surveillance Dell EMC Storage with Verint Nextiva Surveillance Dell EMC Storage with Verint Nextiva Sizing Guide H14897 REV 1.3 Copyright 2016-2017 Dell Inc. or its subsidiaries. All rights reserved. Published September 2017 Dell believes the information

More information

Data Staging: Moving large amounts of data around, and moving it close to compute resources

Data Staging: Moving large amounts of data around, and moving it close to compute resources Data Staging: Moving large amounts of data around, and moving it close to compute resources Digital Preserva-on Advanced Prac--oner Course Glasgow, July 19 th 2013 c.cacciari@cineca.it Definition Starting

More information

A time machine for the OCDB

A time machine for the OCDB A time machine for the OCDB Dario Berzano ALICE Offline Week - July 19, 2017 OCDB source: AliEn Primary source of OCDB Calibration data in multiple ROOT files One XML file mapping run ranges to years Accessed

More information

Comparison: Perforce and Microsoft Visual SourceSafe. Perforce VSS

Comparison: Perforce and Microsoft Visual SourceSafe. Perforce VSS Comparison: Perforce and Microsoft Visual SourceSafe Perforce & VSS Overview This document compares Perforce version 2008.2 with Microsoft Visual SourceSafe (VSS) 2005, and contrasts their significant

More information

Monitoring the Usage of the ZEUS Analysis Grid

Monitoring the Usage of the ZEUS Analysis Grid Monitoring the Usage of the ZEUS Analysis Grid Stefanos Leontsinis September 9, 2006 Summer Student Programme 2006 DESY Hamburg Supervisor Dr. Hartmut Stadie National Technical

More information

FIVE BEST PRACTICES FOR ENSURING A SUCCESSFUL SQL SERVER MIGRATION

FIVE BEST PRACTICES FOR ENSURING A SUCCESSFUL SQL SERVER MIGRATION FIVE BEST PRACTICES FOR ENSURING A SUCCESSFUL SQL SERVER MIGRATION The process of planning and executing SQL Server migrations can be complex and risk-prone. This is a case where the right approach and

More information

Multiple Broker Support by Grid Portals* Extended Abstract

Multiple Broker Support by Grid Portals* Extended Abstract 1. Introduction Multiple Broker Support by Grid Portals* Extended Abstract Attila Kertesz 1,3, Zoltan Farkas 1,4, Peter Kacsuk 1,4, Tamas Kiss 2,4 1 MTA SZTAKI Computer and Automation Research Institute

More information

Monitoring tools in EGEE

Monitoring tools in EGEE Monitoring tools in EGEE Piotr Nyczyk CERN IT/GD Joint OSG and EGEE Operations Workshop - 3 Abingdon, 27-29 September 2005 www.eu-egee.org Kaleidoscope of monitoring tools Monitoring for operations Covered

More information

Grid Computing a new tool for science

Grid Computing a new tool for science Grid Computing a new tool for science CERN, the European Organization for Nuclear Research Dr. Wolfgang von Rüden Wolfgang von Rüden, CERN, IT Department Grid Computing July 2006 CERN stands for over 50

More information

where the Web was born Experience of Adding New Architectures to the LCG Production Environment

where the Web was born Experience of Adding New Architectures to the LCG Production Environment where the Web was born Experience of Adding New Architectures to the LCG Production Environment Andreas Unterkircher, openlab fellow Sverre Jarp, CTO CERN openlab Industrializing the Grid openlab Workshop

More information

UNIT IV PROGRAMMING MODEL. Open source grid middleware packages - Globus Toolkit (GT4) Architecture, Configuration - Usage of Globus

UNIT IV PROGRAMMING MODEL. Open source grid middleware packages - Globus Toolkit (GT4) Architecture, Configuration - Usage of Globus UNIT IV PROGRAMMING MODEL Open source grid middleware packages - Globus Toolkit (GT4) Architecture, Configuration - Usage of Globus Globus: One of the most influential Grid middleware projects is the Globus

More information

Grids and Security. Ian Neilson Grid Deployment Group CERN. TF-CSIRT London 27 Jan

Grids and Security. Ian Neilson Grid Deployment Group CERN. TF-CSIRT London 27 Jan Grids and Security Ian Neilson Grid Deployment Group CERN TF-CSIRT London 27 Jan 2004-1 TOC Background Grids Grid Projects Some Technical Aspects The three or four A s Some Operational Aspects Security

More information

Six Sigma in the datacenter drives a zero-defects culture

Six Sigma in the datacenter drives a zero-defects culture Six Sigma in the datacenter drives a zero-defects culture Situation Like many IT organizations, Microsoft IT wants to keep its global infrastructure available at all times. Scope, scale, and an environment

More information

Globus Online: File Transfer Made Easy!

Globus Online: File Transfer Made Easy! Globus Online: File Transfer Made Easy! Matteo Lanati matteo.lanati@lrz.de Initiative for Globus in Europe Leibniz Supercomputing Centre Outline Introduction and acknowledgments Motivation Demo session

More information

Operational Aspects of Dealing with the Large BaBar Data Set

Operational Aspects of Dealing with the Large BaBar Data Set Operational Aspects of Dealing with the Large BaBar Data Set Tofigh Azemoon, Adil Hasan, Wilko Kröger, Artem Trunov SLAC Computing Services, Stanford, CA 94025, USA On Behalf of the BaBar Computing Group

More information

HPC learning using Cloud infrastructure

HPC learning using Cloud infrastructure HPC learning using Cloud infrastructure Florin MANAILA IT Architect florin.manaila@ro.ibm.com Cluj-Napoca 16 March, 2010 Agenda 1. Leveraging Cloud model 2. HPC on Cloud 3. Recent projects - FutureGRID

More information

Summary of the LHC Computing Review

Summary of the LHC Computing Review Summary of the LHC Computing Review http://lhc-computing-review-public.web.cern.ch John Harvey CERN/EP May 10 th, 2001 LHCb Collaboration Meeting The Scale Data taking rate : 50,100, 200 Hz (ALICE, ATLAS-CMS,

More information

e-infrastructures in FP7 INFO DAY - Paris

e-infrastructures in FP7 INFO DAY - Paris e-infrastructures in FP7 INFO DAY - Paris Carlos Morais Pires European Commission DG INFSO GÉANT & e-infrastructure Unit 1 Global challenges with high societal impact Big Science and the role of empowered

More information

Grid Compute Resources and Job Management

Grid Compute Resources and Job Management Grid Compute Resources and Job Management How do we access the grid? Command line with tools that you'll use Specialised applications Ex: Write a program to process images that sends data to run on the

More information

FlashArray//m. Business and IT Transformation in 3U. Transform Your Business. All-Flash Storage for Every Workload.

FlashArray//m. Business and IT Transformation in 3U. Transform Your Business. All-Flash Storage for Every Workload. Business and IT Transfmation in 3U Transfm IT. Who knew that moving to all-flash stage could help reduce the cost of IT? FlashArray//m makes server and wkload investments me productive, while lowering

More information

Configuring Short RPO with Actifio StreamSnap and Dedup-Async Replication

Configuring Short RPO with Actifio StreamSnap and Dedup-Async Replication CDS and Sky Tech Brief Configuring Short RPO with Actifio StreamSnap and Dedup-Async Replication Actifio recommends using Dedup-Async Replication (DAR) for RPO of 4 hours or more and using StreamSnap for

More information

Cloud Going Mainstream All Are Trying, Some Are Benefiting; Few Are Maximizing Value. An IDC InfoBrief, sponsored by Cisco September 2016

Cloud Going Mainstream All Are Trying, Some Are Benefiting; Few Are Maximizing Value. An IDC InfoBrief, sponsored by Cisco September 2016 All Are Trying, Some Are Benefiting; Few Are Maximizing Value September 2016 Executive Summary Cloud adoption has increased 61% from last year, with 73% pursuing a hybrid cloud strategy and on-premises

More information

Running Splunk on VxRack FLEX

Running Splunk on VxRack FLEX ESG Technical White Paper Running Splunk on VxRack FLEX Background As organizations continue to look for ways to modernize their infrastructures by delivering a cloud-like experience onpremises, hyperconverged

More information

Constant monitoring of multi-site network connectivity at the Tokyo Tier2 center

Constant monitoring of multi-site network connectivity at the Tokyo Tier2 center Constant monitoring of multi-site network connectivity at the Tokyo Tier2 center, T. Mashimo, N. Matsui, H. Matsunaga, H. Sakamoto, I. Ueda International Center for Elementary Particle Physics, The University

More information

Grid Computing. Olivier Dadoun LAL, Orsay. Introduction & Parachute method. Socle 2006 Clermont-Ferrand Orsay)

Grid Computing. Olivier Dadoun LAL, Orsay. Introduction & Parachute method. Socle 2006 Clermont-Ferrand Orsay) virtual organization Grid Computing Introduction & Parachute method Socle 2006 Clermont-Ferrand (@lal Orsay) Olivier Dadoun LAL, Orsay dadoun@lal.in2p3.fr www.dadoun.net October 2006 1 Contents Preamble

More information

The grid for LHC Data Analysis

The grid for LHC Data Analysis The grid for LHC Data Analysis ICAP 2006 Conference Chamonix 5 October 2006 Les Robertson - CERN LHC Computing Grid Project Leader The LHC Computing Challenges 1. Data After reduction by triggers and data

More information

Helix Nebula The Science Cloud

Helix Nebula The Science Cloud Helix Nebula The Science Cloud Title: D5.1 Evaluation of Initial Flagship Deployments Editor: Phil Evans Work Package: 5 Submission Date: 17 th June 2013 Distribution: Public Nature: Report 1 Log Table

More information

The Global Research Council

The Global Research Council The Global Research Council Preamble The worldwide growth of support for research has presented an opportunity for countries large and small to work in concert across national borders. Cooperation and

More information

Backup and archiving need not to create headaches new pain relievers are around

Backup and archiving need not to create headaches new pain relievers are around Backup and archiving need not to create headaches new pain relievers are around Frank Reichart Senior Director Product Marketing Storage Copyright 2012 FUJITSU Hot Spots in Data Protection 1 Copyright

More information

ISTQB Evolution. Gualtiero Bazzana ISTQB President

ISTQB Evolution. Gualtiero Bazzana ISTQB President ISTQB Evolution Gualtiero Bazzana ISTQB President Johannesburg- September 2016 Market trends the future The Sw testing market has a size of over 60B world-wide In accordance to Technavio Reports «Global

More information

FlexPod. The Journey to the Cloud. Technical Presentation. Presented Jointly by NetApp and Cisco

FlexPod. The Journey to the Cloud. Technical Presentation. Presented Jointly by NetApp and Cisco FlexPod The Journey to the Cloud Technical Presentation Presented Jointly by NetApp and Cisco Agenda Alliance Highlights Introducing FlexPod One Shared Vision and Journey FlexPod for the Oracle base base

More information

Connectivity Services, Autobahn and New Services

Connectivity Services, Autobahn and New Services Connectivity Services, Autobahn and New Services Domenico Vicinanza, DANTE EGEE 09, Barcelona, 21 st -25 th September 2009 Agenda Background GÉANT Connectivity services: GÉANT IP GÉANT Plus GÉANT Lambda

More information

FREE SCIENTIFIC COMPUTING

FREE SCIENTIFIC COMPUTING Institute of Physics, Belgrade Scientific Computing Laboratory FREE SCIENTIFIC COMPUTING GRID COMPUTING Branimir Acković March 4, 2007 Petnica Science Center Overview 1/2 escience Brief History of UNIX

More information

PoS(ACAT)020. Status and evolution of CRAB. Fabio Farina University and INFN Milano-Bicocca S. Lacaprara INFN Legnaro

PoS(ACAT)020. Status and evolution of CRAB. Fabio Farina University and INFN Milano-Bicocca   S. Lacaprara INFN Legnaro Status and evolution of CRAB University and INFN Milano-Bicocca E-mail: fabio.farina@cern.ch S. Lacaprara INFN Legnaro W. Bacchi University and INFN Bologna M. Cinquilli University and INFN Perugia G.

More information

EO Ground Segment Evolution Reflections by

EO Ground Segment Evolution Reflections by EO Ground Segment Evolution Reflections by Interoute Jonathan Brown Marketing Director Workshop 2015, 24 th September 2015 ESA/ESRIN Frascati Interoute, from the ground to the cloud 1. Interoute is the

More information

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Copyright 2011, Oracle and/or its affiliates. All rights reserved. The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material,

More information

Bringing ATLAS production to HPC resources - A use case with the Hydra supercomputer of the Max Planck Society

Bringing ATLAS production to HPC resources - A use case with the Hydra supercomputer of the Max Planck Society Journal of Physics: Conference Series PAPER OPEN ACCESS Bringing ATLAS production to HPC resources - A use case with the Hydra supercomputer of the Max Planck Society To cite this article: J A Kennedy

More information

ALICE Grid Activities in US

ALICE Grid Activities in US ALICE Grid Activities in US 1 ALICE-USA Computing Project ALICE-USA Collaboration formed to focus on the ALICE EMCal project Construction, installation, testing and integration participating institutions

More information

Disclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme

Disclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme LHC2384BU VMware Cloud on AWS A Technical Deep Dive Ray Budavari @rbudavari Frank Denneman - @frankdenneman #VMworld #LHC2384BU Disclaimer This presentation may contain product features that are currently

More information

HTCondor Week 2015: Implementing an HTCondor service at CERN

HTCondor Week 2015: Implementing an HTCondor service at CERN HTCondor Week 2015: Implementing an HTCondor service at CERN Iain Steers, Jérôme Belleman, Ulrich Schwickerath IT-PES-PS HTCondor Week 2015 HTCondor at CERN 2 Outline The Move Environment Grid Pilot Local

More information

06 March INVESTOR DAY 2018 Sabre GLBL Inc. All rights reserved. 1

06 March INVESTOR DAY 2018 Sabre GLBL Inc. All rights reserved. 1 06 March 2018 1 Completing Our Technology Evolution Joe DiFonzo, Chief Information Officer Vish Saoji, Chief Technology Officer 06 March 2018 2 We are completing the evolution of our technology to deliver

More information

LEVERAGING A PERSISTENT HARDWARE ARCHITECTURE

LEVERAGING A PERSISTENT HARDWARE ARCHITECTURE WHITE PAPER I JUNE 2010 LEVERAGING A PERSISTENT HARDWARE ARCHITECTURE How an Open, Modular Storage Platform Gives Enterprises the Agility to Scale On Demand and Adapt to Constant Change. LEVERAGING A PERSISTENT

More information

Gain Control Over Your Cloud Use with Cisco Cloud Consumption Professional Services

Gain Control Over Your Cloud Use with Cisco Cloud Consumption Professional Services Solution Overview Gain Control Over Your Cloud Use with Cisco Cloud Consumption Professional Services OPTIMIZE YOUR CLOUD SERVICES TO DRIVE BETTER BUSINESS OUTCOMES Reduce Cloud Business Risks and Costs

More information

Surveillance Dell EMC Storage with FLIR Latitude

Surveillance Dell EMC Storage with FLIR Latitude Surveillance Dell EMC Storage with FLIR Latitude Configuration Guide H15106 REV 1.1 Copyright 2016-2017 Dell Inc. or its subsidiaries. All rights reserved. Published June 2016 Dell believes the information

More information

RESEARCH DATA DEPOT AT PURDUE UNIVERSITY

RESEARCH DATA DEPOT AT PURDUE UNIVERSITY Preston Smith Director of Research Services RESEARCH DATA DEPOT AT PURDUE UNIVERSITY May 18, 2016 HTCONDOR WEEK 2016 Ran into Miron at a workshop recently.. Talked about data and the challenges of providing

More information

Protecting enterprises from potentially disastrous data loss.

Protecting enterprises from potentially disastrous data loss. Singtel Business Product Brochure Managed Backup-as-a-Service Protecting enterprises from potentially disastrous data loss. Singtel Managed Backup-as-a-Service protects enterprises from the consequences

More information

How to install a Crucial SSD in your computer

How to install a Crucial SSD in your computer How to install a Crucial SSD in your computer If you re a first-time installer, there s no need to fear the process is easy and straightforward. It ll take some time, but your computer will be radically

More information

ELASTIC DATA PLATFORM

ELASTIC DATA PLATFORM SERVICE OVERVIEW ELASTIC DATA PLATFORM A scalable and efficient approach to provisioning analytics sandboxes with a data lake ESSENTIALS Powerful: provide read-only data to anyone in the enterprise while

More information

Internet Protocol Version 6

Internet Protocol Version 6 Q&A Internet Protocol Version 6 Last updated: July 2006 Q. What is Internet Protocol Version 6 (IPv6)? A. IPv6 is the next generation of the protocol that runs the Internet. IPv6 is currently a set of

More information

Analisi Tier2 e Tier3 Esperienze ai Tier-2 Giacinto Donvito INFN-BARI

Analisi Tier2 e Tier3 Esperienze ai Tier-2 Giacinto Donvito INFN-BARI Analisi Tier2 e Tier3 Esperienze ai Tier-2 Giacinto Donvito INFN-BARI outlook Alice Examples Atlas Examples CMS Examples Alice Examples ALICE Tier-2s at the moment do not support interactive analysis not

More information

Streamlining CASTOR to manage the LHC data torrent

Streamlining CASTOR to manage the LHC data torrent Streamlining CASTOR to manage the LHC data torrent G. Lo Presti, X. Espinal Curull, E. Cano, B. Fiorini, A. Ieri, S. Murray, S. Ponce and E. Sindrilaru CERN, 1211 Geneva 23, Switzerland E-mail: giuseppe.lopresti@cern.ch

More information

Cloud Going Mainstream All Are Trying, Some Are Benefiting; Few Are Maximizing Value

Cloud Going Mainstream All Are Trying, Some Are Benefiting; Few Are Maximizing Value All Are Trying, Some Are Benefiting; Few Are Maximizing Value Latin America Findings September 2016 Executive Summary Cloud adoption has increased 49% from last year, with 78% of companies in Latin America

More information

Outline. March 5, 2012 CIRMMT - McGill University 2

Outline. March 5, 2012 CIRMMT - McGill University 2 Outline CLUMEQ, Calcul Quebec and Compute Canada Research Support Objectives and Focal Points CLUMEQ Site at McGill ETS Key Specifications and Status CLUMEQ HPC Support Staff at McGill Getting Started

More information

Data Sheet FUJITSU ETERNUS DX60 S3 Disk Storage System

Data Sheet FUJITSU ETERNUS DX60 S3 Disk Storage System Data Sheet FUJITSU ETERNUS DX60 S3 Disk Storage System Data Sheet FUJITSU ETERNUS DX60 S3 Disk Storage System The Economy Storage System for SMBs ETERNUS DX Storage Combining leading performance architecture

More information

Data Analysis in ATLAS. Graeme Stewart with thanks to Attila Krasznahorkay and Johannes Elmsheuser

Data Analysis in ATLAS. Graeme Stewart with thanks to Attila Krasznahorkay and Johannes Elmsheuser Data Analysis in ATLAS Graeme Stewart with thanks to Attila Krasznahorkay and Johannes Elmsheuser 1 ATLAS Data Flow into Analysis RAW detector data and simulated RDO data are reconstructed into our xaod

More information

AGILE OPTICAL NETWORK (AON) CORE NETWORK USE CASES

AGILE OPTICAL NETWORK (AON) CORE NETWORK USE CASES AGILE OPTICAL NETWORK (AON) CORE NETWORK USE CASES PAOLO FOGLIATA - EMEA IP Transport, Director Optical Networking Dec 3, 2014 1830 PSS MARKET MOMENTUM 100G DEPLOYMENTS ACCELERATE 300+/WEEK 400 Manufactured

More information

vsan Disaster Recovery November 19, 2017

vsan Disaster Recovery November 19, 2017 November 19, 2017 1 Table of Contents 1. Disaster Recovery 1.1.Overview 1.2.vSAN Stretched Clusters and Site Recovery Manager 1.3.vSAN Performance 1.4.Summary 2 1. Disaster Recovery According to the United

More information