The LHC Computing Grid

Transcription

1 The LHC Computing Grid Visit of Finnish IT Centre for Science CSC Board Members Finland Tuesday 19 th May 2009 Frédéric Hemmer IT Department Head

2 The LHC and Detectors Outline Computing Challenges Current Grid Usage EGEE & OSG Site Reliability & Middleware

3 LHC: a very large scientific instrument

4 based on advanced technology 23 km of superconducting magnets cooled in superfluid helium at 1.9 K

5 CMS Closed & Ready for First Beam 3 Sept 2008

6 The ATLAS experiment 7000 tons, 150 million sensors generating data 40 millions times per second i.e. a petabyte/s

7 A collision at LHC 7

8 The Data Acquisition 8

9 Tier 0 at CERN: Acquisition, First pass processing Storage & Distribution 1.25 GB/sec (ions) Ian.Bird@cern.ch 9

10 detector event filter (selection & reconstruction) Data Handling and Computation for Physics Analysis reconstruction event summary data processed data raw data event reprocessing analysis batch physics analysis analysis objects (extracted by physics topic) event simulation simulation interactive physics analysis

11 The LHC Computing Challenge Signal/Noise: 10-9 Data volume High rate * large number of channels * 4 experiments 15 PetaBytes of new data each year Compute power Event complexity * Nb. events * thousands users 100 k of (today's) fastest CPUs 45 PB of disk storage Worldwide analysis & funding Computing funding locally in major regions & countries Efficient analysis everywhere GRID technology 11

12 The Worldwide LHC Computing The LHC Grid Service is a worldwide collaboration between: 4 LHC experiments and ~140 computer centres that contribute resources International grid projects providing software and services The collaboration is brought together by a MoU that: Commits resources for the coming years Agrees a certain level of service availability and reliability As of today 33 countries have signed the MoU: CERN (Tier 0) + 11 large Tier 1 sites 130 Tier 2 sites in 60 federations Other sites are expected to participate but without formal commitment 12

13 Tier 0 Tier 1 Tier 2 Tier-0 (CERN): Data recording Initial data reconstruction Data distribution Tier-1 (11 centres): Permanent storage Re-processing Analysis Tier-2 (~130 centres): Simulation End-user analysis Ian.Bird@cern.ch 13

14 Evolution of Grids GriPhyN, ivdgl, PPDG EU DataGrid LCG 1 GRID 3 EGEE 1 OSG WLCG EGEE 2 EGEE 3 LCG Service Challenges Cosmics First physics Data Challenges 14

15 Preparation for accelerator start up Since 2004 WLCG has been running a series of challenges to demonstrate aspects of the system; with increasing targets for: Data throughput Workloads Service availability and reliability Culminating in a 1 month challenge in May with All 4 experiments running realistic work (simulating what will happen in data taking) Demonstrated that we were ready for real data In essence the LHC Grid service has been running for several years 15

16 Recent grid use The grid concept really works all contributions large & small contribute to the overall effort! CPU Usage Early 2008 CERN: 11% Tier 2: 54% Tier 1: 35% CERN BNL TRIUMF FNAL FZK-GRIDKA CNAF CC-IN2P3 RAL ASGC PIC NDGF NL-T1 Tier 2

17 Recent grid activity 350k /day In readiness testing WLCG ran more than10 million jobs /month (1 job is ~ 8 hours use of a single processor) These workloads are at the level anticipated for 2009 data

18 Data transfer out of Tier 0 Full experiment rate needed is 650 MB/s Desire capability to sustain twice that to allow for Tier 1 sites to shutdown and recover Have demonstrated far in excess of that All experiments exceeded required rates for extended periods, & simultaneously All Tier 1s achieved (or exceeded) their target acceptance rates

19 LHCb CMS ATLAS ALICE Number of jobs/month 500k/day Main outstanding issues related to service/site reliability Alice ATLAS CMS LHCb Total Tier-1s % Tier-2s % Total From APEL accounting portal for Aug. 08 to Jan. 09; #s in MSI2k

20 Production Grids WLCG relies on a production quality infrastructure Requires standards of: Availability/reliability Performance Manageability Will be used 365 days a year... (has been for several years!) Tier 1s must store the data for at least the lifetime of the LHC - ~20 years Not passive requires active migration to newer media Vital that we build a fault-tolerant and reliable system That can deal with individual sites being down and recover 20

21 WLCG depends on two major science grid infrastructures. EGEE - Enabling Grids for E-Science OSG - US Open Science Grid... as well as many national grid projects Interoperability & interoperation is vital significant effort in building the procedures to support it 21

22 Impact of the LHC Computing Grid in Europe LCG has been the driving force for the European multiscience Grid EGEE (Enabling Grids for E-sciencE) EGEE is now a global effort, and the largest Grid infrastructure worldwide Co-funded by the European Commission (Cost: ~130 M over 4 years, funded by EU ~70M ) EGEE already used for >100 applications, including Bio-informatics Education, Training Medical Imaging

23 Grid infrastructure project co-funded by the European Commission - now in 3 rd phase with partners in 45 countries Apr-04 Aug-04 Dec-04 Apr-05 Aug-05 Dec-05 Apr-06 Aug-06 Dec-06 Apr-07 Aug-07 Apr-04 Aug-04 Dec-04 Apr-05 Aug-05 Dec-05 Apr-06 Aug-06 Dec-06 Apr-07 Aug-07 Enabling Grids for E-sciencE 240 sites 45 countries 45,000 CPUs 12 PetaBytes > 5000 users > 100 VOs > 100,000 jobs/day Archeology Astronomy Astrophysics Civil Protection Comp. Chemistry Earth Sciences Finance Fusion Geophysics High Energy Physics Life Sciences Multimedia Material Sciences EGEE-II INFSO-RI No. Sites No. CPU

24 EGEE Enabling Grids for E-sciencE The EGEE project Started in April 2004, now in third phase ( ) Brings together more than 240 institutions in 45 countries world-wide EGEE-III Finnish Beneficiary: - Helsingin Yliopisto - CSC - Tieteellinen Laskenta Oy Objectives Large-scale, production-quality grid infrastructure for e-science Attracting new resources and users from industry as well as science Maintain and further improve glite Grid middleware EGEE-II INFSO-RI

25 Enabling Grids for E-sciencE Sustainability Need to prepare for permanent Grid infrastructure Ensure a high quality of service for all user communities Independent of short project funding cycles Infrastructure managed in collaboration with National Grid Initiatives (NGIs) European Grid Initiative (EGI) Supporting Partners in Finland: CSC, the Finnish IT center for Science EGEE-II INFSO-RI Wolfgang von Rüden, CERN, IT Department

26 WLCG Site reliability T1 reliability above target (93%) in May 08 CERN + Tier-1 Reliability Overall average 75-80% Top 50% sites: 95% Tops 20% sites: 98% > 70% resources are at sites > 90% reliability

27 Improving Reliability Monitoring Metrics Workshops Data challenges Experience Systematic problem analysis Priority from software developers

28 Future Middleware an opinion Today s middleware is complex support effort is large (sw & operational) Built a production service from prototypes often too complex for need Web Services promises have not delivered except in proprietary or organised implementations; robust tools do not exist Technology changes rapidly Cloud computing very (too) simple interfaces; virtualisation of environments Global file systems seem a realistic expectation now A reliable messaging system is a better glue for heterogeneous distributed services A cursory analysis suggests that most interactions (even synchronous ) can be implemented We need to work with industry (and opensource) to better integrate remote interfaces to storage and batch systems Some of today s complexity is due to the need to add on external gateways E.g. LSF, SGE, PBS could have (full) BES interface we don t need a CE SRM is too complex as SE interface what is a better way filesystem? We should aim to get as much as possible delivered with the OS, or as standard layered software packages

29 The LHC Grid Service A worldwide collaboration Has been in production for several years Is now being used for real data Is ready to face the computing challenges as LHC gets up to full speed

30 The very first beam-splash event from the LHC in ATLAS on 10:19, 10 th September

31 31