ATLAS Tier-2 Computing in D GridKa-TAB, Karlsruhe, 30.9.2005 München Computing Model Tier-2 Plans for Germany Relations to GridKa/Tier-1 GridKa-TAB, 30.9.05 1
ATLAS Offline Computing ~Pb/sec PC (2004) = ~1 kspecint2k Event Builder 10 GB/sec Some data for calibration and monitoring to institutes Event Filter ~7.5MSI2k 400 MB/sec ~5 PB/year Calibrations flow back Tier 0 T0 ~5MSI2k No simulation 10 sites Tier 1 US Regional Centre Dutch Regional Centre French Regional Centre German Regional Centre (GridKa) 18 MSI2k ~12 PB 622Mb/s links Tier 2 LRZ/RZG Tier2 ~500kSI2k Desy Tier2 ~500kSI2k Fr/Wu Tier2 ~500kSI2k LMU Munich MPI Munich Uni D... Tier 3 Uni M... Physics data cache Desktop GridKa-TAB, 30.9.05 2
Data Volumes and Data Types RAW data for primary reco at Tier-0 (and Tier-1 for reproc) 1.6 MB/event, 2*109 ev/year, 3.2 PB/year 1 copy at Tier-0, 1 copy distributed over ~10 Tier-1 (on tape) ESD (event summary data, reco objects + raw data subset), for physics-group analysis at Tier-1 0.5 MB/event, 1 PB/year 2 copies distributed over ~10 Tier-1 on disk AOD (analysis object data, reconstructed physics objects: jets, leptons, etc) for user analysis at Tier-2 0.1 MB/event, 180 TB/year 1 copy at each Tier-1 and 1 copy shared among ~3 Tier-2 centers TAG (TAG data, basic event-level info) for fast skimming 1 kb/event, 2 TB/year, each T-1/T-2 center Same structure for simulated data, size ~20% of real events GridKa-TAB, 30.9.05 3
Tier-1/Tier-2 Tasks Tier-1: Physics-group ``organized'' analysis of ESD data Calibration-group ``organized'' analysis of ESD data (and Rawd) ATLAS wide re-processing of RAW data 1-2 x / year Main repository for ESD, AOD (real and simulated) No User-level analysis! Tier-2: User-level ``chaotic'' analysis of AOD data Organized Simulation production by ATLAS & Physics groups Analysis of group and user data Repository for AOD, group data sets and some user data... eventually complemented by Tier-3... GridKa-TAB, 30.9.05 4
Average Tier-2 requirements (2008) Disk (TB) Raw 1.5 General ESD (curr.) 0.0 AOD 86.0 TAG 3.0 CPU (ksi2k) Reconstruction 65 Simulation 180 Analysis 290 Total 540 ESD Sim (curr.) 6.0 AOD Sim 20.0 Tag Sim 1.0 User Group 40.0 User Data 60.0 Total 333.0 Proportional scaling not required. Can have Tier-2 focused more on Simulation = CPU AOD analysis=disk User Analysis=both GridKa-TAB, 30.9.05 5
Evolution of Tier-2 requirements T2 Cloud Growth Resource needs basically proportional to accumulated data slight kink in 2009 due to projected high-lumi running Trigger rate const larger events 100000 90000 80000 70000 60000 50000 40000 30000 20000 10000 0 2007 2008 2009 2010 2011 2012 Disk (TB) CPU (ksi2k) Disk (TB) 1606.60 8747.98 15904.56 25815.10 35725.63 45654.33 CPU (ksi2k) 3653.24 19938.74 31767.93 53014.37 71121.85 89229.33 GridKa-TAB, 30.9.05 6
Networking and Tier 2s Tier-2 to Tier-1 networking requirements presumably low 2x / year AOD Tier-1 --> Tier-2 Continuous phys. group sets Tier-1 --> Tier-2 Continuous simul data Tier-2 --> Tier-1 without job traffic ~17.5MB/s for average T2 1 Gbps should be sufficient for peak load and leave some headroom MB/s (nominal) 120.0 100.0 80.0 60.0 40.0 20.0 0.0 ATLAS Average T1 to T2s traffic ATLAS HI ATLAS Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec 2008 2009 2010 Month GridKa-TAB, 30.9.05 7
ATLAS Tier-2 in D - Plans German Tier-2 structure emerging: 3 `average-size Tier-2s planned: DESY standalone Freiburg and Wuppertal as `remote federation Munich RZG and LRZ as `close federation firm pledges from institutional sites: DESY and MPI-M/RZG funding for university sites still unclear started regular meetings share experience with ATLAS specific installation issues next goal is to participate at DC3/Computing Commissioning beginning this winter GridKa-TAB, 30.9.05 8
ATLAS Tier-2 in D - Status Desy fully functional LCG installation & active in LCG SC3 (see Desy talk) Wuppertal several years experience with EDG/LCG/D0Grid operational testbed setup, installation on super-cluster (AliceNext) underway Freiburg LCG 2.6 installed on testbed Munich: LMU/LRZ used existing shared cluster (250 CPU) for DC2/Rome prod via NorduGrid LCG testbed installed Munich: MPG/RZG new 140 CPU/24 TB disk cluster. Shared ATLAS/Magic. LCG testbed installed at MPI, port to RZG cluster in progress. GridKa-TAB, 30.9.05 9
GridKa/Tier-1 Tier-2 Relations Networking Transfer AOD, Sim. Data, Group data sets ~20 MB/sec average between single Tier-2 and Tier-1 Storage for simulated data Primary storage for simulated events at Tier-2/3 are Tier-1 sites. resources already considered in CompModel and planning presumably dataset driven, not a static Tier-2/Tier-1 relation Support for Grid software/deployment crucial to have qualified and responsible support heavily used, cf MPI LCG setup, LMU/LRZ installation GridKa-TAB, 30.9.05 10
GridKa/Tier-1 Tier-2 Relations - cont Support for Operations: SC3, SC4, DC4,... Serve as knowledge center Represent interests of associated Tier-2 GridKa-TAB, 30.9.05 11