Austrian Federated WLCG Tier-2 Peter Oettl on behalf of Peter Oettl 1, Gregor Mair 1, Katharina Nimeth 1, Wolfgang Jais 1, Reinhard Bischof 2, Dietrich Liko 3, Gerhard Walzel 3 and Natascha Hörmann 3 1 Institute of Astro- and Particle Physics, University of Innsbruck 2 Zentraler Informatik Dienst, University of Innsbruck 3 Institute of High Energy Physics, Austrian Academy of Science, Vienna
Content Introduction The Worldwide LHC Computing Grid The Austrian Federated Tier-2 Recent Tests Outlook & Conclusion
Introduction LHC starts operation in fall 2009 Austrian institutes participates in the CMS and ATLAS experiments LHC experiments will produce about 15PB per year Data needs to be stored, processed and made available to over 5000 physicists at more than 500 institutes Worldwide LHC Computing Grid (WLCG) should provide the resources
The WLCG Data storage and analysis infrastructure for the LHC high energy physics community Data from the experiments will be distributed around the globe according to a four-tiered model Tier-0: located at CERN, primary backup on tape, initial processing and data distribution to Tier-1 s Tier-1: 11 large computer centers with round-theclock support, mass storage and processing facilities; data distribution to Tier-2 s
The WLCG continued Tier-2: consisting of one or several collaborating computing facilities with sufficient data storage and adequate computing power for Monte Carlo and analysis tasks Tier-3: Grid access for individual scientists; can be a local department cluster or even individual PC s Based on several Grid infrastructures: EGEE (Enabling Grids for E-sciencE) in Europe OSG (Open Science Grid) in the US NDGF (Nordic Data Grid Facility) in Scandinavia
The WLCG continued Infrastructures support different middleware flavors, but key components (security, accounting, file transfer services) are fully interoperable WLCG provides an interface to seamlessly access these infrastructures LHC experiments developed services on top to operate the infrastructure Workload Management (DIRAC, Alien, Panda,...) Data Management (PhEDEx, DQ2,...) User Analysis (Ganga, CRAB, DIRAC, Alien,...)
Austrian Federated Tier-2 Innsbruck set up their first Grid site in 2003 and participated in ATLAS Data Challenge 2 and large scale production for the workshop in Rome 2005 Innsbruck is associated to ATLAS via the German GridKa cloud (Tier-1) since 2008 Innsbruck currently receives 10% of the data
Austrian Federated Tier-2 cont d Vienna started in 2005 Supports the CMS computing activities with emphasis on user analysis Will store data according to the CMS model 1/3 is general data - real data and simulation 1/3 is group specific data (SUSY and BTag) 1/3 is analysis specific data
Tier-2 Layout - Innsbruck
Tier-2 Layout - Innsbruck cont d Computing Elements 2 x LCG-CE with Torque/Maui Batch System on SL 4.7 28 WN s: 2 x Quad-Core Intel Xeon L5420 CPU s (2.5 GHz), 16 GByte RAM 9 WN s: 2 x Dual-Core Intel Xeon 5160, 8 GByte RAM
Tier-2 Layout - Innsbruck cont d Storage Element, Disk Pool Manager (DPM) 1 DPM Head Node Transtec SUMO RAID, 48 x 1 TByte Extension 48 x 2 TByte projected Starline Easy Raid, 16 x 1 TByte 3 DPM Disk Nodes (2 additional projected) 360 (600) MByte/s between WN s and disks
Tier-2 Layout - Innsbruck cont d Core Service: Top-level BDII (Berkeley Database Information Index) for Central Europe Part of bdii.ce-egee.org DNS pool DNS pool currently contains 6 top-level BDII s for load balancing
Tier-2 Layout - Vienna
Tier-2 Layout - Vienna Computing Element LCG-CE with Torque/Maui Batch System on SL 4.7 50 WN s: Sun blades, 2 x Quad Core Intel Xeon CPU s (2.6 GHz), 16 GByte RAM 50 blades will be added after the upgrade of electric power, cooling and network is finished
Tier-2 Layout - Vienna continued Storage Element, DPM 1 DPM Head Node 4 DPM Disk Nodes 4 Supermicro Raid s á 45 TByte 6 more will be added when the upgrade is finished 2 GBit/s (10 GBit/s) between WN s and disks
Austrian Federated Tier-2 Pledges 2009 2009 pledged ATLAS CMS Total % of pledged CPU [HEP-SPEC06] 4240 1850 3100 4950 117 % Disk [TByte] 295 54 220 274 93 %
Austrian Federated Tier-2 Pledges 2010 2010 pledged ATLAS planned CMS planned Total planned % of pledged CPU [HEP-SPEC06] 4800 1850 7000 8850 184 % Disk [TByte] 330 134 500 634 192 %
Availability and Reliability Tier-2 Reliability Report July 2009 Availability of Innsbruck dropped in July due too network layout improvements AT-HEPHY-VIENNA-UIBK usually within top 10 most reliable sites
Recent Tests STEP09 (Scale Testing for the Experiment Program 2009) HammerCloud (HC) Test July HC Test August HC Test August retested
STEP09 All experiments nominal rate Production User analysis stress test Production: Innsbruck performed good (95% efficiency) User analysis: Innsbruck performed bad Network overload @ many sites HC 432: 76% failure rate Failed HC 430: 62% failure rate Completed
STEP09 - Bottlenecks identified WN s access storage through NAT 2nd cluster s bandwidth to SE Bandwidth to FZK
/012&/'()(*+,-,'345,' 6-"7'('.8)"-9#'4:';7' <8.-)='2'6##>,',#)"' "4'?2@A-#))(!8BB#,,:8*C'&D1;'E' Vienna STEP09
HC July Disk servers connected to internal network now HC 525: 62% failure rate HC 525: gsi-ftp traffic still through NAT HC 531: rfio traffic through internal network HC 531: 1% failure rate Failed Completed Submitted Running
rfio gsi-ftp HC July - continued
HC August HC 574: rfio HC 574: 0% failure rate HC 575: rfcp / FileStager HC 579: Panda HC 575: 0% failure rate Misconfiguration of DPM disk servers HC 579: 97% failure rate
HC August retested HC 585: Panda; limited to around 150-200 concurrent jobs HC 585: 7.7% failure rate HC 600: Panda; limited to around 70-150 concurrent jobs HC 600: 2.8% failure rate Failed Submitted Completed Running
Outlook & Conclusion Austria participates in LHC experiments not only in physics but also in computing Austria setup a medium sized Tier-2 which exceeds the pledges Production is running well Problems with user analysis jobs were identified and are addressed Network bandwidth Need to limit number of concurrent analysis job to the available bandwidth Austrian Federated WLCG Tier-2 will be ready for the LHC start
Thank you for your attention! More information are available here: http://www.uibk.ac.at/austrian-wlcg-tier-2/ Questions?