RDMS CMS Computing Activities before the LHC start

Transcription

1 RDMS CMS Computing Activities before the LHC start RDMS CMS computing model Tiers 1 CERN Collaborative centers: RCC RCC MSU, MSU, RRC RRC KI KI Kharkov Minsk JINR Erevan SINP MSU RDMS CMS Tier2 Tbilisi IHEP ITEP PNPI RAS LPI RAS INR RAS V.Gavrilov 1, I.Golutvin 2, V.Ilyin 3, O.Kodolova 3, V.Korenkov 2, E.Tikhonenko 2, S.Shmatov 2 1- Institute of Theoretical and Experimental Physics, Moscow, Russia 2- Joint Institute for Nuclear Research, Dubna, Russia 3 Skobeltsyn Institute of Nuclear Physics, Moscow, Russia Third International Conference Distributed Computing and GRID Technologies in Science and Education Dubna, Russia, June 30 July 4, 2008

2 Outline RDMS CMS Collaboration RDMS CMS Computing Model RDMS CMS Computing Activities - creation of a common distributed CMS grid-infrastructure at the RDMS CMS institutes - CMS Databases - participation in the ARDA project - participation in CMS Service, Data and Software Challenges - RDMS CMS institutes resources and assignment to physical analysis groups Summary

3 Composition of the RDMS CMS Collaboration RDMS - Russia and Dubna Member States CMS Collaboration Russia Russian Federation Institute for High Energy Physics, Protvino Institute for Theoretical and Experimental Physics, Moscow Institute for Nuclear Research, RAS, Moscow Moscow State University, Institute for Nuclear Physics, Moscow Petersburg Nuclear Physics Institute, RAS, St.Petersburg P.N.Lebedev Physical Institute, Moscow Associated members: High Temperature Technology Center of Research & Development Institute of Power Engineering, Moscow Russian Federal Nuclear Centre Scientific Research Institute for Technical Physics, Snezhinsk Myasishchev Design Bureau, Zhukovsky Electron, National Research Institute, St. Petersburg JINR Joint Institute for Nuclear Research, Dubna Dubna Member States Armenia Yerevan Physics Institute, Yerevan Belarus Georgia High Energy Physics Institute, Tbilisi State University, Tbilisi Institute of Physics, Academy of Science,Tbilisi Ukraine Byelorussian State University, Minsk Research Institute for Nuclear Institute of Single Crystals of National Problems, Minsk Academy of Science, Kharkov National Centre for Particle and High National Scientific Center, Kharkov Energy Physics, Minsk Institute of Physics and Technology, Research Institute for Applied Kharkov Physical Problems, Minsk Kharkov State University, Kharkov Bulgaria Institute for Nuclear Research and Nuclear Energy, BAS, Sofia University of Sofia, Sofia Uzbekistan Institute for Nuclear Physics, UAS, Tashkent the RDMS CMS Collaboration was founded in Dubna in September 1994

4 RDMS Participation in CMS Construction RDMS Full Responsibility RDMS Participation ME1/ 1 ME SE EE HE HF FS

5 RDMS Participation in CMS Project Full responsibility including management, design, construction, installation, commissioning, maintenance and operation for: Endcap Hadron Calorimeter, HE 1st Forward Muon Station, ME1/1 Participation in: Forward Hadron Calorimeter, HF Endcap ECAL, EE Endcap Preshower, SE Endcap Muon System, ME Forward Shielding, FS

6 RDMS activities in CMS Design, production and installation Calibration and alignment H (150 GeV) fi Z 0 Z 0 fi 4 m Reconstruction algorithms Data processing and analysis Monte Carlo simulation

7 desktops portables LHC Computing Model Tier-0 (CERN) Filter raw data small centres Reconstruction summary data (ESD) Record raw data and ESD Distribute raw and ESD to Tier-1 MSU IC JINR ITEP Cambridge Budapest Prague Santiago Tier-2 Weizmann IHEP TRIUMF Legnaro RAL Tier-1 IN2P3 FNAL CNAF FZK CSCS Tier-1 Rome Permanent storage and management of raw, ESD, calibration data, meta-data, analysis data and databases grid-enabled data service Data-heavy analysis Re-processing raw ESD ESD-AOD selection National, regional support Tier-2 Simulation, digitization, calibration of simulated data End-user analysis PIC Kharkov BNL ICEPP Minsk PNPI NIKHEF

8 RuTier2 Cluster RuTier2 Conception: Cluster of institutional computing centers with Tier2 functionality operating for all four experiments - ALICE, ATLAS, CMS and LHCb Participating institutes: Moscow Moscow region St.Petersburg Novosibirsk ITEP, SINP MSU, RRC KI, LPI, MEPhI JINR, IHEP, INR RAS PNPI, SPbSU BINP RDMS CMS Advanced Tier2 Cluster is a part of RuTier2 Basic functions: analysis; simulations; users data support; calibration; reconstruction algorithm development The CMS data needed: ~5-10% of RAW DATA; ESD/DST for AOD selections design and checking; AOD for analysis - the volume of data depending on a concrete task

9 Tiers 1 CERN RDMS CMS computing model: RDMS CMS Advanced Tier2 Cluster as a part of RuTier2 RuTier2 Grid access to CERN 2004: 155 Mbs 2005: 310 Mbs 2007: 1-2 Gbs Data processing&analysis scenario is developing in a context of estimation of resources on a basis of selected physics channels in which the RDMS CMS plans to participate. Collaborative centers: RCC RCC MSU, MSU, RRC RRC KI KI Kharkov Minsk JINR Erevan SINP MSU RDMS CMS Tier2 Tbilisi RDMS CMS computing centers IHEP ITEP PNPI RAS LPI RAS INR RAS Institutes in Russia: IHEP, ITEP, JINR, SINP MSU (as 4 basic centers), LPI RAS, INR RAS, PNPI RAS Collaborative RDMS Institutes: ErPhI (Armenia), HEPI (Georgia) KIPT (Ukraine), NCPHEP (Belarus)

10 CMS Computing model

11 RDMS CMS Tier2 Cluster Features -the facilities (CPU and data storage resources) for: CMS Monte-Carlo production (30% of RDMS resources) - MC simulation of the standard CMS MC samples including the detector simulation and the first path reconstruction. The MC data will be moved to CERN or(and) stored locally. a number of analysis tasks and for detectors calibration, HLT and offline reconstruction algorithms and analysis tools development. - connectivity between the RDMS institutes: >1 Gbit/s Ethernet - replication of databases (conditions) used in reconstruction. Processing of a part of raw data in addition to ordinary functions of Tier-2 (likemaintaining andanalysis of AODs, data simulation and user support) It is needed both for a calibration/alignment of some CMS detector systems for which the RDMS is responsible and for creation and testing of reconstruction software applied to someparticular physics channels. somepart of RAW,ESD/DST and the proper set of AOD data will be transferred and kept at the RDMS CMS Tier2 cluster at the mass storage system (MSS), located in one or two institutes.

12 Creation of a common distributed CMS grid- infrastructure at the RDMS CMS institutes - IHEP, ITEP, JINR, SINP MSU (as 4 basic centers) and INR RAS, LPI RAS, PNPI RAS, KIPT (Ukraine) have computing infrastructure integrated into the global EGEE/WLCG infrastructure and queues for CMS VO enabled; SEs for CMS data storage provided - CMSSW (specialized CMS package for data processing and simulation) installed at all the RDMS grid-sites - CMS VOBOX services provided - Phedex (the data placement and the file transfer system for the CMS experiment servers installed - Squid servers installed (for to access "conditions data" worldwide - see

13 Participation in CMS Databases design and support HE Calibration and Raw Data Flow JINR Dubna API JINR DS DCache XML files in concerned format; Pedestals Leveling coefficient Timing Raw Data by GRID Protocols GeV /ADC ( in progress ) SX 5 API HE DB CERN OMDS - Online Master Data Storage ORCON - Offline Reconstruction Conditions DB ONline subset ORCOF - Offline Reconstruction Conditions DB OFfline subset Conditions for HLT Data storage and data management SX5 Data Storage Disk Pool OMDS Conditions data ORCON ORCOF Conditions data Castor CERN DAQ Trigger 0 HLT Offline reconstruction Raw Data by GRID Protocols CERN - JINR Data management for MTCC: common schema of realization HE Calibration DB Status System is online Full calibration cycle support Integrated into CMS computing environment ~30000 records ~500Mb ~600Gb raw data transferred to JINR Castor CERN Dat a Buffer file Storage With AFS Data Storage Disk Pool with AFS Management and metadata DATA DA TA Data management information system UI GRID { JINR DS DCache Management and metadata it is necessary to provide facilities for DBs replication at the RDMS CMS T2 Cluster DATA

14 Participation in ARDA Project ARDA - A Realisation of Distributed Analysis for LHC ://lcg.web.cern.ch/lcg/activities/arda/arda.htmlarda.html For CMS Dashboard (which is monitoring system for CMS Virtual Organization distributed computing system ): ( ) Monitoring of data transfers between T1 and T2 CMS centers Monitoring of production jobs with MonALISA a) monitoring of errors (unique system for LCG and local farms) b) monitoring of the WN, SE, network, installed software c) participation in the design of the monitoring tables in Dashboard Portability of Monitor system Participation in the CRAB-Task Manager integration ( CRAB - CMS Remote Analysis Builder; ) Development of the methods for the organizing and accessing data for the CMS Dashboard user interface Development of the monitoring package to keep track on Grid jobs running via Condor workload management system the system is now being tested before its deployment on CMS sites

15 CMS Job Monitoring in CMS Dashboard RB R-GMA Monalisa RB CE WN Submission tools Snapshot Statistics Job info. Plots R-GMA Client API Dashboard Web UI PHP Collector (RGMA) Dashboard database sqlite pg Web Service Interface Collector (Monalisa) Postgres sqlite Others? Constantly retrieve job information RDMS CMS staff participates in ARDA activities on monitoring for CMS monitoring development: monitoring of errors due gridenvironment and automatic job restart in these cases; monitoring of a number of events done; the further expansion to private simulation

16 CMS jobs running at JINR WLCG site in 2008 year (information obtained from the CMS dashboard)

17 RDMS CMS Participation in CMS Computing Activities during In the frames of CMS Computing, Software, Data and Analysis challenges (DC04, CSA06, CSA07) and also during Common Computing Readiness Challenge of 2008 (CCRC08) LCG installation of CMSSW; Phedex and Squid servers installation; participation in Phedex data transfers (load test transfers and heart beat transfers; this spring the links to the RDMS CMS sites have been certified at the rate of 20MBs); participation in CMS event simulation: for example, in autumn 2006: 2 RDMS CMS sites (JINR and SINP) produced physical events it is 1 % of total amount (~50 millions events) Job Robot (user analysis jobs) testing at ITEP,JINR,SINP, IHEP and RRC KI The Common Computing Readiness Challenge of 2008 (CCRC08) aimed to test the full scope of the data handling and analysis activities needed for LHC data-taking operations in 2008 See

18 CERN-PROD as T1 center for RDMS (since February, 2007) March 19, 2007 Transfer rate MBs 99% of successful transfers and 990 GB have been transferred October-November, 2006 (CNAF as FTS T1 for JINR) Transfer rates less than 2 MBs Ratio of successful transfers 14-42%

19 Example of Current CERN-JINR Transfer Rates ( )

20 CMS Job Robot testing The JobRobot is a program, currently operated from a machine at CERN, that creates CRAB jobs, submitts them to specific SE sites, and collects them while keeping track of the corresponding information. Its main objective is to test how sites are responding to job processing in order to detect possible problems and correct them as soon as possible see JINR (since ), IHEP (since ) and ITEP (since ) started their participation in CMS Job Robot site testing. During October 2007 March jobs were submitted to the RDMS CMS sites (JINR, ITEP and IHEP) and 91 % of jobs ended successfully (compare: 79.7% of jobs submitted to all the CMS sites ended successfully) SINP and RRC KI successfully joined these activities since May

21 RDMS CMS T2 Cluster Resources and CMS Analysis Tasks By June, 2008 we provide 725 KSI2K CPU resources and 150 TB SE disk space for CMS. A significant increasing of resources is expected by autumn, this year. By the moment Heavy Ions analysis task is assigned to the RDMS CMS T2 cluster and after expanding storage capacities, two more tasks (Exotica and Muons) are expected to be assigned.

22 User Training and Induction Courses for CMS users on submitting jobs to the LCG-infrastructure are regularly conducting

23 SUMMARY RDMS CMS Computing Model has been defined as Tier2 Cluster of institutional computing centers with a partial T1 functionality Data processing & analysis scenario has been developed in a context of estimation of resources on a basis of selected physics channels in which the RDMS CMS plans to participate. The very solution that CERN will serve as T1 center for RDMS would be a strong basement to provide the RDMS CMS computing model requirements. CERN-T1 serves as FTS server for Russian sites since February, The proper RDMS CMS Grid infrastructure has been constructed at RuTier2 and the infrastructure has been successfully tested during years. RDMS RuTier2 CPU and storage resources are sufficient for analysis of first data after the LHC start and for simulation in the 2008 year

24 BACKUP SLIDES

25 CMS Job Robot lessons Starting in October,2007 the participation in CMS JOb Robot testing at JINR, we have encounted JINR local network overloading. That time all 80 computing nodes (4 Core each) (splited by 3 racks) were connected to the main LAN router via 3 1GbE links and 12 SE pool nodes all via 1 1GbE link only. The attempt to improve the situation by connecting all the computing nodes by the only 1Gbs link and each of 12 Se pool nodes by 1 GbE link each did not give a desirable result. In really this time we ended up with complitely overloaded 1GbE links to 3 WN's racks. The fact is that CMS jobs submitted by Job Robot were some kind of a typical CMS job while a small sample of events is extracted from a large.root-file and the new (smaller) file is created for the further physical analysis. These jobs require only several minutes of CPU time but reading of.root-files of > 2GB size causes data transfers 3 times more throughout the local network. As result, local network overloading causes troubles with TCP/IP, SNMP and SSH network protocols and a low efficiency of job execution (CMS jobs of 3 minutes CPU time have more than 1.5 hour of wall clock in the batch).

26 CMS Job Robot lessons (cont.) Thus we had make a decision to reconfigurate the JINR computing center local network structure by constructing a separate dedicated subnetwork for the disk raids, computing farm and a number of the NFS servers. That reconfiguration required to install new Procurve 3500yl- 48G as main LAN router and several Procurve G/48G switches. At presend all racks with WNs and SE pools has been installed with 4-8 1GbE links to the main LAN router (802.3ad - link aggregation/trunking). As result, these actions on the reconfiguration and the attendant upgrade provided the proper environment for LHC jobs and, in particular, for CMS jobs