SPGrid Efforts in Italy

Transcription

1 INFN - Ferrara BaBarGrid Meeting SPGrid Efforts in Italy BaBar Collaboration Meeting - SLAC December 11, 2002 Enrica Antonioli - Paolo Veronesi

2 Topics Ferrara Farm Configuration First SP submissions through the Grid Work in Progress Future Plans Conclusions SLAC - December 11,

3 SPGrid Farm in Ferrara SE 250 GB R A I D 0 LCFG Server Management UI SCSI Data server CE-WN Ferrara - EDG Lock server SLAC - December 11,

4 SPGrid Farm in Ferrara SE 250 GB R A I D 0 UI SCSI Data server CE-WN Ferrara - EDG RB CNAF - Bologna Lock server SLAC - December 11,

5 SPGrid Farm in Ferrara SE 250 GB R A I D 0 UI SCSI Data server CE-WN Ferrara - EDG RB CNAF CERN - Bologna Lock server SLAC - December 11,

6 Configuration INFN Grid Testbed Status: EDG (RedHat 6.2). A BaBar software special release (12.3.2y) has been built and installed to: Write Kanga files Run Moose on RH 6.2 A special tag of ProdTools has been installed to perform tests. A pool of BaBar accounts (babar000, babar001, ) has been created in the EDG farm of Ferrara. Each member of BaBar VO is able to submit jobs to the farm of Ferrara through the RB located at CNAF (grid009g.cnaf.infn.it). SLAC - December 11,

7 Current Status Created a JDL file to run Moose on Grid resources. Created scripts containing EDG commands to submit jobs, to check their status and retrieve output files. An user can submit a range of runs. For each run a job is created and submitted to the Resource Broker, then it is sent to the Ferrara CE (grid0.fe.infn.it). The output file is then transferred to the closest SE (grid2.fe.infn.it). SLAC - December 11,

8 Moose.jdl grid1> more Moose.jdl Executable ="Moose.csh"; InputSandbox ={"Moose.csh",".cshrc","config.csh"}; StdOutput ="Moose.txt"; StdError ="Moose.log"; OutputSandbox ={"Moose.txt","Moose.log"}; General environment configurations Similar to SP standard scripts (Job.Xsh) Config file for BaBar. Similar to SP standard scripts Globus command: To copy output files from WN to SE [ ] tar -czvf run${runnum}.tar.gz *.root globus-url-copy -vb file://`pwd`/run${runnum}.tar.gz \ gsiftp://grid2.fe.infn.it/flatfiles/se00/enrica/run${runnum}.tar.gz SLAC - December 11,

9 The launch script grid1> more launch #!/bin/tcsh num_f = fin = $2 while ( $num_f <= $fin ) ####build the run directories [ ] ####build a config.csh with the appropriate environment variables echo "#\!/bin/tcsh -v" > config.csh [ ] #### now run the job dg-job-submit -o run$num_f.jobid -r \ grid0.fe.infn.it:2119/jobmanager-pbs-long Moose.jdl num_f++ end For each run a job is created runtime Range of runs to submit A config file is created for each run EDG job submission command SLAC - December 11,

10 Job Submission Range of runs to submit grid1>./launch [ ] dg-job-submit -o run$num_f.jobid -r grid0.fe.infn.it:2119/jobmanager-pbs-long Moose.jdl Connecting to host grid009g.cnaf.infn.it, port 7771 Logging to host grid009g.cnaf.infn.it, port CNAF RB ================== dg-job-submit Success ================ The job has been successfully submitted to the Resource Broker. Use dg-job-status command to check job current status. Your job identifier (dg_jobid) is: The dg_jobid has been saved in the following file: /home/enrica/stress/ /run jobid Job ID [ ] grid1> ls Moose.csh Moose.jdl config.csh launch monitor retrieve grid1> ls / Moose.csh Moose.jdl config.csh run jobid grid1> ls / Moose.csh Moose.jdl config.csh run jobid SLAC - December 11,

11 The monitor script grid1> more monitor num_f = fin = $2 while ( $num_f <= $fin ) echo Run $num_f is `dg-job-status -i \ $num_f/run$num_f.jobid grep Status` EDG num_f++ end grid1>./monitor Run Run is isstatus = = OutputReady Scheduled Running Status Status Reason Reason = job = initial accepted terminated Run Run is isstatus = = OutputReady Scheduled Running Status Status Reason Reason = job = initial accepted terminated SLAC - December 11,

12 The retrieve script grid1> more retrieve #!/bin/tcsh num_f = fin = $2 while ( $num_f <= $fin ) cd $num_f #### get logfiles dg-job-get-output -i run$num_f.jobid --dir $PWD #### get rootfiles globus-url-copy \ Globus command: Direct copy of file from SE to UI EDG command gsiftp://grid2.fe.infn.it/flatfiles/se00/enrica/run$num_f.tar.gz \ file://`pwd`/run$num_f.tar.gz tar -xzvf run$num_f.tar.gz rm -f run$num_f.tar.gz #### delete rootfiles form SE globus-job-run grid2.fe.infn.it /bin/rm \ /flatfiles/se00/enrica/run$num_f.tar.gz num_f++ end Globus command: delete file fromse SLAC - December 11,

13 Retrieving Output grid1> ls Moose.csh Moose.jdl config.csh launch monitor retrieve grid1> ls / Moose.jdl rootdef-tru.root Moose.csh config.csh run jobid rootdef-tag.root rootdef-aod.root grid1> ls / Moose.jdl Moose.csh config.csh run jobid rootdef-tru.root rootdef-tag.root rootdef-aod.root grid1> ls / / Moose.log Moose.txt grid1> ls / / Moose.log Moose.txt SLAC - December 11,

14 Work in progress We are now trying to copy files to the SE through the Manchester Replica Catalog. We are checking package dependencies (e.g. flat files, shared libraries) to build a Moose RPM to be automatically installed on Grid machines. We are testing the special tag of ProdTools which includes an option to create and submit Moose jobs to the Grid resources. SLAC - December 11,

15 UI Future Plans Ferrara Lock server CE-WN RPM Data server RB (UK) LCFG Server RPM MOOSE Objectivity DB SE Management 3) Install Objy DB on the SE 2) MOOSE in RPM format 1) Use of IC RB and others SLAC - December 11,

16 CE-WN UI LCFG Server Future Plans Lock server Integration of Moose Application with EDG software releases Data server SE Management SPGrid Farm 3) Install Objy DB on the SE 2) MOOSE in RPM format 1) Use of IC RB and others SLAC - December 11,

17 Summary In order to be integrated with the Grid we need: To produce RPM packages for Moose Application. To install OBJY DB on the SE. To submit the resulting RPMs to the EDG Integration Team. After that we will be able to install and manage Simulation Production software and environment on EDG farms via LCFG server. Thanks to Peter Elmer, Alessandra Forti, Douglas Smith, Matthias Steinke and Steve Traylen for their collaboration. SLAC - December 11,