WLCG Asia Workshop, 2.12.2006 ALICE Grid/Analysis Tutorial Exercise-Solutions Andreas-Joachim Peters CERN www.eu-egee.org cern.ch/lcg http://cern.ch/arda EGEE is a project funded by the European Union under contract IST-2003-508833 GDB Meeting, 08.11. 2006-1
GDB Meeting, 08.11. 2006-2 Exercise 1 Installation 1. Install your GLOBUS certificate in $HOME/.globus Convert to a userkey.pem and usercert.pem file, if you have only one pk12 certificate file 2. Install the ALICE GRID client software Download the alien-installer Run the alien-installer and install the lightweight GRID client packages from release version 2.12 (gshell,xrootd,root) into $HOME/alien-v2.12 3. Setup your PATH and LD_LIBRARY_PATH environment variable: Modifiy your.bashrc or.tcshrc files (or similiar) Add $HOME/alien-v2.12/api/bin to PATH Add $HOME/alien-v2.12/api/lib to LD_LIBRARY_PATH
GDB Meeting, 08.11. 2006-3 Exercise 1 Solution 1. 2. bash> wget alien.cern.ch/alien-installer bash> chmod u+x alien-installer bash> alien-installer -> follow the dialog 3.
GDB Meeting, 08.11. 2006-4 Exercise 2 Authentication 1. Obtain an token by using alien-token-init Don't forget to put your ALICE user name as the 1 st argument If you are prompted for the GRID passphrase, put the password connected to your GLOBUS certificate 2. Inspect the token environment file /tmp/gclient_env_$uid 3. Inspect the token file /tmp/gclient_token_$uid 4. Check your token with alien-token-info
GDB Meeting, 08.11. 2006-5 Exercise 2 Solution 1. bash> alien-token-init <username> 2. bash> cat /tmp/gclient_env_$uid 3. bash> cat /tmp/gclient_token_$uid 4. bash> alien-token-info
GDB Meeting, 08.11. 2006-6 Exercise 3 aliensh - basics Start aliensh 1. Change into your home directory (3 ways possible) 2. Get your username 3. Try to see, who is connected 4. Get your current working directory with a shell command 5. List all the files in your homedirectory 6. Try the tab completion (f.e. ls /ali [tab] ) 7. Navigate through the file catalogue with relative and absolute path names use also.. - ~ / 8. Create the directories ~/bin ~/jdl ~/var ~/xml ~/test 9. Remove the ~/test directory
1. aliensh> 1. cd 2. cd ~ 3. cd $HOME Exercise 3 Solution 2. aliensh> whoami 3. aliensh> who 4. aliensh> pwd 5. aliensh> ls -la 6. aliensh> just do it ;-) 7. aliensh> 1. cd / 2. cd alice 3. cd - 4. Cd ~ 8. aliensh> mkdir ~/bin ~/jdl ~/xml ~/var ~/test 9. aliensh> rmdir ~/test GDB Meeting, 08.11. 2006-7
GDB Meeting, 08.11. 2006-8 Exercise 4.1 File/Data Management 1. Copy the file /alice/cern.ch/tutorial-asia/batch/bin/batch.sh into the bin directory in your home directory Verify the location of the physical file using the whereis command 2. Look at batch.sh using cat, more & less 3. Get information about File permissions GUID MD5 sum 4. Erase your copy and copy the file again into the storage element ALICE::CERN::se01 Verify the location of the physical file using the whereis command 5. Set your EDITOR environment variable to your editor of choice 6. Open the file ~/bin/batch.sh with the aliensh editor Add an empty line in the end and safe the file Try to find the previous version of the file in the file catalogue Clean-up the previous version of the file using the purge command
Exercise 4.1 Solution 1. aliensh> cp /alice/cern.ch/tutorial-asia/batch/bin/batch.sh ~/bin/batch.sh 2. aliensh> 1. cd ~/bin 2. cat batch.sh 3. less batch.sh 4. more batch.sh 3. aliensh> 1. ls -la 2. ls -b 3. ls -m 4. aliensh> 1. erase batch.sh 2. cp /alice/cern.ch/tutorial-asia/batch/bin/batch.sh ~/bin/batch.sh@alice::cern::se01 5. aliensh> export EDITOR='emacs -nw' 6. aliensh> edit batch.sh (add empty line and save) aliensh> ls -la.batch.sh aliensh> cat.batch.sh/v1.0 aliensh> purge./ GDB Meeting, 08.11. 2006-9
Exercise 4.2 File/Data Management 1. Copy the file ~/bin/batch.sh into your local /tmp directory 2. Copy the file ~/bin/batch.sh into your local working directory 3. Erase the file ~/bin/batch.sh 4. Copy a local batch.sh file to the GRID file ~/bin/batch.sh stored in ALICE::CERN::se01 5. Query all files under the path /alice/cern.ch/tutorial-asia by printing on the screen pipe the output into the local file /tmp/querytest Count the number of files using the shell pipe mechanism and 'wc -l' GDB Meeting, 08.11. 2006-10
GDB Meeting, 08.11. 2006-11 1. aliensh> cp ~/bin/batch.sh file:/tmp/batch.sh 2. aliensh> cp ~/bin/batch.sh file:batch.sh 3. aliensh> erase ~/bin/batch.sh Exercise 4.2 Solution 4. aliensh> cp file:/tmp/batch.sh ~/bin/batch.sh@alice::cern::se01 5. 1. aliensh> find /alice/cern.ch/tutorial-asia * 2. aliensh> find /alice/cern.ch/tutorial/asia > /tmp/collection.xml 3. aliensh> find /alice/cern.ch/tutorial/asia wc -l
GDB Meeting, 08.11. 2006-12 Exercise 5 GRID access from ROOT 1. Download the ROOT startup script 'root.sh' from the file catalogue to your local home directory aliensh: /alice/cern.ch/tutorial-asia/root/root.sh 2. Run the root.sh script to startup ROOT 3. Try to authenticate in ROOT using root> TGrid::Connect( alien:// ); 4. Try to open directly an ALICE GRID file root> TFile::Open( alien:///alice/cern.ch/tutorialasia/root/aliesds.root ); 5. Try to inspect the open file with the ROOT Browser root> new TBrowser();
GDB Meeting, 08.11. 2006-13 Exercise 5 Solution 1. aliensh> cp /alice/cern.ch/tutorial-asia/root/root.sh file:root.sh 2. bash> chmod u+x root.sh; root.sh 3. root> TGrid::Connect( alien:// ); 4. root> TFile::Open( alien:///alice/cern.ch/tutorialasia/root/aliesds.root ); 5. root> new TBrowser() 1. click on ROOT files, click on the file and inspect it by clicking more...
GDB Meeting, 08.11. 2006-14 Exercise 6.1 Job Managemante Commands 1. Get a list of available software packages 2. Check your priorities in the queue system 3. Get the 10 highest ranked jobs presently in the queue 4. Display the status of all sites
GDB Meeting, 08.11. 2006-15 Exercise 6.1 Solution 1. aliensh> packages 2. aliensh> queue priority list 3. aliensh> queue priority jobs % 10 4. aliensh> queue list
GDB Meeting, 08.11. 2006-16 Exercise 6.2 Job Management Prepare an analysis job example in your home directory 1. Change into the directory /alice/cern.ch/tutorial-asia/batch 2. Copy all files from var/ xml/ jdl/ to your home directores (~/bin ~/var ~/xml ~/jdl ) into storage element ALICE::CERN::se01 3. Edit the JDL file ~/jdl/batch-analysis.jdl 4. Correct the paths in InputFile & InputDataCollection to the proper paths in your home directory 5. Submit the edited JDL file 6. Follow the progress of the job Monitor with the 'ps' command Retrieve your job JDL with the 'ps' command Get a job trace with the 'trace' command When the job runs, spy on the output of the stdout file in the job sandbox 7. Find the output of the job in the /proc directory Download all output files to your local computer
Exercise 6.2 Solution 1. aliensh> cd /alice/cern.ch/tutorial-asia/batch 2. aliensh> 1. cp bin/ ~/bin@alice::cern::se01 2. cp xml/ ~/xml@alice::cern::se01 3. cp var/ ~/var@alice::cern::se01 3. aliensh> 1. cd ~/bin 2. edit batch.sh 4. correct the paths in InputData & InputDataCollection 5. aliensh> 1. cd ~/jdl 2. submit batch-analysis.xml 6. aliensh 1. ps 2. ps -jdl <jobid> 3. ps -trace <jobid> all 4. spy <jobid> stdout 7. aliensh> find /proc/<username>/<jobid> * aliensh> cp /proc/<username>/<jobid>/subjobs/job-output/ /tmp/ GDB Meeting, 08.11. 2006-17
GDB Meeting, 08.11. 2006-18 1. Modify the JDL to split into 5 sub jobs 2. Submit the Job 3. Wait for your jobs to finish Exercise 6.3 Job Management 4. Merge all ROOT output files using the 'histomerge.c' macro
GDB Meeting, 08.11. 2006-19 Exercise 6.3 Solution 1. aliensh> edit batch-analysis.jdl uncomment line in JDL: 1. MAXNUMBEROFINPUTFILES = 2 2. aliensh> submit batch-analysis.jdl 3. aliensh> ps (until status = D) 4. aliensh> cp /alice/cern.ch/tutorial-asia/root/histomerge.c 5. bash>./root.sh 1. root>.x histomerge.c 2. root> histomerge.c( /proc/<username>/<procid>/, *.root, merged.root ); 3. root> new TBrowser(); 1. browse the result file