glite Data Management System Hands-on

Similar documents
Data Management. Enabling Grids for E-sciencE. Vladimir Slavnic Scientific Computing Laboratory Institute of Physics Belgrade, Serbia

glite Middleware Usage

glite Grid Services Overview

Introduction Data Management Jan Just Keijser Nikhef Grid Tutorial, November 2008

Operating systems fundamentals - B10

glite UI Installation

glite/egee in Practice

Consorzio COMETA - Progetto PI2S2. DMS API glite. Salvatore Scifo Consorzio Cometa (PI2S2) - Catania. Corso introduttivo al Grid Computing

MyProxy Server Installation

Service Availability Monitor tests for ATLAS

Assume that username is cse. The user s home directory will be /home/cse. You may remember what the relative pathname for users home directory is: ~

Understanding StoRM: from introduction to internals

File Access Control Lists (ACLs)

Grid services. Enabling Grids for E-sciencE. Dusan Vudragovic Scientific Computing Laboratory Institute of Physics Belgrade, Serbia

Grid Data Management

EUROPEAN MIDDLEWARE INITIATIVE

CST8207: GNU/Linux Operating Systems I Lab Six Linux File System Permissions. Linux File System Permissions (modes) - Part 1

CSE 390a Lecture 3. Multi-user systems; remote login; editors; users/groups; permissions

Failover procedure for Grid core services

LCG-2 and glite Architecture and components

Working with Basic Linux. Daniel Balagué

Links, basic file manipulation, environmental variables, executing programs out of $PATH

Beob Kyun KIM, Christophe BONNAUD {kyun, NSDC / KISTI

Unix File System. Class Meeting 2. * Notes adapted by Joy Mukherjee from previous work by other members of the CS faculty at Virginia Tech

Scientific data management

Unix Filesystem. January 26 th, 2004 Class Meeting 2

File Security Lock Down Your Data. Brian Reames January 22, 2012

5/8/2012. Encryption-based Protection. Protection based on Access Permission (Contd) File Security, Setting and Using Permissions Chapter 9

CSE 390a Lecture 4. Persistent shell settings; users/groups; permissions

Privileges: who can control what

AMGA metadata catalogue system

CSE 390a Lecture 4. Persistent shell settings; users/groups; permissions

The glite middleware. Ariel Garcia KIT

Files (review) and Regular Expressions. Todd Kelley CST8207 Todd Kelley 1

CSN08101 Digital Forensics. Module Leader: Dr Gordon Russell Lecturers: Robert Ludwiniak

: the User (owner) for this file (your cruzid, when you do it) Position: directory flag. read Group.

Actual4Test. Actual4test - actual test exam dumps-pass for IT exams

Bookkeeping and submission tools prototype. L. Tomassetti on behalf of distributed computing group

Introduction to Linux

User Commands chmod ( 1 )

SLCS and VASH Service Interoperability of Shibboleth and glite

Linux Operating System Environment Computadors Grau en Ciència i Enginyeria de Dades Q2

Grid Computing. Olivier Dadoun LAL, Orsay Introduction & Parachute method. APC-Grid February 2007

The AMGA Metadata Service

StoRM configuration. namespace.xml

bash startup files Linux/Unix files stty Todd Kelley CST8207 Todd Kelley 1

Chelonia User s manual

Unix Introduction to UNIX

GRID COMPANION GUIDE

Seeing file permission/ownership ls -l (Shows the long listing of a file/directory, which allows you to see file permission & ownership)

CSE 303 Lecture 4. users/groups; permissions; intro to shell scripting. read Linux Pocket Guide pp , 25-27, 61-65, , 176

Grid Computing. Olivier Dadoun LAL, Orsay. Introduction & Parachute method. Socle 2006 Clermont-Ferrand Orsay)

AMGA tutorial. Enabling Grids for E-sciencE

read: permitted to read the contents of directory ( view files and sub-directories in that directory ).

Operating Systems. Engr. Abdul-Rahman Mahmood MS, PMP, MCP, QMR(ISO9001:2000) alphapeeler.sf.net/pubkeys/pkey.htm

DIRAC distributed secure framework

dcache Introduction Course

Group Editing of Files and Web Sites

Experience of Data Grid simulation packages using.

Architecture Proposal

National University of Computer and Emerging Sciences Operating System Lab - 02 Lab Manual

Operating Systems. Copyleft 2005, Binnur Kurt

Operating Systems 3. Operating Systems. Content. What is an Operating System? What is an Operating System? Resource Abstraction and Sharing

Basic UNIX commands. HORT Lab 2 Instructor: Kranthi Varala

Argus: The Simplified Policy Language

PESIT Bangalore South Campus

Files and Directories

You can access data using the FTP/SFTP protocol. This document will guide you in the procedures for configuring FTP/SFTP access.

Linux Training. for New Users of Cluster. Georgia Advanced Computing Resource Center University of Georgia Suchitra Pakala

International Collaboration to Extend and Advance Grid Education. glite WMS Workload Management System

Course 144 Supplementary Materials. UNIX Fundamentals

Unix Internal Assessment-2 solution. Ans:There are two ways of starting a job in the background with the shell s & operator and the nohup command.

Basic File Attributes

Using ACLs with Fedora Core 2 (Linux Kernel 2.6.5)

Monitoring tools in EGEE

CS/CIS 249 SP18 - Intro to Information Security

12- File system and access rights

( PROPOSAL ) THE AGATA GRID COMPUTING MODEL FOR DATA MANAGEMENT AND DATA PROCESSING. version 0.6. July 2010 Revised January 2011

Discretionary Access Control

CS Unix Tools. Lecture 2 Fall Hussam Abu-Libdeh based on slides by David Slater. September 10, 2010

A QUICK INTRODUCTION TO VOMS

Deliverable D8.9 - First release of DM services

Introduction. SSH Secure Shell Client 1

Week 2 Lecture 3. Unix

DELL EMC UNITY: DR ACCESS AND TESTING. Dell EMC Unity OE 4.5

SECTION -C. Getting Started with UNIX

HDP HDFS ACLs 3. Apache HDFS ACLs. Date of Publish:

GRID COMPUTING APPLIED TO OFF-LINE AGATA DATA PROCESSING. 2nd EGAN School, December 2012, GSI Darmstadt, Germany

Advanced Job Submission on the Grid

Improving Grid User's Privacy with glite Pseudonymity Service

Operating system security models

User Commands ls ( 1 )

30 Nov Dec Advanced School in High Performance and GRID Computing Concepts and Applications, ICTP, Trieste, Italy

Comparative evaluation of software tools accessing relational databases from a (real) grid environments

DELL EMC UNITY: DR ACCESS AND TESTING. Dell EMC Unity OE 4.3

FILESYSTEMS. Mmmm crunchy

Overview of HEP software & LCG from the openlab perspective

FluidFS in a Multi-protocol (SMB/NFS) Environment

Interconnect EGEE and CNGRID e-infrastructures

Veritas NetBackup Copilot for Oracle Configuration Guide. Release 2.7.3

Transcription:

glite Data Management System Hands-on Antonio Calanducci National Institute of Nuclear Physics (INFN) - Catania EGEE Grid tutorial for Users and Sysadmin Barcelona, 14th-18th April 2008 www.eu-egee.org EGEE-II INFSO-RI-031688 EGEE and glite are registered trademarks

Practicals on LFC and lcg-utils EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 2

Set up your environment Check that the following environment variables, used to specify the catalog type and its location, are set up properly: export LCG_CATALOG_TYPE=lfc export LFC_HOST=lfc-gilda.ct.infn.it Ensure you have created a proxy certificate and it is still valid. If not create it by: voms-proxy-init --voms gilda EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 3

LFC Catalog commands EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 4

LFC Catalog commands Listing the entries of a LFC directory lfc-ls [-cdillrtu] [--comment] path where path specifies the LFC pathname (mandatory) Remember that LFC has a directory tree structure /grid/<vo_name>/<you create it> LFC Namespace Defined by the user All members of a given VO have read-write permissions under their directory -l (it is a lowercase L ) outputs long listing -R lists the contents of directories recursively (don t use it so often!) You can set LFC_HOME to use relative paths LFC_HOME=/grid/eela/myDir /grid/eela/mydir/myfile becomes myfile EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 4

lfc-ls examples EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 5

lfc-ls examples $ lfc-ls l /grid/gilda/tutorial EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 5

lfc-ls examples $ lfc-ls l /grid/gilda/tutorial... drwxrwxr-x 0 117 102 0 Mar 15 16:53 lookee drwxrwxr-x 0 122 102 0 Mar 15 16:54 miacartella dr--rwxrwx 0 114 102 0 Mar 15 16:50 myfolder drwxrwxr-x 0 114 102 0 Mar 15 16:51 myfolder2 -rwxrwxr-x 1 129 102 193 Mar 15 17:06 nome.txt lrwxrwxrwx 1 129 102 0 Mar 15 17:11 nome2.txt -> / grid/gilda/tutorial/nome.txt drwxrwxr-x 0 129 102 0 Mar 15 16:53 p19 drwxrwxr-x 3 124 102 0 Mar 15 17:13 pippo -rwxrwxr-x 1 124 102 318 Mar 15 17:18 pippopro.txt drwxrwxr-x 0 128 102 0 Mar 15 17:00 pollo... EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 5

lfc-ls examples $ lfc-ls l /grid/gilda/tutorial... drwxrwxr-x 0 117 102 0 Mar 15 16:53 lookee drwxrwxr-x 0 122 102 0 Mar 15 16:54 miacartella dr--rwxrwx 0 114 102 0 Mar 15 16:50 myfolder drwxrwxr-x 0 114 102 0 Mar 15 16:51 myfolder2 -rwxrwxr-x 1 129 102 193 Mar 15 17:06 nome.txt lrwxrwxrwx 1 129 102 0 Mar 15 17:11 nome2.txt -> / grid/gilda/tutorial/nome.txt drwxrwxr-x 0 129 102 0 Mar 15 16:53 p19 drwxrwxr-x 3 124 102 0 Mar 15 17:13 pippo -rwxrwxr-x 1 124 102 318 Mar 15 17:18 pippopro.txt drwxrwxr-x 0 128 102 0 Mar 15 17:00 pollo... $ export LFC_HOME=/grid/gilda/tutorial $ lfc-ls l note.txt EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 5

lfc-ls examples $ lfc-ls l /grid/gilda/tutorial... drwxrwxr-x 0 117 102 0 Mar 15 16:53 lookee drwxrwxr-x 0 122 102 0 Mar 15 16:54 miacartella dr--rwxrwx 0 114 102 0 Mar 15 16:50 myfolder drwxrwxr-x 0 114 102 0 Mar 15 16:51 myfolder2 -rwxrwxr-x 1 129 102 193 Mar 15 17:06 nome.txt lrwxrwxrwx 1 129 102 0 Mar 15 17:11 nome2.txt -> / grid/gilda/tutorial/nome.txt drwxrwxr-x 0 129 102 0 Mar 15 16:53 p19 drwxrwxr-x 3 124 102 0 Mar 15 17:13 pippo -rwxrwxr-x 1 124 102 318 Mar 15 17:18 pippopro.txt drwxrwxr-x 0 128 102 0 Mar 15 17:00 pollo... $ export LFC_HOME=/grid/gilda/tutorial $ lfc-ls l note.txt -rwxrwxr-x 1 129 102 193 Mar 15 17:06 /grid/gilda/tutorial/note.txt EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 5

LFC Catalog commands EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 6

LFC Catalog commands Creating a symbolic link lfc-ln -s file linkname lfc-ln -s directory linkname Create a link to the specified file or directory with linkname Example: $ lfc-ln -s /grid/gilda/tutorial/prova.txt /grid/ gilda/tutorial/linktoprova.txt Let s check the link using lfc-ls with long listing (-l) $ lfc-ls -l /grid/gilda/tutorial lrwxrwxrwx 1 101 102 0 Apr 03 11:42 linktoprova.txt -> /grid/ gilda/tutorial/prova.txt EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 6

LFC Catalog commands EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 7

LFC Catalog commands Creating directories in the LFC lfc-mkdir [-m mode] [-p] path... Where path specifies the LFC pathname Remember that while registering a new file (using lcg-cr, for example) the corresponding destination directory must be already created in the catalog Examples: $ lfc-mkdir /grid/gilda/tutorial/examples You can just check the directory with: $ lfc-ls -l /grid/gilda/tutorial EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 7

LFC Catalog commands EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 8

LFC Catalog commands Adding/deleting metadata information lfc-setcomment path comment lfc-delcomment path lfc-setcomment adds/replaces a comment associated with a file/directory in the LFC Catalog lfc-delcomment deletes a comment previously added Example: lfc-setcomment /grid/gilda/tutorial/hello.txt Hello World! Check your job with lfc-ls --comment /grid/gilda/tutorial/hello.txt /grid/gilda/catania hello.txt Hello World! EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 8

LFC Catalog commands EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 9

LFC Catalog commands Example: lfc-delcomment /grid/gilda/tutorial/hello.txt Check if it was successful with lfc-ls l --comment /grid/gilda/tutorial/hello.txt -rw-rw-r-- 1 101 102 13 Apr 03 11:23 /grid/gilda/catania/hello.txt EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 9

Hands-on Session EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 10

Hands-on Session Exercise No.1: EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 10

Hands-on Session Exercise No.1: Log into the UI and initialize your proxy credentials if not already done EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 10

Hands-on Session Exercise No.1: Log into the UI and initialize your proxy credentials if not already done check that your environment variables are properly set up to use the lfc.cecalc.ula.ve catalog EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 10

Hands-on Session Exercise No.1: Log into the UI and initialize your proxy credentials if not already done check that your environment variables are properly set up to use the lfc.cecalc.ula.ve catalog have a look inside the catalog EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 10

Hands-on Session Exercise No.1: Log into the UI and initialize your proxy credentials if not already done check that your environment variables are properly set up to use the lfc.cecalc.ula.ve catalog have a look inside the catalog create a sub-directory, under /grid/gilda/ barcelona, with your surname EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 10

Hands-on Session Exercise No.1: Log into the UI and initialize your proxy credentials if not already done check that your environment variables are properly set up to use the lfc.cecalc.ula.ve catalog have a look inside the catalog create a sub-directory, under /grid/gilda/ barcelona, with your surname put inside the just created dir a link to an existing file (from files in other directories) EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 10

Hands-on Session Exercise No.1: Log into the UI and initialize your proxy credentials if not already done check that your environment variables are properly set up to use the lfc.cecalc.ula.ve catalog have a look inside the catalog create a sub-directory, under /grid/gilda/ barcelona, with your surname put inside the just created dir a link to an existing file (from files in other directories) add a comment to that file and verify it EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 10

LFC Catalog commands Summary of the LFC Catalog commands lfc-chmod lfc-chown lfc-delcomment lfc-getacl lfc-ln lfc-ls lfc-mkdir lfc-rename lfc-rm lfc-setacl lfc-setcomment Change access mode of the LFC file/directory Change owner and group of the LFC file-directory Delete the comment associated with the file/directory Get file/directory access control lists Make a symbolic link to a file/directory List file/directory entries in a directory Create a directory Rename a file/directory Remove a file/directory Set file/directory access control lists Add/replace a comment EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 11

lcg-utils EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 12

lcg-utils The LCG Data Management tools (usually called lcgutils) allow users to copy files between UI, CE, WN and a SE, to register entries in the File Catalogs and replicate files between SEs. EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 12

lcg-utils The LCG Data Management tools (usually called lcgutils) allow users to copy files between UI, CE, WN and a SE, to register entries in the File Catalogs and replicate files between SEs. EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 12

lcg-utils The LCG Data Management tools (usually called lcgutils) allow users to copy files between UI, CE, WN and a SE, to register entries in the File Catalogs and replicate files between SEs. Check if LCG_GFAL_INFOSYS environment variable is correctly set to the local GILDA Information Index (BDII) export LCG_GFAL_INFOSYS=glite-rb.ct.infn.it:2170 EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 12

lcg-utils: lcg-cr EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 13

lcg-utils: lcg-cr Upload a file to a SE and register it into the catalog lcg-cr -d dest_file dest_host -l lfn [-g guid] [-l lfn] [-v --verbose] --vo vo_name src_file where dest_host is the fully qualified hostname of the destination SE dest_file is a valid SURL (both sfn:// or srm:// format are valid) guid specifies the Grid Unique IDentifier. If this option is not present, a GUID is generated internally lfn specifies the Logical File Name associated with the file vo specifies the Virtual Organization the user belongs to src_file specifies the source file name: the protocol can be file:/// or gsiftp:/// EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 13

lcg-utils: lcg-cr EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 14

lcg-utils: lcg-cr To discover which SEs the user is allowed to use, you can use the lcg-infosites command. lcg-infosites --vo gilda se The output is a list of SEs and related information on available/used space lcg-cr usage example: EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 14

lcg-utils: lcg-cr To discover which SEs the user is allowed to use, you can use the lcg-infosites command. lcg-infosites --vo gilda se The output is a list of SEs and related information on available/used space lcg-cr usage example: $ lcg-cr -v -d trigriden01.unime.it -l lfn:/grid/gilda/tutorial/note.txt -- vo gilda file:///home/tcaland/note.txt Using grid catalog type: lfc Using grid catalog : lfc.trigrid.it Source URL: file:///home/tcaland/note.txt File size: 279 VO name: gilda Destination specified: life006.dft.unipa.it Destination URL for copy: gsiftp://trigriden01.unime.it/flatfiles/se00/gilda/generated/2006-04-03/ file2cc930b1-c61f-4747-b55a-bc521c93622b # streams: 1 # set timeout to 0 seconds Alias registered in Catalog: lfn:/grid/gilda/tutorial/note.txt 279 bytes 0.12 KB/sec avg 0.12 KB/sec inst Transfer took 29520 ms Destination URL registered in Catalog: sfn://trigriden01.unime.it/flatfiles/se00/gilda/generated/ 2006-04-03/file2cc930b1-c61f-4747-b55a-bc521c93622b guid:aa5952f7-27b8-4acc-b6ee-0031af0a0f19 EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 14

lcg-utils: lcg-aa and lcg-la EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 15

Adding an alias for a given GUID lcg-aa --vo vo guid lfn lcg-utils: lcg-aa and lcg-la where vo specifies the Virtual Organization the user belongs to guid specifies the Grid Unique Identifier of the file you want to add the alias to lfn specifies the new alias Example: $ lcg-aa --vo gilda guid:aa5952f7-27b8-4acc-b6ee-0031af0a0f19 lfn:/grid/gilda/tutorial/aliastonote.txt To check if the previous command was successful, you can use lcg-la command to list the aliases for a given LFN, GUID or SURL $ lcg-la --vo gilda lfn:/grid/gilda/tutorial/ aliastonote.txt lfn:/grid/gilda/tutorial/note.txt lfn:/grid/gilda/tutorial/aliastonote.txt EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 15

Hands-on session EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 16

Hands-on session Exercise No.2: EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 16

Hands-on session Exercise No.2: verify that your LCG_GFAL_INFOSYS is correctly set up EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 16

Hands-on session Exercise No.2: verify that your LCG_GFAL_INFOSYS is correctly set up create a dummy file EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 16

Hands-on session Exercise No.2: verify that your LCG_GFAL_INFOSYS is correctly set up create a dummy file check the available storage elements EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 16

Hands-on session Exercise No.2: verify that your LCG_GFAL_INFOSYS is correctly set up create a dummy file check the available storage elements copy and register the previous created file into your previously created dir EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 16

Hands-on session Exercise No.2: verify that your LCG_GFAL_INFOSYS is correctly set up create a dummy file check the available storage elements copy and register the previous created file into your previously created dir add an alias to the just uploaded file EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 16

Hands-on session Exercise No.2: verify that your LCG_GFAL_INFOSYS is correctly set up create a dummy file check the available storage elements copy and register the previous created file into your previously created dir add an alias to the just uploaded file check if the alias was assigned correctly EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 16

lcg-utils commands for replicas (I) Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 17

lcg-utils commands for replicas (I) Enabling Grids for E-sciencE Copying a file from one SE to another one and register its replica in the Catalog where lcg-rep -d dest_file dest_host [-v --verbose] --vo vo src_file dest_host is the fully qualified hostname of the destination SE dest_file is a valid SURL (both sfn:// or srm:// are valid) vo specifies the Virtual Organization the user belongs to src_file specifies the source file name: the protocol can be LFN, GUID or SURL. An SURL scheme can be sfn: for a classical SE or srm: EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 17

lcg-utils commands for replicas (I) Enabling Grids for E-sciencE Copying a file from one SE to another one and register its replica in the Catalog where lcg-rep -d dest_file dest_host [-v --verbose] --vo vo src_file dest_host is the fully qualified hostname of the destination SE dest_file is a valid SURL (both sfn:// or srm:// are valid) vo specifies the Virtual Organization the user belongs to src_file specifies the source file name: the protocol can be LFN, GUID or SURL. An SURL scheme can be sfn: for a classical SE or srm: $ lcg-rep -v -d grid009.ct.infn.it --vo gilda lfn:/grid/gilda/tutorial/note.txt Using grid catalog type: lfc Using grid catalog : lfc.trigrid.it Source URL: lfn:/grid/gilda/tutorial/note.txt File size: 279 VO name: gilda Destination specified: grid009.ct.infn.it Source URL for copy: gsiftp://gildase.oact.inaf.it/flatfiles/se00/gilda/generated/2006-04-03/ file02c844ad-1299-44db-a347-75ed3003c11e Destination URL for copy: gsiftp://grid009.ct.infn.it/flatfiles/se00/gilda/generated/2006-04-03/ file1e0415e0-3486-42bc-947a-824451f69dc4 # streams: 1 # set timeout to 0 0 bytes 0.00 KB/sec avg 0.00 KB/sec inst Transfer took 2020 ms Destination URL registered in LRC: sfn://grid009.ct.infn.it/flatfiles/se00/gilda/generated/2006-04-03/ file1e0415e0-3486-42bc-947a-824451f69dc4 EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 17

lcg-utils commands for replicas (II) Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 18

lcg-utils commands for replicas (II) Enabling Grids for E-sciencE Listing of replicas for a given LFN, GUID or SURL lcg-lr --vo vo_name file where vo_name specifies the Virtual Organization the user belongs to file specifies the Logical File Name, the Grid Unique IDentifier or the Site URL. An SURL scheme can be sfn: for a classical SE or srm: Example: $ lcg-lr --vo gilda lfn:/grid/gilda/tutorial/note.txt sfn://gildase.oact.inaf.it/flatfiles/se00/gilda/generated/2006-04-03/file02c844ad-1299-44dba347-75ed3003c11e sfn://grid009.ct.infn.it/flatfiles/se00/gilda/generated/2006-04-03/ file1e0415e0-3486-42bc-947a-824451f69dc4 sfn://life006.dft.unipa.it/flatfiles/se00/gilda/generated/2006-04-03/file2cc930b1-c61f-4747- b55a-bc521c93622b we can have the same output using the GUID: $ lcg-lr --vo gilda guid:aa5952f7-27b8-4acc-b6ee-0031af0a0f19 EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 18

lcg-utils commands for replicas (III) Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 19

lcg-utils commands for replicas (III) Enabling Grids for E-sciencE Deleting replicas lcg-del [ -a ] [ -s se ] [ -v --verbose ] --vo vo file where a is used to delete all replicas of the given file se specifies the SE from which you want to remove the replica vo specifies the Virtual Organization the user belongs to file specifies the Logical File Name, the Grid Unique IDentifier or the Site URL. An SURL scheme can be sfn: for a classical SE or srm:. Example: delete one replica $ lcg-del --vo gilda -s grid009.ct.infn.it lfn:/grid/gilda/ tutorial/note.txt delete all the replicas $ lcg-del -a --vo gilda lfn:/grid/gilda/tutorial/note.txt let s check if the previous command was successful $ lcg-lr --vo gilda lfn:/grid/gilda/tutorial/note.txt lcg_lr: No such file or directory or by lfs-ls /grid/gilda/tutorial (you will not see anymore note.txt and its alias) EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 19

lcg-utils: lcg-cp EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 20

lcg-utils: lcg-cp Downloading a Grid file in a SE to a local destination lcg-cp [ -v --verbose ] --vo vo src_file dest_file where vo specifies the Virtual Organization the user belongs to src_file specifies the source file name: the protocol can be LFN, GUID, SURL or local file. An SURL scheme can be sfn: for a classical SE or srm: dest_file specifies the destination. The protocol can be file:/// or gsiftp:/// Example: $ lcg-cp --vo gilda lfn:/grid/gilda/tutorial/note.txt file:/home/claudio/note2.txt Using grid catalog type: lfc Using grid catalog : lfc.trigrid.it Source URL: lfn:/grid/gilda/tutorial/note.txt File size: 279 VO name: gilda Source URL for copy: gsiftp://gildase.oact.inaf.it/flatfiles/se00/gilda/ generated/2006-04-03/file02c844ad-1299-44db-a347-75ed3003c11e Destination URL: file:/home/claudio/note2.txt # streams: 1 # set timeout to 0 (seconds) 0 bytes 0.00 KB/sec avg 0.00 KB/sec inst Transfer took 1030 ms EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 20

Hands-on session EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 21

Hands-on session Exercise No.3: EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 21

Hands-on session Exercise No.3: Create two replicas of the file you previously uploaded (you could also use the alias to point it out) EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 21

Hands-on session Exercise No.3: Create two replicas of the file you previously uploaded (you could also use the alias to point it out) Check if the operation was successful EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 21

Hands-on session Exercise No.3: Create two replicas of the file you previously uploaded (you could also use the alias to point it out) Check if the operation was successful Download the file back in your UI EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 21

Hands-on session Exercise No.3: Create two replicas of the file you previously uploaded (you could also use the alias to point it out) Check if the operation was successful Download the file back in your UI Delete just one replica and verify that EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 21

Hands-on session Exercise No.3: Create two replicas of the file you previously uploaded (you could also use the alias to point it out) Check if the operation was successful Download the file back in your UI Delete just one replica and verify that Delete all the replicas and verify that EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 21

Hands-on session Exercise No.3: Create two replicas of the file you previously uploaded (you could also use the alias to point it out) Check if the operation was successful Download the file back in your UI Delete just one replica and verify that Delete all the replicas and verify that Verify if the entry is still into the catalog EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 21

Handling entries permissions Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 22

Handling entries permissions Enabling Grids for E-sciencE Get LFC entries (file/directories) permissions and ACLs EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 22

Handling entries permissions Enabling Grids for E-sciencE Get LFC entries (file/directories) permissions and ACLs lfc-getacl path... EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 22

Handling entries permissions Enabling Grids for E-sciencE Get LFC entries (file/directories) permissions and ACLs lfc-getacl path... EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 22

Handling entries permissions Enabling Grids for E-sciencE Get LFC entries (file/directories) permissions and ACLs lfc-getacl path... where path is a LFC file or directory EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 22

Handling entries permissions Enabling Grids for E-sciencE Get LFC entries (file/directories) permissions and ACLs lfc-getacl path... where path is a LFC file or directory Example: $ lfc-getacl /grid/gilda/tutorial # file: /grid/gilda/tutorial # owner: /C=IT/O=GILDA/OU=Personal Certificate/L=CATANIA/CN=CATANIA01/ Email=roberto.barbera@ct.infn.it # group: gilda user::rwx group::rwx #effective:rwx other::r-x default:user::rwx default:group::rwx default:other::r-x EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 22

Handling entries permissions Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 23

Handling entries permissions Enabling Grids for E-sciencE Change the permission of a file/directory lfc-chmod absolute_mode path... where absolute_mode is the permission pattern expressed in octal mode path is a LFC file or directory Example: $ lfc-chmod 750 /grid/gilda/tutorial/note.txt $ lfc-getacl /grid/gilda/tutorial/note.txt # file: /grid/gilda/tutorial/note.txt # owner: /C=IT/O=GILDA/OU=Personal Certificate/L=INFN Catania/CN=Tony Calanducci/ Email=tony.calanducci@ct.infn.it # group: gilda user::rwx group::r-x #effective:r-x other::--- EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 23

Setting ACLs (I) EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 24

Setting ACLs (I) Set LFC directory/file access control list lfc-setacl [-d] [-m] [-s] acl_entries path... where acl_entries is a comma separated list of entries. Each entry has colon separated fields: ACL type, id (uid or gid), permission. The entries look like: user::perm user:uid:perm group::perm group:gid:perm mask:perm other:perm default:user::perm default:user:uid:perm default:group::perm default:group:gid:perm default:mask:perm default:other:perm The ACL type can be abbreviated with the first letter EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 24

Setting ACLs (I) Set LFC directory/file access control list lfc-setacl [-d] [-m] [-s] acl_entries path... where acl_entries is a comma separated list of entries. Each entry has colon separated fields: ACL type, id (uid or gid), permission. The entries look like: user::perm user:uid:perm group::perm group:gid:perm mask:perm other:perm default:user::perm default:user:uid:perm default:group::perm default:group:gid:perm default:mask:perm default:other:perm user and group -> owner permission The ACL type can be abbreviated with the first letter EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 24

Setting ACLs (I) Set LFC directory/file access control list lfc-setacl [-d] [-m] [-s] acl_entries path... where acl_entries is a comma separated list of entries. Each entry has colon separated fields: ACL type, id (uid or gid), permission. The entries look like: user::perm user:uid:perm group::perm group:gid:perm mask:perm other:perm default:user::perm default:user:uid:perm default:group::perm default:group:gid:perm default:mask:perm default:other:perm user and group -> owner permission user:gid and group:gid -> specific users and groups The ACL type can be abbreviated with the first letter EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 24

Setting ACLs (I) Set LFC directory/file access control list lfc-setacl [-d] [-m] [-s] acl_entries path... where acl_entries is a comma separated list of entries. Each entry has colon separated fields: ACL type, id (uid or gid), permission. The entries look like: user::perm user:uid:perm group::perm group:gid:perm mask:perm other:perm default:user::perm default:user:uid:perm default:group::perm default:group:gid:perm default:mask:perm default:other:perm user and group -> owner permission user:gid and group:gid -> specific users and groups mask -> maximum permissions to specific users/groups The ACL type can be abbreviated with the first letter EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 24

Setting ACLs (I) Set LFC directory/file access control list lfc-setacl [-d] [-m] [-s] acl_entries path... where acl_entries is a comma separated list of entries. Each entry has colon separated fields: ACL type, id (uid or gid), permission. The entries look like: user::perm user:uid:perm group::perm group:gid:perm mask:perm other:perm default:user::perm default:user:uid:perm default:group::perm default:group:gid:perm default:mask:perm default:other:perm user and group -> owner permission user:gid and group:gid -> specific users and groups mask -> maximum permissions to specific users/groups default permissions (only for a directory) are inherited by the files or sub-directories of that dir The ACL type can be abbreviated with the first letter EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 24

Setting ACLs (II) EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 25

Setting ACLs (II) path specifies the LFC pathname uid can be given as the DN or the corresponding numeric id. gid can be given as the VO name or the corresponding numeric id. perm can be expressed as a combination of characters rwx- or as a value between 0 and 7. OPTIONS -d remove ACL entries. The "perm" field is ignored. -m modify existing ACL entries or add new entries. -s set the ACL entries. The complete set of ACL entries is replaced. EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 25

ACLs example EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 26

ACLs example Example: $ lfc-setacl -m u:'/c=it/o=gilda/ou=personal Certificate/L=INFN/CN=Diego Scardaci/ Email=diego.scardaci@ct.infn.it':rwx,g:trigrid:rx,m:rwx /grid/gilda/tutorial/note.txt $ lfc-getacl /grid/gilda/tutorial/note.txt # file: /grid/gilda/tutorial/note.txt # owner: /C=IT/O=GILDA/OU=Personal Certificate/L=INFN Catania/ CN=Tony Calanducci/Email=tony.calanducci@ct.infn.it # group: gilda user::rwx user:/c=it/o=gilda/ou=personal Certificate/L=INFN/CN=Diego Scardaci/Email=diego.scardaci@ct.infn.it:rwx #effective:rwx group::r-x #effective:r-x group:trigrid:r-x #effective:r-x mask::rwx other::--- EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 26

Exercise No. 4 EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 27

Exercise No. 4 Create an entry in your working FC directory. Take a look to the default permission of it. EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 27

Exercise No. 4 Create an entry in your working FC directory. Take a look to the default permission of it. Change the permission in a way that only you (the owner) can delete the entry EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 27

Exercise No. 4 Create an entry in your working FC directory. Take a look to the default permission of it. Change the permission in a way that only you (the owner) can delete the entry Ask your neighbor to delete that entry (he should not be able) EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 27

Exercise No. 4 Create an entry in your working FC directory. Take a look to the default permission of it. Change the permission in a way that only you (the owner) can delete the entry Ask your neighbor to delete that entry (he should not be able) Ask again your neighbor his DN and add it as an ACL entry granting full permission. EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 27

Exercise No. 4 Create an entry in your working FC directory. Take a look to the default permission of it. Change the permission in a way that only you (the owner) can delete the entry Ask your neighbor to delete that entry (he should not be able) Ask again your neighbor his DN and add it as an ACL entry granting full permission. Ask now to delete it (it should be able to do that now) delete with lcg-del -s STORAGEHOST EGEE-II INFSO-RI-031688 EGEE Tutorial - Barcelona 14th - 18th April 2008, 27