International Collaboration to Extend and Advance Grid Education. glite WMS Workload Management System
|
|
- Mervin Leonard
- 5 years ago
- Views:
Transcription
1 International Collaboration to Extend and Advance Grid Education glite WMS Workload Management System Marco Pappalardo Consorzio COMETA & INFN Catania, Italy ITIS Ferraris, Acireale, Tutorial GRID per gli Insegnanti, INFSO-SSA-26637
2 Workload Management System (WMS) components and services User Interface (RB) Resource Broker (RB) Logging and Bookkeeping (LB) Computing Element (CE) Contents Job Description Language (JDL) JDL document Sandboxes attachment Type of jobs Type of requests
3 Overview of the Architecture
4 Architecture overview Output Sandbox Replicas info Authentication & authirization User Interface Job submit event Input Sandbox Job status Network Server (Resource Broker) Input Sandbox Output Sandbox Publish LHC File Catalogue Information Index Logging & Bookkeeping Computing Element Workload Management System components Storage Element
5 Components and Services User Interface (UI) terminal to access to all grid facilities, including the WMS Command Line Interface (CLI) Grid Portals (such as GENIUS) Network Server(NS) / Resource Broker (RB) WMS access point. Dispatches jobs across computing resources Implements some scheduling algorithm Logging and Bookkeeping (LB) Keeps track of any WMS status or action Computing Element (CE) Is the actual computing resource It is an interface. We don t know what stays beyond
6 More on Computing Elements Logically, it is a queue of pending jobs Physically, it is a farm of Worker Nodes (WNs) Stays on top of a Local Resource Management System (LRMS) PBS, Condor, LSF CCS in the future? It exposes a common interface independent of the underlying LRMS
7 Job Life Cycle (1/4) X The grid user describes a job via a Job Description Language (JDL) document. Some input files (Input Sandbox) can be attached to the JDL doc. The grid user submits the JDL job using the CLI and waits for reply. The Resource Broker gets and stores the JDL document together with attached input files. The just generated jobid is sent back to the user to refer to that job univocally in the future
8 Job Life Cycle (2/4) The Resources Broker executes a special algorithm (MatchMaking) and selects a Computing Element according to best-fit rules. The job is handed to chosen CE together with the Input SandBox The Computing Element accepts the job and queues it. The job starts execution over the Local Resources Management System (LRMS)
9 Job Life Cycle (3/4) When the job terminates, the produced output is sent back to the Resource Broker The Resource Broker gets the results and the Output Sandbox and stores them in the local repository At the same time, the Computing Element notifies the Logging & Bookkeeping Now the job output is available on the Resource Broker
10 Job Life Cycle (4/4) Job-status? Terminated The User queries the L&B to have a look on his/her jobs and realizes that the job has terminated. The User gets Output SandBox from the Resource Broker. The Resource Broker clears all no more needed info from its repository. The job life cycle has terminated (either well or not)!!!
11 Job State Machine Submitted: The job has been created on the UI but not yet sent to the Resource Broker Waiting: The job is now being processed by the Resource Broker Ready: The job has been processed but not yet sent to chosen CE Scheduled: Job is now queued on the CE and is waiting to be executed Running: The job is running on the Computing Element Done: The job has terminated its execution Aborted: The job has been aborted by the WMS Cancelled: The job has been cancelled by the user Cleared: The job has terminated and the output has been retrieved
12 Matchmaking Algorithm The Matchmaking Algorithm (within the RB) Decides how to dispatch jobs across resources (where). Uses the Information System as resource discovery system. First phase: Selection In this phase, the algorithm chooses which Computing Elements are suitable for executing a given job Requirements JDL attribute is evaluated for any candidate CE Second Phase: Ranking A fitness function (Rank JDL attribute) is evaluated over suitable CEs. The CE that maximizes the above function is chosen. The job is submitted to the selected CE.
13 Job Description Language (JDL)
14 Job Description Language The Job Description Language (JDL) is a language to describe job is composed mainly of a collection of attribute-value pairs allows the attachment of files
15 Type of jobs Normal A batch executable Interactive Requires interaction of the user MPICH Needs the Message Passing Interface (MPI) installed on the computing resource Partitionable Can be partitioned into more sub-jobs. Deprecated. Checkpointable Execution can be marked at some specific position of the code (checkpoints) to be resumed later. Deprecated. Parametric a job whose JDL contains parametric attributes (e.g. Arguments, StdInput etc.).
16 Types of requests The JDL allows description of the following types of requests: JOB a simple job (default) DAG a Direct Acyclic Graph of dependent jobs Collection a set of independent jobs
17 JDL format A JDL file consists of lines having the format: Attribute = expression; and terminated by a semicolon. Expressions can span several lines, but only the last one must be terminated by a semicolon. Comments must have a sharp character (#) or a double slash (//) at the beginning of each line. Comments spanning multiple lines can be specified enclosing the text between /* and */.
18 Type Example The Type attribute is a string representing the type of the request described by the JDL, e.g. Type = Job ; Possible values are: Job DAG Collection The value for this attribute is case insensitive. If this attribute is not specified in the JDL description, the WMS will set it to Job. Default: Job
19 JobType Example The JobType attribute is a string representing the type of the job described by the JDL, e.g.: JobType = Interactive ; Possible values are: Normal Interactive MPICH Checkpointable Partitionable Parametric This attribute only makes sense when the Type attribute equals to Job. The value for this attribute is case insensitive. Default: Normal
20 Example of JDL file scriptls.jdl VirtualOrganisation= gilda ; Executable = "ls.sh"; // this will run on the endpoint StdError = "stderr.log"; // redirect stderror to this file StdOutput = "stdout.log"; // redirect stdout to this file InputSandbox = "ls.sh"; // attach this file to the JDL OutputSandbox = // these files will be the output {"stderr.log", stdout.log"}; // for this job ls.sh #!/bin/sh /bin/ls // simply executes ls on the final // computing resource
21 JDL Attributes This is a very not exhaustive list of JDL attributes Type Job or DAG JobType Interactive, MPICH... Executable Arguments StdInput StdOutput and StdError InputSandbox The command line to execute String used as argument for the executable A file attached as standard input Files to which redirect standard output and standard error List of files attached to the JDL OutputSandboxList of files to be retrieved as output Requirements Rank RetryCount Boolean expression to select suitable CEs Fitness function to evaluate over candidate CEs Retry matchmaking
22 Requirements = < logic expression > Expression that uses C-like operators. It represents job requirements on resources. Requirements The Requirements expression can contain attributes that describe the CE in the IS which are prefixed with other.. e.g.: Requirements = other.glueceinfolrmstype == "PBS" && other.glueceinfototalcpus > 2); Rank = < floating point expression > Fitness function that uses C-like operators to select the best CE The Rank expression can contain attributes that describe the CE in the IS which are prefixed with other.. e.g. Rank = other.gluecepolicymaxrunningjobs other.gluecestaterunningjobs
23 Command Line Interface
24 edg-job-list-match [glite edg]-job-* commands Gets the list of CEs that satisfy requirements to execute the job edg-job-submit Submits the job to the Resources Broker and returns the just generated job_id edg-job-status Retrieves the status of the given job edg-job-cancel Cancels a submitted jobs edg-job-get-output Retrieves the output only if the job has terminated
25 Example: hostname.jdl (i) $> cat hostname.jdl Type = Job ; JobType = Normal ; It is a standard job Executable = /bin/sh/ ; Arguments = start_hostname.sh ; The executable to run StdError = stderr.log ; StdOutput = stdout.log ; Redirect standard output and standard error to these files InputSandbox = start_hostname.sh ; OutputSandbox = { stderr.log, stdout.log }; RetryCount = 7; If the job fails the execution, retrys for at most 7 times $> cat start_hostname.sh #!/bin/sh sleep 5 hostname f Attach this file to the JDL document Thesefileshavetoberetrieved when the job terminates
26 Example: hostname.jdl $> edg-job-submit -o jobid hostname.jdl Selected Virtual Organisation name (from proxy certificate extension): gilda Connecting to host glite-rb.ct.infn.it, port 7772 Logging to host glite-rb.ct.infn.it, port 9002 ================== glite-job-submit Success ============================== The job has been successfully submitted to the Network Server. Use edg-job-status command to check job current status. Your job identifier is: - just generated job id The job identifier has been saved in the following file: /home/fscibi/glite/other/jobid ===================================================================== option -o jobid $> cat jobid ###Submitted Job Ids###
27 Example: hostname.jdl $> edg-job-status -i jobid ************************************************************* BOOKKEEPING INFORMATION: Status info for the Job : Current Status: Done (Success) Terminated Exit code: 0 Status Reason: Job terminated successfully Destination: grid004.iucc.ac.il:2119/jobmanager-lcgpbs-short Submitted: Mon Apr 3 12:27: CEST Computing ************************************************************* Element where the job executed
28 Esempio: hostname.jdl (v) edg-job-status -v 3 -i jobid ************************************************************* BOOKKEEPING INFORMATION: Status info for the Job : Current Status: Cleared Status Reason: user retrieved output sandbox Destination: grid004.iucc.ac.il:2119/jobmanager-lcgpbs-short Submitted: Mon Apr 3 12:27: CEST stateentertimes = Submitted : Mon Apr 3 12:27: CEST Waiting : Mon Apr 3 12:27: CEST Ready Scheduled Running Done Cleared Aborted : --- Cancelled : --- Unknown : --- : Mon Apr 3 12:27: CEST : Mon Apr 3 12:28: CEST : Mon Apr 3 12:28: CEST : Mon Apr 3 12:30: CEST : Mon Apr 3 15:36: CEST Job status variation
29 Esempio: hostname.jdl (vi) edg-job-get-output -i jobid Retrieving files from host: glite-rb.ct.infn.it ( for ) ********************************************************************************* JOB GET OUTPUT OUTCOME Output sandbox files for the job: - have been successfully retrieved and stored in the directory: /tmp/glite/glite-ui/fscibi_lb6lihd93s7vyz1rvbcp8a Default dir where retrieved output is stored ********************************************************************************* To specify a different $> edg-job-get-output -i jobid --dir <dirname> directory
30 Esempio: sphere.jdl (i) $> cat sphere.jdl #author: Type = "Job"; JobType = "Normal"; Executable = "/bin/sh"; MyProxyServer="lxshare0207.cern.ch"; StdOutput = "sphere.out"; StdError = "sphere.err"; InputSandbox = {"start_sphere.sh","sphere1.pov","sphere1.ini"}; OutputSandbox = {"sphere.out","sphere.err","final_sphere.gif"}; RetryCount = 7; Arguments = "start_sphere.sh"; Requirements = Member("POVRAY-3.5",other.GlueHostApplicationSoftwareRunTimeEnvironment); Select only CEs where POVRAY is installed
31 Esempio: sphere.jdl (ii) $> edg-job-list-match sphere.jdl Selected Virtual Organisation name (from proxy certificate extension): gilda Connecting to host glite-rb.ct.infn.it, port 7772 *************************************************************************** COMPUTING ELEMENT IDs LIST The following CE(s) matching your job requirements have been found: *CEId* dgt01.ui.savba.sk:2119/jobmanager-lcgpbs-infinite dgt01.ui.savba.sk:2119/jobmanager-lcgpbs-long dgt01.ui.savba.sk:2119/jobmanager-lcgpbs-short egee008.cnaf.infn.it:2119/blah-pbs-infinite egee008.cnaf.infn.it:2119/blah-pbs-long egee008.cnaf.infn.it:2119/blah-pbs-short fenrir.uniandes.edu.co:2119/blah-pbs-infinite fenrir.uniandes.edu.co:2119/blah-pbs-long *************************************************************************** CEs where POVRAY is installed
32 Esempio: sphere.jdl (iii) $> edg-job-submit -o jobid sphere.jdl Connecting to host glite-rb.ct.infn.it, port 7772 Logging to host glite-rb.ct.infn.it, port 9002 ====================== glite-job-submit Success ========================= The job has been successfully submitted to the Network Server. Use glite-job-status command to check job current status. Your job identifier is: - The job identifier has been saved in the following file: /home/fscibi/glite/other/jobid ====================================================================
33 Esempio: pds2jpg-asar-demo.jdl (i) $> cat pds2jpg-asar-demo.jdl [ ] VirtualOrganisation = "gilda"; Executable = "/bin/bash"; Arguments = "pds2jpg_asar_install.sh ASA_APG_1PXPDE _093043_ _00394_02452_0000"; StdOutput = "pds2jpg_asar.out"; StdError = "pds2jpg_asar.err"; OutputSandbox = { "ASA_APG_1PXPDE _093043_ _00394_02452_0000-b1.jpg", "ENVISAT_Product_courtesy_of_European_Space_Agency", "pds2jpg_asar.out", "pds2jpg_asar.err }; RetryCount = 3; JobType = "normal"; Type = "Job"; InputSandbox = {"./pds2jpg_asar_install.sh","./beam20.tar.gz"}; rank = (-other.gluecestateestimatedresponsetime); requirements = (other.gluecestatestatus=="production") The installer is attached to the JDL Fitness function to select the best CE
34 Esempio: pds2jpg-asar-demo.jdl (ii) $> cat pds2jpg_asar_install.sh echo Staging Input Data \(Courtesy of European Space Agency\); #edg-rm --vo=gilda copyfile lfn:$1.n1 file://$pwd/$1.n1; lcg-cp --vo=gilda lfn:$1.n1 file://$pwd/$1.n1; echo Staging Application; gunzip beam20.tar.gz; tar xvf beam20.tar; cd beam-2.0/bin echo Starting Application;./pds2jpg-ASAR-run.sh $1; mv $1-b*.jpg../.. cd../.. rm -fr beam-2.0; rm -fr $PWD/$1.N1; rm -fr $PWD/beam20.tar; echo Input ENVISAT Product courtesy of European Space Agency touch ENVISAT_Product_courtesy_of_European_Space_Agency echo No Output Packaging; echo Done!;
35 Questions
Architecture of the WMS
Architecture of the WMS Dr. Giuliano Taffoni INFORMATION SYSTEMS UNIT Outline This presentation will cover the following arguments: Overview of WMS Architecture Job Description Language Overview WMProxy
More informationProblemi di schedulazione distribuita su Grid
Problemi di schedulazione distribuita su Grid Ivan Porro Università degli Studi di Genova, DIST, Laboratorio BioLab pivan@unige.it 010-3532789 Riadattato da materiale realizzato da INFN Catania per il
More informationAdvanced Job Submission on the Grid
Advanced Job Submission on the Grid Antun Balaz Scientific Computing Laboratory Institute of Physics Belgrade http://www.scl.rs/ 30 Nov 11 Dec 2009 www.eu-egee.org Scope User Interface Submit job Workload
More informationHow to use computing resources at Grid
How to use computing resources at Grid Nikola Grkic ngrkic@ipb.ac.rs Scientific Computing Laboratory Institute of Physics Belgrade, Serbia Academic and Educat ional Gr id Init iat ive of S er bia Oct.
More informationDataGrid. Document identifier: Date: 16/06/2003. Work package: Partner: Document status. Deliverable identifier:
DataGrid JDL ATTRIBUTES Document identifier: Work package: Partner: WP1 Datamat SpA Document status Deliverable identifier: Abstract: This note provides the description of JDL attributes supported by the
More informationglite Advanced Job Management
The EPIKH Project (Exchange Programme to advance e-infrastructure Know-How) glite Advanced Job Management Porting Application School Cesar Fernández (cesar.fernandez@usm.cl) Valparaíso, 30 November 2010
More informationGergely Sipos MTA SZTAKI
Application development on EGEE with P-GRADE Portal Gergely Sipos MTA SZTAKI sipos@sztaki.hu EGEE Training and Induction EGEE Application Porting Support www.lpds.sztaki.hu/gasuc www.portal.p-grade.hu
More informationDataGrid. Document identifier: Date: 28/10/2003. Work package: Partner: Document status. Deliverable identifier:
DataGrid JDL ATTRIBUTES Document identifier: Work package: Partner: WP1 Datamat SpA Document status Deliverable identifier: Abstract: This note provides the description of JDL attributes supported by the
More informationDataGrid. Document identifier: Date: 24/11/2003. Work package: Partner: Document status. Deliverable identifier:
DataGrid WMS GUI USER G UIDE Document identifier: Work package: Partner: WP1 Datamat SpA Document status Deliverable identifier: Abstract: This document provides a description of all functionalities provided
More informationGrid Computing. Olivier Dadoun LAL, Orsay. Introduction & Parachute method. Socle 2006 Clermont-Ferrand Orsay)
virtual organization Grid Computing Introduction & Parachute method Socle 2006 Clermont-Ferrand (@lal Orsay) Olivier Dadoun LAL, Orsay dadoun@lal.in2p3.fr www.dadoun.net October 2006 1 Contents Preamble
More informationGrid Computing. Olivier Dadoun LAL, Orsay Introduction & Parachute method. APC-Grid February 2007
Grid Computing Introduction & Parachute method APC-Grid February 2007 Olivier Dadoun LAL, Orsay http://flc-mdi.lal.in2p3.fr dadoun@lal.in2p3.fr www.dadoun.net October 2006 1 Contents Machine Detector Interface
More informationglite/egee in Practice
glite/egee in Practice Alex Villazon (DPS, Innsbruck) Markus Baumgartner (GUP, Linz) With material from www.eu-egee.org ISPDC 2007 5-8 July 2007 Hagenberg, Austria EGEE-II INFSO-RI-031688 Overview Introduction
More informationglite Middleware Usage
glite Middleware Usage Dusan Vudragovic dusan@phy.bg.ac.yu Scientific Computing Laboratory Institute of Physics Belgrade, Serbia Nov. 18, 2008 www.eu-egee.org EGEE and glite are registered trademarks Usage
More informationLCG-2 and glite Architecture and components
LCG-2 and glite Architecture and components Author E.Slabospitskaya www.eu-egee.org Outline Enabling Grids for E-sciencE What are LCG-2 and glite? glite Architecture Release 1.0 review What is glite?.
More information30 Nov Dec Advanced School in High Performance and GRID Computing Concepts and Applications, ICTP, Trieste, Italy
Advanced School in High Performance and GRID Computing Concepts and Applications, ICTP, Trieste, Italy Why the Grid? Science is becoming increasingly digital and needs to deal with increasing amounts of
More informationGRID COMPANION GUIDE
Companion Subject: GRID COMPANION Author(s): Miguel Cárdenas Montes, Antonio Gómez Iglesias, Francisco Castejón, Adrian Jackson, Joachim Hein Distribution: Public 1.Introduction Here you will find the
More informationGrid services. Enabling Grids for E-sciencE. Dusan Vudragovic Scientific Computing Laboratory Institute of Physics Belgrade, Serbia
Grid services Dusan Vudragovic dusan@phy.bg.ac.yu Scientific Computing Laboratory Institute of Physics Belgrade, Serbia Sep. 19, 2008 www.eu-egee.org Set of basic Grid services Job submission/management
More informationParallel Computing in EGI
Parallel Computing in EGI V. Šipková, M. Dobrucký, and P. Slížik Ústav informatiky, Slovenská akadémia vied 845 07 Bratislava, Dúbravská cesta 9 http://www.ui.sav.sk/ {Viera.Sipkova, Miroslav.Dobrucky,
More informationAGATA Analysis on the GRID
AGATA Analysis on the GRID R.M. Pérez-Vidal IFIC-CSIC For the e682 collaboration What is GRID? Grid technologies allow that computers share trough Internet or other telecommunication networks not only
More informationJ O B D E S C R I P T I O N L A N G U A G E A T T R I B U T E S S P E C I F I C A T I O N
. E G E E J O B D E S C R I P T I O N L A N G U A G E A T T R I B U T E S S P E C I F I C A T I O N F O R T H E G L I T E W O R K L O A D M A N A G E M E N T S Y S T E M Document identifier: WMS-JDL.odt
More informationWMS overview and Proposal for Job Status
WMS overview and Proposal for Job Status Author: V.Garonne, I.Stokes-Rees, A. Tsaregorodtsev. Centre de physiques des Particules de Marseille Date: 15/12/2003 Abstract In this paper, we describe briefly
More informationA Login Shell interface for INFN-GRID
A Login Shell interface for INFN-GRID S.Pardi2,3, E. Calloni1,2, R. De Rosa1,2, F. Garufi1,2, L. Milano1,2, G. Russo1,2 1Università degli Studi di Napoli Federico II, Dipartimento di Scienze Fisiche, Complesso
More informationThe glite middleware. Ariel Garcia KIT
The glite middleware Ariel Garcia KIT Overview Background The glite subsystems overview Security Information system Job management Data management Some (my) answers to your questions and random rumblings
More informationEUROPEAN MIDDLEWARE INITIATIVE
EUROPEAN MIDDLEWARE INITIATIVE VOMS CORE AND WMS SECURITY ASSESSMENT EMI DOCUMENT Document identifier: EMI-DOC-SA2- VOMS_WMS_Security_Assessment_v1.0.doc Activity: Lead Partner: Document status: Document
More informationglite Grid Services Overview
The EPIKH Project (Exchange Programme to advance e-infrastructure Know-How) glite Grid Services Overview Antonio Calanducci INFN Catania Joint GISELA/EPIKH School for Grid Site Administrators Valparaiso,
More informationDataGrid EDG-BROKERINFO USER GUIDE. Document identifier: Date: 06/08/2003. Work package: Document status: Deliverable identifier:
DataGrid Document identifier: Date: 06/08/2003 Work package: Document status: WP1-WP2 DRAFT Deliverable identifier: Abstract: This document presents what is the BrokerInfo file, the edg-brokerinfo command
More informationDataGrid D EFINITION OF ARCHITECTURE, TECHNICAL PLAN AND EVALUATION CRITERIA FOR SCHEDULING, RESOURCE MANAGEMENT, SECURITY AND JOB DESCRIPTION
DataGrid D EFINITION OF ARCHITECTURE, SECURITY AND JOB DESCRIPTION Document identifier: Work package: Partner: WP1: Workload Management INFN Document status DRAFT Deliverable identifier: DataGrid-D1.2
More informationFuture of Grid parallel exploitation
Future of Grid parallel exploitation Roberto Alfieri - arma University & INFN Italy SuperbB Computing R&D Workshop - Ferrara 6/07/2011 1 Outline MI support in the current grid middleware (glite) MI and
More informationJob submission and management through web services: the experience with the CREAM service
Journal of Physics: Conference Series Job submission and management through web services: the experience with the CREAM service To cite this article: C Aiftimiei et al 2008 J. Phys.: Conf. Ser. 119 062004
More informationParallel Job Support in the Spanish NGI! Enol Fernández del Cas/llo Ins/tuto de Física de Cantabria (IFCA) Spain
Parallel Job Support in the Spanish NGI! Enol Fernández del Cas/llo Ins/tuto de Física de Cantabria (IFCA) Spain Introduction (I)! Parallel applica/ons are common in clusters and HPC systems Grid infrastructures
More informationA Practical Approach for a Workflow Management System
A Practical Approach for a Workflow Management System Simone Pellegrini, Francesco Giacomini, Antonia Ghiselli INFN Cnaf Viale B. Pichat, 6/2 40127 Bologna {simone.pellegrini francesco.giacomini antonia.ghiselli}@cnaf.infn.it
More informationEGEE. Grid Middleware. Date: June 20, 2006
EGEE Grid Middleware HANDOUTS FOR STUDENTS Author(s): Fokke Dijkstra, Jeroen Engelberts, Sjors Grijpink, David Groep, Jeff Templon Abstract: These handouts are provided for people to learn how to use the
More informationglite UI Installation
The EPIKH Project (Exchange Programme to advance e-infrastructure Know-How) glite UI Installation Antonio Calanducci INFN Catania Joint GISELA/EPIKH School for Grid Site Administrators Valparaiso, Chile,
More informationProgramming the Grid with glite
Programming the Grid with glite E. Laure 1, C. Grandi 1, S. Fisher 2, A. Frohner 1, P. Kunszt 3, A. Krenek 4, O. Mulmo 5, F. Pacini 6, F. Prelz 7, J. White 1 M. Barroso 1, P. Buncic 1, R. Byrom 2, L. Cornwall
More informationMPI SUPPORT ON THE GRID. Kiril Dichev, Sven Stork, Rainer Keller. Enol Fernández
Computing and Informatics, Vol. 27, 2008, 213 222 MPI SUPPORT ON THE GRID Kiril Dichev, Sven Stork, Rainer Keller High Performance Computing Center University of Stuttgart Nobelstrasse 19 70569 Stuttgart,
More informationInterconnect EGEE and CNGRID e-infrastructures
Interconnect EGEE and CNGRID e-infrastructures Giuseppe Andronico Interoperability and Interoperation between Europe, India and Asia Workshop Barcelona - Spain, June 2 2007 FP6 2004 Infrastructures 6-SSA-026634
More informationSPGrid Efforts in Italy
INFN - Ferrara BaBarGrid Meeting SPGrid Efforts in Italy BaBar Collaboration Meeting - SLAC December 11, 2002 Enrica Antonioli - Paolo Veronesi Topics Ferrara Farm Configuration First SP submissions through
More informationA unified user experience for MPI jobs in EMI
A unified user experience for MPI jobs in EMI Enol Fernández (CSIC) glite MPI PT Outline Parallel Jobs EMI middleware stacks approaches How to execute a simple MPI job with 16 process with ARC/gLite/UNICORE?
More informationGRID COMPUTING APPLIED TO OFF-LINE AGATA DATA PROCESSING. 2nd EGAN School, December 2012, GSI Darmstadt, Germany
GRID COMPUTING APPLIED TO OFF-LINE AGATA DATA PROCESSING M. KACI mohammed.kaci@ific.uv.es 2nd EGAN School, 03-07 December 2012, GSI Darmstadt, Germany GRID COMPUTING TECHNOLOGY THE EUROPEAN GRID: HISTORY
More informationDIRAC Documentation. Release integration. DIRAC Project. 09:29 20/05/2016 UTC
DIRAC Documentation Release integration DIRAC Project. 09:29 20/05/2016 UTC Contents 1 User Guide 3 1.1 Getting Started.............................................. 3 1.2 Web Portal Reference..........................................
More informationWMS Application Program Interface: How to integrate them in your code
The EPIKH Project (Exchange Programme to advance e-infrastructure Know-How) WMS Application Program Interface: How to integrate them in your code Fabrizio Pistagna (fabrizio.pistagna@ct.infn.it) Kolkata,
More informationParallel computing on the Grid
arallel computing on the Grid and application porting examples Roberto Alfieri - arma University & INFN Italy Comput-er.it Meeting - arma 21/06/2011 1 Current MI support in Grid (glite middleware) Issue
More informationDr. Giuliano Taffoni INAF - OATS
Query Element Demo The Grid Query Element for glite Dr. Giuliano Taffoni INAF - OATS Overview What is a G-DSE? Use and Admin a DB: the Query Element; Upcoming Features; Working on QE People: Edgardo Ambrosi
More informationALHAD G. APTE, BARC 2nd GARUDA PARTNERS MEET ON 15th & 16th SEPT. 2006
GRID COMPUTING ACTIVITIES AT BARC ALHAD G. APTE, BARC 2nd GARUDA PARTNERS MEET ON 15th & 16th SEPT. 2006 Computing Grid at BARC Computing Grid system has been set up as a Test-Bed using existing Grid Technology
More informationGrid Scheduling Architectures with Globus
Grid Scheduling Architectures with Workshop on Scheduling WS 07 Cetraro, Italy July 28, 2007 Ignacio Martin Llorente Distributed Systems Architecture Group Universidad Complutense de Madrid 1/38 Contents
More informationTroubleshooting Grid authentication from the client side
Troubleshooting Grid authentication from the client side By Adriaan van der Zee RP1 presentation 2009-02-04 Contents The Grid @NIKHEF The project Grid components and interactions X.509 certificates, proxies
More informationCE+WN+siteBDII Installation and configuration
The EPIKH Project (Exchange Programme to advance e-infrastructure Know-How) CE+WN+siteBDII Installation and configuration Andrea Cortellese (andrea.cortellese@ct.infn.it) National Institute of Nuclear
More informationGrid Documentation Documentation
Grid Documentation Documentation Release 1.0 Grid Support Nov 06, 2018 Contents 1 General 3 2 Basics 9 3 Advanced topics 25 4 Best practices 81 5 Service implementation 115 6 Tutorials
More informationGrid Infrastructure For Collaborative High Performance Scientific Computing
Computing For Nation Development, February 08 09, 2008 Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi Grid Infrastructure For Collaborative High Performance Scientific
More informationE UFORIA G RID I NFRASTRUCTURE S TATUS R EPORT
E UFORIA G RID I NFRASTRUCTURE S TATUS R EPORT DSA1.1 Document Filename: Activity: Partner(s): Lead Partner: Document classification: EUFORIA-DSA1.1-v1.0-CSIC SA1 CSIC, FZK, PSNC, CHALMERS CSIC PUBLIC
More informationA Hands-On Tutorial: RNA Sequencing Using High-Performance Computing
A Hands-On Tutorial: RNA Sequencing Using Computing February 11th and 12th, 2016 1st session (Thursday) Preliminaries: Linux, HPC, command line interface Using HPC: modules, queuing system Presented by:
More informationGetting Started with OSG Connect ~ an Interactive Tutorial ~
Getting Started with OSG Connect ~ an Interactive Tutorial ~ Emelie Harstad , Mats Rynge , Lincoln Bryant , Suchandra Thapa ,
More informationMulti-thread and Mpi usage in GRID Roberto Alfieri - Parma University & INFN, Gr.Coll. di Parma
SuperB Computing R&D Workshop Multi-thread and Mpi usage in GRID Roberto Alfieri - Parma University & INFN, Gr.Coll. di Parma Ferrara, Thursday, March 11, 2010 1 Outline MPI and multi-thread support in
More informationUoW HPC Quick Start. Information Technology Services University of Wollongong. ( Last updated on October 10, 2011)
UoW HPC Quick Start Information Technology Services University of Wollongong ( Last updated on October 10, 2011) 1 Contents 1 Logging into the HPC Cluster 3 1.1 From within the UoW campus.......................
More informationALICE Grid/Analysis Tutorial Exercise-Solutions
WLCG Asia Workshop, 2.12.2006 ALICE Grid/Analysis Tutorial Exercise-Solutions Andreas-Joachim Peters CERN www.eu-egee.org cern.ch/lcg http://cern.ch/arda EGEE is a project funded by the European Union
More informationAliEn Resource Brokers
AliEn Resource Brokers Pablo Saiz University of the West of England, Frenchay Campus Coldharbour Lane, Bristol BS16 1QY, U.K. Predrag Buncic Institut für Kernphysik, August-Euler-Strasse 6, 60486 Frankfurt
More informationCERN LCG. LCG Short Demo. Markus Schulz. FZK 30 September 2003
LCG Short Demo Markus Schulz LCG FZK 30 September 2003 LCG-1 Demo Outline Monitoring tools and where to get documentation Getting started Running simple jobs Using the information system More on JDL Data
More informationProgramming the Grid with glite *
COMPUTATIONAL METHODS IN SCIENCE AND TECHNOLOGY 12(1), 33-45 (2006) Programming the Grid with glite * E. Laure 1, S. M. Fisher 2, A. Frohner 1, C. Grandi 3, P. Kunszt 4, A. Krenek 5, O. Mulmo 6, F. Pacini
More informationDataGrid TECHNICAL PLAN AND EVALUATION CRITERIA FOR THE RESOURCE CO- ALLOCATION FRAMEWORK AND MECHANISMS FOR PARALLEL JOB PARTITIONING
DataGrid D EFINITION OF THE ARCHITECTURE, WP1: Workload Management Document identifier: Work package: Partner(s): Lead Partner: Document status WP1: Workload Management INFN INFN APPROVED Deliverable identifier:
More informationTutorial for CMS Users: Data Analysis on the Grid with CRAB
Tutorial for CMS Users: Data Analysis on the Grid with CRAB Benedikt Mura, Hartmut Stadie Institut für Experimentalphysik, Universität Hamburg September 2nd, 2009 In this part you will learn... 1 how to
More informationOn the employment of LCG GRID middleware
On the employment of LCG GRID middleware Luben Boyanov, Plamena Nenkova Abstract: This paper describes the functionalities and operation of the LCG GRID middleware. An overview of the development of GRID
More informationCREAM-WMS Integration
-WMS Integration Luigi Zangrando (zangrando@pd.infn.it) Moreno Marzolla (marzolla@pd.infn.it) JRA1 IT-CZ Cluster Meeting, Rome, 12 13/9/2005 1 Preliminaries We had a meeting in PD between Francesco G.,
More informationg-eclipse A Framework for Accessing Grid Infrastructures Nicholas Loulloudes Trainer, University of Cyprus (loulloudes.n_at_cs.ucy.ac.
g-eclipse A Framework for Accessing Grid Infrastructures Trainer, University of Cyprus (loulloudes.n_at_cs.ucy.ac.cy) EGEE Training the Trainers May 6 th, 2009 Outline Grid Reality The Problem g-eclipse
More informationDataGRID EDG TUTORIAL. EDMS id: Lead Partner: Document status: Version 3.2.1
DataGRID EDG TUTORIAL HANDOUTS FOR PARTICIPANTS FOR EDG RELEASE 2.X Document identifier: DataGrid-08-TUT-V3.2.1 EDMS id: Work package: Partner(s): Lead Partner: EDG Collaboration EDG Collaboration EDG
More informationBookkeeping and submission tools prototype. L. Tomassetti on behalf of distributed computing group
Bookkeeping and submission tools prototype L. Tomassetti on behalf of distributed computing group Outline General Overview Bookkeeping database Submission tools (for simulation productions) Framework Design
More informationISTITUTO NAZIONALE DI FISICA NUCLEARE
ISTITUTO NAZIONALE DI FISICA NUCLEARE Sezione di Perugia INFN/TC-05/10 July 4, 2005 DESIGN, IMPLEMENTATION AND CONFIGURATION OF A GRID SITE WITH A PRIVATE NETWORK ARCHITECTURE Leonello Servoli 1,2!, Mirko
More informationIntroduction to Grid Infrastructures
Introduction to Grid Infrastructures Stefano Cozzini 1 and Alessandro Costantini 2 1 CNR-INFM DEMOCRITOS National Simulation Center, Trieste, Italy 2 Department of Chemistry, Università di Perugia, Perugia,
More informationDataGRID EDG TUTORIAL. Document identifier: EDMS id: Date: April 4, Work package: Partner(s): Lead Partner: Document status: Version 2.6.
DataGRID EDG TUTORIAL HANDOUTS FOR PARTICIPANTS FOR EDG RELEASE 1.4.X Document identifier: DataGrid-08-TUT-V2.6 EDMS id: Work package: Partner(s): Lead Partner: EDG Collaboration EDG Collaboration EDG
More informationGrid Compute Resources and Job Management
Grid Compute Resources and Job Management How do we access the grid? Command line with tools that you'll use Specialised applications Ex: Write a program to process images that sends data to run on the
More informationGanga The Job Submission Tool. WeiLong Ueng
Ganga The Job Submission Tool WeiLong Ueng wlueng@twgrid.org Objectives This tutorial gives users to understand Why require Ganga in Grid environment What advantages of Ganga The Architecture of Ganga
More informationMyProxy Server Installation
MyProxy Server Installation Emidio Giorgio INFN First Latin American Workshop for Grid Administrators 21-25 November 2005 www.eu-egee.org Outline Why MyProxy? Proxy Renewal mechanism. Remote authentication
More informationISTITUTO NAZIONALE DI FISICA NUCLEARE Sezione di Padova
ISTITUTO NAZIONALE DI FISICA NUCLEARE Sezione di Padova INFN/TC-09/03 May 05, 2009 DESIGN AND IMPLEMENTATION OF THE GLITE CREAM JOB MANAGEMENT SERVICE Cristina Aiftimiei, Paolo Andreetto, Sara Bertocco,
More informationBOINC extensions in the SZTAKI DesktopGrid system
BOINC extensions in the SZTAKI DesktopGrid system József Kovács smith@sztaki.hu BOINC Workshop, Grenoble, 10/09/2008 SZTAKI Desktop Grid: BOINC project http://szdg.lpds.sztaki.hu/szdg SZTAKI Desktop Grid:
More informationLesson 6: Portlet for job submission
Lesson 6: Portlet for job submission Mario Torrisi University of Catania - Italy (mario.torrisi@ct.infn.it) Sci-GaIA Winter School This project has received funding from the European Union s Horizon 2020
More informationArgus Vulnerability Assessment *1
Argus Vulnerability Assessment *1 Manuel Brugnoli and Elisa Heymann Universitat Autònoma de Barcelona June, 2011 Introduction Argus is the glite Authorization Service. It is intended to provide consistent
More information( PROPOSAL ) THE AGATA GRID COMPUTING MODEL FOR DATA MANAGEMENT AND DATA PROCESSING. version 0.6. July 2010 Revised January 2011
( PROPOSAL ) THE AGATA GRID COMPUTING MODEL FOR DATA MANAGEMENT AND DATA PROCESSING version 0.6 July 2010 Revised January 2011 Mohammed Kaci 1 and Victor Méndez 1 For the AGATA collaboration 1 IFIC Grid
More informationDIRAC pilot framework and the DIRAC Workload Management System
Journal of Physics: Conference Series DIRAC pilot framework and the DIRAC Workload Management System To cite this article: Adrian Casajus et al 2010 J. Phys.: Conf. Ser. 219 062049 View the article online
More informationInstalling and running COMSOL 4.3a on a Linux cluster COMSOL. All rights reserved.
Installing and running COMSOL 4.3a on a Linux cluster 2012 COMSOL. All rights reserved. Introduction This quick guide explains how to install and operate COMSOL Multiphysics 4.3a on a Linux cluster. It
More informationbwunicluster Tutorial Access, Data Transfer, Compiling, Modulefiles, Batch Jobs
bwunicluster Tutorial Access, Data Transfer, Compiling, Modulefiles, Batch Jobs Frauke Bösert, SCC, KIT 1 Material: Slides & Scripts https://indico.scc.kit.edu/indico/event/263/ @bwunicluster/forhlr I/ForHLR
More informationQosCosGrid Middleware
Domain-oriented services and resources of Polish Infrastructure for Supporting Computational Science in the European Research Space PLGrid Plus QosCosGrid Middleware Domain-oriented services and resources
More informationA Virtual Observatory for Pulsar Astronomy
A Virtual Observatory for Pulsar Astronomy Supervisor: Dr. John Brooke MASTERS PROJECT INITIAL REPORT 2008-2009 Akshay M. Sangodkar ID: 7348050 1 Table of Contents Abstract...3 Introduction...3 Background...4
More informationMessage Passing Interface (MPI-1)
IPGP INSTITUT DE PHYSIQUE DU GLOBE DE PARIS Parallel Programming: Message Passing Interface (MPI-1) Geneviève Moguilny moguilny@ipgp.jussieu.fr Institut de Physique du Globe de Paris December 2005 Architecture
More informationMonitoring the Usage of the ZEUS Analysis Grid
Monitoring the Usage of the ZEUS Analysis Grid Stefanos Leontsinis September 9, 2006 Summer Student Programme 2006 DESY Hamburg Supervisor Dr. Hartmut Stadie National Technical
More informationHistory of SURAgrid Deployment
All Hands Meeting: May 20, 2013 History of SURAgrid Deployment Steve Johnson Texas A&M University Copyright 2013, Steve Johnson, All Rights Reserved. Original Deployment Each job would send entire R binary
More informationXSEDE High Throughput Computing Use Cases
XSEDE High Throughput Computing Use Cases 31 May 2013 Version 0.3 XSEDE HTC Use Cases Page 1 XSEDE HTC Use Cases Page 2 Table of Contents A. Document History B. Document Scope C. High Throughput Computing
More informationWorkload Management. Stefano Lacaprara. CMS Physics Week, FNAL, 12/16 April Department of Physics INFN and University of Padova
Workload Management Stefano Lacaprara Department of Physics INFN and University of Padova CMS Physics Week, FNAL, 12/16 April 2005 Outline 1 Workload Management: the CMS way General Architecture Present
More informationGridbus Portlets -- USER GUIDE -- GRIDBUS PORTLETS 1 1. GETTING STARTED 2 2. AUTHENTICATION 3 3. WORKING WITH PROJECTS 4
Gridbus Portlets -- USER GUIDE -- www.gridbus.org/broker GRIDBUS PORTLETS 1 1. GETTING STARTED 2 1.1. PREREQUISITES: 2 1.2. INSTALLATION: 2 2. AUTHENTICATION 3 3. WORKING WITH PROJECTS 4 3.1. CREATING
More informationFirst evaluation of the Globus GRAM Service. Massimo Sgaravatto INFN Padova
First evaluation of the Globus GRAM Service Massimo Sgaravatto INFN Padova massimo.sgaravatto@pd.infn.it Draft version release 1.0.5 20 June 2000 1 Introduction...... 3 2 Running jobs... 3 2.1 Usage examples.
More informationA Simplified Access to Grid Resources for Virtual Research Communities
Consorzio COMETA - Progetto PI2S2 UNIONE EUROPEA A Simplified Access to Grid Resources for Virtual Research Communities Roberto BARBERA (1-3), Marco FARGETTA (3,*) and Riccardo ROTONDO (2) (1) Department
More informationChapter 2 Introduction to the WS-PGRADE/gUSE Science Gateway Framework
Chapter 2 Introduction to the WS-PGRADE/gUSE Science Gateway Framework Tibor Gottdank Abstract WS-PGRADE/gUSE is a gateway framework that offers a set of highlevel grid and cloud services by which interoperation
More informationThe PanDA System in the ATLAS Experiment
1a, Jose Caballero b, Kaushik De a, Tadashi Maeno b, Maxim Potekhin b, Torre Wenaus b on behalf of the ATLAS collaboration a University of Texas at Arlington, Science Hall, PO Box 19059, Arlington, TX
More informationClient tools know everything
Scheduling, clients Client tools know everything Application database Certificate Certificate Authorised users directory Certificate Policies Grid job management service Data Certificate Certificate Researcher
More informationMigrating from Zcluster to Sapelo
GACRC User Quick Guide: Migrating from Zcluster to Sapelo The GACRC Staff Version 1.0 8/4/17 1 Discussion Points I. Request Sapelo User Account II. III. IV. Systems Transfer Files Configure Software Environment
More informationbwunicluster Tutorial Access, Data Transfer, Compiling, Modulefiles, Batch Jobs
bwunicluster Tutorial Access, Data Transfer, Compiling, Modulefiles, Batch Jobs Frauke Bösert, SCC, KIT 1 Material: Slides & Scripts https://indico.scc.kit.edu/indico/event/263/ @bwunicluster/forhlr I/ForHLR
More informationFREE SCIENTIFIC COMPUTING
Institute of Physics, Belgrade Scientific Computing Laboratory FREE SCIENTIFIC COMPUTING GRID COMPUTING Branimir Acković March 4, 2007 Petnica Science Center Overview 1/2 escience Brief History of UNIX
More informationGrid Examples. Steve Gallo Center for Computational Research University at Buffalo
Grid Examples Steve Gallo Center for Computational Research University at Buffalo Examples COBALT (Computational Fluid Dynamics) Ercan Dumlupinar, Syracyse University Aerodynamic loads on helicopter rotors
More informationPerformance of R-GMA for Monitoring Grid Jobs for CMS Data Production
2005 IEEE Nuclear Science Symposium Conference Record N14-207 Performance of R-GMA for Monitoring Grid Jobs for CMS Data Production R. Byrom, D. Colling, S. M. Fisher, C. Grandi, P. R. Hobson, P. Kyberd,
More informationCHARON System Framework for Applications and Jobs Management in Grid Environment
CHARON System Framework for Applications and Jobs Management in Grid Environment Jan Kmuníček, 1,2 Petr Kulhánek, 1,3 and Martin Petřek 1,3 1. CESNET z.s.p.o., Zikova 4, CZ-16000 Praha, Czech Republic
More informationThe ATLAS PanDA Pilot in Operation
The ATLAS PanDA Pilot in Operation P. Nilsson 1, J. Caballero 2, K. De 1, T. Maeno 2, A. Stradling 1, T. Wenaus 2 for the ATLAS Collaboration 1 University of Texas at Arlington, Science Hall, P O Box 19059,
More informationIntroduction to Linux Workshop 1
Introduction to Linux Workshop 1 The George Washington University SEAS Computing Facility Created by Jason Hurlburt, Hadi Mohammadi, Marco Suarez hurlburj@gwu.edu Logging In The lab computers will authenticate
More information