A Practical Approach for a Workflow Management System
|
|
- Giles Sims
- 6 years ago
- Views:
Transcription
1 A Practical Approach for a Workflow Management System Simone Pellegrini, Francesco Giacomini, Antonia Ghiselli INFN Cnaf Viale B. Pichat, 6/ Bologna {simone.pellegrini francesco.giacomini antonia.ghiselli}@cnaf.infn.it
2 Outline Workflow Management Systems overview A practical approach for real workflows Implementation issues A case study: JDL to GWorkflowDL conversion
3 Outline Workflow Management Systems overview A practical approach for real workflows Implementation issues A case study: JDL to GWorkflowDL conversion
4 Workflow Management Systems Overview A lot of interest around WfMSs exist Thanks to workflows, the processes' business logic can be easily expressed using graphs Appealing for users with limited programming skills However, the lack of a recognized standard causes incompatibility among WfMSs: several languages for workflow description exist: Based on different modeling formalisms (DAGs, Petri Nets, Pi Calculus...);
5 Problems in Workflow Management The choice of a WfMS usually binds users to a specific workflow language Change of the WfMS has a high cost: Legacy workflows must be rewritten. Interoperability among WfMSs is still an open issue Difficulty to manage large, complex and real life scientific processes
6 Outline Workflow Management Systems overview A practical approach for real workflows Implementation issues A case study: JDL to GWorkflowDL conversion
7 A practical approach for real workflows Introducing a WfMS: Petri Nets based Formal semantics Turing complete (deals with Workflow Patterns) Build time analysis tools (reachability, boundedness...) Independent from the underlying Grid middleware Multi language: GWorkflowDL used as internal representation Deals with interoperability
8 The WfMS Architecture Aims at language independence Layered architecture Workflow Management System Workflow Gateway Workflow Engine Grid Abstraction Layer Grid Middleware/s Execution nodes Storage nodes Language Interoperability JDL, GworkflowDL, BPEL Engine Interoperability Aims at Grid middleware independence
9 Grid Abstraction Layer Abstracts the basic functions of a Grid providing portability of the Workflow engine over different Grid middlewares Grid Abstraction Layer Dispatcher Data Transfer Dispatcher: Job submission/cancellation Reservation Data Transfer: Move data between Grid nodes Observer: Monitor submitted job status Reservation: Resource reservation Observer
10 Workflow Gateway Provides language and model converters in order to achieve compatibility with legacy workflows and legacy WfMSs Parser: Extracts the model from a workflow description Workflow Gateway Dag Pi Calculus Model Translator Parser Compiler BPEL glite (classad) JSDL Compiler: Produces a workflow representation using a target language (JDL, GWorkflowDL, BPEL, )
11 Language Interoperability The Gateway solves some interoperability issues in workflow management: Part of a workflow (or a sub workflow) can be translated and successively delegated to a thirdparty WfMS Legacy workflows can be executed on our WfMS without being rewritten Every process can be expressed in terms of a Petri Net (Turing complete)
12 The Workflow Engine Our goal is to keep the engine as simple as possible and concentrate on interoperability issues Main characteristics Petri Net base Micro Kernel architecture: aims at modularity and extendibility Distributed
13 Outline Workflow Management Systems overview A practical approach for real workflows Implementation issues A case study: JDL to GWorkflowDL conversion
14 EGEE/gLite as a Grid Middleware Choice of EGEE/gLite middleware because of: Services maturity Reliability Large adoption Job management is done by the Workload Management System (WMS) Job monitoring is done by the Logging and Bookkeeping Service (LB)
15 Project Goal: A WfMS over the WMS The practical outcome of our work is to build a WfMS relying on the WMS + LB Both WMS and LB provide a Web Service interface simplify interaction with Grid services
16 WfMS Deploy (1/2) WfMS running on a dedicated server: Client sends the workflow description to the server; WfMS server manages workflows execution; Client Workflow running instance Monitoring WFMS Server Workflow Description Grid Submitted Jobs Interface to the Grid is provided by the WMS API.
17 WfMS Deploy (2/2) WfMS running as a Grid job: Client submits a workflow to the Grid via the WMS and monitors it via the LB The WfMS ends up running on a Grid node The WfMS instance submits workflow tasks to the Grid using the WMS and monitors them via the LB Client Monitoring Workflow Description submit Grid WFMS running instances Taking full advantage from the Grid resources and facilities (e.g. checkpointing)
18 Outline Workflow Management Systems overview A practical approach for real workflows Implementation issues A case study: JDL to GWorkflowDL conversion
19 The Job Description Language (JDL) The Job Description Language describes a job to be execution on the Grid The JDL adopted within the glite middleware is based upon Condor's Classified Advertisements (ClassAds) record like structure composed of a finite number of attributes separated by semi colon (;) Attribute = Value;
20 A JDL DAG Workflow JDL allows workflow (DAG based) definition: [ ] Type = dag ; [... ] nodes = [ father = [... ]; son1 = [... ]; son2 = [... ]; final = [... ]; dependencies = { {father, {son1, son2}}, ]; }; {son1, final}, {son2, final} Dependencies JDL parser + Model extractor son1 DAG Model father final son2 Job
21 DAGs in glite Many legacy workflows expressed in JDL exist; JDL workflows are managed by the Condor DAG Manager (DAGMan): Acts as a Meta Scheduler for Condor jobs Submits job respecting their inter dependencies In case of job failure, DAGMan continues until it can no longer make progress.
22 DAG -> Petri Net The current DAG model is very limited, e.g. lack of error handling lack of task types other than computation DAGs can be easily described using Petri Nets a DAG node can be represented by a Petri Net transition the flow of data among DAG nodes is modeled using data tokens
23 DAG -> Petri Net Petri Net Model DAG Model init father father P1 P2 son1 son2 son1 son2 final DAG to Petri Net Model Converter P3 final P4 end
24 Petri Net -> GWorkflowDL Petri Net Model init father P1 P2 son1 son2 P3 P4 final end <workflow [...]> <place ID= init /> <place ID= P1 /> <place ID= P2 /> [...] Compiler <place ID= end /> <transition ID= father > <inputplace placeid= init /> <outputplace placeid= P1 /> <outputplace placeid= P2 /> <operation /> </transition> [...] </workflow> DEMO
25 From abstract to concrete workflow (1/4) The workflow description needs a refinement process in order to perform concrete task operations In the case of the glite middleware task execution is asynchronous The WMS serves job submission request returning an ID that identifies the job in its task queue The ID is used to query (or register for notifications by) the LB service until task termination (or failure)
26 From abstract to concrete workflow (2/4) Task Execution P1 InputSandbox jdl jobid P1 P1 1 JobExecute P1 2 JobStart P1 3 JobRegister data movement JobExecute Recovery Strategy if(result.fail) result P1 4 P2 P2 P1 5 wait_for_termination data movement if(result.success)
27 From abstract to concrete workflow (3/4) Waits until the task is done (or failed) using polling: getjobstate() operation of the LB service. wait_for_termination (polling) if(s.job_done s.job_fail) P1 3 jobid N sec getjobstate P1 3 1 s if(s.job_running) P1 4
28 From abstract to concrete workflow (4/4) Same sub workflow implementation using notifications; wait_for_termination (notify) P1 3 P1 3 jobid result P1 4 N sec do_polling
29 Conclusions and Future Work Workflow Gateway as a standalone component Other WfMS can easily take advantage from languages conversions customizable target language depending on the underlying WfMS A lightweight WfMS with basic functionalities solves low level aspects of workflow management Investigate WfMSs engine interoperability
30 Thank You!
Architecture of the WMS
Architecture of the WMS Dr. Giuliano Taffoni INFORMATION SYSTEMS UNIT Outline This presentation will cover the following arguments: Overview of WMS Architecture Job Description Language Overview WMProxy
More informationOverview of WMS/LB API
Overview of WMS/LB API Emidio Giorgio www.eu-egee.org Outline API vs. CLI Overview of WMProxy API Overview of Logging and Bookkeeping API Usage examples API vs. CLI Users applications can be integrated
More informationGrid Programming: Concepts and Challenges. Michael Rokitka CSE510B 10/2007
Grid Programming: Concepts and Challenges Michael Rokitka SUNY@Buffalo CSE510B 10/2007 Issues Due to Heterogeneous Hardware level Environment Different architectures, chipsets, execution speeds Software
More informationDataGrid D EFINITION OF ARCHITECTURE, TECHNICAL PLAN AND EVALUATION CRITERIA FOR SCHEDULING, RESOURCE MANAGEMENT, SECURITY AND JOB DESCRIPTION
DataGrid D EFINITION OF ARCHITECTURE, SECURITY AND JOB DESCRIPTION Document identifier: Work package: Partner: WP1: Workload Management INFN Document status DRAFT Deliverable identifier: DataGrid-D1.2
More informationCREAM-WMS Integration
-WMS Integration Luigi Zangrando (zangrando@pd.infn.it) Moreno Marzolla (marzolla@pd.infn.it) JRA1 IT-CZ Cluster Meeting, Rome, 12 13/9/2005 1 Preliminaries We had a meeting in PD between Francesco G.,
More informationMultiple Broker Support by Grid Portals* Extended Abstract
1. Introduction Multiple Broker Support by Grid Portals* Extended Abstract Attila Kertesz 1,3, Zoltan Farkas 1,4, Peter Kacsuk 1,4, Tamas Kiss 2,4 1 MTA SZTAKI Computer and Automation Research Institute
More informationAdvanced Job Submission on the Grid
Advanced Job Submission on the Grid Antun Balaz Scientific Computing Laboratory Institute of Physics Belgrade http://www.scl.rs/ 30 Nov 11 Dec 2009 www.eu-egee.org Scope User Interface Submit job Workload
More informationGergely Sipos MTA SZTAKI
Application development on EGEE with P-GRADE Portal Gergely Sipos MTA SZTAKI sipos@sztaki.hu EGEE Training and Induction EGEE Application Porting Support www.lpds.sztaki.hu/gasuc www.portal.p-grade.hu
More informationg-eclipse A Framework for Accessing Grid Infrastructures Nicholas Loulloudes Trainer, University of Cyprus (loulloudes.n_at_cs.ucy.ac.
g-eclipse A Framework for Accessing Grid Infrastructures Trainer, University of Cyprus (loulloudes.n_at_cs.ucy.ac.cy) EGEE Training the Trainers May 6 th, 2009 Outline Grid Reality The Problem g-eclipse
More informationInterconnect EGEE and CNGRID e-infrastructures
Interconnect EGEE and CNGRID e-infrastructures Giuseppe Andronico Interoperability and Interoperation between Europe, India and Asia Workshop Barcelona - Spain, June 2 2007 FP6 2004 Infrastructures 6-SSA-026634
More informationglite Grid Services Overview
The EPIKH Project (Exchange Programme to advance e-infrastructure Know-How) glite Grid Services Overview Antonio Calanducci INFN Catania Joint GISELA/EPIKH School for Grid Site Administrators Valparaiso,
More informationLCG-2 and glite Architecture and components
LCG-2 and glite Architecture and components Author E.Slabospitskaya www.eu-egee.org Outline Enabling Grids for E-sciencE What are LCG-2 and glite? glite Architecture Release 1.0 review What is glite?.
More informationGrid Scheduling Architectures with Globus
Grid Scheduling Architectures with Workshop on Scheduling WS 07 Cetraro, Italy July 28, 2007 Ignacio Martin Llorente Distributed Systems Architecture Group Universidad Complutense de Madrid 1/38 Contents
More informationThree Fundamental Dimensions of Scientific Workflow Interoperability: Model of Computation, Language, and Execution Environment
Three Fundamental Dimensions of Scientific Workflow Interoperability: Model of Computation, Language, and Execution Environment Erik Elmroth Francisco Hernández Johan Tordsson UMINF 09.05 Dept. of Computing
More informationGrid services. Enabling Grids for E-sciencE. Dusan Vudragovic Scientific Computing Laboratory Institute of Physics Belgrade, Serbia
Grid services Dusan Vudragovic dusan@phy.bg.ac.yu Scientific Computing Laboratory Institute of Physics Belgrade, Serbia Sep. 19, 2008 www.eu-egee.org Set of basic Grid services Job submission/management
More informationWMS overview and Proposal for Job Status
WMS overview and Proposal for Job Status Author: V.Garonne, I.Stokes-Rees, A. Tsaregorodtsev. Centre de physiques des Particules de Marseille Date: 15/12/2003 Abstract In this paper, we describe briefly
More informationInternational Collaboration to Extend and Advance Grid Education. glite WMS Workload Management System
International Collaboration to Extend and Advance Grid Education glite WMS Workload Management System Marco Pappalardo Consorzio COMETA & INFN Catania, Italy ITIS Ferraris, Acireale, Tutorial GRID per
More informationUser Tools and Languages for Graph-based Grid Workflows
User Tools and Languages for Graph-based Grid Workflows User Tools and Languages for Graph-based Grid Workflows Global Grid Forum 10 Berlin, Germany Grid Workflow Workshop Andreas Hoheisel (andreas.hoheisel@first.fraunhofer.de)
More informationProblemi di schedulazione distribuita su Grid
Problemi di schedulazione distribuita su Grid Ivan Porro Università degli Studi di Genova, DIST, Laboratorio BioLab pivan@unige.it 010-3532789 Riadattato da materiale realizzato da INFN Catania per il
More informationGrid Computing Middleware. Definitions & functions Middleware components Globus glite
Seminar Review 1 Topics Grid Computing Middleware Grid Resource Management Grid Computing Security Applications of SOA and Web Services Semantic Grid Grid & E-Science Grid Economics Cloud Computing 2 Grid
More informationMinutes of WP1 meeting Milan (20 th and 21 st March 2001).
Minutes of WP1 meeting Milan (20 th and 21 st March 2001). Meeting held INFN Milan, chaired by Francesco Prelz. Welcome (F. Prelz) Report from GGF1 (M. Sgaravatto) (http://www.pd.infn.it/~sgaravat/grid/report_amsterdam.ppt)
More informationWorkflow, Planning and Performance Information, information, information Dr Andrew Stephen M c Gough
Workflow, Planning and Performance Information, information, information Dr Andrew Stephen M c Gough Technical Coordinator London e-science Centre Imperial College London 17 th March 2006 Outline Where
More informationBENCHFLOW A FRAMEWORK FOR BENCHMARKING BPMN 2.0 WORKFLOW MANAGEMENT SYSTEMS
BENCHFLOW A FRAMEWORK FOR BENCHMARKING BPMN 2.0 WORKFLOW MANAGEMENT SYSTEMS, Ana Ivanchikj, Cesare Pautasso Faculty of Informatics University of Lugano (USI) Switzerland BPMN 2.0: A Widely Adopted Standard
More informationWMS Application Program Interface: How to integrate them in your code
The EPIKH Project (Exchange Programme to advance e-infrastructure Know-How) WMS Application Program Interface: How to integrate them in your code Fabrizio Pistagna (fabrizio.pistagna@ct.infn.it) Kolkata,
More informationDataGrid TECHNICAL PLAN AND EVALUATION CRITERIA FOR THE RESOURCE CO- ALLOCATION FRAMEWORK AND MECHANISMS FOR PARALLEL JOB PARTITIONING
DataGrid D EFINITION OF THE ARCHITECTURE, WP1: Workload Management Document identifier: Work package: Partner(s): Lead Partner: Document status WP1: Workload Management INFN INFN APPROVED Deliverable identifier:
More informationJob submission and management through web services: the experience with the CREAM service
Journal of Physics: Conference Series Job submission and management through web services: the experience with the CREAM service To cite this article: C Aiftimiei et al 2008 J. Phys.: Conf. Ser. 119 062004
More informationA Login Shell interface for INFN-GRID
A Login Shell interface for INFN-GRID S.Pardi2,3, E. Calloni1,2, R. De Rosa1,2, F. Garufi1,2, L. Milano1,2, G. Russo1,2 1Università degli Studi di Napoli Federico II, Dipartimento di Scienze Fisiche, Complesso
More informationBuilding E-Business Suite Interfaces using BPEL. Asif Hussain Innowave Technology
Building E-Business Suite Interfaces using BPEL Asif Hussain Innowave Technology Agenda About Innowave Why Use BPEL? Synchronous Vs Asynchronous BPEL Adapters Process Activities Building EBS Interfaces
More informationGrid Compute Resources and Job Management
Grid Compute Resources and Job Management How do we access the grid? Command line with tools that you'll use Specialised applications Ex: Write a program to process images that sends data to run on the
More informationGrid Compute Resources and Grid Job Management
Grid Compute Resources and Job Management March 24-25, 2007 Grid Job Management 1 Job and compute resource management! This module is about running jobs on remote compute resources March 24-25, 2007 Grid
More information1Z0-560 Oracle Unified Business Process Management Suite 11g Essentials
1Z0-560 Oracle Unified Business Process Management Suite 11g Essentials Number: 1Z0-560 Passing Score: 650 Time Limit: 120 min File Version: 1.0 http://www.gratisexam.com/ 1Z0-560: Oracle Unified Business
More informationPoS(EGICF12-EMITC2)081
University of Oslo, P.b.1048 Blindern, N-0316 Oslo, Norway E-mail: aleksandr.konstantinov@fys.uio.no Martin Skou Andersen Niels Bohr Institute, Blegdamsvej 17, 2100 København Ø, Denmark E-mail: skou@nbi.ku.dk
More informationWorkload Management. Stefano Lacaprara. CMS Physics Week, FNAL, 12/16 April Department of Physics INFN and University of Padova
Workload Management Stefano Lacaprara Department of Physics INFN and University of Padova CMS Physics Week, FNAL, 12/16 April 2005 Outline 1 Workload Management: the CMS way General Architecture Present
More informationDynamic Workflows for Grid Applications
Dynamic Workflows for Grid Applications Dynamic Workflows for Grid Applications Fraunhofer Resource Grid Fraunhofer Institute for Computer Architecture and Software Technology Berlin Germany Andreas Hoheisel
More informationLesson 5 Web Service Interface Definition (Part II)
Lesson 5 Web Service Interface Definition (Part II) Service Oriented Architectures Security Module 1 - Basic technologies Unit 3 WSDL Ernesto Damiani Università di Milano Controlling the style (1) The
More informationThe Problem of Grid Scheduling
Grid Scheduling The Problem of Grid Scheduling Decentralised ownership No one controls the grid Heterogeneous composition Difficult to guarantee execution environments Dynamic availability of resources
More informationISTITUTO NAZIONALE DI FISICA NUCLEARE Sezione di Padova
ISTITUTO NAZIONALE DI FISICA NUCLEARE Sezione di Padova INFN/TC-09/03 May 05, 2009 DESIGN AND IMPLEMENTATION OF THE GLITE CREAM JOB MANAGEMENT SERVICE Cristina Aiftimiei, Paolo Andreetto, Sara Bertocco,
More informationDIRAC pilot framework and the DIRAC Workload Management System
Journal of Physics: Conference Series DIRAC pilot framework and the DIRAC Workload Management System To cite this article: Adrian Casajus et al 2010 J. Phys.: Conf. Ser. 219 062049 View the article online
More information30 Nov Dec Advanced School in High Performance and GRID Computing Concepts and Applications, ICTP, Trieste, Italy
Advanced School in High Performance and GRID Computing Concepts and Applications, ICTP, Trieste, Italy Why the Grid? Science is becoming increasingly digital and needs to deal with increasing amounts of
More informationDataGrid. Document identifier: Date: 24/11/2003. Work package: Partner: Document status. Deliverable identifier:
DataGrid WMS GUI USER G UIDE Document identifier: Work package: Partner: WP1 Datamat SpA Document status Deliverable identifier: Abstract: This document provides a description of all functionalities provided
More informationHow to use computing resources at Grid
How to use computing resources at Grid Nikola Grkic ngrkic@ipb.ac.rs Scientific Computing Laboratory Institute of Physics Belgrade, Serbia Academic and Educat ional Gr id Init iat ive of S er bia Oct.
More informationPetri Nets ~------~ R-ES-O---N-A-N-C-E-I--se-p-te-m--be-r Applications.
Petri Nets 2. Applications Y Narahari Y Narahari is currently an Associate Professor of Computer Science and Automation at the Indian Institute of Science, Bangalore. His research interests are broadly
More informationCloudBATCH: A Batch Job Queuing System on Clouds with Hadoop and HBase. Chen Zhang Hans De Sterck University of Waterloo
CloudBATCH: A Batch Job Queuing System on Clouds with Hadoop and HBase Chen Zhang Hans De Sterck University of Waterloo Outline Introduction Motivation Related Work System Design Future Work Introduction
More informationAdvanced Topics in Software Engineering (02265) Ekkart Kindler
Advanced Topics in Software Engineering (02265) Recapitulation (I. Introduction ) 2 Levels of models ClassDiagram is an instance of PetriNet * Object * Class 1 start 1 end * Association concrete syntax
More informationA Software Developing Environment for Earth System Modeling. Depei Qian Beihang University CScADS Workshop, Snowbird, Utah June 27, 2012
A Software Developing Environment for Earth System Modeling Depei Qian Beihang University CScADS Workshop, Snowbird, Utah June 27, 2012 1 Outline Motivation Purpose and Significance Research Contents Technology
More informationGeneralized Document Data Model for Integrating Autonomous Applications
6 th International Conference on Applied Informatics Eger, Hungary, January 27 31, 2004. Generalized Document Data Model for Integrating Autonomous Applications Zsolt Hernáth, Zoltán Vincellér Abstract
More informationEclipse Technology Project: g-eclipse
(Incubation) Document classification: Made available under the Eclipse Public License v1.0. Date: September 11, 2007 Abstract: This document contains the Release Review Documentation for the Eclipse Technology
More informationTelecooperation. Application of Subject-oriented Modeling in Automatic Service Composition. Erwin Aitenbichler. Technische Universität Darmstadt
Technische Universität Darmstadt Telecooperation Application of Subject-oriented Modeling in Automatic Service Composition Erwin Aitenbichler erwin@informatik.tu-darmstadt.de Area Head Smart Environments
More informationPegasus Workflow Management System. Gideon Juve. USC Informa3on Sciences Ins3tute
Pegasus Workflow Management System Gideon Juve USC Informa3on Sciences Ins3tute Scientific Workflows Orchestrate complex, multi-stage scientific computations Often expressed as directed acyclic graphs
More informationAutomatic Parallelization of Sequential C Code
Automatic Parallelization of Sequential C Code Pete Gasper Department of Mathematics and Computer Science South Dakota School of Mines and Technology peter.gasper@gold.sdsmt.edu Caleb Herbst Department
More informationEnterprise Integration with Workflow Management
Read in Notes Mode Enterprise Integration with Fred A. Cummins November 1,1999 EDS, 1999 1, using message brokers, has emerged as a commonly accepted approach to integration of independently developed
More informationUnderstanding StoRM: from introduction to internals
Understanding StoRM: from introduction to internals 13 November 2007 Outline Storage Resource Manager The StoRM service StoRM components and internals Deployment configuration Authorization and ACLs Conclusions.
More informationAMGA metadata catalogue system
AMGA metadata catalogue system Hurng-Chun Lee ACGrid School, Hanoi, Vietnam www.eu-egee.org EGEE and glite are registered trademarks Outline AMGA overview AMGA Background and Motivation for AMGA Interface,
More informationBOSCO Architecture. Derek Weitzel University of Nebraska Lincoln
BOSCO Architecture Derek Weitzel University of Nebraska Lincoln Goals We want an easy to use method for users to do computational research It should be easy to install, use, and maintain It should be simple
More informationModel Driven Engineering (MDE)
Model Driven Engineering (MDE) Yngve Lamo 1 1 Faculty of Engineering, Bergen University College, Norway 26 April 2011 Ålesund Outline Background Software Engineering History, SE Model Driven Engineering
More informationGrid Computing Fall 2005 Lecture 5: Grid Architecture and Globus. Gabrielle Allen
Grid Computing 7700 Fall 2005 Lecture 5: Grid Architecture and Globus Gabrielle Allen allen@bit.csc.lsu.edu http://www.cct.lsu.edu/~gallen Concrete Example I have a source file Main.F on machine A, an
More informationGenerating Petri Nets from AADL descriptions. Thomas Vergnaud
Generating Petri Nets from AADL descriptions Designing a Distribution Runtime for the AADL work at ENST focuses on the building of communication middlewares manage communication
More informationTutorial 4: Condor. John Watt, National e-science Centre
Tutorial 4: Condor John Watt, National e-science Centre Tutorials Timetable Week Day/Time Topic Staff 3 Fri 11am Introduction to Globus J.W. 4 Fri 11am Globus Development J.W. 5 Fri 11am Globus Development
More informationACET s e-research Activities
18 June 2008 1 Computing Resources 2 Computing Resources Scientific discovery and advancement of science through advanced computing Main Research Areas Computational Science Middleware Technologies for
More informationPoS(ACAT)020. Status and evolution of CRAB. Fabio Farina University and INFN Milano-Bicocca S. Lacaprara INFN Legnaro
Status and evolution of CRAB University and INFN Milano-Bicocca E-mail: fabio.farina@cern.ch S. Lacaprara INFN Legnaro W. Bacchi University and INFN Bologna M. Cinquilli University and INFN Perugia G.
More informationIntroduction to Grid Infrastructures
Introduction to Grid Infrastructures Stefano Cozzini 1 and Alessandro Costantini 2 1 CNR-INFM DEMOCRITOS National Simulation Center, Trieste, Italy 2 Department of Chemistry, Università di Perugia, Perugia,
More informationKestrel An XMPP-Based Framework for Many Task Computing Applications
Kestrel An XMPP-Based Framework for Many Task Computing Applications Lance Stout Mike Murphy Sebastien Goasguen HISTORY/PURPOSE Kestrel s Goals Lightweight / Easy to set up Run cross-platform without re-compiling
More informationDistributed systems. Distributed Systems Architectures. System types. Objectives. Distributed system characteristics.
Distributed systems Distributed Systems Architectures Virtually all large computer-based systems are now distributed systems. Information processing is distributed over several computers rather than confined
More informationGRID COMPANION GUIDE
Companion Subject: GRID COMPANION Author(s): Miguel Cárdenas Montes, Antonio Gómez Iglesias, Francisco Castejón, Adrian Jackson, Joachim Hein Distribution: Public 1.Introduction Here you will find the
More informationA Composable Service-Oriented Architecture for Middleware-Independent and Interoperable Grid Job Management
A Composable Service-Oriented Architecture for Middleware-Independent and Interoperable Grid Job Management Erik Elmroth and Per-Olov Östberg Dept. Computing Science and HPC2N, Umeå University, SE-901
More informationCloud Computing. Up until now
Cloud Computing Lecture 4 and 5 Grid: 2012-2013 Introduction. Up until now Definition of Cloud Computing. Grid Computing: Schedulers: Condor SGE 1 Summary Core Grid: Toolkit Condor-G Grid: Conceptual Architecture
More informationQosCosGrid Middleware
Domain-oriented services and resources of Polish Infrastructure for Supporting Computational Science in the European Research Space PLGrid Plus QosCosGrid Middleware Domain-oriented services and resources
More informationBookkeeping and submission tools prototype. L. Tomassetti on behalf of distributed computing group
Bookkeeping and submission tools prototype L. Tomassetti on behalf of distributed computing group Outline General Overview Bookkeeping database Submission tools (for simulation productions) Framework Design
More informationDr. Giuliano Taffoni INAF - OATS
Query Element Demo The Grid Query Element for glite Dr. Giuliano Taffoni INAF - OATS Overview What is a G-DSE? Use and Admin a DB: the Query Element; Upcoming Features; Working on QE People: Edgardo Ambrosi
More informationChapter 2 Introduction to the WS-PGRADE/gUSE Science Gateway Framework
Chapter 2 Introduction to the WS-PGRADE/gUSE Science Gateway Framework Tibor Gottdank Abstract WS-PGRADE/gUSE is a gateway framework that offers a set of highlevel grid and cloud services by which interoperation
More informationTutorial for CMS Users: Data Analysis on the Grid with CRAB
Tutorial for CMS Users: Data Analysis on the Grid with CRAB Benedikt Mura, Hartmut Stadie Institut für Experimentalphysik, Universität Hamburg September 2nd, 2009 In this part you will learn... 1 how to
More informationL3.4. Data Management Techniques. Frederic Desprez Benjamin Isnard Johan Montagnat
Grid Workflow Efficient Enactment for Data Intensive Applications L3.4 Data Management Techniques Authors : Eddy Caron Frederic Desprez Benjamin Isnard Johan Montagnat Summary : This document presents
More informationglideinwms architecture by Igor Sfiligoi, Jeff Dost (UCSD)
glideinwms architecture by Igor Sfiligoi, Jeff Dost (UCSD) Outline A high level overview of the glideinwms Description of the components 2 glideinwms from 10k feet 3 Refresher - HTCondor A Condor pool
More informationPegasus WMS Automated Data Management in Shared and Nonshared Environments
Pegasus WMS Automated Data Management in Shared and Nonshared Environments Mats Rynge USC Information Sciences Institute Pegasus Workflow Management System NSF funded project and developed
More informationGeant4 on Azure using Docker containers
http://www.geant4.org Geant4 on Azure using Docker containers Andrea Dotti (adotti@slac.stanford.edu) ; SD/EPP/Computing 1 Outlook Motivation/overview Docker + G4 Azure + G4 Conclusions 2 Motivation/overview
More informationWhat s new in HTCondor? What s coming? HTCondor Week 2018 Madison, WI -- May 22, 2018
What s new in HTCondor? What s coming? HTCondor Week 2018 Madison, WI -- May 22, 2018 Todd Tannenbaum Center for High Throughput Computing Department of Computer Sciences University of Wisconsin-Madison
More informationIntegration of the guse/ws-pgrade and InSilicoLab portals with DIRAC
Journal of Physics: Conference Series Integration of the guse/ws-pgrade and InSilicoLab portals with DIRAC To cite this article: A Puig Navarro et al 2012 J. Phys.: Conf. Ser. 396 032088 Related content
More informationTopics of Discussion
CPET 581 Cloud Computing: Technologies and Enterprise IT Strategies Lecture on NIST Cloud Computing Definition, Standards & Roadmap, Security & Privacy Guidelines Spring 2013 A Specialty Course for Purdue
More informationThe glite middleware. Ariel Garcia KIT
The glite middleware Ariel Garcia KIT Overview Background The glite subsystems overview Security Information system Job management Data management Some (my) answers to your questions and random rumblings
More informationImplementing GRID interoperability
AFS & Kerberos Best Practices Workshop University of Michigan, Ann Arbor June 12-16 2006 Implementing GRID interoperability G. Bracco, P. D'Angelo, L. Giammarino*, S.Migliori, A. Quintiliani, C. Scio**,
More informationDeploying virtualisation in a production grid
Deploying virtualisation in a production grid Stephen Childs Trinity College Dublin & Grid-Ireland TERENA NRENs and Grids workshop 2 nd September 2008 www.eu-egee.org EGEE and glite are registered trademarks
More informationSemantic-enabled CARE Resource Broker (SeCRB) for managing grid and cloud environment
J Supercomput (2014) 68:509 556 DOI 10.1007/s11227-013-1047-z Semantic-enabled CARE Resource Broker (SeCRB) for managing grid and cloud environment Thamarai Selvi Somasundaram Kannan Govindarajan Usha
More informationDataGrid. Logging and Bookkeeping Service for the DataGrid. Date: October 8, Workpackage: Document status: Deliverable identifier:
DataGrid Logging and Bookkeeping Service for Document identifier: DataGrid-01-TEN-0109-1_0 Workpackage: Partner: WP1 CESNET Document status: DRAFT Deliverable identifier: IST-2000-25182 PUBLIC 1/40 From
More informationCHAPTER 3 GRID MONITORING AND RESOURCE SELECTION
31 CHAPTER 3 GRID MONITORING AND RESOURCE SELECTION This chapter introduces the Grid monitoring with resource metrics and network metrics. This chapter also discusses various network monitoring tools and
More informationOverview. Distributed Systems. Distributed Software Architecture Using Middleware. Components of a system are not always held on the same host
Distributed Software Architecture Using Middleware Mitul Patel 1 Overview Distributed Systems Middleware What is it? Why do we need it? Types of Middleware Example Summary 2 Distributed Systems Components
More informationHigh Throughput WAN Data Transfer with Hadoop-based Storage
High Throughput WAN Data Transfer with Hadoop-based Storage A Amin 2, B Bockelman 4, J Letts 1, T Levshina 3, T Martin 1, H Pi 1, I Sfiligoi 1, M Thomas 2, F Wuerthwein 1 1 University of California, San
More informationIndependent Software Vendors (ISV) Remote Computing Usage Primer
GFD-I.141 ISV Remote Computing Usage Primer Authors: Steven Newhouse, Microsoft Andrew Grimshaw, University of Virginia 7 October, 2008 Independent Software Vendors (ISV) Remote Computing Usage Primer
More informationEGEE and Interoperation
EGEE and Interoperation Laurence Field CERN-IT-GD ISGC 2008 www.eu-egee.org EGEE and glite are registered trademarks Overview The grid problem definition GLite and EGEE The interoperability problem The
More informationParallel Computing in EGI
Parallel Computing in EGI V. Šipková, M. Dobrucký, and P. Slížik Ústav informatiky, Slovenská akadémia vied 845 07 Bratislava, Dúbravská cesta 9 http://www.ui.sav.sk/ {Viera.Sipkova, Miroslav.Dobrucky,
More informationGrid Infrastructure For Collaborative High Performance Scientific Computing
Computing For Nation Development, February 08 09, 2008 Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi Grid Infrastructure For Collaborative High Performance Scientific
More informationXSEDE High Throughput Computing Use Cases
XSEDE High Throughput Computing Use Cases 31 May 2013 Version 0.3 XSEDE HTC Use Cases Page 1 XSEDE HTC Use Cases Page 2 Table of Contents A. Document History B. Document Scope C. High Throughput Computing
More informationImplementing Problem Resolution Models in Remedy
CERN-IT-2000-001 15 February 2000 Implementing Problem Resolution Models in Remedy Miguel Marquina, Raúl Ramos, José Padilla CERN, IT Division, User Support Group This paper defines the concept of Problem
More informationTopics in Object-Oriented Design Patterns
Software design Topics in Object-Oriented Design Patterns Material mainly from the book Design Patterns by Erich Gamma, Richard Helm, Ralph Johnson and John Vlissides; slides originally by Spiros Mancoridis;
More informationThe Opal Toolkit. Wrapping Scientific Applications as Web Services
The Opal Toolkit Wrapping Scientific Applications as Web Services Outline!Introduction!Motivation!Opal!Summary 1 What is Opal?! Opal is a toolkit for wrapping scientific applications as Web services on
More informationMONTE CARLO SIMULATION FOR RADIOTHERAPY IN A DISTRIBUTED COMPUTING ENVIRONMENT
The Monte Carlo Method: Versatility Unbounded in a Dynamic Computing World Chattanooga, Tennessee, April 17-21, 2005, on CD-ROM, American Nuclear Society, LaGrange Park, IL (2005) MONTE CARLO SIMULATION
More informationTask Management Service
UMEÅ UNIVERSITY Department of Author:, Contents 1 Abstract 1 2 About this document 1 3 Introduction 2 4 Overview 2 5 Use cases 3 5.1 The TMS point of view....................... 3 5.2 The users / clients
More informationResource Allocation in computational Grids
Grid Computing Competence Center Resource Allocation in computational Grids Riccardo Murri Grid Computing Competence Center, Organisch-Chemisches Institut, University of Zurich Nov. 23, 21 Scheduling on
More informationTesting an Open Source installation and server provisioning tool for the INFN CNAF Tier1 Storage system
Testing an Open Source installation and server provisioning tool for the INFN CNAF Tier1 Storage system M Pezzi 1, M Favaro 1, D Gregori 1, PP Ricci 1, V Sapunenko 1 1 INFN CNAF Viale Berti Pichat 6/2
More informationResource Discovery in IoT: Current Trends, Gap Analysis and Future Standardization Aspects
Resource Discovery in IoT: Current Trends, Gap Analysis and Future Standardization Aspects Soumya Kanti Datta Research Engineer, EURECOM TF-DI Coordinator in W3C WoT IG Email: dattas@eurecom.fr Roadmap
More informationA Next Generation Hypervisor for the Embedded Market. Whitepaper
A Next Generation Hypervisor for the Embedded Market Whitepaper 1 Table of Contents Background: Current Generation of Hypervisors... 3 Limitations of Current Generation Hypervisors... 4 Next Generation
More information