The GridWay. approach for job Submission and Management on Grids. Outline. Motivation. The GridWay Framework. Resource Selection

Size: px
Start display at page:

Download "The GridWay. approach for job Submission and Management on Grids. Outline. Motivation. The GridWay Framework. Resource Selection"

Transcription

1 The GridWay approach for job Submission and Management on Grids Eduardo Huedo Rubén S. Montero Ignacio M. Llorente Laboratorio de Computación Avanzada Centro de Astrobiología (INTA - CSIC) Associated to NASA Astrobiology Institute Distributed Systems Architecture & Security Group Dpto. de Arquitectura de Computadores y Automática Facultad de Informática (UCM) Outline Motivation The GridWay Framework Selection Adaptive Job Execution Example: Opportunistic Job Migration Summary

2 Motivation I Globus Toolkit Enables secure multiple domain operation with different resource managers and access policies Globus components: Management (GRAM) Data Management (GridFTP & Replica Catalog) Grid Security Infrastructure (GSI) Information Service (MDS) User: Where do I execute my job? What do I need (files,...)? How do I execute my job? How is my job doing? Can I move my job to a better host? How do I retrieve job output? resource selection resource preparation job submission monitoring migration termination Motivation II High Fault Rate Network Dynamic Cost time of the day (working / nonworking hours) resource load (peak/off-peak) Dynamic Availability Job cancellation by remote administrator Addition and removal of resources A Grid Dynamic Load Shared resources Idle hosts become saturated, and vice versa. Job must be able to migrate among grid resources to obtain application performance and fault tolerance

3 The GridWay Framework Project Goal: Easy and efficient execution of jobs on heterogeneous and dynamic grids in a submit & forget fashion Design Guidelines: Easily Adaptable (modular design) Easily Scalable (decentralized architecture) Easily Deployable (user privileges, standard services) Easily Extensible (use of non-standard services) Easily Applicable (ready to use for a wide range of applications) The GridWay Framework User Interface: gwps: display job information and status JID AID TID DM SM GSM STIME ETIME EXETIME EXIT HOST TEMPLATE submitted prologue -- --:-- --:-- --:-- -- columba job_template zombie done -- 27:37 28:07 00:30 0 ursa job_template pending done -- --:-- --:-- --:-- -- draco job_template gwhistory: display job execution history HOST RANK STIME ETIME EXETIME MIGRATION_REASON columba.dacya.ucm.es :-- --:-- --:-- -- ursa.dacya.ucm.es/jobmanager-grd 50 27:41 27:52 00:11 discovery timeout gwkill: signals a job (kill, stop, resume, reschedule) gwsubmit: submits a job, or an array job gwwait: waits for zombie state of a job (any, all, set) Client API: Allows the interaction with each module, (DMRAA subset)

4 The GridWay Architecture Selector MDS GIIS/GRIS requirements Rank expression Dispatch Manager Request Manager Performance Monitor Performance Degradation Evaluator Performance Profile Submission Agent Job Pool Submission Manager Job Files Executable I/O files Checkpoint GridFTP Client Host GRAM request GRAM callback GASS requirements Rank expression GateKeeper JobManager JOB JOB Performance Profile Execution Host Selector I Rank Expression Requirements (&(Mds-Computer-isa=sparc) (Mds-Memory-Ram-free256)) FQDN stage-rm ursa.ucm.es jobmanager draco.ucm.es jobmanager exec-rm rank jobmanager-sge 50 jobmanager 25 LDAP Filter Static Information (S.O., architecture) User-provided Requirements Authorization test Dispatch Manager Discovery Globus Monitoring and Discovery Service (MDS) Filtered LDAP search GRIS Dynamic Information (CPU load, ) Rank expression User provided executable Characterize discovered hosts Selection LDAP queries GRIIS GRIIS GRIIS

5 Selector II Estimated execution time (lowest is best) Rank = T exe (h n,h n ) = T cpu (h n,h n ) + T xfer (h n,h n ) Estimated Computational Time: Computational work already performed Dynamic performance of the host Estimated File Transfer Time between: Client host and candidate execution host Job submission and monitoring File staging (executable, input/output files) File server and candidate execution host Input/output files Candidate execution host and current execution host Restart files Adaptive Job Execution Job Adaptation is achieved by automatic job migration when: A new better resource is discovered (opportunistic migration) The remote host or its network connection fails The job is cancelled or suspended A performance degradation is detected The requirements of the application change (self-migration) Migration gain (opportunistic migration and performance degradation): G m Rank ( h = n 1, t n 1 ) Rank ( h Rank ( h, t n 1, t n 1 ) n n ) User threshold

6 Example: Opportunistic Migration Experimental testbed: Host Model Speed OS Memory Network ursa Sun Blade Mhz Solaris 8 256MB LAN draco Sun Ultra 1 167Mhz Solaris 8 128MB LAN columba Intel Pentium MMX 233MHz Linux MB LAN cepheus Intel Pentium Pro 200MHz Linux MB LAN solea Sun Enterprise MHz Solaris 8 256MB MAN Client host Execution host File server Experiment: CFD code to solve the 3D Navier-Stokes equations using an iterative multigrid method Initially the application is submitted to draco Re-schedules when columba and solea becomes available at different iterations of the application running on draco #Job template EXECUTABLE = NS3D.$GW_ARCH ARGUMENTS = input INPUT= gsftp://cepheus/mesh input OUTPUT=profile STDOUT=stdout.$GW_JOB_ID RESTART_FILE=checkpoint REQUIREMENTS=host_req.ldif RANK=rank.sh Example: Opportunistic Migration Dynamic ranks of solea and columba at different execution points Measured Execution Profile, of the application when it is actually migrated at different iterations 1. Migration to solea is profitable until Iteration 2 is reached 2. From fourth iteration the best host is columba (nearest) 3. From fifth iteration the performance gain is not high enough to compensate the file transfer overhead

7 Summary Related Work: Job management within the same administration domain: Condor Load Sharing Facility (LSF) Sun Grid Engine Portable Batch System (PBS) Job management for interconnection of multiple domains: Sun Grid Engine Enterprise Edition Condor Flocking Globus middleware for job management: Condor/G AppLes Nimrod/G Job Adaptation: Cactus Worm GrADS GridWay

A framework for adaptive execution in grids

A framework for adaptive execution in grids SOFTWARE PRACTICE AND EXPERIENCE Softw. Pract. Exper. 2004; 34:631 651 (DOI: 10.1002/spe.584) A framework for adaptive execution in grids Eduardo Huedo 1, Ruben S. Montero 2,, and Ignacio M. Llorente 1,2

More information

Developing Grid-Aware Applications with DRMAA on Globus-based Grids

Developing Grid-Aware Applications with DRMAA on Globus-based Grids Developing Grid-Aware Applications with DRMAA on Globus-based Grids. Herrera 1, E. Huedo 2, R.S. Montero 1, and I.M. Llorente 1,2 1 Departamento de Arquitectura de Computadores y Automática, Universidad

More information

Layered Architecture

Layered Architecture The Globus Toolkit : Introdution Dr Simon See Sun APSTC 09 June 2003 Jie Song, Grid Computing Specialist, Sun APSTC 2 Globus Toolkit TM An open source software toolkit addressing key technical problems

More information

GridWay interoperability through BES

GridWay interoperability through BES interoperability through BES EGI Technical Congreso Forum 2012 Prague, Cuidad, Czech Spain Republic September May 15, 17-21, 20072012 Dr Ismael Marín Carrión 1, Dr Eduardo Huedo 1, Dr Stephen Crouch 2,

More information

Grid Architectural Models

Grid Architectural Models Grid Architectural Models Computational Grids - A computational Grid aggregates the processing power from a distributed collection of systems - This type of Grid is primarily composed of low powered computers

More information

Grid Scheduling Architectures with Globus

Grid Scheduling Architectures with Globus Grid Scheduling Architectures with Workshop on Scheduling WS 07 Cetraro, Italy July 28, 2007 Ignacio Martin Llorente Distributed Systems Architecture Group Universidad Complutense de Madrid 1/38 Contents

More information

Submission, Monitoring and Control of Jobs

Submission, Monitoring and Control of Jobs Submission, Monitoring and Control of Jobs GridWay HPC SYSADMIN Congreso MEETING'12 Barcelona, Cuidad, Spain October May 15-16, 15, 2007 2012 Dr Ismael Marín Carrión Distributed Systems Architecture Group

More information

An Experimental Framework for Executing Applications in Dynamic Grid Environments

An Experimental Framework for Executing Applications in Dynamic Grid Environments NASA/CR-2002-211960 ICASE Report No. 2002-43 An Experimental Framework for Executing Applications in Dynamic Grid Environments Eduardo Huedo Centro de Astrobiologia, CSIC-INTA, Torrej6n de Ardoz, Spain

More information

Transparent Access to Grid-Based Compute Utilities

Transparent Access to Grid-Based Compute Utilities Transparent Access to Grid-Based Compute Utilities Tino Vázquez, Javier Fontán, Eduardo Huedo, Rubén S. Montero, and Ignacio M. Llorente Departamento de Arquitectura de Computadores y Automática Facultad

More information

Grid Computing Fall 2005 Lecture 5: Grid Architecture and Globus. Gabrielle Allen

Grid Computing Fall 2005 Lecture 5: Grid Architecture and Globus. Gabrielle Allen Grid Computing 7700 Fall 2005 Lecture 5: Grid Architecture and Globus Gabrielle Allen allen@bit.csc.lsu.edu http://www.cct.lsu.edu/~gallen Concrete Example I have a source file Main.F on machine A, an

More information

Technologies for Grid Computing

Technologies for Grid Computing October 20, 2005 Grid Workshop Technologies for Grid Computing asds.dacya.ucm.es/nacho Grupo de Arquitectura de Sistemas Distribuidos Departamento de Arquitectura de Computadores y Automática Universidad

More information

Development and execution of an impact cratering application on a computational Grid 1

Development and execution of an impact cratering application on a computational Grid 1 Scientific Programming 13 (2005) 19 30 19 IOS Press Development and execution of an impact cratering application on a computational Grid 1 E. Huedo a,, A. Lepinette a, R.S. Montero b, I.M. Llorente a,b

More information

Cloud Computing. Up until now

Cloud Computing. Up until now Cloud Computing Lecture 4 and 5 Grid: 2012-2013 Introduction. Up until now Definition of Cloud Computing. Grid Computing: Schedulers: Condor SGE 1 Summary Core Grid: Toolkit Condor-G Grid: Conceptual Architecture

More information

Porting of scientific applications to Grid Computing on GridWay 1

Porting of scientific applications to Grid Computing on GridWay 1 Scientific Programming 13 (2005) 317 331 317 IOS Press Porting of scientific applications to Grid Computing on GridWay 1 J. Herrera a,, E. Huedo b, R.S. Montero a and I.M. Llorente a,b a Departamento de

More information

Grid services. Enabling Grids for E-sciencE. Dusan Vudragovic Scientific Computing Laboratory Institute of Physics Belgrade, Serbia

Grid services. Enabling Grids for E-sciencE. Dusan Vudragovic Scientific Computing Laboratory Institute of Physics Belgrade, Serbia Grid services Dusan Vudragovic dusan@phy.bg.ac.yu Scientific Computing Laboratory Institute of Physics Belgrade, Serbia Sep. 19, 2008 www.eu-egee.org Set of basic Grid services Job submission/management

More information

Benchmarking of high throughput computing applications on Grids q

Benchmarking of high throughput computing applications on Grids q Parallel Computing 32 (26) 267 279 www.elsevier.com/locate/parco Benchmarking of high throughput computing applications on Grids q R.S. Montero a, *, E. Huedo b, I.M. Llorente a,b a Departamento de Arquitectura

More information

Grid Computing. MCSN - N. Tonellotto - Distributed Enabling Platforms

Grid Computing. MCSN - N. Tonellotto - Distributed Enabling Platforms Grid Computing 1 Resource sharing Elements of Grid Computing - Computers, data, storage, sensors, networks, - Sharing always conditional: issues of trust, policy, negotiation, payment, Coordinated problem

More information

An Experimental Framework for Executing Applications in Dynamic Grid Environments

An Experimental Framework for Executing Applications in Dynamic Grid Environments NASA/CR-2002-211960 ICASE Report No. 2002-43 An Experimental Framework for Executing Applications in Dynamic Grid Environments Eduardo Huedo Centro de Astrobiología, CSIC-INTA, Torrejón de Ardoz, Spain

More information

Day 1 : August (Thursday) An overview of Globus Toolkit 2.4

Day 1 : August (Thursday) An overview of Globus Toolkit 2.4 An Overview of Grid Computing Workshop Day 1 : August 05 2004 (Thursday) An overview of Globus Toolkit 2.4 By CDAC Experts Contact :vcvrao@cdacindia.com; betatest@cdacindia.com URL : http://www.cs.umn.edu/~vcvrao

More information

Adaptive Cluster Computing using JavaSpaces

Adaptive Cluster Computing using JavaSpaces Adaptive Cluster Computing using JavaSpaces Jyoti Batheja and Manish Parashar The Applied Software Systems Lab. ECE Department, Rutgers University Outline Background Introduction Related Work Summary of

More information

Grid Computing Middleware. Definitions & functions Middleware components Globus glite

Grid Computing Middleware. Definitions & functions Middleware components Globus glite Seminar Review 1 Topics Grid Computing Middleware Grid Resource Management Grid Computing Security Applications of SOA and Web Services Semantic Grid Grid & E-Science Grid Economics Cloud Computing 2 Grid

More information

30 Nov Dec Advanced School in High Performance and GRID Computing Concepts and Applications, ICTP, Trieste, Italy

30 Nov Dec Advanced School in High Performance and GRID Computing Concepts and Applications, ICTP, Trieste, Italy Advanced School in High Performance and GRID Computing Concepts and Applications, ICTP, Trieste, Italy Why the Grid? Science is becoming increasingly digital and needs to deal with increasing amounts of

More information

The Integration of Grid Technology with OGC Web Services (OWS) in NWGISS for NASA EOS Data

The Integration of Grid Technology with OGC Web Services (OWS) in NWGISS for NASA EOS Data The Integration of Grid Technology with OGC Web Services (OWS) in NWGISS for NASA EOS Data Liping Di, Aijun Chen, Wenli Yang and Peisheng Zhao achen6@gmu.edu; achen@laits.gmu.edu Lab for Advanced Information

More information

The University of Oxford campus grid, expansion and integrating new partners. Dr. David Wallom Technical Manager

The University of Oxford campus grid, expansion and integrating new partners. Dr. David Wallom Technical Manager The University of Oxford campus grid, expansion and integrating new partners Dr. David Wallom Technical Manager Outline Overview of OxGrid Self designed components Users Resources, adding new local or

More information

Grid Compute Resources and Job Management

Grid Compute Resources and Job Management Grid Compute Resources and Job Management How do we access the grid? Command line with tools that you'll use Specialised applications Ex: Write a program to process images that sends data to run on the

More information

An Introduction to Virtualization and Cloud Technologies to Support Grid Computing

An Introduction to Virtualization and Cloud Technologies to Support Grid Computing New Paradigms: Clouds, Virtualization and Co. EGEE08, Istanbul, September 25, 2008 An Introduction to Virtualization and Cloud Technologies to Support Grid Computing Distributed Systems Architecture Research

More information

Chapter 4:- Introduction to Grid and its Evolution. Prepared By:- NITIN PANDYA Assistant Professor SVBIT.

Chapter 4:- Introduction to Grid and its Evolution. Prepared By:- NITIN PANDYA Assistant Professor SVBIT. Chapter 4:- Introduction to Grid and its Evolution Prepared By:- Assistant Professor SVBIT. Overview Background: What is the Grid? Related technologies Grid applications Communities Grid Tools Case Studies

More information

The Problem of Grid Scheduling

The Problem of Grid Scheduling Grid Scheduling The Problem of Grid Scheduling Decentralised ownership No one controls the grid Heterogeneous composition Difficult to guarantee execution environments Dynamic availability of resources

More information

GEMS: A Fault Tolerant Grid Job Management System

GEMS: A Fault Tolerant Grid Job Management System GEMS: A Fault Tolerant Grid Job Management System Sriram Satish Tadepalli Thesis submitted to the faculty of the Virginia Polytechnic Institute and State University in partial fulfillment of the requirements

More information

IMAGE: An approach to building standards-based enterprise Grids

IMAGE: An approach to building standards-based enterprise Grids IMAGE: An approach to building standards-based enterprise Grids Gabriel Mateescu 1 and Masha Sosonkina 2 1 Research Computing Support Group 2 Scalable Computing Laboratory National Research Council USDOE

More information

Architecture Proposal

Architecture Proposal Nordic Testbed for Wide Area Computing and Data Handling NORDUGRID-TECH-1 19/02/2002 Architecture Proposal M.Ellert, A.Konstantinov, B.Kónya, O.Smirnova, A.Wäänänen Introduction The document describes

More information

Chapter 3. Design of Grid Scheduler. 3.1 Introduction

Chapter 3. Design of Grid Scheduler. 3.1 Introduction Chapter 3 Design of Grid Scheduler The scheduler component of the grid is responsible to prepare the job ques for grid resources. The research in design of grid schedulers has given various topologies

More information

Grid Programming: Concepts and Challenges. Michael Rokitka CSE510B 10/2007

Grid Programming: Concepts and Challenges. Michael Rokitka CSE510B 10/2007 Grid Programming: Concepts and Challenges Michael Rokitka SUNY@Buffalo CSE510B 10/2007 Issues Due to Heterogeneous Hardware level Environment Different architectures, chipsets, execution speeds Software

More information

CMS HLT production using Grid tools

CMS HLT production using Grid tools CMS HLT production using Grid tools Flavia Donno (INFN Pisa) Claudio Grandi (INFN Bologna) Ivano Lippi (INFN Padova) Francesco Prelz (INFN Milano) Andrea Sciaba` (INFN Pisa) Massimo Sgaravatto (INFN Padova)

More information

Knowledge Discovery Services and Tools on Grids

Knowledge Discovery Services and Tools on Grids Knowledge Discovery Services and Tools on Grids DOMENICO TALIA DEIS University of Calabria ITALY talia@deis.unical.it Symposium ISMIS 2003, Maebashi City, Japan, Oct. 29, 2003 OUTLINE Introduction Grid

More information

First evaluation of the Globus GRAM Service. Massimo Sgaravatto INFN Padova

First evaluation of the Globus GRAM Service. Massimo Sgaravatto INFN Padova First evaluation of the Globus GRAM Service Massimo Sgaravatto INFN Padova massimo.sgaravatto@pd.infn.it Draft version release 1.0.5 20 June 2000 1 Introduction...... 3 2 Running jobs... 3 2.1 Usage examples.

More information

Corral: A Glide-in Based Service for Resource Provisioning

Corral: A Glide-in Based Service for Resource Provisioning : A Glide-in Based Service for Resource Provisioning Gideon Juve USC Information Sciences Institute juve@usc.edu Outline Throughput Applications Grid Computing Multi-level scheduling and Glideins Example:

More information

PARALLEL PROGRAM EXECUTION SUPPORT IN THE JGRID SYSTEM

PARALLEL PROGRAM EXECUTION SUPPORT IN THE JGRID SYSTEM PARALLEL PROGRAM EXECUTION SUPPORT IN THE JGRID SYSTEM Szabolcs Pota 1, Gergely Sipos 2, Zoltan Juhasz 1,3 and Peter Kacsuk 2 1 Department of Information Systems, University of Veszprem, Hungary 2 Laboratory

More information

An Evaluation of Alternative Designs for a Grid Information Service

An Evaluation of Alternative Designs for a Grid Information Service An Evaluation of Alternative Designs for a Grid Information Service Warren Smith, Abdul Waheed *, David Meyers, Jerry Yan Computer Sciences Corporation * MRJ Technology Solutions Directory Research L.L.C.

More information

APPLICATION LEVEL SCHEDULING (APPLES) IN GRID WITH QUALITY OF SERVICE (QOS)

APPLICATION LEVEL SCHEDULING (APPLES) IN GRID WITH QUALITY OF SERVICE (QOS) APPLICATION LEVEL SCHEDULING (APPLES) IN GRID WITH QUALITY OF SERVICE (QOS) CH V T E V Laxmi 1, Dr. K.Somasundaram 2 1,Research scholar, Karpagam University, Department of Computer Science Engineering,

More information

Grid Compute Resources and Grid Job Management

Grid Compute Resources and Grid Job Management Grid Compute Resources and Job Management March 24-25, 2007 Grid Job Management 1 Job and compute resource management! This module is about running jobs on remote compute resources March 24-25, 2007 Grid

More information

UNIT IV PROGRAMMING MODEL. Open source grid middleware packages - Globus Toolkit (GT4) Architecture, Configuration - Usage of Globus

UNIT IV PROGRAMMING MODEL. Open source grid middleware packages - Globus Toolkit (GT4) Architecture, Configuration - Usage of Globus UNIT IV PROGRAMMING MODEL Open source grid middleware packages - Globus Toolkit (GT4) Architecture, Configuration - Usage of Globus Globus: One of the most influential Grid middleware projects is the Globus

More information

Distributed and Cloud Computing

Distributed and Cloud Computing Distributed and Cloud Computing K. Hwang, G. Fox and J. Dongarra Chapter 2: Computer Clusters for Scalable parallel Computing Adapted from Kai Hwang University of Southern California March 30, 2012 Copyright

More information

A Data-Aware Resource Broker for Data Grids

A Data-Aware Resource Broker for Data Grids A Data-Aware Resource Broker for Data Grids Huy Le, Paul Coddington, and Andrew L. Wendelborn School of Computer Science, University of Adelaide Adelaide, SA 5005, Australia {paulc,andrew}@cs.adelaide.edu.au

More information

igrid: a Relational Information Service A novel resource & service discovery approach

igrid: a Relational Information Service A novel resource & service discovery approach igrid: a Relational Information Service A novel resource & service discovery approach Italo Epicoco, Ph.D. University of Lecce, Italy Italo.epicoco@unile.it Outline of the talk Requirements & features

More information

Designing a Resource Broker for Heterogeneous Grids

Designing a Resource Broker for Heterogeneous Grids Designing a Resource Broker for Heterogeneous Grids Srikumar Venugopal, Krishna Nadiminti, Hussein Gibbins and Rajkumar Buyya Grid Computing and Distributed Systems Laboratory Dept. of Computer Science

More information

Introduction to Grid Computing

Introduction to Grid Computing Milestone 2 Include the names of the papers You only have a page be selective about what you include Be specific; summarize the authors contributions, not just what the paper is about. You might be able

More information

Motivation. Threads. Multithreaded Server Architecture. Thread of execution. Chapter 4

Motivation. Threads. Multithreaded Server Architecture. Thread of execution. Chapter 4 Motivation Threads Chapter 4 Most modern applications are multithreaded Threads run within application Multiple tasks with the application can be implemented by separate Update display Fetch data Spell

More information

Multiprocessor Scheduling. Multiprocessor Scheduling

Multiprocessor Scheduling. Multiprocessor Scheduling Multiprocessor Scheduling Will consider only shared memory multiprocessor or multi-core CPU Salient features: One or more caches: cache affinity is important Semaphores/locks typically implemented as spin-locks:

More information

NUSGRID a computational grid at NUS

NUSGRID a computational grid at NUS NUSGRID a computational grid at NUS Grace Foo (SVU/Academic Computing, Computer Centre) SVU is leading an initiative to set up a campus wide computational grid prototype at NUS. The initiative arose out

More information

Data Management 1. Grid data management. Different sources of data. Sensors Analytic equipment Measurement tools and devices

Data Management 1. Grid data management. Different sources of data. Sensors Analytic equipment Measurement tools and devices Data Management 1 Grid data management Different sources of data Sensors Analytic equipment Measurement tools and devices Need to discover patterns in data to create information Need mechanisms to deal

More information

GT 4.2.0: Community Scheduler Framework (CSF) System Administrator's Guide

GT 4.2.0: Community Scheduler Framework (CSF) System Administrator's Guide GT 4.2.0: Community Scheduler Framework (CSF) System Administrator's Guide GT 4.2.0: Community Scheduler Framework (CSF) System Administrator's Guide Introduction This guide contains advanced configuration

More information

Fault tolerance based on the Publishsubscribe Paradigm for the BonjourGrid Middleware

Fault tolerance based on the Publishsubscribe Paradigm for the BonjourGrid Middleware University of Paris XIII INSTITUT GALILEE Laboratoire d Informatique de Paris Nord (LIPN) Université of Tunis École Supérieure des Sciences et Tehniques de Tunis Unité de Recherche UTIC Fault tolerance

More information

Grid Computing: Status and Perspectives. Alexander Reinefeld Florian Schintke. Outline MOTIVATION TWO TYPICAL APPLICATION DOMAINS

Grid Computing: Status and Perspectives. Alexander Reinefeld Florian Schintke. Outline MOTIVATION TWO TYPICAL APPLICATION DOMAINS Grid Computing: Status and Perspectives Alexander Reinefeld Florian Schintke Schwerpunkte der Informatik" Ringvorlesung am 05.06.2003 1 Outline MOTIVATION o What s a Grid? Why using Grids? TWO TYPICAL

More information

OAR batch scheduler and scheduling on Grid'5000

OAR batch scheduler and scheduling on Grid'5000 http://oar.imag.fr OAR batch scheduler and scheduling on Grid'5000 Olivier Richard (UJF/INRIA) joint work with Nicolas Capit, Georges Da Costa, Yiannis Georgiou, Guillaume Huard, Cyrille Martin, Gregory

More information

Grid Computing Systems: A Survey and Taxonomy

Grid Computing Systems: A Survey and Taxonomy Grid Computing Systems: A Survey and Taxonomy Material for this lecture from: A Survey and Taxonomy of Resource Management Systems for Grid Computing Systems, K. Krauter, R. Buyya, M. Maheswaran, CS Technical

More information

PROOF-Condor integration for ATLAS

PROOF-Condor integration for ATLAS PROOF-Condor integration for ATLAS G. Ganis,, J. Iwaszkiewicz, F. Rademakers CERN / PH-SFT M. Livny, B. Mellado, Neng Xu,, Sau Lan Wu University Of Wisconsin Condor Week, Madison, 29 Apr 2 May 2008 Outline

More information

A Brief Survey on Resource Allocation in Service Oriented Grids

A Brief Survey on Resource Allocation in Service Oriented Grids A Brief Survey on Resource Allocation in Service Oriented Grids Daniel M. Batista and Nelson L. S. da Fonseca Institute of Computing State University of Campinas Avenida Albert Einstein, 1251 13084-971

More information

EGEE and Interoperation

EGEE and Interoperation EGEE and Interoperation Laurence Field CERN-IT-GD ISGC 2008 www.eu-egee.org EGEE and glite are registered trademarks Overview The grid problem definition GLite and EGEE The interoperability problem The

More information

THE GLOBUS PROJECT. White Paper. GridFTP. Universal Data Transfer for the Grid

THE GLOBUS PROJECT. White Paper. GridFTP. Universal Data Transfer for the Grid THE GLOBUS PROJECT White Paper GridFTP Universal Data Transfer for the Grid WHITE PAPER GridFTP Universal Data Transfer for the Grid September 5, 2000 Copyright 2000, The University of Chicago and The

More information

Building Campus HTC Sharing Infrastructures. Derek Weitzel University of Nebraska Lincoln (Open Science Grid Hat)

Building Campus HTC Sharing Infrastructures. Derek Weitzel University of Nebraska Lincoln (Open Science Grid Hat) Building Campus HTC Sharing Infrastructures Derek Weitzel University of Nebraska Lincoln (Open Science Grid Hat) HCC: Campus Grids Motivation We have 3 clusters in 2 cities. Our largest (4400 cores) is

More information

CSCE 313: Intro to Computer Systems

CSCE 313: Intro to Computer Systems CSCE 313 Introduction to Computer Systems Instructor: Dr. Guofei Gu http://courses.cse.tamu.edu/guofei/csce313/ Programs, Processes, and Threads Programs and Processes Threads 1 Programs, Processes, and

More information

Extensible Job Managers for Grid Computing

Extensible Job Managers for Grid Computing Extensible Job Managers for Grid Computing Paul D. Coddington Lici Lu Darren Webb Andrew L. Wendelborn Department of Computer Science University of Adelaide Adelaide, SA 5005, Australia Email: {paulc,andrew,darren}@cs.adelaide.edu.au

More information

CSCE 313 Introduction to Computer Systems. Instructor: Dezhen Song

CSCE 313 Introduction to Computer Systems. Instructor: Dezhen Song CSCE 313 Introduction to Computer Systems Instructor: Dezhen Song Programs, Processes, and Threads Programs and Processes Threads Programs, Processes, and Threads Programs and Processes Threads Processes

More information

George Mason University

George Mason University George Mason University CONCEPTUAL COMPARATIVE STUDY OF JOB MANAGEMENT SYSTEMS A Report for the NSA LUCITE Task Order Productive Use of Distributed Reconfigurable Computing Tarek El-Ghazawi, P.I. Kris

More information

Types of Virtualization. Types of virtualization

Types of Virtualization. Types of virtualization Types of Virtualization Emulation VM emulates/simulates complete hardware Unmodified guest OS for a different PC can be run Bochs, VirtualPC for Mac, QEMU Full/native Virtualization VM simulates enough

More information

Operating Systems CS3502 Spring 2018

Operating Systems CS3502 Spring 2018 Operating Systems CS3502 Spring 2018 Presented by Dr. Guoliang Liu Department of Computer Science College of Computing and Software Engineering Kennesaw State University Computer Systems See Appendix G

More information

GRAM: Grid Resource Allocation & Management

GRAM: Grid Resource Allocation & Management Copyright (c) 2002 University of Chicago and The University of Southern California. All Rights Reserved. This presentation is licensed for use under the terms of the Globus Toolkit Public License. See

More information

Job Management System Extension To Support SLAAC-1V Reconfigurable Hardware

Job Management System Extension To Support SLAAC-1V Reconfigurable Hardware Job Management System Extension To Support SLAAC-1V Reconfigurable Hardware Mohamed Taher 1, Kris Gaj 2, Tarek El-Ghazawi 1, and Nikitas Alexandridis 1 1 The George Washington University 2 George Mason

More information

Advanced School in High Performance and GRID Computing November Introduction to Grid computing.

Advanced School in High Performance and GRID Computing November Introduction to Grid computing. 1967-14 Advanced School in High Performance and GRID Computing 3-14 November 2008 Introduction to Grid computing. TAFFONI Giuliano Osservatorio Astronomico di Trieste/INAF Via G.B. Tiepolo 11 34131 Trieste

More information

Elastic Management of Cluster-based Services in the Cloud

Elastic Management of Cluster-based Services in the Cloud Elastic Management of Cluster-based Services in the Cloud Rafael Moreno-Vozmediano, Ruben S. Montero, Ignacio M. Llorente Dept. Arquitectura de Computadores y Automática Universidad Complutense de Madrid

More information

A FRAMEWORK FOR THE DYNAMIC RECONFIGURATION OF SCIENTIFIC APPLICATIONS IN GRID ENVIRONMENTS

A FRAMEWORK FOR THE DYNAMIC RECONFIGURATION OF SCIENTIFIC APPLICATIONS IN GRID ENVIRONMENTS A FRAMEWORK FOR THE DYNAMIC RECONFIGURATION OF SCIENTIFIC APPLICATIONS IN GRID ENVIRONMENTS By Kaoutar El Maghraoui A Thesis Submitted to the Graduate Faculty of Rensselaer Polytechnic Institute in Partial

More information

HPC learning using Cloud infrastructure

HPC learning using Cloud infrastructure HPC learning using Cloud infrastructure Florin MANAILA IT Architect florin.manaila@ro.ibm.com Cluj-Napoca 16 March, 2010 Agenda 1. Leveraging Cloud model 2. HPC on Cloud 3. Recent projects - FutureGRID

More information

enanos Grid Resource Broker

enanos Grid Resource Broker enanos Grid Resource Broker Ivan Rodero, Julita Corbalán, Rosa M. Badia, and Jesús Labarta CEPBA-IBM Research Institute, Technical University of Catalonia (UPC), Spain {irodero, juli, rosab, jesus}@ac.upc.es

More information

Dynamic fault tolerant grid workflow in the water threat management project

Dynamic fault tolerant grid workflow in the water threat management project Rochester Institute of Technology RIT Scholar Works Theses Thesis/Dissertation Collections 2010 Dynamic fault tolerant grid workflow in the water threat management project Young Suk Moon Follow this and

More information

The EU DataGrid Fabric Management

The EU DataGrid Fabric Management The EU DataGrid Fabric Management The European DataGrid Project Team http://www.eudatagrid.org DataGrid is a project funded by the European Union Grid Tutorial 4/3/2004 n 1 EDG Tutorial Overview Workload

More information

Real Parallel Computers

Real Parallel Computers Real Parallel Computers Modular data centers Background Information Recent trends in the marketplace of high performance computing Strohmaier, Dongarra, Meuer, Simon Parallel Computing 2005 Short history

More information

University of Castilla-La Mancha

University of Castilla-La Mancha University of Castilla-La Mancha A publication of the Computing Systems Department Grid Metascheduling Using Network Information: A Proof-of-Concept Implementation by Luis Tomás, Agustín Caminero, Blanca

More information

Predicting the Performance of a GRID Environment: An Initial Effort to Increase Scheduling Efficiency

Predicting the Performance of a GRID Environment: An Initial Effort to Increase Scheduling Efficiency Predicting the Performance of a GRID Environment: An Initial Effort to Increase Scheduling Efficiency Nuno Guerreiro and Orlando Belo Department of Informatics, School of Engineering, University of Minho

More information

Pegasus. Automate, recover, and debug scientific computations. Rafael Ferreira da Silva.

Pegasus. Automate, recover, and debug scientific computations. Rafael Ferreira da Silva. Pegasus Automate, recover, and debug scientific computations. Rafael Ferreira da Silva http://pegasus.isi.edu Experiment Timeline Scientific Problem Earth Science, Astronomy, Neuroinformatics, Bioinformatics,

More information

GridNEWS: A distributed Grid platform for efficient storage, annotating, indexing and searching of large audiovisual news content

GridNEWS: A distributed Grid platform for efficient storage, annotating, indexing and searching of large audiovisual news content 1st HellasGrid User Forum 10-11/1/2008 GridNEWS: A distributed Grid platform for efficient storage, annotating, indexing and searching of large audiovisual news content Ioannis Konstantinou School of ECE

More information

Managing MPICH-G2 Jobs with WebCom-G

Managing MPICH-G2 Jobs with WebCom-G Managing MPICH-G2 Jobs with WebCom-G Padraig J. O Dowd, Adarsh Patil and John P. Morrison Computer Science Dept., University College Cork, Ireland {p.odowd, adarsh, j.morrison}@cs.ucc.ie Abstract This

More information

University of Alberta

University of Alberta University of Alberta PLACEHOLDER SCHEDULING FOR OVERLAY METACOMPUTING by Christopher James Pinchak A thesis submitted to the Faculty of Graduate Studies and Research in partial fulfillment of the requirements

More information

MONITORING OF GRID RESOURCES

MONITORING OF GRID RESOURCES MONITORING OF GRID RESOURCES Nikhil Khandelwal School of Computer Engineering Nanyang Technological University Nanyang Avenue, Singapore 639798 e-mail:a8156178@ntu.edu.sg Lee Bu Sung School of Computer

More information

Dynamic Workflows for Grid Applications

Dynamic Workflows for Grid Applications Dynamic Workflows for Grid Applications Dynamic Workflows for Grid Applications Fraunhofer Resource Grid Fraunhofer Institute for Computer Architecture and Software Technology Berlin Germany Andreas Hoheisel

More information

Globus Toolkit 4 Execution Management. Alexandra Jimborean International School of Informatics Hagenberg, 2009

Globus Toolkit 4 Execution Management. Alexandra Jimborean International School of Informatics Hagenberg, 2009 Globus Toolkit 4 Execution Management Alexandra Jimborean International School of Informatics Hagenberg, 2009 2 Agenda of the day Introduction to Globus Toolkit and GRAM Zoom In WS GRAM Usage Guide Architecture

More information

Grid-Based Data Mining and the KNOWLEDGE GRID Framework

Grid-Based Data Mining and the KNOWLEDGE GRID Framework Grid-Based Data Mining and the KNOWLEDGE GRID Framework DOMENICO TALIA (joint work with M. Cannataro, A. Congiusta, P. Trunfio) DEIS University of Calabria ITALY talia@deis.unical.it Minneapolis, September

More information

Work Queue + Python. A Framework For Scalable Scientific Ensemble Applications

Work Queue + Python. A Framework For Scalable Scientific Ensemble Applications Work Queue + Python A Framework For Scalable Scientific Ensemble Applications Peter Bui, Dinesh Rajan, Badi Abdul-Wahid, Jesus Izaguirre, Douglas Thain University of Notre Dame Distributed Computing Examples

More information

EFFICIENT SCHEDULING TECHNIQUES AND SYSTEMS FOR GRID COMPUTING

EFFICIENT SCHEDULING TECHNIQUES AND SYSTEMS FOR GRID COMPUTING EFFICIENT SCHEDULING TECHNIQUES AND SYSTEMS FOR GRID COMPUTING By JANG-UK IN A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR

More information

VMware View Upgrade Guide

VMware View Upgrade Guide View 4.0 View Manager 4.0 View Composer 2.0 This document supports the version of each product listed and supports all subsequent versions until the document is replaced by a new edition. To check for

More information

High Performance Computing Course Notes Grid Computing I

High Performance Computing Course Notes Grid Computing I High Performance Computing Course Notes 2008-2009 2009 Grid Computing I Resource Demands Even as computer power, data storage, and communication continue to improve exponentially, resource capacities are

More information

A Resource Discovery Algorithm in Mobile Grid Computing Based on IP-Paging Scheme

A Resource Discovery Algorithm in Mobile Grid Computing Based on IP-Paging Scheme A Resource Discovery Algorithm in Mobile Grid Computing Based on IP-Paging Scheme Yue Zhang 1 and Yunxia Pei 2 1 Department of Math and Computer Science Center of Network, Henan Police College, Zhengzhou,

More information

Grid Computing Fall 2005 Lecture 10 and 12: Globus V2. Gabrielle Allen

Grid Computing Fall 2005 Lecture 10 and 12: Globus V2. Gabrielle Allen Grid Computing 7700 Fall 2005 Lecture 10 and 12: Globus V2 Gabrielle Allen allen@bit.csc.lsu.edu http://www.cct.lsu.edu/~gallen/ Globus 4 Primer Required Reading Coursework Essay: 4 pages Describe the

More information

Scientific Computing with UNICORE

Scientific Computing with UNICORE Scientific Computing with UNICORE Dirk Breuer, Dietmar Erwin Presented by Cristina Tugurlan Outline Introduction Grid Computing Concepts Unicore Arhitecture Unicore Capabilities Unicore Globus Interoperability

More information

Usage of LDAP in Globus

Usage of LDAP in Globus Usage of LDAP in Globus Gregor von Laszewski and Ian Foster Mathematics and Computer Science Division Argonne National Laboratory, Argonne, IL 60439 gregor@mcs.anl.gov Abstract: This short note describes

More information

LIGO Virtual Data. Realizing. UWM: Bruce Allen, Scott Koranda. Caltech: Kent Blackburn, Phil Ehrens, Albert. Lazzarini, Roy Williams

LIGO Virtual Data. Realizing. UWM: Bruce Allen, Scott Koranda. Caltech: Kent Blackburn, Phil Ehrens, Albert. Lazzarini, Roy Williams Realizing LIGO Virtual Data Caltech: Kent Blackburn, Phil Ehrens, Albert Lazzarini, Roy Williams ISI: Ewa Deelman, Carl Kesselman, Gaurang Mehta, Leila Meshkat, Laura Pearlman UWM: Bruce Allen, Scott Koranda

More information

CHAPTER 2 LITERATURE REVIEW AND BACKGROUND

CHAPTER 2 LITERATURE REVIEW AND BACKGROUND 8 CHAPTER 2 LITERATURE REVIEW AND BACKGROUND 2.1 LITERATURE REVIEW Several researches have been carried out in Grid Resource Management and some of the existing research works closely related to this thesis

More information

Announcements Processes: Part II. Operating Systems. Autumn CS4023

Announcements Processes: Part II. Operating Systems. Autumn CS4023 Operating Systems Autumn 2018-2019 Outline Announcements 1 Announcements 2 Announcements Week04 lab: handin -m cs4023 -p w04 ICT session: Introduction to C programming Outline Announcements 1 Announcements

More information

Multiprocessor Scheduling. Multiprocessor Scheduling

Multiprocessor Scheduling. Multiprocessor Scheduling Multiprocessor Scheduling Will consider only shared memory multiprocessor or multi-core CPU Salient features: One or more caches: cache affinity is important Semaphores/locks typically implemented as spin-locks:

More information

Multiprocessor Scheduling

Multiprocessor Scheduling Multiprocessor Scheduling Will consider only shared memory multiprocessor or multi-core CPU Salient features: One or more caches: cache affinity is important Semaphores/locks typically implemented as spin-locks:

More information