By Ian Foster. Zhifeng Yun

Transcription

1 By Ian Foster Zhifeng Yun

2 Outline Introduction Globus Architecture Globus Software Details Dev.Globus Community Summary Future Readings

3 Introduction Globus Toolkit v4 is the work of many Globus Alliance members, at Argonne Natl. Lab & U.Chicago USC Information Sciences Institute National Center for Supercomputing Applns U. Edinburgh Swedish PDC Univa Corporation Other contributors at other institutions Supported by DOE, NSF, UK EPSRC, and other sources

4 Overview First version released in late 1990s Now Web Service-based GT4 released Globus is:! A community of users and developers! The software the Globus Toolkit! The infrastructure that supports this community

5 Globus Toolkit The software provides: A set of services implementations Tools for building new Web Services A security infrastructure for authentication and authorization Client APIs and command line programs Detailed documentation of various components and interfaces, etc.

6 Globus Architecture CLIENT Your Your Java Java Client Client Your Your C Client Client Your Your Python Client Client Your Your Java Java Client Client Your Your C Client Client Your Your Python Client Client Interoperable WS-I-compliant SOAP messaging X.509 credentials = common authentication Your Your Java Java Service GRAM RFT Delegation Index Trigger Archiver CAS OGSA-DAI GTCP Your Python Service pyglobus WS Core Your C Service C WS Core SimpleCA MyProxy RLS Pre-WS GRAM Pre-WS MDS SERVER Java Services in Apache Axis Plus GT Libraries and Handlers Python hosting, GT Libraries C Services using GT Libraries and Handlers

7 Globus Toolkit version 2 Web Services Components Pre-WS Authentication Authorization Grid Resource Alloc. Mgmt (GRAM) Monitoring & Discovery (MDS) C Common Libraries Non-WS Components Security Data Mgmt Execution Mgmt Info Services Common Runtime

8 Globus Toolkit version 3 Community Authorization WS Authentication Authorization Data Access & Integration Reliable File Transfer Grid Resource Alloc. Mgmt (WS GRAM) MDS3 Java WS Core Web Services Components Pre-WS Authentication Authorization Replica Location Grid Resource Alloc. Mgmt (GRAM) Monitoring & Discovery (MDS) C Common Libraries extensible IO (XIO) Non-WS Components Security Data Mgmt Execution Mgmt Info Services Common Runtime

9 Delegation Globus Toolkit version 4 Data Replication Grid Telecontrol Protocol Community Scheduling Framework WebMDS Python WS Core Community Authorization Authentication Authorization Data Access & Integration Reliable File Transfer Workspace Management Grid Resource Allocation & Management Trigger Index C WS Core Java WS Core Web Services Components Pre-WS Authentication Authorization Credential Mgmt Replica Location Pre-WS Grid Resource Alloc. & Mgmt Pre-WS Monitoring & Discovery C Common Libraries extensible IO (XIO) Non-WS Components Security Data Mgmt Execution Mgmt Info Services Common Runtime

10 Globus Toolkit: Open Source Grid Infrastructure Credential Mgmt Delegation Community Authorization Data Replication Replica Location Data Access & Integration Reliable File Transfer Grid Telecontrol Protocol Community Scheduling Framework Workspace Management Globus Toolkit v4 WebMDS Trigger Python Runtime C Runtime Authentication Authorization Grid Resource Allocation & Management Index Java Runtime Security Data Mgmt Execution Mgmt Info Services Common Runtime

11 GT4 Data Management Stage/move large data to/from nodes, Reliable File Transfer (RFT) Alone, and integrated with GRAM Locate data of interest Replica Location Service (RLS) Replicate data for performance/reliability Distributed Replication Service (DRS) Provide access to diverse data sources File systems, parallel file systems, hierarchical storage: Databases: OGSA DAI

12 A high-performance, secure, reliable data transfer protocol optimized for high-bandwidth wide-area networks server ~ high performance FTP server with GSI Multiple nodes work together and act as a single server Each node moves (reads or writes) only the pieces of the file that it is responsible for. Pluggable Front-end: e.g., future WS control channel Back-end: e.g., HPSS, cluster file systems Transfer: e.g., UDP, NetBLT transport

13 Striped Service A distributed service that runs on a storage cluster Every node of the cluster is used to transfer data into/out of the cluster Head node coordinates transfers Lead to very high performance Maximizes use of Gbit+ WANs Parallel Filesystem Parallel Transfer Fully utilizes bandwidth of network interface on single nodes. Parallel Filesystem Striped Transfer Fully utilizes bandwidth of Gb+ WAN using multiple nodes.

14 Fire-and-forget transfer Web services interface Many files & directories Reliable File Transfer: Third Party Transfer Integrated failure recovery SOAP Messages RFT Client RFT Service Notifications (Optional) Server Server Master DSI Protocol Interpreter Data Channel Data Channel Protocol Interpreter Master DSI IPC Link IPC Link IPC Receiver Slave DSI Data Channel Data Channel Slave DSI IPC Receiver

15 Globus Toolkit: Open Source Grid Infrastructure Credential Mgmt Delegation Community Authorization Data Replication Replica Location Data Access & Integration Reliable File Transfer Grid Telecontrol Protocol Community Scheduling Framework Workspace Management Globus Toolkit v4 WebMDS Trigger Python Runtime C Runtime Authentication Authorization Grid Resource Allocation & Management Index Java Runtime Security Data Mgmt Execution Mgmt Info Services Common Runtime

16 Execution Management (GRAM) Common WS interface to schedulers Condor, LSF, PBS, More generally: interface for process execution management Lay down execution environment Stage data Monitor & manage lifecycle Kill it, clean up A basis for application-driven provisioning

17 GT4 WS GRAM Architecture Service host(s) and compute element(s) functions ClientJob Delegate GT4 Java Container Delegation GRAM services RFT File Transfer Transfer request Job events Local job control Delegate FTP control sudo SEG GRAM adapter FTP data Compute element Local scheduler User job Remote storage element(s)

18 functions ClientJob Delegate GT4 WS GRAM Architecture GT4 Java Container Delegation GRAM services RFT File Transfer Service host(s) and compute element(s) Transfer request Job events Local job control Delegate FTP control Delegated credential can be: Made available to the application sudo SEG GRAM adapter FTP data Compute element Local scheduler User job Remote storage element(s)

19 functions ClientJob Delegate GT4 WS GRAM Architecture GT4 Java Container Delegation GRAM services RFT File Transfer Transfer request Delegated credential can be: Used to authenticate with RFT Service host(s) and compute element(s) Job events Local job control Delegate FTP control sudo SEG GRAM adapter FTP data Compute element Local scheduler User job Remote storage element(s)

20 functions ClientJob Delegate GT4 WS GRAM Architecture GT4 Java Container Delegation GRAM services RFT File Transfer Service host(s) and compute element(s) Transfer request Job events Local job control Delegate FTP control Delegated credential can be: Used to authenticate with sudo SEG GRAM adapter FTP data Compute element Local scheduler User job Remote storage element(s)

21 Choosing User Accounts You may be authorized to use more than one account at the remote site By default, the first listed in the gridmapfile will be used You may request a specific user account using the <localuserid> element

22 Summary

23 The Globus Toolkit is a Collection of Components A set of loosely-coupled components, with: Services and clients Libraries Development tools GT components are used to build Grid-based applications and services GT can be viewed as a Grid SDK GT4 use WS protocols for service interactions GT4 services work according to WSRF behavior paradigms

24 Further readings Service Oriented Architecture What is Service-Oriented Architecture?. Hao He. Service-Oriented Architecture: A Primer. Michael S. Pallos. The Benefits of a Service-Oriented Architecture. Michael Stevens. Web services Web Services Specifications - OGSA, WSRF The Physiology of the Grid. Ian Foster, Carl Kesselman, Jeffrey M. Nick, Steven Tuecke. The Anatomy of the Grid. Ian Foster, Carl Kesselman, Steven Tuecke. Web Services Resource Framework -

25 Reference B. Clifford and S. S. Bharathi, Globus Toolkit Use Case: A Monitoring System for the Earth System Grid esday/mds-tg07.ppt Question?