Zukünftige Dienste im D-Grid: Neue Anforderungen an die Rechenzentren? Alexander Reinefeld Zuse-Institut Berlin Humboldt Universität zu Berlin ZKI Herbsttagung in Heilbronn, 29.09.2004 1
Contents 1 What is the Grid? 2 Grid Concepts 3 Grid Standards 4 Challenges for Computer Services 2
1 What is the Grid? 3
The Three Grids Information Grid web HTML file sharing Service Grid search engines OGSA SOAP, WSDL, UDDI XML network Resource Grid storage computing access, usage publication of meta information 4
What is The Grid? Web = HTML Grid =??? [The Grid] intends to make access to computing power, scientific data repositories and experimental facilities as easy as the Web makes access to information. Tony Blair, 2002 5
What is The Web? A fractal thing? It is fundamentally a decentralized thing and when we really use it practically, it becomes a fractal thing. Tim Berners-Lee 6
A Fractal Thing? Fractals (Latin fractus = broken) combine structure and irregularity. Features: Self-similarity Infinite detail regardless of magnification In the eyes of computer scientists: ideal scalability (top down) no global state (difficult to control bottom up) 7
More Definitions of The Grid You're able to get what you want, when you want it. You don't have to concern yourself with the infrastructure, the resources simply appear on demand. You pay only for what you use, as reflected on your monthly bill. Hype versus reality 8
Grid Characteristics Cannot affect site policies. Difficult orchestration, coordination. Resources may differ significantly. local autonomy heterogeneity scalability dynamics Grid may be local or worldwide. Communication latency, bottlenecks due to hierarchy, synchronization. Form and properties of resources may change during the lifespan of the application. 9
2 Grid Concepts 10
Transparency Transparency Access Location Migration Relocation Replication Concurrency Failure Persistence Description Hide differences in data representation and how a resource is accessed Hide where a resource is located Hide that a resource may move to another location Hide that a resource may be moved to another location while in use Hide that a resource may be replicated for concurrent access Hide that a resource may be shared by several competitive users Hide the failure and recovery of a resource Hide whether a (software) resource is in memory, on disk, in an archive 11
Transparency by Virtualization Supercomputer Experiments Today: Monolithic, vertically integrated, proprietary solutions. PC-Cluster Scientist/user Supercomputer Tomorrow: Flexible, adaptable, interchangeable one-stop-shop solutions via standard interfaces. Scientist/user Archives M i d d l e w a r e PC-Cluster Supercomputer PC-Cluster Archive PC-Cluster Analysis Analysis Archive Archives Experiment Supercomputer Analysis J. Taylor, modified by Hoffmann, Putzer, Reinefeld 12
Virtualization hides the complexity of distribution, provides a user-centric view, provides synergy, QoS? and complicates the system architecture! 13
3 Grid Standards: WS, OGSA, OGSI, WSRF 14
1995 Globus project proposal Globus Toolkit - Timeline 1998 GT 1.0: GRAM, MDS, 2001 GT 2.0: GridFTP, packaging, reliability, (toolkit approach) Standards 2002 GT 3.0 Technology Preview (tracking OGSI definition, substantial extensions to web services) 6/03 GT 3.0: OGSI-based, GT2 functionality OGSA v0.01: 9/02 v0.17: 6/04 OGSI 1st spec. 07/03 1/05 GT 4.0: WSRF WSRF intro. 01/2004 (back to web services, family of standards) 15
Web Services The famous XML-family WSDL (Web Services Description Language) UDDI (Universal Description, Discovery and Integration) SOAP (Simple Object Access protocol) and other members 16
17
Web Services vs. Grid Services Web Services address discovery & invocation of persistent, stateless services. Grid Services support transient, stateful service instances. Significant implications for how services are managed, named, discovered, and used! OGSA Open Grid Service Architecture proposed at GGF 2002, physiology paper 18
OGSA - Open Grid Services Architecture Refactor Globus protocol suite to enable common base Service orientation to virtualize resources, services, information Embrace Web Service technologies for standard IDL Result: Standard interfaces & behaviors: Grid Service. 19
OGSI Components and their Interplay 1. Factory suchen Registry 2. Auftrag zur Instanziierung Factory 3. Eintrag eines neuen Handle Client Grid Service Handle Resolver 5. Aufruf Instanz (Zustand) 4. Neue Dienstinstanz meldet sich an dauerhafte Handles kurzlebige Handles nicht in OGSI spezifiziert 20
Three Major Concerns about OGSI and how they are addressed by WSRF Too much stuff in one specification WSRF is a family of composable specifications. Does not work well with existing Web services tooling WSRF tones down the usage of XML schema. Too object oriented WSRF distinguishes between a service and the stateful resources acted upon by that service. 21
From OGSI to WSRF OGSI HandleResolver porttype Service data definition & access GridService lifetime mgmt ServiceGroup porttypes Base fault type Grid Service Reference Grid Service Handle Notification porttypes WSRF WS-RenewableReferences WS-ResourceProperties WS-ResourceLifetime WS-ServiceGroup WS-BaseFaults WS-Addressing Endpoint Reference WS-Addressing Endpoint Reference WS-Notification WSRF Specs other Specs 22
WSRF Components and their Interplay discovery, interaction introspection status notification initial endpoint reference Interface Web Service WS Resource status: context A S runtime environment context B WS Resource status: R 23
Convergence (?) Grid GT1 GT2 Started far apart in apps & tech Web Have been converging HTTP WSDL, WS-* OGSI WSDL 2, WSDM WSRF The definition of WSRF means that Grid and Web communities can move forward on a common base. [Ian Foster 1/2004] 24
3 Challenges for Computer Services 25
Challenges dynamic formation & management of virtual organizations discovery & online negotiation of access to services: who, what, why, when, how configuration of applications and systems that are able to deliver multiple QoS autonomic management of distributed infrastructures, services and applications management of distributed state open, extensible infrastructure 26
Challenges for Computer Services Service Virtualization will cause trouble! o How to guarantee QoS for a service that I do not provide locally? Dynamic behavior means more trouble! o No self-configuration yet. Not even policies. The world is your customer not your local campus student. A gentle warning to our politicians: Grid Resources must be paid for! Grids will improve system utilization, but there will be more demand. 27
Computer services will need more qualified staff, not less. increasing complexity increasing remote accesses 28
4 Summary 29
Lessons Learned From Each Chapter 1 What is the Grid? Web (and Grid) are fractal things [Berners-Lee]. Current Grid implementations are not. 2 Grid Concepts Transparency through virtualization. Difficult system design. 3 Grid Standards: WS, OGSA, OGSI, WSRF Will probably stabilize on WS. But it really doesn t matter. 4 Challenges for Computer Services More work ahead. Need more qualified staff. Many open problems. 30
Information A. Reinefeld, F. Schintke. Dienste und Standards für das Grid Computing. In: J. von Knop, W. Haferkamp (Hrsg.), 18. DFN Arbeitstagung über Kommunikationsnetze, Düsseldorf, Lecture Notes in Informatics, 2004, vol. P-55, pp. 293-304. NGG2 Expert Group, Next Generation Grids 2 Requirements and Options for European Grids: Research 2005-2010 and Beyond. NGG2 Report, July 2004, http://www.cordis.lu/ist/grids. e-science in Deutschland: F&E-Rahmenprogramm 2005 bis 2009. Vorgelegt von der D-Grid-Initiative, 6. Juli 2004. http://www.d-grid.de www.zib.de/csr 31