High Performance Computing Course Notes 2008-2009 2009 Grid Computing I
Resource Demands Even as computer power, data storage, and communication continue to improve exponentially, resource capacities are failing to keep up with what scientists demand of them A personal computer in 2001 is as fast as a supercomputer of 1990, but 10 years ago, biologists were happy to compute a single molecular structure. Now, they want to simulate the activities of human brain Personal computers now ship with more than 100 gigabytes (GB) of storage--as much as an entire 1990 supercomputer center. But by 2006, several physics projects, CERN's Large Hadron Collider (LHC) among them, will produce multiple petabytes (10 15 byte) of data per year Some wide area networks can now operate at 100 megabits per second (Mb/s), three orders of magnitude faster than the state-of-the-art 56 kilobits per second (Kb/s) that connected US supercomputer centers in 1985. But to work with colleagues across the world on petabyte data sets, scientists now demand tens of gigabits per second (Gb/s). 2
Technology trends Computer power doubles every 18 month Storage capacity doubles every 12 month Performance of wide area networks doubles every 9 months If the trend continues, the communication will become essentially free, which changes how we think about and undertake collaboration, that is, using remote resources to do things that we cannot do easily at home Not just conventional C/S model, but a large-scale, heterogeneous, dynamic infrastructure; we needs to deal with Different administrative domains Different architectures Different security and access control policies 3
Grid Infrastructure Built on the Internet and the World Wide Web, the Grid is a new class of infrastructure providing scalable, secure, high-performance mechanisms for discovering and negotiating access to remote resources make it possible for scientific collaborations to share resources on an unprecedented scale, and for geographically distributed groups to work together in ways that were previously impossible 4
What Grid can do? Using remote resources involves: Discover the resource Negotiate access Configuration of local hardware and software protect my own security and respect the security of the remote resources Implementing these steps requires uniform mechanisms for: creating and managing services on remote resources supporting single sign-on to distributed resources transferring large volume of data at high speeds forming large distributed virtual organisations Maintaining information about status and usage policies of the resources in the virtual organisations 5
What Grid can do? Today's Internet and Web technologies address basic communication requirements, but not the tasks just outlined The aim of Grid is to provide the infrastructure and tools that make large-scale, secure resource sharing possible and straightforward Grid tools are concerned with resource discovery, data management, Job scheduling, security, and so forth. 6
How Grid works The users enter grid by using the software interface running on their own computers Perform security validation Interact with resource broker Resource broker inquires information service to know which resources and services are available to process your program Resource broker inquires replica catalog to know all existing data Once the suitable resources are allocated, the resource broker assigns the job to the resources for execution 7
Grid Infrastructure 8
Grid Infrastructure Fabric layer contains the physical devices or resources that Grid users want to share and access Connectivity and Resource layer Connectivity layer Contains core communication and authentication protocols Communication protocols enable the exchange of data between resources authentication protocols provide cryptographically secure mechanisms Resource layer contains protocols that exploit communication and authentication protocols to enable the secure submission, initiation, monitoring, and control of resource-sharing operations Implementation: Globus 9
Grid Infrastructure collective layer contains protocols, services, and APIs that implement interactions across collections of resources. Exemplar services include brokering and information services for resource discovery and allocation; data replication services; diagnostic services; membership and policy services for keeping track of who in a community is allowed to access resources. user applications Call on the components in any other layer For example obtaining authentication credentials (connectivity layer protocols) querying an information system and replica catalog (collective services) submitting requests to appropriate resources (resource protocols) monitoring the progress of the various computations (resource protocols). 10
Security in Grid environments Traditional security technologies are concerned primarily with securing the interactions between clients and servers a client and a server mutually authenticate each other's identity the server determines whether to authorize requests issued by the client In Grid environments, the situation is more complex The distinction between client and server tends to disappear Single sign-on a user authenticate once and then assign to the computation the right to operate on his or her behalf achieved through the creation of a proxy credential Mapping to local security mechanisms Delegation Community authorization and policy It is infeasible for each resource to keep track of identity of every user Instead, users can be verified using group membership, which can be achieved with a cryptographic credential issued by a trusted third party (a community representative) 11
Security in Grid environments 12
Reference Ian Foster, The Grid: A new Infrastructure for 21st Century Science, http://www.aip.org/pt/vol-55/iss- 2/p42.html 13