THE VEGA PERSONAL GRID: A LIGHTWEIGHT GRID ARCHITECTURE

Transcription

1 THE VEGA PERSONAL GRID: A LIGHTWEIGHT GRID ARCHITECTURE Wei Li, Zhiwei Xu, Bingchen Li, Yili Gong Institute of Computing Technology of Chinese Academy of Sciences Beijing China, {zxu, liwei, libingchen, gongyili}@ict.ac.cn ABSTRACT computing has emerged as a technology to solve the problem of sharing and cooperating of computational resources in wide area. In this paper, we propose a architecture called Vega Personal (the Vega PG). The key feature of the Vega PG is that it is a usercentered lightweight architecture. We make a mapping between traditional computer systems and the Vega PG, which comprises virtual hardware, operating system and User Environment. The virtual hardware comprises various resources distributed in wide area. The operating system (GOS) is a middleware based on the virtual hardware and responsible for the management of the virtual hardware and the activities of the computations in a. The User Environment comprises several software tools enabling end users to utilize resources conveniently. We also propose four important principles, which are Versatile Services, Enabling Intelligence, Global Uniformity and Autonomous Control, to evaluate the architecture design of the Vega PG. KEY WORDS Vega, Personal, Architecture 1. Introduction During the history of computers, how to share computational resources efficiently is a main target to computer system designers. computing [3], which derives from the concept of metacomputing [10], has emerged as a technology to solve the problem of sharing computational resources in wide area. We think that the emergence of the is due to following reasons: the worldwide distribution of computational resources, the maturity of interconnecting technologies in wide area (such as the Internet) and the increasing demands on resource sharing. In such condition, the fundamental problems to architecture designers are how to organize the resources effectively to provide the aggregated computational power. In this paper, we address above problems by an architecture called Vega Personal (Vega PG). The main idea of the Vega PG is to construct the as a virtual computer in wide area. One important feature of the Vega PG is the user-centered philosophy, that is, users can utilize resources as their own will. Another important feature is that the Vega PG does not aim at building a complex system. On the contrary, the Vega PG adopts a lightweight architecture, which uses simplified structures and protocols. The Vega PG also does not aim at rebuilding the from the hardware to the top application entirely; we just make a mapping between the and traditional computer systems and use the virtual computer concept to build the Vega PG on legacy systems. The benefit of this method is that we can borrow many mature technologies from the traditional computer architecture design. Here we give several important concepts in the Vega PG: virtual hardware is an abstraction to low-level components in the Vega PG, and it contains processors, the system bus and virtual devices. operating system (GOS) is a middleware based on the virtual hardware and it is responsible for managing the virtual hardware and computations. The GOS is running on processors logically and provides a developing and running environment for applications. User Environment comprises several software modules by which end users can explore and utilize various resources in a. The main components are Browser, Service Markup Language (GSML), Server, Service Request Protocol (GSRP), Mapper and Community. The paper is organized as follows: Section 2 introduces related works on architectures and the key feature of the Vega PG. Section 3 proposes four principles to evaluate the architecture design. Section 4 provides the overview of the Vega PG architecture. Section 5 describes the detail of the virtual hardware. Section 6 gives the internal design of the Vega GOS. Section 7 gives the Vega User Environment. Section 8 gives the conclusion and the future plan of the Vega PG. 2. Related Works Up to now several architectures have been proposed by various research organizations. As a research project, Globus [2] has been aiming at building the computational

2 and focusing on sharing and cooperating of computational resources. It has proposed a ed architecture [4] and its implementation, the Globus Toolkit [1], has been widely deployed and used. The goal of Legion Project [6] is to integrate resources into a single world wide metasystem. Legion uses the object-oriented technology in their architecture design, which is a ed virtual machine. Web Service is based on Web technology and supported by many industry powers [8]. The model [8] of Web Service describes its basic roles and operations. Bring the concepts and technologies of the Globus Toolkit and Web Service together, the Globus Project proposes an Open Services Architecture [5] recently. OGSA adopts service-oriented conceptions and represents everything as a service. The Vega PG project is developed at Institute of Computing Technology, Chinese Academy of Sciences. The main features of the Vega PG are as follows: The Vega PG is a user-centered architecture. In a Client-Server model or Browser-Server model, the user s activity is controlled or predefined by servers. When a user accesses to a Web server, he must follow the schema defined by this server and can t operate according to one s own inclinations. In the Vega PG, the user is the master of the. The only supplies resources and its activities are controlled by end users. How to use these resources is left to end users via the client side programming method. In another hands, most other systems are aiming at building the infrastructure, whose works are mostly at server side. The Vega PG mainly focuses on developing protocols and tools at client side. The Vega PG is a lightweight architecture. The Vega PG does not aim at building a complex system. We simplify the architecture of the Vega PG as much as possible, which can make the Vega PG very small and efficient. For example, when solving the problem of resource discovering, we simply assign every resource a unique integer, by which resource routers can locate resources quickly [9]. 3. Principles for the Design of Vega PG When designing the Vega PG architecture, we propose following principles called VEGA [12] [13] to evaluate the Vega PG architecture design: Versatile Services. The should have the ability to support various services. That is, the should be constructed as an infrastructure to provide a developing and running environment supporting various applications, various using patterns and various platforms. Also all applications and interactions should use the service mechanism and all development, deployment and management should be abstracted to services. Enabling Intelligence. The should have the ability to support the intelligent computing. The should be more intelligent than the Internet and it can provide the automatic production of information, knowledge and services. The itself is not the subject that provides the intelligence while it can assist the people to develop the intelligent applications. Global Uniformity. From the viewpoint of users, the should be a single virtual computer. The should supply Single System Image (SSI), Single Sign-On and other related technologies. Autonomous Control. The should not be governed by a central administration. All components in the can freely join or leave at their own will. For members that provide the resources, they have full rights to control their resources exported; and for members that use the resources, they have rights to use the resources as their will within the purview of their rights. 4. The Architecture Overview Similar to traditional computer systems, we divide the Vega PG virtual computer in three parts, which are virtual hardware, operating system and User Environment. The virtual here means that the virtual hardware in the Vega PG is not real hardware; they are logic components that comprise both hardware and software. The operating system is responsible for the management of virtual hardware in the Vega PG; based on the GOS, end users can use the Vega Browser to utilize resources. Three- architecture of the Vega PG is illustrated in Figure 1. Vega Browser GSML GSRP GSML Server Other Servers: Database, Web, Applications User Environment Layer Operating System Layer: OGSA, Globus, Web Service, Vega GOS Virtual Hardware Layer: Dawning Enabling Superservers Storage, Database, software, Processor Fig. 1 The Vega Personal Architecture The virtual hardware has three types of components: processor is an abstraction of a resource consumer that controls the activities of the virtual hardware. The processor is a logical concept and can be implemented as either software or hardware. virtual device is an abstraction of a resource provider in the. Different resource providers are viewed as different virtual devices that provide various services. For example, we are developing Dawning 4000 superservers, which are clusters with enabling technology to support platforms and applications [11] [14].

3 system bus is a communicating channel used to interconnect all virtual hardware together. Different from the Internet whose main function is transferring data, the role of the system bus is transferring computing power. The [9] enables application-level connectivity and allows resources to be efficiently deployed and discovered. The GOS includes system software and middleware, such as Globus, OGSA, web services, and other commercial software, as well as technologies of the Vega PG. The User Environment includes various application software servers, such as database servers, web servers, and business application servers. The Vega PG adds two new components at this, one at the client side and one at the server side. The Vega Browser is different from a traditional web browser, in that it allows users to write to and to operate the. The Vega Server (the GSML server) is a portal to the, which provides a logically single entry point for users to interact with the, and handles processing tasks that are common to all services. The Vega Browser and the Vega Server interact through a new protocol, called the Service Request Protocol (GSRP). Another new feature is the Service Markup Language (GSML), which allows users (not necessarily programmers) to specify services and user interface in an easy to use fashion. 5. Virtual Hardware In a environment, we think the most important components are resource consumers and resource producers, which are represented by processors and virtual devices. Another important role in virtual hardware is the system bus that links all processors and virtual devices together, and it can transfer computations from processors to virtual devices. The distribution topology of the virtual hardware is illustrated in Figure 2. We can see that the system bus is composed of resource routers, which interconnect with each other randomly. processors and virtual devices link together by connecting to nearby resource routers Processors The processor is a logical concept dissimilar to the traditional CPU. In practice, processors can be personal computers, special client devices or software at client side. In the Vega PG, the processor mainly works as a control center, which manages the activities of virtual hardware. virtual devices provide only basic services and users can use the services to custom their activities freely. The conception of the processor embodies the important philosophy of usercentered but not server-centered The System Bus As we know, the Internet based on the TCP/IP technology is a channel to transfer information. For example, we can use a web browser to obtain the needed information from a web server. Also we can add a HTML page to publish our information. Although a web server can use Common Gateway Interface (CGI) and other similar technologies to provide various services, these methods are not natural ways to transfer computing power from producers to consumers. Consider the PCI [7] system bus in a computer system: basic functions of the PCI are addressing, controlling and data transferring. Compared with the PCI system bus, the Internet does not have standard controlling functions. In a environment, we need a new communication channel, the system bus, to transfer computing powers. The system bus not only provides data transferring functions, but also supplies controlling functions, by which processors can manipulate the activities of virtual devices directly. Based on the Internet, the system bus comprises many resource routers and uses a protocol called Computing Protocol (GCP), which defines the communication pattern between processors and virtual devices s Processor System Bus Fig. 2 The topology of the virtual hardware Virtual Device The base of the system bus is the Internet, but it needs new protocols and software to support computation transferring because the Internet is lack of such support. The resource router [9] is the backbone of the system bus. It is a bridge linking all processors and virtual devices together. All processors and virtual devices can dynamically join or leave the by connecting or disconnecting to resource routers. When more resource routers linked together, the can expand to huge scale. The can be constructed as a fully distributed resource network via resource routers. The resource router is a transfer station for resource discovery request, and it can collect the information of virtual devices and give a path for resource requests from processors to virtual devices [9].

4 Computing Protocols The proposed Computing Protocol contains two basic s: the Computation Transfer Protocol () and Computation Representation Protocol (CRP). The protocol is designed to transfer computations from processors to virtual devices. As we know, the electric power in an electrical can move freely in electrical networks. Thus an appliance only needs an electrical outlet to receive the electric power. But in a computational, the computing power can t move from here to there. So we need to transfer the computation to virtual devices that provide the computing power. That is why we call it as the Computation Transfer Protocol. To transfer the computation to virtual devices, the protocol must be able to know the computing power of virtual devices and their locations. A processor must acquire the resource information of virtual devices before transferring the computation to a proper destination. Also there should have some mechanisms to locate virtual devices, which is fulfilled by resource routers. routers will gather the resource information of virtual devices by which processors can know where the computing power can be obtained. routers also record the routing information [9] about virtual devices by which resource requests can be transferred to proper destinations. In the Vega PG, the protocol messages can be divided into following types: Register/Unregister/Update messages, by which virtual devices can register/unregister/update the resource information to a resource router. Routing information update messages, by which resource routers can exchange routing information between each other. Search messages, by which processors can find virtual devices that provide the needed resources. Control messages, by which the processors can manipulate the activities of virtual devices. Interrupt messages, by which virtual devices can send interrupt requests to processors. Another protocol of the GCP is the CRP protocol, which is used to perform a computation after finding out a proper virtual device. In this, we use several abstractions to represent a computation as well as its behaviors. That is why we call it as a Representation. The first abstraction is active data, which represents a computation in a environment. Another abstraction is vega primitives, which represent the process to fulfill a computation. These two abstractions will be discussed in detail later. The CRP protocol mainly does following works: After finding out a virtual device via the protocol, the user can perform a computation, that is, a user can submit a computation to a virtual device. When a computation is running, a user can manipulate this computation at one s own will. For processors and virtual devices, they all need to communicate with each other both at the and the CRP. The resource router only has resource discovery functions and we need to implement only the protocol on resource routers. Figure 3 gives the relationships between the virtual hardware and the GCP protocol hierarchy. CRP Processor CRP protocol protocol CRP Virtual Device Fig. 3 The relationships between the GCP and the virtual hardware 5.3. Virtual Devices In the Vega PG, we abstract a resource provider as a virtual device, which can be an application, a database, a storage server or an instrument. Similar to the device in a traditional computer system, the virtual device is abstracted as an interface and a series of operations, and we do not care for their internals. In the Vega PG, the virtual device management is accomplished by GOS. 6. Internal of Operating System The main function of the GOS is to manage the virtual hardware, such as the system bus and virtual devices, to work harmoniously. Another important function of the GOS is to manage the active data, which represents a computation Virtual Hardware Management As we view a resource producer as a special kind of virtual hardware, we face the similar problem of device management in traditional operating systems. In the Vega PG, when controlling virtual devices, the GOS will issue commands, catch interrupts, and handle errors. GOS also provides a virtual device access interface that is simple and easy to use. To support the expansibility, the interface is same for all virtual devices. A key concept in the virtual hardware management is the virtual device independence, which means that it should be possible to access different virtual devices via the same interface. The GOS should take care of the problems caused by the fact that these devices implementations are different. In the GOS, we provide several fundamental abstractions for the operations of virtual devices and provide related interfaces.

5 One problem related to the virtual device independence is the uniform naming for virtual devices. In UNIX operating systems, all devices are viewed as files and they adopt the naming convention of file systems. The only thing a user can see about the device is a file name that hides the physical property of this device. In the Vega PG, we use a unique resource ID [9] to name a virtual device. Synchronous and asynchronous transfer is another important issue. In the traditional operating system, most devices are asynchronous. In the Vega PG, the virtual devices are implemented as asynchronous mode because it can save the waiting time of processors. Additionally, we use the soft-interrupt method to simulate the hardware interrupt mechanism. The GOS also provides the basic support to manage the system bus. We implement GCP protocols on different components including processors, resource routers and virtual devices. In addition to implementing the GCP protocols, we also implement the management of the system bus, such as initialization, error detection and recovery, as well as monitoring Active Data Management In a environment,we need an entity like process in the UNIX operating system to represent and manage the lifecycle and activities of a computation. As we view a resource provider as a virtual device, we think the computation transferred to a virtual device can be viewed as a special kind of data, which can be "written to or read from a virtual device. Unlike the data in a real hardware device, this data is dynamical, so we can view it as active data The Structure of Active Data The active data is an abstraction to a computation and mainly has following components: section, which is used to describe what resource the current computation needs. This section determines what kind of virtual device is needed. The information in this section will be sent to the system bus in order to find out the proper virtual devices. Representation section, which is used to describe the image of a computation. This image contains the data needed for executing a computation, such as the program file, command arguments, environment variables, the working path, input/output files and etc. Context section, which is used to record the running environment of current computation. This section contains the status information of current computation and a buffer for input and output operations between the user and the computation. Stub section, which is used to record the information about virtual devices. This section contains the address of the virtual device, which performs the computation represented by this active data. There are some other miscellaneous data in the structure of the active data and we will not refer them here. We think the active data with above sections is allinclusive for a computation Lifetime of Active Data As we know, a computation is a series of actions, changes, or functions and it has its own lifetime. In an active data, we use status information in the context section to describe the transformation of a computation. We also define a set of primitives to manage the running process of an active data. In our design, an active data has two states: one is the open state and another is the close state. Open state means that the user is now operating on this active data. Close state means that the user has finished the operations on this active data this time. Although an active data is closed, the computation it represented may still be running. In such condition, the user can load this active data again. In addition to these two basic states, we also define several other states invisible to users to indicate which phase a computation is in. An active data can transform from one phase to another via special primitives. Figure 4 gives the lifetime cycle of an active data and meanings of these primitives are described as follows: Vega primitive, which is used to create a new active data, load or delete an existed active data. Bind primitive, which is used to find a proper virtual device and establish a connection between active data and virtual devices. Read primitive, which is used to get the information of an active data from virtual devices. Write primitive, which is used to put the control command to manipulate the activities of active data. Close primitive, which is used to disconnect user from current active data. This primitive does not influence the running of current computation. Processor vega bind write read close Virtual Device Fig. 4 Operations of active data and the handshaking process between processors and virtual devices

6 7. Vega User Environment The Vega User Environment is a set of tools and protocols that enable end user to use resources conveniently, and it is illustrated in Figure 5. GSRP Mapper s different components of a traditional computer system. This method helps us construct the to a world wide virtual computer. We also make an abstraction to the computation in a environment, which is called active data. This abstraction helps us manage various computations in the. Vega GOS also provides a set of primitives enabling developers to build a application easily. In the top, we develop the GSML and related tools and protocols to enable users access resources conveniently. In the next step, we will implement the unfinished functions of the Vega PG. Another important work is consolidating the Vega PG with other systems, such as OGSA, Web Service and Globus. Browser GSML Page Community Acknowledgement GSML Server Fig. 5 The structure of Vega User Environment When designing the Vega User Environment, we mainly focus on two important issues: how to supply users the ability of programming and how to make a small resource view that is accordant to users requirement. To the first issue, we propose the Service Markup Language (GSML), which includes a set of tags to describe various resources such contents, services, databases and etc. The GSML supplies the programming ability to end users, who can construct a custom resource view easily and quickly. To the second issue, our method is to map the global resource view to a user-specific view. We use a threelevel conversion to realize this goal: the first one is the resource level, which includes all resources in the ; the second one is the Community level, which is a subset of resources; the third one is the GSML page level, which is a user-specific resource collection. The role of the Mapper is to convert the distributed heterogeneous resource to standard GSML descriptions. After this three-level conversion, the content of a GSML page can be a small resource collection and user-specific, which enable users make a custom resource view easily. In addition to above modules, we also develop a graphic user interface called browser to view the GSML page. We also develop a GSML Server containing different GSML pages and Communities. The GSML server also handles the request from browsers. We propose a protocol called Service Request Protocol to support the interaction between GSML Servers and Browsers. 8. Conclusions and Future Works In this paper we propose a user-centered lightweight architecture accordant to VEGA principles. Our architecture maps different components of the to We are grateful to Hua Huang, Haiying Tang, Jun Zhang, Fangpeng Dong, Yanbin Li and Baiming Feng for their helpful works in Vega Personal project. References [1] I. Foster, C. Kesselman, Globus: A Metacomputing Infrastructure Toolkit, International Journal of Supercomputer Applications, 11(2), 1997, pp [2] I. Foster, C. Kesselman, The Globus Project: A Status Report, Proc. IPPS/SPDP '98 Heterogeneous Computing Workshop, 1998, pp [3] I. Foster, C. Kesselman (Eds), The : Blueprint for a New Computing Infrastructure, Morgan Kaufmann Publishers, [4] I. Foster, C. Kesselman, S. Tuecke, The Anatomy of the : Enabling Scalable Virtual Organizations, Int l Journal of Supercomputer Applications, 15(3), 2001, pp [5] I. Foster, C. Kesselman, J. Nick, S. Tuecke, Services for Distributed Systems Integration, IEEE Computer, 35 (6), 2002, pp [6] A. S. Grimshaw, W. A. Wulf, J. C. French, A. C. Weaver, P. F. Reynolds, Legion: The next logical step toward a nationwide virtual computer, Technical Report CS-94-21, University of Virginia, [7] Intel Corporation, PCI Local Bus Specification, [8] H. Kreger, IBM Software Group, Web Services Conceptual Architecture (WSCA 1.0). [9] W. Li, Z. Xu, F. Dong, J. Zhang, Discovery Based on a Routing-Transferring Model, The 3rd International Workshop on Computing, November, [10] L. Smarr, C. Catlett, Metacomputing, Communication of the ACM, 35, 1992, pp [11] N.H. Sun, T.Y. Liu, Enabling Clusters, Journal of Computer Research and Development, 39 (8), 2002, pp [12] Z. Xu, W. Li, The Research on Vega Architecture, Journal of Computer Research and Development, 39 (8), 2002, pp [13] Z. Xu, W. Li, H. Fu, Z. Zeng, The Vega and - Based Education, The 1st International Conference on Web- Based Learning, HK, China, 2002, pp [14] Z. Xu, N. Sun, D. Meng, W. Li, Cluster and Superservers: The Dawning Experience in China, Proc. of the 3rd IEEE Int l Conf. on Cluster Computing, 2001, pp