Research on the Key Technologies of Geospatial Information Grid Service Workflow System

Transcription

1 Research on the Key Technologies of Geospatial Information Grid Service Workflow System Lin Wan *, Zhong Xie, Liang Wu Faculty of Information Engineering China University of Geosciences Wuhan, China * Corresponding author: wanyixue@gmail.com Jiayuan Lin Institute of Mountain Hazards and Environment Chinese Academy of Sciences Chengdu, China Abstract This paper discussed the key technologies for building a Geospatial Information Grid service workflow and describes the design and implementation of a Grid GIS service integration and hybrid invocation prototype framework: the MapGIS Grid Workflow Framework (MGWF). First, we discussed the WSRF based GIS function encapsulation details for legacy GIS system. Then we researched on the Grid GIS services dynamic monitoring and discovery mechanism and did some modifications to the Globus Toolkit 4 MDS (The Monitoring and Discovery System) tool to make it fit to our workflow framework better. The service integration and hybrid invocation mechanism in our prototype framework was described in the next part. Finally, we introduced the three main components of MGWF: a simple but visualized, easy-to-use Grid service workflow modeling IDE, a distributed process executor cluster and a pragmatic process template library. And we also give a Grid GIS service workflow examples to demonstrate the process of GIS workflow s composition, generation and execution monitoring. Keywords- Geospatial Information Grid; Grid Service; Workflow; WSRF; Grid GIS; Globus Toolkit 4 I. INTRODUCTION The spatial information sharing and integration of legacy GIS business systems based on Grid service is a core component of the spatial information service chain in Geospatial Information Grid [1]. With the sharing and integration platform, we can compose multiple spatial and nonspatial services into one GIS service chain. In the service chain, common GIS services and Grid GIS services can invoke each other easily. As the important technical support of Geospatial Information Grid, the spatial Grid service workflow management system can provide versatile facilities for users to utilize many kinds of spatial information resources in Grid environment [2]. Through service capabilities sharing, task collaboration and parallelizing mechanism, the workflow system can achieve dynamic cluster organization and load balance ability flexibly, and provide a highly abstract unified view for user. It also can help Grid GIS system to overcome many problems in traditional GIS system. WSRF (Web Services Resource Framework) is a de facto industrial standard of Grid service [3]. Based on WSRF services, we can effectively achieve the integration of multiple heterogeneous legacy GIS systems and build a robust GIS service chain in Grid environment easily [4]. There are several Grid service reference implementations, such as the Globus Toolkit 4 WSRF Service [5] and the WSRF.NET Service [6], which have been widely used in many large Geospatial Information Grid applications. However, the bottom implementation mechanism of these reference implementation models are different, the combination and interaction of GIS services based on these different Grid service implementation is very difficult, which has brought difficulties to construct the GIS service chain in Geospatial Information Grid. In addition, it is necessary to implement the hybrid invocation mechanism for Grid services and some legacy GIS services. This paper mainly discussed these critical technologies of Geospatial Information Grid service workflow system. We designed and implemented a Grid GIS services integration and invocation prototype system-the MapGIS Grid Workflow Framework (MGWF). It provides the basic function of Grid GIS services workflow composition, execution and monitoring. As an easyto-use framework, it can integrate different kind of Grid GIS services into a concentrated workflow management platform. Just by a simple mouse drag-and-drop style operation, we can easily create workflow processes and make hybrid invocations among the heterogeneous WSRF services and ordinary Web Services. II. KEY TECHNOLOGIES OF GEOSPATIAL INFORMATION GRID SERVICE WORKFLOW SYSTEM The key technologies for implementing an Geospatial Information Grid service workflow system mainly include: the effective encapsulation mechanism of GIS services on legacy GIS node, which should make slight changes to the legacy system at low cost in the process of building a Grid workflow system; the Grid GIS service dynamic monitoring and discovery mechanism, it is very helpful to quickly find dynamic changes of spatial information resource and to make fast updates during the GIS workflow execution process; different kinds of Grid service combination and interinvocation methods in Grid GIS system require a tool to integrate multiple kinds of GIS services and enable the hybrid invocation among these services. This work was supported by the National High Technology Development 863 Program of China (Grant No. 2007AA120503)

2 A. Encapsulation Mechanism of Grid GIS Service Nowadays, more and more legacy GIS business system built on different Web GIS vendors' products. These GIS platforms, which provide the basic function of deploying geographic information service on the Internet, chiefly have two types of GIS service: one is platform-dependent GIS service, another is OGC-compliant GIS service [7]. The latter implements a series of standardized OGC service interfaces: Web Map Service (WMS) [8], Web Feature Service (WFS) [9], Web Coverage Service (WCS) [10] and Web Processing Service (WPS) [11]. However, whether the platform-dependent GIS services of legacy system or the OGC-compliant GIS services, the implementation technologies are both based on traditional Web Service standards, which are unable to meet the requirements for building the Grid GIS workflow system: the stateless GIS service can not keep the last invocation results for the next invocation. Moreover, the service efficiency is very lower, for example, a simple WMS service request URI string which contains too long parameters (display range, map size and layer ID...), will repeat sending these parameters as long as users make a request. This will increase the server load, lower the efficiency of GIS service or even cause denial-of-service condition; different vendor-dependent GIS services can not implement the interoperability since their diverse realization mechanisms [12]. WSRF based services provide stateful Web Service solutions by introducing the resource state. As an accepted standard, it is extensively used in many large Grid project. Therefore, we adopt the Globus Toolkit 4 (i.e. GT 4, which includes a complete WSRF implementation) as our Geospatial Information Grid construction tools. On condition that this shall not affect the GIS functions of legacy system, we use WSRF to encapsulate the GIS functions which are deployed as GIS services. Two steps are involved in the process as follows: Figure 1. The grid encapsulation of heterogeneous legacy gis node 1) Create OGC services: each Grid node must expose the standard OGC-compliant service interfaces, such as WMS, WFS, and WCS etc. in order to meet the basic service interoperability for each other. We use ZondyCyber MapGIS IMS 7.3 [13] as our GIS platform; through the encapsulation of bottom spatial data access interfaces of GIS server, basic GIS functions were published as OGC serivces on network. 2) WSRF services encapsulation: every OGC service interfaces must add the resource states. We can separate/insert some necessary resources from/to the original service. The selection of resource state elements, which vary with the type of service, at least depends on two principles: Figure 2. Grid based ogc service design model a) Service-dependent characteristics: for example, we can extract the parameters-layers, Styles, which change unfrequently during a chain of user map request actions, from the WMS Service and put them into a separate resource home facility. These resource factories can dynamic updating these parameters on demand. So the invocation efficiency can be effectively improved by reducing unnecessary handling costs. b) Workflow-related characteristics: in our prototype system, we designed our Grid GIS service using Singleton design pattern- each service has only one resource, And there are several resource properties per resource. Considering the need of service workflow system, each resource has at least four properties: the response time of last invocation(ms), the invocation count of the service, the distinguish name of users proxy credential and the client IP address. Through the WSRF based encapsulation of OGC service, clients can easily access the exposed service endpoint interface, simply instantiate the interface, then invoke Grid GIS services just like call local Java methods. All spatial data which underlie the service have guaranteed higher safety as well as better easy-use quality in dynamic services process composition. B. Dynamic monitoring and discovery mechanism In the process of GIS service workflow execution, service state has ever been changing. Therefore, we need fast find available services and invoke the proper service. The key to solving this problem is implementing the dynamic monitoring and integration of Grid service resources. As is stated above, we build our Grid GIS service by GT 4 WSRF technology. In that environment, service registration, discovery, query and locating are all realized by using Web MDS (Monitoring & Discovery Service) [14]. We can consider MDS as a dynamic UDDI registration center [15]. However, it

3 has several shortcomings which can not meet the requirements of our workflow system: for its' tree structure, lower nodes information aggregated into higher centralized nodes hierarchically, so it is easy to lead to the single point of failure; it is difficult to improve the performance since every service node will periodically send all service information by means of XML file data, the updating rate is very slow and frequent updating can even cause severe network traffic congestion. Therefore, we made some corresponding modifications to the GT 4 MDS component for solving these problems as shown in figure 3. Figure 3. The overview of dynamic MDS cluster system 1) Using MDS backup nodes list: every time when we create the dynamic Virtual Organization (VO) [16], one XML list which contains the specific MDS backup node addresses will be sent to Local Information Manager (LIM) Service of each member node. There may be several MDS nodes at the same level for resource nodes to choose in our system. If one primary MDS node is unavailable to access, the LIM service will automatically find the second backup nodes from the list, then the third, etc. Moreover, MDS nodes with the same rank adopt WSRF based message queues and information notifications for synchronization. This effectively prevents the single point of failure. 2) Higher nodes active polling scheme: the updating method of our system has been designed to better match the fast change of Grid node services. After the first time VO member services information update is complete, we consider that the VO enters a temporal stable state. Then we switch the higher MDS nodes into an active polling state, these nodes will periodically send heart beat polling signals to lower nodes and wait for the corresponding responses in order to check whether nodes are still alive or not. Provided one member node does not reply to the polling within the given time, the MDS node will add one to the failure counter. If the failure counter's value reached the Freezing threshold- 5 times, service information item about that node will be keep in the MDS node but be recognized temp-frozen. Further more, when the failure counter's value accumulated to 10 times, the related entries will be removed from the MDS node permanently. 3) Member nodes incremental updating scheme: to reduce the cost of change treatment so as to accelerate the client updating process, the change updating action will send less updating data than the whole service information list. No updating messages will be signaled from the member node side if there are no any changes within the fixed polling time. As long as the service state changes, the LIM service will trigger an incremental updating service (IUS) to send the change details quickly to the specific MDS aggregator nodes. The amount of the updating data depends on the service changes and will be very small than the traditional all-sending method. At the same time, the efficiency of the MDS system will be highly improved. In our prototype system, the dynamic discovery and integration facility for Geospatial Information Grid services is mainly based on the above modified MDS system. We use additional WSRF services, for example, IUS service, to extend and wrap the core GT 4 MDS service. And we also develop several assistant WSRF services like LIM service to help improve the performance of the whole updating process. The implementation provides the pragmatic functions for Grid node service information registration and dynamic query on demand. It mainly includes: Service information dynamic registration. The LIM service of each node need register in the resource information center node, and must provide the available service metadata in the node for completing the registration process. On the other hand, the resource information center node deal with applications from the member nodes, maintain a dynamic service information list object. Service metadata information dynamic query. Client can obtain the current Grid node list as well as available service metadata since the service endpoint expose the query interface to supply dynamic query action. The interface also can accept flexible and diversified query forms to inquire the service information list on certain node, the list of nodes which have published one specific service or even more details accurate to layer properties of nodes related to one service, according to the combination of multiple query constraints. C. Service Integration and Invocation tool In actual Geospatial Information Grid application chains, there are many heterogeneous GIS services based on GT 4 WSRF, WSRF.NET or common Web Services, and we also need some assistant Web Services. So to supply the integration and hybrid invocation of different types of service in the workflow is not only important but also necessary. In our system, we have considered three different service types: Java based GT 4 WSRF, Microsoft.Net based WSRF.NET and W3C standard Web Service. In order to deal with the service state resources of Grid service, we designed a dynamic integration and invocation tool in our prototype framework. It is mainly made up of two parts described as figure 4.

4 1) The process modeling tool. In order to improve the throughput of workflow framework and implement the dynamic load equilibrium method, the core component of the workflow engine is divided into two parts. The first partprocess parser, monitor and controller are integrated into the modeling tool which is extended from the MapGIS workflow engine [19]. This model generator can generate the process description file by simply drag-and-drop mouse operation, the straightforward "Wizard" method or directly choosing from the process template library. Figure 4. Components of service integration and invocation tool 1) Service Integrator. The kernel of service aggregator is Service Proxy Generator. Proxy Generator provide the basic function of service proxy generating, it can identify different Grid service implementation type, use the WSDL files [17] to dynamic create the GT 4 WSRF, WSRF.Net or common Web Service proxy classes. These generated classes will be put into a temporary storage area, and their metadata can be collected into a Proxy type object by the integrator. The Hybrid Invoker tool (mentioned below) will use this Proxy object to invoke the corresponding Service. 2) Hybrid Invoker. This tool realized the unified invocation funtion for WSRF based service and common Web Service. Every service to be invoked in the workflow will store the information required for invoking into a service object we called Invoker. As an indispensable element for WSRF service invocation, the endpoint reference resource address must be recorded during the service proxy classes generating process. The Invoker object, which stores the WSRF service endpoint reference object [18] in addition to the basic service information, can be managed by an endpoint reference (EPR) manager service. The EPR manager service can serialize/deserialize the EPR for the WSRF service execution. The function of Hybrid Invoker is exposed as three Web Service interfaces: invokeinit, invoke and getsvcoperinfo. At the first time the service is called, the workflow engine can use invokeinit method do some initialization work, such as setting service context parameters, creating necessary resources. The invoke method is called during the execution process, it can accept a Parameter type object which is passed to Hybrid Invoker tool, then return the service invocation result object after finish current service invocation. The getsvcoperinfo method will present a manifest which describes the web invoke interfaces for a service, the workflow engine can choose any one of these web methods to execute. III. THE MAPGIS GRID SERVICE WORKFLOW FRAMEWORK Based on the above key technologies for building a flexible Grid GIS service workflow system, we designed a distributed Geospatial Information Grid service workflow prototype system - the MapGIS Workflow Framework. As shown in Figure 5, the framework implements a loose coupling architecture model, it mainly consists of three components: Figure 5. The architecture of the mapgis workflow framework When we finished setting the required service parameters for each member node, the framework will invoke the process parser to verify the process elements and the execution sequences automatically. If the process instance passed the check, the execution controller will start to instruct the remote executors to invoke the services in the order predefined by the process. The execution monitor can interact with our MDS componet so that the dynamic change can be feed back to the controller in time. According to the feedback information, the controller can adjust the execution behavior at an opportune time. Figure 6. The integrated modeling environment 2) The process executor. As the second part of the workflow engine, we separate the task executor from the process engine. There may have several service executors distributed in different Grid nodes. These executors, which consist of the service integrator and invoker components, are implemented as Web Services in order to be deployed

5 crossing-platform. They can call different kind of services and collect the invocation results. The MDS nodes can interact with the executor nodes to get the work load information of every node. The information can be accessed by the execution controller, and then the controller decided which executor node is best for current task execution. The different executors can communicate with each other to synchronize and update their state using JXTA [20] Technology. 3) The process template library. This component is used to store some user-defined GIS business process description file templates and also provide more convenience for the users. Users can directly use these templates to construction their workflow process or customize these templates based on their own requirements. Finally, as shown in Figure 7, we present an example to demonstrate a Geospatial Information Grid service workflow modeling process. In the MapGIS Grid workflow framework modeling IDE, we quickly set up a spatial information service workflow application. The process, which is made up of six service nodes, including four WSRF service nodes and two common Web Services, is used to generate a global map view in which data is coming from four different GIS nodes. Figure 7. A simple workflow execution example IV. CONCLUSION AND FUTURE WORK Based on the research of the Grid GIS service encapsulation, the Grid service dynamic monitoring and updating mechanism and the heterogeneous services hybrid invocation implementation technologies, we designed and implemented a Geospatial Information Grid service workflow prototype framework, which provides process modeling, dynamic service invocation between different kinds of GIS function services and performs well in terms of load balancing and high throughput. The dynamic service binding mechanism deserves our more deep research. Moreover, the implementation of our prototype system doesn t concern the interaction between common GIS services and secure Grid services which may include the GSI related information. Therefore, those will become a study emphasis in our further research. REFERENCES [1] Z. Xie, M. M. Song, and X. G. Luo, Resource and Information Sharing Mechanism Based on Geospatial Information Grid, Grid and Cooperative Computing, GCC '08. Seventh International Conference on, vol., no., pp , Oct [2] J. T. Zhang et al., Integrating Data Grid and Web Services for E- Science Applications: A Case Study of Exploring Species Distributions, e-science and Grid Computing, e-science '06. Second IEEE International Conference on, vol., no., pp.31-31, Dec [3] Web Services Resource Framework (WSRF) Primer, [4] Z. M. Gui, Z. Huang, X. T. Xie, D. X. Tian, L. X. Liu, X. N. Wang, Building Grid GIS Service with WSRF, Advanced Geographic Information Systems, Applications, and Services (GEOPROCESSING), 2010 Second International Conference on, vol., no., pp.52-55, Feb [5] I. Foster and C. Kesselman, Globus: A metacomputing infrastructure toolkit, Intl. Journal of Supercomputing Applications, vol. 2, pp , November [6] M. Humphrey and G. Wasson, Architectural Foundations of WSRF.NET, International Journal of Web Services Research. 2(2), pp , April-June [7] OGC services, [8] Web Map Service, [9] Web Feature Service, [10] Web Coverage Service, [11] P. Schut, OpenGIS Web Processing Service, OpenGIS Standard, Version: 1.0.0, OGC r7, [12] Y. Sun, G. Q. Li, Interoperability Research of Heterogeneous GIS Based on Geospatial Information Grid, Computer Science and Software Engineering, 2008 International Conference on, vol.3, no., pp.41-44, Dec [13] MapGIS IMS, [14] J. M. Schopf et al., Monitoring and discovery in a web services framework: Functionality and performance of the globus toolkit s mds4, Technical report, Argonne National Laboratory, 2005 [15] Universal Description, Discovery and Integration, [16] I. Foster, C. Kesselman, and S. Tuecke, The anatomy of the Grid: Enabling scalable virtual organizations, International Journal of Supercomputing Applications, vol. 15, no. 3, pp , [17] WSDL V1.1 specification, [18] B. Sotomayor, The Globus Toolkit 4 Programmer s Tutorial, [19] MapGIS workflow, [20] JXTA project,