REsources linkage for E-scIence - RENKEI -

Transcription

1 REsources linkage for E-scIence - - hlp:// sciren.org/ REsources linkage for E- science () is a research and development project for new middleware technologies to enable e- science communi?es. "" is Japanese word meaning "federa?on". The goal of the project is to develop middleware to federate/share resources (computers, storages, databases and applica?ons) distributed among mul?ple organiza?ons, such as research laboratories, na?onal computer centers and interna?onal grids. The research ac?vi?es consist of five research themes presented in the figure below. This project is supported by the Ministry of Educa?on, Culture, Sports, Science and Technology in Japan. computa?on intensive applica?ons users database users applica?on developers (1) compu?ng resource federa?on, and applica?on hos?ng (3) federa?on, and federa?on with ID management systems (4) API for mul?ple grid middleware laboratory (LLS) grid middleware (e.g. NAREGI) computer centers (NIS) interna0onal inter- opera0on grid middleware (5) evalua?on and collabora?on with users computer users (2) file sharing, and file catalogue federa?on computa?on/data intensive applica?on users #1143 #1143 #3029 #1143 #3213 #3013 #1143 #1127

2 -Workflow Tool(WFT) WFT(WorkFlow Tool) provides end users with a single seamless interface to manage workflow jobs. They use WFT to discover appropriate resources, to specify workflow jobs consisting of applications deployed by AHS on various grid resources, to launch jobs, to monitor running jobs, to terminate jobs, to retrieve files to client machine and to transfer files between them with the GUI interface. System features Ø Workflow engine controls over a single workflow execu?on by submi_ng jobs to compu?ng resources on LLS (Laboratory Level System) and NIS (Na?onal Infrastructure System, Super computer centers). Ø Limited communica?on protocols(hlps, myproxy, gfarm) between LLS and NIS resources for communica?ng through firewalls Ø Security Infrastructure (GSI) used as an authen?ca?on mechanism for submi_ng jobs to both resources. User Management Server (UMS) for archiving user cer?ficates and MyProxy Server in NIS are shared between LLS and NIS. Ø File sharing and stage- in/out of files between LLS and NIS by the file system (Gfarm). pre sim 1 sim. 2 sim. 3 post WFT AHS Workflow LLS Portal WF engine Cloud Client SAM WS- GRAM Cloud Nodes LLS Resources Deploy and boot pre post sim 1 SS WF engine Portal (WFT) Super Scheduler NIS Resources(NAREGI) VM VM VOMS,UMS MyProxy sim. 2 sim. 3 IS Gfarm meta server, file server

3 Interoperation compu?ng enables resource federa?on among organiza?ons. Scien?sts and engineers are ge_ng many benefits from resource sharing among organiza?ons. However, barriers s?ll exist. Because different grid middleware are implemented using different specifica?ons, resources cannot be shared among them. Many grid projects including are making effort to achieve grid interopera?on. User in Another Information Service in another Job Submission Client / Service in Another HPCBP Client HPCBP Service Computing Resource in Another Another Infrastructure -IS Schema Translator -SS SS Services HPCBP-SC HPCBP Client HPCBP-Service HPCBP Server LRMS SC NAREGI VM IS NAS / CDAS NIS Computing Resource Portal NAREGI user -IS LRPS Modification/Addition from NAREGI Infrastructure (NIS) Fig. 1 Interopera?on Architecture Interna?onal standards defined by the OGF (Open Forum) are used for grid interopera?on. " HPCBP: HPC Basic Profile " PGI: Produc?on Infrastructure " BES : Basic Execu?on Service " JSDL: Job Submission Descrip?on Language " GLUE: Informa?on schema etc. Five grid projects in the world tried and succeeded an interopera?on test. Web Server Oxford e-research Centre Pre-Processing: Generate input data (Perl) Main-Processing: Data Staging Job Submission to s Post-Processing-1: Choice Best Result (Perl) Post-Processing-2: Upload results to Web : HPCBP(BES, JSDL, etc.) : FTP or FTP UNICORE (DEISA) SAM (UK-NGS/OMII-UK) ARC (Nordu) Genesis II (U of Virginia) (NII) Local System Environments Internet Fig. 2 A Interopera?on Test The client can submit jobs to resources on different grids in a single workflow. Fig. 3 Using Mul?ple s in a Workflow

4 Applica0on Hos0ng Service (AHS) AHS provides a repository of applica?on informa?on to the research community created by grid virtual organiza?on (VO). Community members can share the applica?ons and their execu?on environment on hierarchical grid environments as shown in the right Figure. In a research community, applica?on users need to know only the applica?on specifica?ons, such as the applica?on name or keywords for selec?ng the applica?ons. Architecture A Univ. Virtual Organizations Application Developers Groups Application Users Joint Research Community AHS basic architecture is constructed as a set of Web/ service infrastructure on top of WSRF (Web Services Resource Framework) specifica?ons. WSRF is a messaging model and provides the ability to model stateful resources in a framework of Web services. AHS operates coopera?vely with the workflow tool and access to SS (Super Scheduler) for the NIS or SAM for the LLS as shown in the bolom Figure. A Lab. B Lab. B Univ. C Lab. C Univ. DUniv. D Lab. NIS LLS

5 U0lizing Cloud and for Research and Educa0on NII carries out two projects, edubase Cloud and, to federate cloud and grid infrastructures for research and education. The edubase Cloud offers platforms for educational programs and daily computing resources to educators and researchers. The middleware enable users to run their applications utilizing resources both in a cloud and a grid, e.g. running pre/post es in a cloud (Iaas) and large-scale simulation in a grid. ü plahorms for HPC applica?ons grid cloud service ü plahorms for educa?onal programs service ü plahorms for daily compu?ng ü a gateway to supercompu?ng services educators researchers edubase Cloud prj_educloud.html