OGF25/EGEE User Forum SMOA Computing approach to HPC Basic Profile DRMAA + gsoap Mariusz Mamoński Krzysztof Kurowski {krzysztof.kurowski,mamonski}@man.poznan.pl
OpenDSP - Retrospection In the mid 2005 at Poznan Supercomputing and Networking Center OpenDSP project was launched aiming to: give consistent, remote, multi-user access to various DRM systems using standardized DRMAA interface add Authentication, Authorization and Accounting layers, which are out of scope of DRMAA specification use JSDL as the job description language provide high performance: C + gsoap + DRMAA C binding (which usually relies on internal DRMS API)
OpenDSP - Performance Tests %$! %$%! % #$&! '()*+,,-)./0+123003()0/,-04()0-/.32- :32-/=0> #$! #$%! # "$&! "$! "$%! 54-)678 9:; "!" #"" #!" '()*+,,-)./*<3-).0
OpenDSP - Summary more then 300 downloads all together (versions 0.1, 1.0 and 1.1) at sourceforge.net exploitation: InteliGRID EU project Tele-HVEM Seoul National University project tested with SGE, Condor, PBS/Torque, Condor and Xgrid DRMAA libraries
From OpenDSP to SMOA Computing migration from OpenDSP specific Web Service interface to OGSA-BES (HPC BES Basic Profile) refined architecture: privilege separation instead of setuid binaries JSDL used as the only format for internal job representation support for modules written in Python more extensions points added exposed Advance Reservation capability of underlying DRMS (LSF, SGE) file staging support as the part of job life cycle separate interface for lightweight, direct file staging (SOAP attachments)
SMOA Computing - Architecture SMOA Computing Authorization Modules Remote Interfaces BES Factory BES Managment ARES Factory Factory Attributes Modules JSDL Filter Modules ARDL Filter Modules session processes manager user session process Submision Module Reservation Module DRMAA Library ARES Library Message Level Authentication Modules SMOA Staging Transport Level Authentication Module Database Module WS-Notification Module
Authentication and Authorization Transport level security: none (plain http) SSL (with or without client authentication) GSI Message level security: WS-Security Username Token Profile WS-Security SAML Token Profile (Bearer) WS-Security X509 Token Profile Authorization modules: Globus-like mapfile Anonymous (fixed mapping/jsdl-hpc:username) Connection rate limit (per user/ip) Authorization Modules Message Level Authentication Modules Transport Level Authentication Module
Accounting SMOA Computing stores all vital information in database: submitted/post processed JSDL documents (GetActivityDocuments) local user account name (GetActivityDocuments) exit status (GetActivityStatuses) job resource usage information
JSDL to DRMAA mapping <jsdl:jobname> DRMAA_JOB_NAME <jsdl-hpcpa:executable> DRMAA_REMOTE_COMMAND <jsdl-hpcpa:argument>* DRMAA_V_ARGV <jsdl-hpcpa:environment>* DRMAA_V_ENV <jsdl-hpcpa:workingdirectory> DRMAA_WD <jsdl-hpcpa:input> DRMAA_INPUT_PATH <jsdl-hpcpa:output> DRMAA_OUTPUT_PATH <jsdl-hpcpa:error> DRMAA_ERROR_PATH other JSDL elements (e.g. TotalCpuCount) are mapped to DRMAA_NATIVE_SPECIFICATION by dedicated SMOA Computing JSDL Filter module JSDL Filter Modules
State model.,#/"--"$!"#$%#& '(##%#& )%#%*+"$ 12,&"3%# 0("("$ 67"/(2%#& 12,&"38(2 4"-$ 1(*5"#$"$ ),%-"$
AdvanceREServation Factory CreateReservation GetReservationStatus GetReservationDocument TerminateReservation GetActiveReservations ARES Factory Reservation Module ARES Library <ardl:reservationdefinition xmlns:ardl="http://schemas.smoa-project.com/comp/ 2009/01/ardl"> <ardl:timewindow> <ardl:starttime>2009-03-21t21:00:00+02:00</ardl:starttime> <ardl:endtime>2009-03-21t22:00:00+02:00</ardl:endtime> </ardl:timewindow> <ardl:resources> <ardl:reservedslotscount>1</ardl:reservedslotscount> </ardl:resources> </ardl:reservationdescription> </ardl:reservationdefinition>
SMOA Computing in BREIN DRM system: Sun Grid Engine 6.2 Advance Reservation mechanism being used for ensuring SLA fulfilling (e.g.: job start time) while submitting jobs to shared cluster additional staging module for BREIN specific data transfer protocol
SMOA Computing in QosCosGrid DRM system: LSF 7.0.3 Advance Reservation mechanism used for submitting parallel cross-cluster jobs Fig. 2. The QosCosGrid multi-tier architecture connecting different middleware services as well as parallel application development and deployment tools. Below we briefly describe two main programming and execution environments, ProActive and OpenMPI, which have been successfully integrated with OpenDSP and
SMOA Computing - Summary elastic - dynamically loadable module efficient - preliminary tests (with SSL authentication) shown ability to handle 30 job submit requests per second standard - DRMAA on the one side and BES on the other makes maintenance and integration easier
SMOA Project Other SMOA Project components: SMOA Core - common library SMOA Notification - open implementation of the WS-Notification 1.3 OASIS specification http://sourceforge.net/projects/smoa-project/
Thank you