Deployment of e-science Infrastructure and Applications for PNC

Transcription

1 Deployment of e-science Infrastructure and Applications for PNC Eric Yen Academia Sinica, Taiwan 16 Aug

2 Outline The common keywords of PNC and e-science: Sharing and Collaboration How readiness is the e-science Infrastructure and Applications What are the Core Services Lessons Learned from WLCG/EGEE of Taiwan Metrics and Operation Summary 2

3 Motivation e-science and PNC share the same vision of sharing and collaboration how should PNC take advantage of Grid? How mature/readiness is the e-science infrastructure, and how to build up/migrate applications upon it? Key Issues Infrastructure: glite + OSG (why not Globus?) Identify Core Services Data Management Resource Discovery Security VO (Role-based rights management and collaboration) O&M Foster user communities, such as ECAI, PRDLA, etc. Identify the common requirements of each application domain Application Development Services How to measure the Success? 3

4 Deployment of Production Middleware Infrastructure glite + OSG Interoperation Data Management embedded data management subsystem in glite/osg SRM interface to integrate with other mass storage systems, such as SRB, Castor, and dcache, etc. Resource Discovery customized based on the domain specific workflow Application Specific Services Long-term Preservation Virtual Screening Geospatial data management and hazards mitigation Digital Collection Federation and other services 4

5 Enabling Grids for E-sciencE EGEE Infrastructures Production service Scaling up the infrastructure with resource centres around the globe Stable, well-supported infrastructure, running only well-tested and reliable middleware Pre-production service Run in parallel with the production service (restricted nr of sites) First deployment of new versions of the glite middleware Test-bed for applications and other external functionality T-Infrastructure (Training&Education) Complete suite of Grid elements and application (Testbed, CA, VO, monitoring, support, ) Everyone can register and use GILDA for training and testing 20 sites on 3 continents EGEE-II INFSO-RI EGEE - A Large-scale Production Grid Infrastructure 5

6 Enabling Grids for E-sciencE Geographically distributed responsibility for operations: There is no central operation Regional Operation Centers Responsible or resource centers in their region Tools are developed/hosted at different sites: GOC DB (RAL), SFT (CERN), GStat (Taipei), CIC Portal (Lyon) Grid operator on duty 6 teams working in weekly rotation CERN, IN2P3, INFN, UK/I, Ru,Taipei Crucial in improving site stability and management Expanding to all ROCs in EGEE-II Operations coordination Weekly operations meetings Regular ROC managers meetings Series of EGEE Operations Workshops Nov 04, May 05, Sep 05, June 06 EGEE Operations Process Procedures described in Operations Manual Introducing new sites Site downtime scheduling Suspending a site Escalation procedures; etc. EGEE-II INFSO-RI EGEE - A Large-scale Production Grid Infrastructure 6

7 Enabling Grids for E-sciencE Production Grid Middleware EGEE-II INFSO-RI EGEE - A Large-scale Production Grid Infrastructure 7

8 Enabling Grids for E-sciencE Production Grid Middleware Key factors in EGEE Grid Middleware Development: EGEE-II INFSO-RI EGEE - A Large-scale Production Grid Infrastructure 7

9 Enabling Grids for E-sciencE Production Grid Middleware Key factors in EGEE Grid Middleware Development: 1. Strict software process Use industry standard software engineering methods Software configuration management, version control, defect tracking, automatic build system, EGEE-II INFSO-RI EGEE - A Large-scale Production Grid Infrastructure 7

10 Enabling Grids for E-sciencE Production Grid Middleware Key factors in EGEE Grid Middleware Development: 1. Strict software process Use industry standard software engineering methods Software configuration management, version control, defect tracking, automatic build system, 2. Conservative approach in what software to use Avoid cutting-edge software Deployment on over 100 sites cannot assume a homogenous environment middleware needs to work with many underlying software flavors Avoid evolving standards Evolving standards change quickly (and sometime significantly cf. OGSI vs. WSRF) impossible to keep pace on > 100 sites EGEE-II INFSO-RI EGEE - A Large-scale Production Grid Infrastructure 7

11 Enabling Grids for E-sciencE Production Grid Middleware Key factors in EGEE Grid Middleware Development: 1. Strict software process Use industry standard software engineering methods Software configuration management, version control, defect tracking, automatic build system, 2. Conservative approach in what software to use Avoid cutting-edge software Deployment on over 100 sites cannot assume a homogenous environment middleware needs to work with many underlying software flavors Avoid evolving standards Evolving standards change quickly (and sometime significantly cf. OGSI vs. WSRF) impossible to keep pace on > 100 sites Long (and tedious) path from prototypes to production EGEE-II INFSO-RI EGEE - A Large-scale Production Grid Infrastructure 7

12 Enabling Grids for E-sciencE glite Grid Middleware Services Overview paper EGEE-II INFSO-RI EGEE - A Large-scale Production Grid Infrastructure 8

13 Directives Enabling Grids for E-sciencE glite Software Process JRA1 Development Software Error Fixing Serious problem SA3 Integration SA3 Testing & Certification Deployment Packages SA1 Pre- Production Problem Fail Testbed Deployment SA1 Production Infrastructure Integration Tests Pass Fail Pre-Production Deployment Functional Tests Pass Fail Release Installation Guide, Release Notes, etc Pass Scalability Tests EGEE-II INFSO-RI EGEE - A Large-scale Production Grid Infrastructure 9

14 OSG Middleware Layering Infrastructure Applications LIGO Data Grid CMS Services & Framework ATLAS Services & Framework CDF, D0 SamGrid & Framework OSG Release Cache: VDT + Configuration, Validation, VO management Virtual Data Toolkit (VDT) Common Services NMI + VOMS, CEMon (common EGEE components), MonaLisa, Clarens, AuthZ NSF Middleware Initiative (NMI): Condor, Globus, Myproxy

15 OSG Middleware Pipeline Domain science requirements. Condor, Globus, EGEE etc OSG stakeholders and middleware developer (joint) projects. Test on VO specific grid Integrate into VDT Release. Deploy on OSG integration grid Test Interoperability With EGEE and TeraGrid Provision in OSG release & deploy to OSG production.

16 GH5780""I 29 July CERN power-off 08:00-24:00 - SAM failed through 31 July. Other sites that were active before and after power-off assumed to be fully available available throughout power-off and SAM failure. Availability on 29 July before 08:00 adjusted pro-rata. Data from SAM monitoring. Site availability as agreed in WLCG MB on 07mar06 and 04apr06. Service considered unavailable from time of first failure to time of next success Scheduled interruptions are considered as unavailable. %*(&%'(8O%558B4$(BP!"# $%&'($ ))# B4$(8%*(&%'(8T959H&8T9U41'V88W8+"#89:8$%&'($8888!8+"#89:8$%&'($888!8$%&'($ 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% % 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% % 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% =DEF@JEXM8 %*(&%'( +"# NYZ@<=>08 %*(&%'(,-# KF0J.@==8 %*(&%'( )!# 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% % 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% % 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% KFNF@?/8 %*(&%'(./# E3<@<=>08 %*(&%'(!.# A3E3@R3?EK[8 %*(&%'( )-# 100% 100% 100% 90% 90% 90% 80% 80% 80% 70% 70% 70% 60% 60% 60% 50% 40% 30% 20% 10% 50% 40% 30% 20% 10% 50% 40% 30% 20% 10% SAM tests fail due to dcache function failure that does not affect CMS jobs. The problem is understood and is being worked on 0% % % ?EKQRN@<=>08 %*(&%'( )"#?%4S%1@<=>08 %*(&%'( +)# QA=RA@NF3<@;=/8 %*(&%'( 0"# 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% % 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% % 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% JK= %*(&%'( )!# LF< %*(&%'( 12% FM>N %*(&%'( 12% CERN + Tier-1 Availability 2 Printed on 10/08/2006 at 12:41 12

17 WLCG Availability of CERN & T1s Target is to have >=8 sites reaching 88% availability by end of Sep After that, full WLCG services should be in operation. 13

18 WLCG Availability of CERN & T1s Target is to have >=8 sites reaching 88% availability by end of Sep After that, full WLCG services should be in operation. CERN- PROD FZK- LCG2 IN2P3- CC INFN-T1 RAL- LCG2 SARA- MATRIX TRIUMF- LCG2 Taiwan- LCG2 USCMS- FNAL- WC1 PIC average - all Tier-1s May'06 89% 85% 83% 89% 68% 58% 77% 87% 68% 61% 77% June'06 92% 15% 89% 62% 76% 49% 88% 75% 64% 88% 70% July'06 90% 54% 87% 31% 73% 84% 80% 98% 20% 87% 70% average 90% 53% 86% 61% 72% 64% 81% 87% 50% 78% 72% 13

19 EGEE OSG inter-operability Agree on a common Virtual Organization Management System (VOMS) Active Joint Security groups: leading to common policies and procedures. Condor-G interfaces to multiple remote job execution services (GRAM, Condor-C). File Transfers using GridFTP. SRM V1.1 for managed storage access. SRM V2.1 in test. Publish OSG BDII to shared BDII for Resource Brokers to route jobs across the two grids. Automate ticket routing between GOCs.

20 OSG/LCG resource integration Mature tech help integrating resources GCB introduced to help integrating with OSG computing resources CDF/OSG users can submit jobs by gliding-in into GCB box Access ASGC T1 computing resources from twgrid VO Customized UI to help accessing back-end storage resources Help local users not ready for grid HEP users access T1 resources

21 Data Management 16

22 Requirements: User s viewpoint Find Data Registries & Human communication Understand data Metadata description, Standard / familiar formats & representations, Standard value systems & ontologies Data Access Find how to interact with data resource Obtain permission (authority) Make connection Make selection Move Data In bulk or streamed (in increments) Ischia, Italy July 2006

23 Requirements: User s viewpoint 2 Transform Data To format, organisation & representation required for computation or integration Combine data Standard DB operations + operations relevant to the application model Present results To humans: data movement + transform for viewing To application code: data movement + transform to the required format To standard analysis tools, e.g. R To standard visualisation tools, e.g. Spotfire Ischia, Italy July 2006

24 Metadata Services for Grid Efficient management of millions of files spread over several sites User and applications need an efficient mechanism to find the files of interest discover and query information about their contents Could be provided by associating descriptive attributes (metadata) to files exposing this information in catalogues, accessible and searchable by user and client application Quality of Services Hide network latency Disconnected computing: local replicas for offline access DBMS independent: Grid environment is heterogeneous Reliability: no single point of failure Scalability: support hundreds/thousands of concurrent users 19

25 Grid Metadata Services Grid Metadata Service (GMS) is not just for data management, but also for job management (e.g., WLCG) and resource management in general GMS is not exclusive to the (file) catalog service. File catalog keeps track in which storage element a particular object is located (but provides no way to query on their contents) GMS provides the repository of detailed information for each object, and makes queries on them Advanced Features (Research Topics) Split big files among several SEs (different chunks stored in different SEs) data security enforcement upload/download bandwidth increased Automatic extraction and population of metadata 20

26 Integrated service for Data & Metadata Ischia, Italy July 2006

27 Enabling Grids for E-sciencE Introduction to VOMS VOMS Features Single login using (proxy-init) only at the beginning of a session VOMS extensions are attached to user proxy Expiration time The authorization information is only valid for a limited period of time as the proxy certificate itself Multiple VO User may log-in into multiple VOs and create an aggregate proxy certificate, which enables him/her to access resources in any one of them Support for Group and Roles Group membership is automatically inserted when requesting voms proxy Role has to be requested explicitly Backward compatibility The extra VO related information is in the user s proxy certificate User s proxy certificate can be still used with non VOMS-aware service Security All client-server communications are secured and authenticated INFSO-RI Catania, NA4 Generic Applications Meeting, January 10th,

28 VOMS Web interface Enabling Grids for E-sciencE VO user can Query membership details Register himself in the VO You will need a valid certificate Track his requests VO manager can Handle request from users Administer the VO INFSO-RI Catania, NA4 Generic Applications Meeting, January 10th,

29 Enabling Grids for E-sciencE Groups The number of users of a VO can be very high: E.g. the experiment ATLAS has 2000 member Make VO manageable by organizing users in groups: Examples: VO GILDA Group Catania INFN o Group Barbera University Group Padua VO GILDA /GILDA/TUTORS /GILDA/STUDENT can write to normal storage only write to volatile space Groups can have a hierarchical structure, undefinitely deep INFSO-RI Catania, NA4 Generic Applications Meeting, January 10th,

30 Enabling Grids for E-sciencE Roles Roles are specific roles a user has and that distinguishes him from others in his group: Software manager VO-Administrator Difference between roles and groups: Roles have no hierarchical structure there is no sub-role Roles are not used in normal operation They are not added to the proxy by default when running voms-proxy-init But they can be added to the proxy for special purposes when running vomsproxy-init Example: User Emidio has the following membership VO=gilda, Group=tutors, Role=SoftwareManager During normal operation the role is not taken into account, e.g. Emidio can work as a normal user For special things he can obtain the role Software Manager INFSO-RI Catania, NA4 Generic Applications Meeting, January 10th,

31 Grid Security Fundamentals Key terms that are typically associated with security Authentication Authorisation Audit/accounting Integrity Fabric Management Confidentiality Privacy Trust All are important for Grids but some applications may have more emphasis on certain concepts than others ISSGC06, Ischia July 2006

32 Grid Security Infrastructure (GSI) Standard mechanism for interfacing Grids Supports X509 proxy certificates for Authentication Created with the grid-proxy-init command Proxy certificate stored in /tmp directory Establishes connections between services by Mutual Authentication Preliminary messages are exchanged and encrypted/decrypted Signatures and CAs are verified If this checks out then both parties know who they are talking to Uses an Access Control List called a grid-mapfile for Authorisation But still can use GGF SAML Callout to other AuthZ functions Stored in the /etc/grid-security/ directory ISSGC06, Ischia July 2006

33 Federated Trust Local authentication infrastructures are vital e.g. Campus student directories Support existing infrastructures (e.g. registration, human resources) Will normally have enrolled IN PERSON at the institution» With standard identity (birth certificate, exam results) Will be (reasonably) well known by local staff Also the Regional Operators for a CA Required decentralisation of credential verification due to travel/time restrictions National CA would be impossible without this Remote authentication information will always be out of date Don t want to have to learn lots of usernames/passwords The best entity to authenticate a person is their home institution/company Info will be up to date They will always know a person better than a remote site Remote site may not know if user is still valid or not ISSGC06, Ischia July 2006

34 Summary Security is a combination of technical implementation and sociological behaviour There can be no overall security policy for the Grid integrate existing site policies The establishment of identity on the Grid (authentication) is achieved through the use of PKI Certificates and Proxies ISSGC06, Ischia July 2006

35 Application Development 30

36 CMS Experiment - an exemplar community grid OS G EGEE CERN Taiwan Italy UK USA Germany France Purdue UNL Wisconsin Caltech UCSD Florida MIT Data & jobs moving locally, regionally & globally within CMS grid. Transparently across grid boundaries from campus to global.

37 Computing Models Cover Data model Output stages, formats, sizes, rates, distribution Analysis model Workflow, streams, (re)processing, data movement, interactivity Distributed deployment strategy Computing tier roles, data management & processing across the tiers Specifications for Capacity (processors, storage, network etc.) Capability (middleware and other services) 32

38 Common Needs for a New Community/Application Understand what the grid is about Try it out, get a feeling of how it works (grid user temporary access) Understand what could be the added value for their applications Discover/identify new kind of applications of value for their community Deployment of middleware on their machines Join an existing VO/infrastructure Experience with grid programming: look and learn from example code of applications that use the middleware Understand what the middleware does and what it does not do (know about current bugs and their workarounds in the middleware) Be informed about/get in contact with other applications Ask questions to other grid developers Set up their own infrastructure (identify the infrastructure requirements) Organize their own VO Understand project positions with respect to standards Have a plan about what will be available and when for adoption Provide relevant requirements/feedback in a coordinated way Be constantly informed about relevant events Be involved in the discussions about the hot topics (concertation events) Source: EGEE Application Migration Report, EGEE-DNA v doc 33

39 e-science Applications in Taiwan High Energy Physics: WLCG Bioinformatics: mpiblast-g2 Biomedicine: Distributing AutoDock tasks on the Grid using DIANE Digital Archive: Data Grid for Digital Archive Longterm preservation Atmospheric Science Geoscience: GeoGrid for data management and hazards mitigation Ecology Research and Monitoring: EcoGrid BioPortal Biodiversity: TaiBIF/GBIF e-science Application Framework Development 34

40 WLCG Services in Taiwan PHEDEX Transfer All! ASGC ASGC! All 35

41 EGEE Biomed DC II Large Scale Virtual Screening of Drug Design on the Grid Biomedical goal accelerating the discovery of novel potent inhibitors thru minimizing nonproductive trial-and-error approaches improving the efficiency of high throughput screening Grid goal aspect of massive throughput: reproducing a grid-enabled in silico process (exercised in DC I) with a shorter time of preparation aspect of interactive feedback: evaluating an alternative light-weight grid application framework (DIANE) Grid Resources: AuverGrid, BioinfoGrid, EGEE-II, Embrace, & TWGrid Problem Size: around 300 K compounds from ZINC database and a chemical combinatorial library, need ~ 137 CPU years in 4 weeks a world-wide infrastructure providing over than 5,000 CPUs

42 Distributed Data Management & Long-term Preservation of NDAP Long-term Preservation Automatic remote replication with 3 copies in different sites Effective migration based on metadata not just the digitized contents were archived, but als o their metadata, methods/procedures, standard format, and management information Separation of data representation and presentation Secure Access Reduce the total cost of management Data Management Framework could be shared for contentbased applications, e.g., federation etc. Sustainable Operation and Services 37

43 Architecture NDAP LTP Infrastructure Storage Resource Network: = 8 Storage Resource Centre + IDC + TWAREN/GSN Middleware replication, fail-over, uniform namespace, metadata catalog federation Security Infrastructure Operation and Management Interface to Applications Discovery Management 38

44 SRB-based Data Grid System Architecture for NDAP

45 Taiwan GeoGrid Applications Grid for Geoscience, Earth Science and Environmental Research and Applications Land Use and Natural Resources Plan/Management Hazards Mitigation Typhoon Earthquake Flood Coast line changes Landslide/Debris flow On-the-fly overlay of base maps and thematic maps, from distributed data sources (of variant resolution, types, and time) based on Grid Data Management WebGIS/Goole Earth based UI Integration of Applications with Grid

46 ASGC e-science Application Focus Grid Portal common data sharing environment as a one stop shop to search for and access data from difference administrative domains on heterogeneous system in a UNIFORM way Content Analysis & Management Metadata model, Content management framework, data federation Security Framework PKI based authentication, authorization, accounting and encryption Storage Resource Broker (collaborate with SDSC) pure distributed data management system, integration to Grid infrastructure, development of SRB-SRM Long-Term Preservation (LTP) & Data Curation Persistent archive, Mass storage technology, sustainable operation/business model

47 Common Framework for Application Development Data Management Web-Based Portal User Interface Job repository User/Grid Proxy Manager DataBank/Storage Element Virtual Queuing System Computing Grid Agent Grid Computing Element

48 How to Measure the Success Increasing the number of infrastructure users by increasing awareness Dissemination and outreach Training and education Increasing the number of applications by improving application support and middleware functionality Improved usability through high level grid middleware extensions Increasing the grid infrastructure Incubating related projects Ensuring interoperability between projects Protecting user investments Towards a sustainable grid infrastructure 43

49 A Service-Oriented Grid Grid middleware services Job-Submit Service Brokering Service Registry Service Advertise Notify Virtualized resources CPU Compute Resource Service Data Service Application Service Printer Service Hiro Kishimoto: Keynote GGF17 Ischia, Italy July

50 Unreliability Counter Measures Requires much R&D Continuous arms race as scale of Grids grow Ideal of a continuously available stable service Not achievable recognise that drops in response and local failures must be dealt with Design resilient architectures Design resilient algorithms Improve reliability of each component Distribute the responsibility For failure detection For recovery action Ischia, Italy July 2006

51 Summary e-science infrastructure is available and will be more robust and stable very soon by 2008 Have to take advantage of the core services provided by e-infrastructure and develop the best application model in Grid way based on domain consensus. Communication between application and e-infrastructure is indispensable. e- Science services need to be driven by application rather than the technology. Build up the production quality Grid Infrastructure for WLCG and e-science Applications in Taiwan, and happy to work with all PNC members to Have the e-infrastructure in place collaboratively Identify the core services required (collection of use cases) Foster grid applications DAGrid, Geospatial DataGrid, etc., and moving toward semantic content discovery Training and workshops 46