EUROPEAN MIDDLEWARE INITIATIVE

EUROPEAN MIDDLEWARE INITIATIVE DJRA1.6.1 INTEGRATION WORK PLAN AND STATUS REPORT EC DELIVERABLE: D5.6.1 Document identifier: EMI-DJRA1.6.1-1277592- Integration_Work_Plan_v1.0.doc Activity: Lead Partner: Document status: Document link: JRA1 JUELICH Final http://cdsweb.cern.ch/record/1277592?ln=en Abstract: This deliverable contains the detailed work plan of the integration activities and objectives compliant with the overall EMI Technical Development Plan and aligned with the work-plans of other tasks within this work package. The plan is released early in the project life and updated every year including a status report on the achievements of the past 12 months compared to the planned objectives.

Copyright notice: Copyright (c) Members of the EMI Collaboration. 2010. See http://www.eu-emi.eu/about/partners/ for details on the copyright holders. EMI ( European Middleware Initiative ) is a project partially funded by the European Commission. For more information on the project, its partners and contributors please see http://www.eu-emi.eu. This document is released under the Open Access license. You are permitted to copy and distribute verbatim copies of this document containing this copyright notice, but modifying this document is not allowed. You are permitted to copy this document in whole or in part into other documents if you attach the following reference to the copied elements: "Copyright (C) 2010. Members of the EMI Collaboration. http://www.eu-emi.eu ". The information contained in this document represents the views of EMI as of the date they are published. EMI does not guarantee that any information contained herein is error-free, or up to date. EMI MAKES NO WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, BY PUBLISHING THIS DOCUMENT. INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 2 / 41

Delivery Slip Name Partner / Activity Date Signature From André Giesler JUELICH/JRA1 03/01/2011 Reviewed by Balazs Konya, Massimo Sgaravatto, Jozef Cernak LU/NA1, INFN/SA1, UPJS/SA2 12/02/2011 Approved by PEB Document Log Issue Date Comment Author / Partner 0.1 28/11/2010 Structure and Table of Content Alberto Di Meglio/CERN 0.2 06/12/2010 Starting with content André Giesler 0.3 15/12/2010 Defining integration test plan André Giesler 0.4 20/12/2010 Draft version for showing up progress André Giesler 0.5 23/12/2010 Refined draft version André Giesler 1.0 08/02/2011 Revised final version André Giesler Document Change Record Issue Item Reason for Change INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 3 / 41

TABLE OF CONTENTS 1 INTRODUCTION... 6 1.1 PURPOSE... 6 1.2 DOCUMENT ORGANISATION... 6 1.3 REFERENCES... 6 1.4 DOCUMENT AMENDMENT PROCEDURE... 8 1.5 TERMINOLOGY... 8 2 EXECUTIVE SUMMARY... 10 3 STATUS REPORT... 11 4 OVERVIEW OF REQUIRED WORK... 13 4.1 EMI 1 TIMELINES... 13 4.2 EMI INTEGRATION AND INTEROPERABILITY OBJECTIVES... 13 4.2.1 Cross all areas... 13 4.2.2 Compute Area... 14 4.2.3 Data Area... 14 4.2.4 Infrastructure Area... 15 4.2.5 Security Area... 15 4.3 OVERVIEW OF PRODUCT CHANGES... 15 4.3.1 Compute Area... 15 4.3.2 Data Area... 16 5 SOFTWARE INTEGRATION AND INTEROPERABILITY STRATEGIES... 18 5.1 SYSTEM INTEGRATION AND TESTING... 18 5.1.1 How does Integration Testing fit into the Software Development Life Cycle?... 18 5.1.2 Prerequisites for Integration Testing... 18 5.1.3 Integration Testing Steps... 19 5.1.4 What is an Integration Test Plan?... 19 5.1.5 How to write an Integration Test Case?... 19 5.2 UNIT INTEGRATION AND TESTING... 20 5.3 INTEROPERABILITY TESTING... 21 5.4 STANDARD COMPLIANCE TESTING... 22 5.4.1 Computer Hardware Resource Utilization... 22 5.4.2 Access for Review... 23 6 PLANS FOR PERFORMING DETAILED SOFTWARE INTEGRATION AND INTEROPERABILITY ACTIVITIES... 24 6.1 PROJECT PLANNING AND OVERSIGHT... 24 6.1.1 Integration Plan... 24 6.1.2 Interoperability Plan... 24 6.1.3 Standard Compliance Plan... 25 6.2 SOFTWARE DEVELOPMENT ENVIRONMENT... 25 6.2.1 Software Engineering Environment... 25 6.2.2 Software Test Environment... 26 6.2.3 Software Integration Tasks Tracking... 28 6.3 INTEGRATION, INTEROPERABILTY AND STANDARD COMPLIANCE TASKS... 28 6.3.1 ARC Computing Element (A-REX)... 29 6.3.2 WMS... 29 6.3.3 CREAM... 30 INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 4 / 41

6.3.4 UNICORE compute elements... 31 6.3.5 ARC* Client... 31 6.3.6 UCC Client... 32 6.3.7 DPM... 33 6.3.8 dcache... 34 6.3.9 StoRM... 35 6.3.10 UNICORE data... 37 6.4 OTHER SOFTWARE DEVELOPMENT ACTIVITIES... 37 6.4.1 Risk Management... 37 7 SCHEDULES AND ACTIVITY NETWORK... 39 8 CONCLUSIONS... 41 INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 5 / 41

1 INTRODUCTION 1.1 PURPOSE This document describes the EMI Integration Plan for the first year of EMI. It is applicable to the development and test activities within JRA1 that lead to the availability of software products conformant to the EMI technical objectives for Year 1 and compliant with the EMI Software Quality Assurance Plan and related procedures. This document is a specialization of the general EMI Development and Testing Plan describing in details the integration and interoperability strategies [R10]. 1.2 DOCUMENT ORGANISATION This document is organized as follows: Chapter 1 - Introduction: this section, explaining the purpose, scope and organization of the document Chapter 2 - Executive Summary: this section contains a high-level description of the document. It gives a summary of the most important points described in each main section. Chapter 3 Status Report: this section describes the actual outcome of the integration tasks in the previous development cycle. Since we are considering the first cycle, an overview of the state of the art is given. Chapter 4 - Overview of Required Work: this section describes the overall goals and timelines of the EMI-1 development activities. Chapter 5 - Software Integration and Interoperability Strategies: this section describes the processes, tools and constraints applicable to the EMI-1 integration activities. Chapter 6 - Plans for Performing Detailed Software Integration and Interoperability Activities: this section describes the actual implementation of the general processes and the detailed tasks to be performed by each Product within the scope of the EMI-1 integration activities. Chapter 7 Schedules and Activity Network: this section provides a global overview of the integration activities schedule. Chapter 8 Development Organization and Resources: this section gives a description of the integration organization, the roles and responsibilities. 1.3 REFERENCES R1 EMI JRA1.7 Wiki Page https://twiki.cern.ch/twiki/bin/view/emi/emijra1t7integration R2 EMI Testbed Wiki Page https://twiki.cern.ch/twiki/bin/view/emi/testbed R3 EMI JRA1 Wiki Page https://twiki.cern.ch/twiki/bin/view/emi/jra1 R4 Technical Development Plan https://twiki.cern.ch/twiki/bin/view/emi/deliverabledna131 INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 6 / 41

R5 R6 R7 R8 R9 R10 R11 R12 R13 R14 R15 R16 R17 R18 R19 R20 R21 R22 R23 EMI Certification and Test Plan https://twiki.cern.ch/twiki/pub/emi/emisa2certtestguidelines/emi_sa2_certificationandte sting_v_1_0.pdf Compute Area Work Plan and Status Report https://twiki.cern.ch/twiki/bin/view/emi/deliverabledjra111 Data Area Work Plan and Status Report https://twiki.cern.ch/twiki/bin/view/emi/deliverabledjra121 Security Area Work Plan and Status Report https://twiki.cern.ch/twiki/bin/view/emi/deliverabledjra131 Infrastructure Area Work Plan and Status Report https://twiki.cern.ch/twiki/bin/view/emi/deliverabledjra141 JRA1 Development and Test Plan https://twiki.cern.ch/twiki/pub/emi/internaldeliverableemi1releasedevplans/2010-12- 17_EMI_1_Development_Test_Plan-v0.4_MRi_.doc GLUE2 Specification http://www.ogf.org/documents/gfd.147.pdf ETICS Build and Test System https://etics.cern.ch/eticsportal/#legbs Build Configuration and Integration Guidelines https://twiki.cern.ch/twiki/bin/view/emi/emisa2configurationintegrationguidelines ETICS Nightly Builds http://etics-repository.cern.ch/repository/reports/name/emi_r_0/-/reports/index.html JRA1 EMI -1 Technical Objectives https://twiki.cern.ch/twiki/bin/view/emi/emijra1t1emi1technicalobjectives JRA1 Wiki pages with GANTT charts https://twiki.cern.ch/twiki/bin/view/emi/emijra1t1systemwpprogresstrackingprocess GLUE Integration Testing Plan https://twiki.cern.ch/twiki/bin/view/emi/emijra1t1emi1teststrategyglueinformation SRM HTTPS Binding Test https://twiki.cern.ch/twiki/bin/view/emi/emijra1t1emi1teststrategysrmcompliance SES Access Protocols Test Plan https://twiki.cern.ch/twiki/bin/view/emi/emijra1t1emi1teststrategysesaccessprotocols GSTAT http://www.gstat.org Total Integration Matrix https://twiki.cern.ch/twiki/pub/emi/emijra1t7integration/integrationmatrix.xlsx Integration Compute Matrix https://twiki.cern.ch/twiki/pub/emi/emijra1t7integration/integrationcompute.xlsx Integration Data Matrix https://twiki.cern.ch/twiki/pub/emi/emijra1t7integration/integrationdata.xlsx INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 7 / 41

R24 R25 R26 R27 R28 R29 R30 R31 Integration Infrastructure Matrix https://twiki.cern.ch/twiki/pub/emi/emijra1t7integration/integrationinfrastructure.xlsx Integration Security Matrix https://twiki.cern.ch/twiki/pub/emi/emijra1t7integration/integrationsecurity.xlsx Total Interoperability Matrix https://twiki.cern.ch/twiki/pub/emi/emijra1t7integration/interopmatrix.xlsx Interoperability Compute Matrix https://twiki.cern.ch/twiki/pub/emi/emijra1t7integration/interopcompute.xlsx Interoperability Data Matrix https://twiki.cern.ch/twiki/pub/emi/emijra1t7integration/interopdata.xlsx Interoperability Infrastructure Matrix https://twiki.cern.ch/twiki/pub/emi/emijra1t7integration/interopinfrastructure.xlsx Interoperability Security Matrix https://twiki.cern.ch/twiki/pub/emi/emijra1t7integration/interopsecurity.xlsx Project Technical Board https://twiki.cern.ch/twiki/bin/view/emi/ptb 1.4 DOCUMENT AMENDMENT PROCEDURE This document can be amended by the EMI JRA1 Integration Task Leader further to any feedback from other teams or people. Minor changes, such as spelling corrections, content formatting or minor text re-organisation not affecting the content and meaning of the document can be applied by the JRA1 Integration Task Leader without peer review. Other changes must be submitted to peer review and to the EMI PTB for approval. When the document is modified for any reason, its version number shall be incremented accordingly. The document version number shall follow the standard EMI conventions for document versioning. The document shall be maintained in the CERN CDS repository and be made accessible through the OpenAIRE portal. 1.5 TERMINOLOGY DPM Disk Pool Manager GLUE Grid Laboratory Uniform Environment JSDL Job Submission and Description Language GIN Grid Interoperation Now KPI Key Performance Indicator OGF Open Grid Forum PGI Production Grid Infrastructure SAGA Simple API for Grid Applications SDTP JRA1 Software Development Test Plan SRM Storage Resource Manager INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 8 / 41

TSI Target System Interface UAS UNICORE Atomic Services WMS Workload Management System WebDAV Web-based Distributed Authoring and Versioning XNJS Enhanced Network Job Supervisor INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 9 / 41

2 EXECUTIVE SUMMARY The EMI 1 Integration Work Plan and Status Report contain the detailed work plan of the integration activities and objectives compliant with the overall EMI Technical Development Plan and aligned with the work-plans of other tasks within JRA1. This document is closely intermeshed with the EMI 1 Software Development Test Plan (SDTP) [R10] which targets a couple of guidelines concerning the development and testing process of components. Although the plan is released early in the project life it is updated every year including a status report on the achievements of the past 12 months compared to the planned objectives. The PTB has defined a number of technical objectives to be addressed as part of the EMI-1 release in addition to standard changes and improvements planned as part of the continuous product improvement activities [R31]. These objectives as far as they include integration and testing activities are covered in this plan. Chapter 3 outlines an overview about the state-of-the-art concerning integration activities. Additionally it is described how JRA1.7 has enabled a continuous status request concept concerning the integration and interoperability tasks of EMI product teams. Chapter 4 lists the particular objectives and highlights all affected EMI components in this regard. The overall strategies to test integration and interoperability in EMI are described in chapter 5. Developers and software testers will find here an approach how JRA1.7 performs integration, unit, and standard compliance testing. Chapter 6 provides a fine-grained, detailed plan in which manner the strategies are adapted to the identified objectives from chapter 4. The detailed project schedule and activity network is described in the Section 7. INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 10 / 41

3 STATUS REPORT The current EMI development cycle is called EMI 0. It can be considered as an exercise release designed to understand how to apply the agreed procedures, how to find any problem about the used build tools, and in general to fine tune the EMI software engineering process before the first real EMI software release. The EMI 0 release is not designed for external users. The goal is instead to prepare a consistent, coherent repository of non-conflicting software packages by the end of 2010 without any specific commitment on functionality. For this reason, almost no developments were explicitly done for this release. So, EMI components, which are integrated in EMI 0, represent in principle the same development or production status as before EMI and can be considered as the latest state-of-the-art. In some cases product teams defined already release candidates for EMI 1 in their EMI 0 configurations. Since project start we have worked on the integration and interoperability matrices and their continuous updates by inquiring the individual product teams and checking their working plans. Integration refers to a setup where one software component inherently uses another component to provide a dedicated functionality. Interoperability within this EMI task means that one component is interoperable with another if both integrating the same interface. Chapter 5 provides a more detailed introduction to integration and interoperability strategies in this task. So, a client who is compliant with that interface is able to communicate with both components without the need of any transformation in the protocol. In the matrices we are tracking information about integration and interoperability relations between EMI components, interfaces, or external components in terms of their current integration status or release plans, supported platforms, and their integration in the ETICS- and EMI-testbed system as well. In particular, the status of an integration/interoperability relation can have the following conditions: Not-applicable (e.g. an EMI component is incompatible with SRM standard) Not-integrated (e.g. it is not planned to integrate a specific EMI component with SRM standard within the EMI project) Fully integrated (e.g. an EMI component integrates the GLUE 2.0 standard) emi-0, emi-1, emi-2, emi-x: Integration is planned in EMI-0, EMI-1, EMI-2, or EMI-X which means later in project but not yet specified precisely The statuses for the interoperability evaluation have quite the same definitions: Not-applicable (e.g. an EMI component is not interoperable to with another EMI component) Not-interoperable (e.g. it is not planned to integrate a specific EMI component with SRM standard) Fully interoperable (e.g. an EMI component is fully interoperable with another component by using a defined standard like HTTPS) emi-0, emi-1, emi-2, emi-x: Interoperability is planned in EMI-0, EMI-1, EMI-2, or EMI-X which means later in project but not yet specified precisely Figure 1 shows an excerpt of the integration matrix. This tool is based on MS Excel tables and it shows exactly where points of interest with reference to integration of components are, while the latest status can be tracked via the wiki page [R1]. While many fields of these matrices are empty, those that are filled indicate integration activity between. As being the first report, we describe the major areas of work for component integrations and we expect to refine these descriptions as well as the availability of integration tests in updates of this INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 11 / 41

report, including important cross-area integrations (e.g. where compute hits security, etc.). This list is not complete and expected to be re-fined with the future updates of this report, including also the addition of emerging components (i.e. EMI Registry, messaging, etc.). However, we define major relevant areas of work that are important to ensure that already the EMI-1 release has relevant integration tests in place that in turn realize a high quality integrated set of release candidates. Note that the details of these working areas are encoded in the integration matrices on the wiki [R1]. Figure 1: Excerpt of the integration matrix The following table provides references to the individual integration and interoperability matrices. Type Area Reference Integration Total (all areas covered) [R21] Integration Compute [R22] Integration Data [R23] Integration Infrastructure [R24] Integration Security [R25] Interoperability Total (all areas covered) [R26] Interoperability Compute [R27] Interoperability Data [R28] Interoperability Infrastructure [R29] Interoperability Security [R30] INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 12 / 41

4 OVERVIEW OF REQUIRED WORK 4.1 EMI 1 TIMELINES This section describes the overall EMI 1 timelines within the context of the EMI Development Roadmap. The EMI Technical Development Plan [R4] of the EMI project provides defined technical objectives for the EMI development by representing a definition of the EMI software portfolio, and a comprehensive technical formulation of development objects coupled to components and delivery dates. In addition to this, this document provides low level, implementation-specific details of the technical objectives as well as a timeline of the development tasks within the context of the first EMI project year. So, this document covers all scheduled developments to be released in EMI 1 with a particular focus on standardization and integration plans. The development and integration process starting from the state-of-the-art to EMI 1 includes goals for the four technical areas plus a specific area for cross-area concerns. Section [4.2.] will cover all technical objectives with a deadline for the EMI 1 release concerning real physical components and their system integration aspects. EMI 1 will be released in April 2011. 4.2 EMI INTEGRATION AND INTEROPERABILITY OBJECTIVES This section gives a brief overview of the EMI technical objectives in year 1 for each technical area with particular focus on their integration and interoperability aspects. Examples of such objectives are those involving different implementations of the same capability, the adoption of common interfaces, and the use of reusable or external components across sets of different products. In addition, this section covers the general requirements of integration with standard operating systems. Technical objectives that will not include any development and integration activity are not listed here. Compare chapter 3 of the SDTP [R10] to get a complete list of all physical and not physical technical objectives. 4.2.1 Cross all areas The following objectives are improvements for EMI components that are not covered by the Technical Development Plan for Year 1. All of the listed objectives are already in development or in beta productions status. If these component improvements will be really released in time for EMI 1 or in later minor releases towards EMI 1.1 and higher depends on the progress in the responsible product teams. Providing WebDAV access For EMI 1 dcache will provide WebDAV authentication via user/password. Authenticated Web Interface For EMI 1 dcache will provide an authenticated web interface to manage pools and other system components. Re-balancing data on pools For EMI 1 dcache will provide the functionality of Re-balancing data on pools after new pools are added or filled unbalanced. Improved drain performance For EMI 1 DPM will speed up the draining of data pools in case of hardware maintenance or decommissioning. INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 13 / 41

Disk server monitoring In preparation of the EMI common task monitoring the DPM storage element will improve the internal monitoring and will provide more information to track down problems. Disk server management For EMI 1 DPM will provide mechanisms to steer the number of streams from individual pools to better manage resources. Support for roles DPM will enable that priorities can be given to particular roles. High availability clusters For EMI 1 StoRM will provide a cluster solution for high availability. Log analyzer For EMI 1 StoRM will provide an administrative tool to interrogate StoRM log files. Replacement of update-crls with fetch-crl For EMI 1 StoRM will provide an administrative tool to interrogate StoRM log files. EES development The integration of the Argus policies into the mapfile generator should be completed. 4.2.2 Compute Area Glue 2.0 support in job management services and client tools Several EMI components in the compute area are going to integrate the GLUE 2.0 model for taking advantage of this information schema. It is prerequisite that the EMI computing elements and the systems that offer access to computing systems should expose pieces of information about computing resources in a consistent manner. The integration of GLUE will be performed on unit level of the affected compute components by implementing the schema as XML or LDAP in the appropriate units. The implementations will be integrated in the ETICS system. The interoperability between components using GLUE as an information model will be tested by standard compliance tests which have to be defined within an integration testing plan. Chapter 6 provides a more detailed approach of the GLUE testing plans concerning EMI components. 4.2.3 Data Area Publishing GLUE 2.0 information in storage components All three major storage elements within EMI named as dcache, DPM, and StoRM will integrate several information provider components. The goal is that EMI storage elements and the systems that offer access to storage systems should expose pieces of information about storage resources in a consistent manner. To ensure that the integration of these components is seamless, we also need to explore and execute integration tests in this context. Using https for the SRM protocol The SRM (Storage Resource Manager) is a protocol for Storage Resource Management. SRM is used to ask a Mass Storage System to make a file ready for transfer, or to create space in a disk cache to which a file can be uploaded. A prototype implementation should be created for EMI 1 where SRM is INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 14 / 41

using https as transport protocol instead of httpg. This prototype development explores the use of the SRM protocol without requiring the proprietary GSI. We will perform integration tests as soon as the prototype is available in the ETICS system. HTTP(s) support for all storage elements In order to be more in-line with modern Web-based access methods it is intended to provide HTTPS support for all EMI storage elements. The integration of this will be verified by functionality tests defined in a detailed test plan. Supporting "file://" access protocol All storage elements should offer at least a prototype-level support for the standard "file://" access protocol. The integration of this will be verified by functionality tests. File Catalogue Access from UNICORE Traditional the UNICORE system was not enabled to use any form of file catalogues and thus it is required to augment this functionality in order to fully function in the larger EMI ecosystem. 4.2.4 Infrastructure Area No objectives identified for EMI 1 in this area. 4.2.5 Security Area No objectives identified for EMI 1 in this area. 4.3 OVERVIEW OF PRODUCT CHANGES This section gives a brief overview of the EMI Products affected by the defined Technical Objectives as described in the overall EMI Software Development and Test Plan [R10] focusing on their integration and interoperability aspects. The tables in the following subsections provide information about the concerned EMI components, the current development status, the deadline of the task, and the destination platforms. However, the platform column shows which platforms are currently supported by the affected component. It had not yet been decided which platforms have to be supported in EMI 1 at the time of composing this document. 4.3.1 Compute Area Support for Glue 2.0 in job management services and client tools Several EMI components are going to integrate the GLUE model for taking advantage of that information model standard. The following table lists the EMI components planning to integrate GLUE2.0 for the EMI 1 release Component Status Integration due Platforms A-REX In development M12 RHEL5, Deb5, Scientific Linux CREAM In development M12 Scientific Linux UNICORE TSI / U. XNJS / U.UAS/ WSRFLite Beta version available M12 All platforms supported by Java INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 15 / 41

WMS In development M12 Scientific Linux Libarcclient/ Arc* Beta version available M12 All Linux system, partly on Mac and Solaris UCC (OGSA-BES plugin)/ HILA Beta version available M12 All platforms supported by Java 4.3.2 Data Area Publishing GLUE 2.0 information in storage components All EMI storage elements will publish initial GLUE 2.0 storage information, and a storage client or library should be capable to consume that information. The following components are affected in the context of this JRA1 technical objective. Component Status Integration due Platforms DPM Beta version available M12 Scientific Linux dcache Beta version available M12 All platforms supported by Java StoRM Beta version available M12 Scientific Linux UAS-D Beta version available M12 All platforms supported by Java Using https for the SRM protocol The following component is affected in the context of this technical objective. Component Status Integration due Platforms dcache (Client and Server) Beta version available M12 All platforms supported by Java HTTP(s) support for all storage elements The following components are affected in the context of this JRA1 technical objective. Component Status Integration due Platforms DPM Not addressed by M12 Scientific Linux developers yet or already developed dcache Production quality M12 All platforms supported by Java StoRM Beta version available M12 Scientific Linux All storage elements offering at least a prototype-level support for the "file://" access protocol The following components are affected in the context of this JRA1 technical objective. INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 16 / 41

Component Status Integration due Platforms DPM Beta version available M12 Scientific Linux dcache Beta version available M12 All platforms supported by Java StoRM In development M12 Scientific Linux File Catalogue Access from UNICORE Component In development Integration due Platforms UAS-D Beta version available M12 All platforms supported by Java INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 17 / 41

5 SOFTWARE INTEGRATION AND INTEROPERABILITY STRATEGIES The subsections of this section describe the principles of different integration and interoperability strategies covered by this plan. 5.1 SYSTEM INTEGRATION AND TESTING This section describes the scope and goals of the system integration activity and the adopted testing methods and tools. Integration refers to a setup where an EMI software component inherently uses another software component to provide a dedicated functionality. The way how an EMI component integrates another one varies regarding the use of common interfaces and schemas (i.e. information model via GLUE) or simply the usage of libraries. Based on that definition integration tests are functionality tests involving the interaction among different components, generally developed by different Product Teams and which may be deployed on different hardware instances or as modules on the same instance. Integration testing is a logical extension of unit testing. In its simplest form, two units that have already been tested are combined into a component and the interface between them is tested. A component, in this sense, refers to an integrated aggregate of more than one unit. In a realistic scenario, many units are combined into components, which in turn could be aggregated into even larger parts of the system. The idea is to test combinations of pieces and eventually expand the process to test particular modules with those of other components. Eventually all the modules making up a process are tested together. Beyond that, if the component is composed of more than one process, they should be tested in pairs rather than all at once. Integration testing identifies problems that occur when units are combined. By using a test plan that requires EMI to test each unit and ensure the viability of each before combining units. This method reduces the number of possibilities to a far simpler level of analysis. 5.1.1 How does Integration Testing fit into the Software Development Life Cycle? Even if a software component is successfully unit tested, in an enterprise n-tier distributed application it is of little or no value if the component cannot be successfully integrated with the rest of the application. Once unit tested components are delivered they are integrated together. These integrated components are tested to weed out errors and bugs caused due to the integration. This is a very important step in the Software Development Life Cycle. It is possible that different programmers developed different components, so it is not unusual that a lot of bugs emerge during the integration step. In most cases a dedicated testing team focuses on Integration Testing. JRA1.7 will coordinate that process. 5.1.2 Prerequisites for Integration Testing The integration of components in EMI is initialised by adding appropriate modules to the EMI ETICS system. The responsible product teams create then ETICS configurations representing a specific version of an integrated component. These configurations contain information about used dependencies as well as build and test functions. The remote builds and tests are executed periodically from the ETICS build system, so that issues on that level are already logged and registered by the ETICS error reporting system. Additionally, ETICS is using specific plug-ins for static code analysis providing an analysis about the quality of the integrated code. Before JRA1.7 begins with Integration Testing it is important that all the components have been successfully unit tested. This implies that the component has been already ported to the EMI ETICS system which is executing the unit tests periodically [section 5.2]. Another prerequisite is that the responsible product team has already produced a Software Verification and Validation Plan according to the EMI Testing and Certification guidelines [R5]. INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 18 / 41

JRA1.7 has already registered all relevant integration tasks for EMI-1 in the integration matrices. These integration relations can be identified in the ETICS system by analyzing the appropriate configurations in regard to the used internal or external dependencies under reserve that the integrated task is represented by a library, service or other dependency. For example, the AMGA component integrates the BDII GLUE model. This integration task is represented by the emi.bdii.glue-schema dependency within the AMGA configuration emi.amga.glite-amga_postgres. 5.1.3 Integration Testing Steps We have created a defined procedure which should on the one hand formalize the testing process, and on the other hand should also provide a manual for developers, testers and product coordinators who are involved in the process. The EMI integration testing for major releases typically involves the following steps: 1. Create a Test Plan (JRA1.7 and PTs) 2. Create Test Cases and Test Data (JRA1.7 and PTs) 3. Agreement with EMI SA2.6 and affected PTs how to utilize hardware resources and access the testbed (JRA1.7, PTs, and SA2) 4. If applicable create scripts to run test cases (JRA1.7 and PTs) 5. Once the components have been integrated execute the test cases (JRA1.7 and PTs) 6. Fix bugs if any and re-test the code (PTs) 7. Repeat the test cycle until the components have been successfully integrated (JRA1.7 and PTs) These steps will be performed in close collaboration between JRA1.7, who will coordinate the testing process, the affected product teams, and the EMI-testbed administrators. While step 3 is described in section 5.4 and steps 4 to 7 are self-explanatory, the first two steps need more detailed description. 5.1.4 What is an Integration Test Plan? This plan describes recurring parameters in a typical integration test case. How the tests will be carried out. The list of things to be tested Roles and Responsibilities Prerequisites to begin Testing Test Environment Assumptions What to do after a test is successfully carried out. What to do if test fails Glossary 5.1.5 How to write an Integration Test Case? In short, a Test Case describes exactly how the test should be carried out. The Integration test cases specifically focus on the flow of data/information/control from one component to the other. So the Integration Test cases should typically focus on scenarios where one component is being called from another. Also the overall component functionality should be tested to make sure the software product works when the different components are brought together. The various Integration Test Cases are INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 19 / 41

concatenated together forming an Integration Test Suite. Each suite may have a particular focus, which is usually defined by the technical objective. In other words different Test Suites may be created for on EMI component to focus on different technical objectives it is involved. As mentioned before a dedicated Testing Team may be created to execute the Integration test cases. Since JRA1.7 is not familiar with technical details of all EMI components, a strong collaboration with the appropriate PTs is required here. The Integration Test Cases should be as detailed as possible. Below is listed a sample Test Case table as planned for the integration testing. Test Case ID Test Case Description Input Data Expected Result Actual Result Pass/Fail Remarks Additionally the following information will also be captured: a) Test Suite Name b) Tested by c) Date d) Test Iteration (One or more iterations of Integration testing may be performed) When performing functionality tests defined in an Integration Test Case, JRA1.7 will go back to the Software Verification and Validation Plan of the component which has to be created by the responsible product teams according to Certification and Testing Guidelines [R5]. This report should already contain details of the available functionality tests of the affected component. In particular, it is listed what should be tested and how. If possible JRA1.7 will make use of existing test cases, or rather adopt them to the requirements of the integration testing. JRA1.7 checks that every predefined test concerning the integration task is running properly. The results will be added to the Software Verification and Validation Report [R5]. Integration testing is only part of the entire software development process. The latter is described comprehensively in the SDTP [R3]. 5.2 UNIT INTEGRATION AND TESTING This section describes the scope and goals of the unit integration activity and the adopted testing methods and tools. Unit tests are intended to test the correctness of individual units or a group of related units. Since the components integrated in EMI are implemented in different programming languages, also the definition of a unit can vary accordingly. However, in general the methods of unit testing are well standardized. Tools like CPPUnit, JUnit, and PyUnit are predominantly used in EMI software components. A unit is the smallest testable part of a software entity. In this respect, JRA1.7 will identify unit test cases in components affected in the integration activity that were explicitly implemented to validate the functionality. In a given case that means for instance, if an EMI component of the compute area has implemented the GLUE 2.0 schema successfully, there should exist as well some unit test cases which are validating the correct execution of the implementation. JRA1.7 expects that appropriate unit tests in regard to the integration activity will be made available by the product teams along with the releases of the developments. Since ETICS is used in EMI as the common building and testing tool, each PT should provide already by default unit tests in their INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 20 / 41

appropriate component configurations. The unit tests should be also declared within the Software Verification and Validation Plan of the component [R5]. If no unit tests can be identified, we will request the developers to provide or if they are not sufficient to modify these tests. However, since EMI demands by default high unit test code coverage according to the Certification and Test Guidelines [R5], it can be expected that adequate unit test cases will be provided by the responsible developers. JRA1.7 will add the unit tests concerning the technical objectives defined in the SDTP to the overall integration test plan [section 5.2] to keep related unit and integration tests in a single document. 5.3 INTEROPERABILITY TESTING This section describes the scope and goals of the interoperability activity and the adopted testing methods and tools. Interoperability within this EMI task can be considered as follows. As shown exemplarily in Figure 2 service component A is interoperable with service component B since both integrate the same standard interface. In other words, any client that is compliant with this Standard Service Interface is able to communicate with both services without the need of any transformations in the protocol. In this particular context we can speak of a native interoperability between both components. In contrast to the aforementioned form of interoperability, there is another type of interoperability that refers to the client to service communication. As also shown in Figure 2, the client application is interoperable with service component A and B, since client and server implement the same standard interface. While the client application uses a client implementation of this interface, the server components uses the server-side implementation of the interface. Figure 2: Illustration of various interoperable components realized by adopting the same standard interface on the server-side and on the client-side. INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 21 / 41

In principle, we will perform functional tests to check the interoperability between EMI components. So, JRA1.7 will create standardized tests which should be able to run on each EMI component which is implemented to process the used standard interface. For instance, we will create standardized jobs which will be submitted to all components in the compute area that are implementing a specific interface. In this respect the EMI testbed will be used predominantly as soon as the software products are available there. For EMI-1 all interoperability tests can be categorized as compliance checkers. The following section outlines how the technical objectives concerning interoperability are tested in a standard compliance testing approach. 5.4 STANDARD COMPLIANCE TESTING This section describes the scope and goals of the standard compliance activity and the adopted testing methods and tools. Since the project start we have already worked on the interoperability matrices and their continuous updates. In parallel, we have also started within this sub-task and identified a couple of initial standard compliance checkers together with JRA1.8 that make it possible to validate several interoperability works within the EMI components. During the course of the project we foresee new standard compliance checkers to be created where necessary, for example, when specifications of the Production Grid Infrastructure (PGI) set of specifications/profiles will be available. JRA1.7 will closely work together with the developers in the product teams that require the existence of standard compliance checkers in order to ensure a high quality of their components. In this context, the interoperability matrix will point to relevant working areas and will enable to track progress towards the use of standard compliance checks. SRM Standard Compliance Checks This sub-task will also work together with the product teams in the data area that adopt the SRM interface. There is an SRMv2 compliance test suite available that is of major relevance for the EMI data component stack. The relevant component in context is dcache that already performs tests with this compliance test suite on a regular basis. Here it makes sense to broaden the usage of this test suite to other SRM-compliant systems such as DPM and StoRM. IPV6 Compliance Checks This sub-task will work together with those product teams of components that will take advantage of the emerging IPv6 technology. There is an IPv6 compliance test suite available and relevant for those components. The relevant component in context is currently the Logging and Bookkeeping service that already performs tests with this compliance test suite on a regular basis. Here it makes also sense to broaden the usage of this test suite to other EMI components that will make use of IPv6 during the course of the project. LDAP and GLUE Compliance Checks Early work exists within the EMI project to check the compliance to LDAP and GLUE in context. This sub-task will closely work together with those product teams that use LDAP. The relevant component in context is currently the set of BDII systems that are based on LDAP and take advantage of the GLUE information model standard. It will be explored how the set of components can be extended that take advantage of these tests as they are also extended towards the test of GLUE2 compliance. The particular tool is GStat [R20] that provides suitable validation probes. 5.4.1 Computer Hardware Resource Utilization This paragraph describes the approach to be followed for allocating computer hardware resources and monitoring their utilization. EMI has created an integration testbed with a set of allocated hardware resources to enable continuous software development and testing. EMI Integration Testbed indicates the whole set of geographically distributed infrastructural (hardware, networking, software) and operational (procedures, tools, documentation, communication tool, support handling etc.) resources made available for integration testing purposes. An overview of the EMI integration testbed and how INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 22 / 41

to use it in particular is described in [R3]. EMI task SA2.4 is in charge of the EMI integration testbed [R5]. JRA1.7 needs to perform functionality tests to check if an integrated task is really in place and working properly. These tests can only be done manually by using an appropriate installation of the affected EMI components and having authenticated/authorized access to them. Depending on the component, it may include interaction with other components developed by the same or a different product team. The right place to perform functional tests is usually the EMI integration testbed [R2], which provides operational resources made explicitly available for the continuous integration testing process of EMI software products. JRA1.7 will request access to the needed resources in the testbed to check if the integration task is working properly according to the defined Integration Test Plan [section 5.1.4] and thus tested successfully for a major EMI release. Finally, if system integration tasks cannot be tested by accessing installed resources within the EMI testbed for whatever reasons, JRA1.7 will try to install the identified components itself for further analysis. JRA1.7 will hand out a special form of the integration matrix to EMI SA2.4 that contains all identified integration tasks of EMI components concerning EMI-1. The matrix provides information about which groups of components are affected in an integration task, which hardware resources need to be allocated, and which operation systems are in the scope. 5.4.2 Access for Review This paragraph describes the approach to be followed for providing project members access to developer work for review of software products and activities. Refer to ETICS public results, nightly builds results, test reports, etc. Initially, project members can use the ETICS build and test system [R12] as a central point to get access to the available software components of EMI. Users can simply browse through the component tree of the ETICS web portal (or alternatively by using the ETICS command line client) to find the appropriate components and to examine the current configuration of the component. A quick check in the ETICS workspace will give an overview of used dependencies and if unit tests are enabled in remote builds and tests. According to the Build Configuration and Integration Guidelines [R13] the formatted names of the ETICS configurations characterize their state. For instance, the letter R describes that the configuration is a release version of the component. All component configurations associated with an EMI-release candidate are executed periodically in nightly builds. For EMI-0 this automatic build is already activated and are summarized in an overall ETICS report [R14] which is freely accessible for EMI project members. This constant updated webdocument provides detailed build and test reports of all adopted components. Additionally, JRA1 provides comprehensive information at the TWIKI pages. A good starting point about ongoing developments for EMI releases, the technical objectives, and testing strategies is the JRA1 EMI -1 Technical Objectives page [R15]. Finally, the development activities are tracked via GANTT charts that are publicly available on a quarterly basis in the JRA1 Wiki pages [R16]. This enables the EMT, PTB, and PEB the access to the status and results of the developments. INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 23 / 41

6 PLANS FOR PERFORMING DETAILED SOFTWARE INTEGRATION AND INTEROPERABILITY ACTIVITIES Several parts of this section refer to the corresponding sections in the EMI-1 Development and Test Plan. 6.1 PROJECT PLANNING AND OVERSIGHT This section describes the types and content of the development and test plans of each EMI component with a particular focus on the EMI-1 release. For each product the available plans are included by reference or marked as missing if still in development. 6.1.1 Integration Plan Each affected Product Team must develop and record plans for conducting the integration activities required by this document and by other software-related requirements in the project. This planning must be consistent with system-level planning and shall include all applicable items in the SDTP. In particular it must describe the system and unit integration test cases and test plans. These integration plans should be part of the Software Development Plans which are already requested from each Product team as defined in the JRA1 SDTP [R3]. In these plans, each of the PTs provides a detailed description of their system and integration test cases and test plans of the affected EMI components. JRA1 are actually part of the work plans for each technical area within EMI and can be found via the following references. Integration Activity Technical Area Relevant Components of PTs Plan Support for Glue 2.0 Compute A-REX, CREAM, U. TSI, U. XNJS, UAS-C, WSRFlite, WMS, libarcclient, arc*, UCC [R17] Publishing GLUE 2.0 Data DPM, StoRM, dcache, UAS-D [R17] Using https for the SRM protocol HTTP(s) support for all storage elements Support for the "file://" access protocol File catalogue access for UNICORE Data dcache (client and server) [R18] Data DPM, dcache, StoRM [R19] Data DPM, dcache, StoRM [R19] Data UAS-D In development From SDTP: Note that with respect to EMI-1, the infrastructure area has no implementations that directly address EMI technical objectives as part of this plan, but plans of these PTs can be obtained via [R8, R9] and will be included in the next update of the development plan for a EMI minor release or the next major release. 6.1.2 Interoperability Plan Each affected Product Team must develop and record plans for conducting the interoperability activities required by this document and by other software-related requirements in the project. This planning must be consistent with system-level planning and shall include all applicable items in the SDTP. In particular it must describe the interoperability test cases and test plans. No interoperability tasks are planned for EMI 1. INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 24 / 41

6.1.3 Standard Compliance Plan Each affected Product Team must develop and record plans for conducting the standard compliance activities required by this document and by other software-related requirements in the project. This planning must be consistent with system-level planning and shall include all applicable items in the SDTP. In particular it must describe the standard compliance test cases and test plans. The standard compliance testing in EMI-1 is included in the appropriate integration testing plans. The following table lists the compliance testing tasks for EMI-1, affected components, and the associated test plan. Compliance Testing Technical Area Relevant Components of PTs Plan GLUE Compute A-REX, CREAM, U. TSI, U. XNJS, UAS-C, WSRFlite, WMS, libarcclient, arc*, UCC, HILA [R17] GLUE Data DPM, StoRM, dcache, UAS-D [R17] SRM Data dcache (client and server) [R18] HTTP(s) Data DPM, dcache, StoRM [R19] "file://" access protocol Data DPM, dcache, StoRM [R19] 6.2 SOFTWARE DEVELOPMENT ENVIRONMENT This section describe how the integration, interoperability and standard compliance testing methods and tools described in Section 5 are instantiated in practice for the development and testing of the Products in EMI-1. 6.2.1 Software Engineering Environment The test development activities of the identified EMI components are summarized in the table below. It is shown which EMI product is using which software engineering environment. In particular it describes: Product a. The ETICS configurations to be used. If not yet known for EMI-1 the appropriate ETICS component is listed. b. The target platforms (It is not yet decided which target platforms will be used for EMI-1 at this stage, so the currently available platforms are listed.) c. Specific versions of programming or scripting languages or development tools to be used (if available per platform). d. Specific version of external software packages to be used as far as known (if available per platform). ETICS Component/ Configuration Platforms with languages and development tools A-Rex emi.arc.arc1 All platforms supporting Java. Built by Maven. Current target platforms: Deb5 (64bit), SL5 (64bit) WMS emi.wms (current conf: emi- Development tools / Programming languages Maven / Java + Make / C External Dependencies (links to list) Maven controlled dependencies (no ETICS dependencies) SL4 Make / C https://twiki.cern.ch/twiki/ bin/view/emi/jobmanpt INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 25 / 41

CREAM UNICORE CE ARC* Client UCC Client DPM dcache Server dcache Client StoRM UNICORE data wms_b_3_3) emi.cream-ce (current conf: emi-creamce_b_1_13) emi.unicore.unico rex (current conf: emi.unicore.unico rex.unicoreunicorex_b_tru NK) Missing, but expected emi.unicore.ucc (emi.unicore.ucc.e mi-unicoreucc_b_trunk) emi.lcgdm (many singular ETICS components) emi.dcache.server (emi-dcacheserver_r_head) emi.dcache.srmcli ent (mi-dcachesrmclient_r_he AD) emi.storm (emistormbackend_r_1_6) emi.unicore.unico rex (current conf: emi.unicore.unico rex.unicoreunicorex_b_tru NK) ExternalDeps SL5, Deb5 Maven / Java https://twiki.cern.ch/twiki/ bin/view/emi/jobmanpt ExternalDeps All platforms supporting Java. Current target platforms for EMI-1: Deb5 (64bit), SL5 (64bit) Missing, but expected All platforms supporting Java. Current target platforms for EMI-1: Deb5 (64bit), SL5 (64bit) missing (given later) Maven / Java Maven controlled dependencies (no ETICS dependencies) Missing, but expected Maven / Ant / Java Missing, but expected Missing, but expected Maven controlled dependencies (no ETICS dependencies) https://twiki.cern.ch/twiki/ bin/view/egee/dmcomp onentslist#dependencies_ List SL5 Ant / Java Missing, but expected SL5 Ant / Java Missing, but expected SL5 Ant / Java https://twiki.cern.ch/twiki/ bin/view/emi/stormpte xternaldeps All platforms supporting Java. Current target platforms for EMI-1: Deb5 (64bit), SL5 (64bit) Maven / Java Maven controlled dependencies (no ETICS dependencies) 6.2.2 Software Test Environment This section describes the specific environment where testing activities must take place. The environment for each product is listed in the table below. In particular it describes: a. The ETICS project configurations to be used. b. The target platforms. For EMI-1 SL5 is the mandatory platform. Probably Debian 5 will also be target platform. c. The reference testbed configuration to be used and any required endpoint as far as known from the EMI testbed activity [R2] INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 26 / 41

d. Specific test formats or test tools to be used (possibly per platform) e. Specific version of external software packages to be used (possibly per platform) Product ETICS Component/ Configurati on Target Platforms A-Rex emi.arc.arc1 SL5, (Deb5) WMS CREAM UNICO RE CE ARC* Client UCC Client DPM emi.wms (current conf: emiwms_b_3_3 ) emi.creamce (current conf: emicreamce_b_1_13) emi.unicore. unicorex (emi.unicore.unicorex.uni coreunicorex_b_ TRUNK) missing emi.unicore. ucc (emi.unicore.ucc.emiunicoreucc_b_tru NK) emi.lcgdm (many Testbed OS: SLC5.3 x86, Debian Lenny x8 SL5, (Deb5) Testbed OS: SL4 SL5, (Deb5) Testbed OS: SLC4 x86_64, SL5 x86 SL5, (Deb5) Testbed OS: opensuse 11.3 SL5, (Deb5), Testbed OS: Ubuntu Hardy i386 SL5, (Deb5) Testbed OS: opensuse 11.3 Testbed availability Available Version 1.1 Available (3 instances) Version glite 3.1 Available Production Version (LSF) 3.2; 3.1.24 Production Available 6.3.1 Production Available Release 0.8.2 Production Available 6.3.1 Production Test formats/tools n/a In development Planned for Integration tests: Unit tests Planned for compliance tests: GSTAT [R20] n/a In development Planned for Integration tests: Unit tests Planned for compliance tests: GSTAT [R20] n/a In development Planned for Integration tests: Unit tests Planned for compliance tests: GSTAT [R20] n/a In development Planned for Integration tests: Unit tests Planned for compliance tests: GSTAT [R20] n/a In development Planned for Integration tests: Unit tests Planned for compliance tests: GSTAT [R20] n/a In development Planned for Integration tests: Unit tests Planned for compliance tests: GSTAT [R20] External Software Packages Maven controlled dependencies (no ETICS dependencies) https://twiki.cern.ch/t wiki/bin/view/emi/j obmanptexternalde ps https://twiki.cern.ch/t wiki/bin/view/emi/j obmanptexternalde ps Maven controlled dependencies (no ETICS dependencies) Missing, but expected Maven controlled dependencies (no ETICS dependencies) SL5, Available n/a In development https://twiki.cern.ch/t wiki/bin/view/egee INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 27 / 41

singular ETICS components) (Deb5) Testbed OS: SLC4.8 x86 Production 3.2.5 Planned for Integration tests: Unit tests /DMComponentsList #Dependencies_List dcache Server emi.dcache.s erver (emidcacheserver_r_h EAD) SL5, (Deb5) Testbed OS: SL5 Available Version A + B n/a In development Planned for Integration tests: Unit tests Missing, but expected dcache Client emi.dcache.s rmclient (midcachesrmclient_r _HEAD) SL5, (Deb5) Testbed OS: SL5 Available Version A + B n/a In development Planned for Integration tests: Unit tests Missing, but expected StoRM emi.storm (emi-stormbackend_r_ 1_6_0_5) SL5, (Deb5) Testbed OS: SL5 Available 3.1.0-0_ig50_sl4 Production Version n/a In development Planned for Integration tests: Unit tests https://twiki.cern.ch/t wiki/bin/view/emi/s tormptexternaldep s UNICO RE data emi.unicore. unicorex (current conf: unicoreunicorex_b_ TRUNK) SL5, (Deb5) Testbed OS: opensuse 11.3 Available 6.3.1 Production n/a In development Planned for Integration tests: Unit tests Maven controlled dependencies (no ETICS dependencies) 6.2.3 Software Integration Tasks Tracking This section describes the change management tracker instance to be used to track EMI-1 integration tasks. The integration and testing tasks are tracked at JRA1.7 [R1] and the superior JRA1 Wiki page [R3]. In particular all testing strategies can be found under [R3] since they are related to the general development of the Technical Objectives in JRA1. An overview of the state of the art integration and interoperability statuses is available under the JRA1.7 page. There will be also a table available which summarizes all integration and testing tasks with links to external reports in one piece of information. Columns in this table will indicate and track the current status of the integration testing. 6.3 INTEGRATION, INTEROPERABILTY AND STANDARD COMPLIANCE TASKS This section describes the expected input and output of the testing activity for each Product within the scope of the EMI-1 Integration Plans. The section is organized in subsections, one subsection for each Product. Each subsection contains a description of the integration, interoperability and standard compliance test cases and their implementation. If the information is available in external documents, it is added by reference. If information is not available from the PT, it marked as expected, but missing. If it is not applicable in general or in this development phase, it is marked as n/a (Not Applicable). INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 28 / 41

6.3.1 ARC Computing Element (A-REX) This ARC computing element (CE) implements job execution functions used for a large variety of computational resources. The following technical objectives are addressed by this component also providing insights in the adopted test plans and the expected outcome of the testing activities. Constraints are reported where given. Technical Objectives Compute - Glue 2.0 support in job management services and client tools Test Plans Expected Outcome Constraints RC Date Integration: Testing the integration of GLUE 2.0 in A-REX Interoperability/Standard Compliance: Testing compliance with other GLUE 2.0 enabled compute components A-REX component is able to expose GLUE2 compliant pieces of information about computational resources. All affected EMI compute components are compliant with GLUE 2.0 and will be interoperable in respect to this schema. n/a 2011-02- 28 6.3.1.1 Integration testing The testing strategy for the GLUE 2.0 support for EMI compute elements is outlined at [R17]. The integration testing strategy is still under development, and may be refined until EMI-1. The expected availability of the test implementations is: 2011-01-31 The expected deployment within the EMI testbed is: 2011-01-31 6.3.1.2 Standard compliance testing The EMI compute elements which are supporting GLUE 2.0 should be interoperable in regard to this information schema. The standard compliant testing for GLUE 2.0 in regard to compute elements is still in evaluation. It is planned to create a test suite with using the GSTAT tool. 6.3.2 WMS The WMS receives requests for a job execution from a client, finds the required appropriate resources, then dispatches and follows them until completion. The following technical objectives are addressed by this component also providing insights in the adopted test plans and the expected outcome of the testing activities. Constraints are reported where given. Technical Objectives Compute - Glue 2.0 support in job management services and client tools Test Plans Expected Outcome Constraints RC Date Integration: Testing the integration of GLUE 2.0 in WMS Interoperability/Standard Compliance: Testing WMS component is able to expose GLUE2 compliant pieces of information about computational resources. All affected EMI compute components are compliant n/a 2011-02- 28 INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 29 / 41

compliance with other GLUE 2.0 enabled compute components with GLUE 2.0 and will be interoperable in respect to this schema. 6.3.2.1 Integration testing The testing strategy for the GLUE 2.0 support for EMI compute elements is outlined at [R17]. The integration testing strategy is still under development, and may be refined until EMI-1. The expected availability of the test implementations is: 2011-01-31 The expected deployment within the EMI testbed is: 2011-01-31. 6.3.2.2 Standard compliance testing The EMI compute elements which are supporting GLUE 2.0 should be interoperable in regard to this information schema. The standard compliant testing for GLUE 2.0 in regard to compute elements is still in evaluation. It is planned to create a test suite with using the GSTAT tool. 6.3.3 CREAM This service is a simple, lightweight service for job management operations and represents the CE of glite. The following technical objectives are addressed by this component also providing insights in the adopted test plans and the expected outcome of the testing activities. Constraints are reported where given. Technical Objectives Compute - Glue 2.0 support in job management services and client tools Test Plans Expected Outcome Constraints RC Date Integration: Testing the integration of GLUE 2.0 in CREAM Interoperability/Standard Compliance: Testing compliance with other GLUE 2.0 enabled compute components CREAM component is able to expose GLUE2 compliant pieces of information about computational resources. All affected EMI compute components are compliant with GLUE 2.0 and will be interoperable in respect to this schema. Dynamic GLUE2 information might be experimental (info provider problem) 2011-02- 28 6.3.3.1 Integration testing The testing strategy for the GLUE 2.0 support for EMI compute elements is outlined at [R17]. The integration testing strategy is still under development, and may be refined until EMI 1. The expected availability of the test implementations is: 2011-01-31 The expected deployment within the EMI testbed is: 2011-01-31 INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 30 / 41

6.3.3.2 Standard compliance testing The EMI compute elements which are supporting GLUE 2.0 should be interoperable in regard to this information schema. The standard compliant testing for GLUE 2.0 in regard to compute elements is still in evaluation. It is planned to create a test suite with using the GSTAT tool. 6.3.4 UNICORE compute elements The UNICORE compute services provide several Web services that can be used for computational job submission and management including data-staging. It consists of the following components: U.TSI, U.XNJS, WSFRLite, and another internal component that is the common information provider (CIP). The following technical objectives are addressed by this component also providing insights in the adopted test plans and the expected outcome of the testing activities. Constraints are reported where given. Technical Objectives Compute - Glue 2.0 support in job management services and client tools Test Plans Expected Outcome Constraints RC Date Integration: Testing the integration of GLUE 2.0 in UNICORE compute elements. Interoperability/Standard Compliance: Testing compliance with other GLUE 2.0 enabled compute components. UNICORE component is able to expose GLUE2 compliant pieces of information about computational resources. All affected EMI compute components are compliant with GLUE 2.0 and will be interoperable in respect to this schema. n/a 2011-02- 28 6.3.4.1 Integration testing The testing strategy for the GLUE 2.0 support for EMI compute elements is outlined at [R17]. The integration testing strategy is still under development, and may be refined until EMI-1. The expected availability of the test implementations is: 2011-01-31 The expected deployment within the EMI testbed is: 2011-01-31 6.3.4.2 Standard compliance testing The EMI compute elements which are supporting GLUE 2.0 should be interoperable in regard to this information schema. The standard compliant testing for GLUE 2.0 in regard to compute elements is still in evaluation. It is planned to create a test suite with using the GSTAT tool. 6.3.5 ARC* Client This component is a client solution capable of working with the ARC computing Element and several storage elements. The libarclient is one library integrated within the component. The following technical objectives are addressed by this component also providing insights in the adopted test plans and the expected outcome of the testing activities. Constraints are reported where given. INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 31 / 41

Technical Objectives Compute - Glue 2.0 support in job management services and client tools Test Plans Expected Outcome Constraints RC Date Integration: Testing the integration of GLUE 2.0 in ARC* client compute elements. Interoperability/Standard Compliance: Testing compliance with other GLUE 2.0 enabled compute components ARC* client components are able to expose GLUE2 compliant pieces of information about computational resources. All affected EMI compute components are compliant with GLUE 2.0 and will be interoperable in respect to this schema. n/a 2011-02- 28 6.3.5.1 Integration testing The testing strategy for the GLUE 2.0 support for EMI compute elements is outlined at [R17]. The integration testing strategy is still under development, and may be refined until EMI-1. The expected availability of the test implementations is: 2011-01-31 The expected deployment within the EMI testbed is: 2011-01-31 6.3.5.2 Standard compliance testing The EMI compute elements which are supporting GLUE 2.0 should be interoperable in regard to this information schema. The standard compliant testing for GLUE 2.0 in regard to compute elements is still in evaluation. It is planned to create a test suite with using the GSTAT tool. 6.3.6 UCC Client The UCC is the command-line client of the UNICORE technology. The following technical objectives are addressed by this component also providing insights in the adopted test plans and the expected outcome of the testing activities. Constraints are reported where given. Technical Objectives Compute - Glue 2.0 support in job management services and client tools Test Plans Expected Outcome Constraints RC Date Integration: Testing the integration of GLUE 2.0 in UCC client. Interoperability/Standard Compliance: Testing compliance with other GLUE 2.0 enabled compute components The UCC component is able to expose GLUE2 compliant pieces of information about computational resources. All affected EMI compute components are compliant with GLUE 2.0 and will be interoperable in respect to this schema. n/a 2011-02- 28 6.3.6.1 Integration testing The testing strategy for the GLUE 2.0 support for EMI compute elements is outlined at [R17]. The integration testing strategy is still under development, and may be refined until EMI-1. INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 32 / 41

The expected availability of the test implementations is: 2011-01-31 The expected deployment within the EMI testbed is: 2011-01-31 6.3.6.2 Standard compliance testing The EMI compute elements which are supporting GLUE 2.0 should be interoperable in regard to this information schema. The standard compliant testing for GLUE 2.0 in regard to compute elements is still in evaluation. It is planned to create a test suite with using the GSTAT tool. 6.3.7 DPM The Disk Pool Manager (DPM) is a lightweight solution for disk storage management (ease of installation, configuration, low effort of maintenance), while providing all required functionality for a grid storage solution (support for multiple disk server nodes, different space types, multiple file replicas in disk pools, etc.) The following technical objectives are addressed by this component also providing insights in the adopted test plans and the expected outcome of the testing activities. Constraints are reported where given. Technical Objectives Data 1 - All storage elements publishing initial GLUE 2.0 storage information. Data 3 - All storage elements offering support for the http(s) protocol. Data 4 - All storage elements offering support for the "file://" access protocol. Test Plans Expected Outcome Constraints RC Date Integration: Testing the integration of GLUE 2.0 in DPM. Interoperability/Standard Compliance: Testing compliance with other GLUE 2.0 enabled data components Integration: Testing the integration of the HTTPS support in DPM. Interoperability/Standard Compliance: Testing compliance with other HTTPS enabled data components Integration: Testing the integration of the file protocol in DPM. Interoperability/Standard Compliance: Testing compliance with other file enabled data components DPM component is able to expose GLUE2 compliant pieces of information about storage resources. All affected EMI data components are compliant with GLUE 2.0 and will be interoperable in respect to this schema. DPM component should be able to use HTTP(S) All affected EMI data components are compliant with HTTPS and will be interoperable in respect to this protocol. DPM component will be able to support the file:// protocol via NFS4.1 All affected EMI data components are compliant with the file protocol and will be interoperable in respect to this protocol. GLUE2 support is experimental while GLUE1.3 remains production 2011-02- 28 n/a 2011-02- 28 Experimental 2011-02- 28 INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 33 / 41

6.3.7.1 Integration testing The testing strategy for the GLUE 2.0 support of EMI data elements is outlined at [R17]. The integration testing strategy is still under development, and may be refined until EMI-1. The testing strategy for the HTTPS support of EMI data elements is outlined at [R18]. The integration testing strategy is still under development, and may be refined until EMI-1. The testing strategy for the file protocol support of EMI data elements is outlined at [R19]. The integration testing strategy is still under development, and may be refined until EMI-1. The expected availability of the test implementations is: 2011-01-31 The expected deployment within the EMI testbed is: 2011-01-31 6.3.7.2 Standard compliance testing The EMI data elements which are supporting GLUE 2.0, HTTPS, and the file protocol should be interoperable in regard to these standards. The standard compliant testing for these schemas and protocols in regard to data elements is still in evaluation. 6.3.8 dcache System for storing and retrieving huge amounts of data, distributed among a large number of heterogeneous server nodes, under a single virtual file system tree with a variety of standard access methods. The following technical objectives are addressed by this component also providing insights in the adopted test plans and the expected outcome of the testing activities. Constraints are reported where given. Technical Objectives Data 1 - All storage elements publishing initial GLUE 2.0 storage information. Data 2 - Using https instead of httpg for the SRM protocol as a prototype implementation in one storage element and client (library). Data 3 - All storage elements Test Plans Expected Outcome Constraints RC Date Integration: Testing the integration of GLUE 2.0 in dcache. Interoperability/Standard Compliance: Testing compliance with other GLUE 2.0 enabled data components Integration: Testing the integration of https in dcache. Integration: Testing the integration of the dcache component is able to expose GLUE2 compliant pieces of information about storage resources. All affected EMI data components are compliant with GLUE 2.0 and will be interoperable in respect to this schema. dcache server and client can work together without the use of httpg and the GSI. n/a 2011-02- 28 n/a 2011-02- 28 dcache component should n/a 2011-02- INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 34 / 41

offering support for the http(s) protocol. Data 4 - All storage elements offering support for the "file://" access protocol. HTTPS support in dcache. Interoperability/Standard Compliance: Testing compliance with other HTTP(S) enabled data components Integration: Testing the integration of the file protocol in dcache. Interoperability/Standard Compliance: Testing compliance with other file enabled data components be able to use HTTP(S) 28 All affected EMI data components are compliant with HTTP(S) and will be interoperable in respect to this protocol. dcache component will be able to support the file:// protocol via NFS4.1 All affected EMI data components are compliant with the file protocol and will be interoperable in respect to this protocol. Experiment al 2011-02- 28 6.3.8.1 Integration testing The testing strategy for the GLUE 2.0 support of EMI data elements is outlined at [R17]. The integration testing strategy is still under development, and may be refined until EMI-1. The testing strategy for replacing httpg in SRM protocol is included in [R18]. dcache already performs tests with a SRM compliance test suite on a regular basis. Here it makes sense to broaden the usage of this test suite to other SRM-compliant systems such as DPM and StoRM. The integration testing strategy is still under development, and may be refined until EMI-1. The testing strategy for the HTTP(S) support of EMI data elements is outlined at [R18]. The integration testing strategy is still under development, and may be refined until EMI-1. The testing strategy for the file protocol support of EMI data elements is outlined at [R19]. The integration testing strategy is still under development, and may be refined until EMI-1. The expected availability of the test implementations is: 2011-01-31 The expected deployment within the EMI testbed is: 2011-01-31 6.3.8.2 Standard compliance testing The EMI data elements which are supporting GLUE 2.0, HTTP(S) in SRM protocol, HTTP(S) in general and the file protocol should be interoperable in regard to these standards. The standard compliant testing for these schemas and protocols in regard to data elements is still in evaluation. 6.3.9 StoRM StoRM is a service compliant with the open standard Storage Resource Manager (SRM) v2.2 being able to manage any POSIX file systems supporting the Access Control List (ACL) mechanism, including high performance storage systems based on cluster file system (such as GPFS from IBM and Lustre from Oracle/Sun Microsystems). The following technical objectives are addressed by this component also providing insights in the adopted test plans and the expected outcome of the testing activities. Constraints are reported where given. INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 35 / 41

Technical Objectives Data 1 - All storage elements publishing initial GLUE 2.0 storage information. Data 3 - All storage elements offering support for the http(s) protocol. Data 4 - All storage elements offering support for the "file://" access protocol. Test Plans Expected Outcome Constraints RC Date Integration: Testing the integration of GLUE 2.0 in StoRM. Interoperability/Standard Compliance: Testing compliance with other GLUE 2.0 enabled data components Integration: Testing the integration of the HTTPS support in StoRM. Interoperability/Standard Compliance: Testing compliance with other HTTPS enabled data components Integration: Testing the integration of the file protocol in StoRM. Interoperability/Standard Compliance: Testing compliance with other file enabled data components StoRM component is able to expose GLUE2 compliant pieces of information about storage resources. All affected EMI data components are compliant with GLUE 2.0 and will be interoperable in respect to this schema. StoRM component should be able to use HTTP(S) All affected EMI data components are compliant with HTTPS and will be interoperable in respect to this protocol. StoRM component will be able to support the file:// protocol via GPFS and Lustre. All affected EMI data components are compliant with the file protocol and will be interoperable in respect to this protocol. GLUE2 support is experimental while GLUE1.3 remains production 2011-02- 28 n/a 2011-02- 28 Experimental 2011-02- 28 6.3.9.1 Integration testing The testing strategy for the GLUE 2.0 support of EMI data elements is outlined at [R17]. The integration testing strategy is still under development, and may be refined until EMI-1. The testing strategy for the HTTPS support of EMI data elements is outlined at [R18]. The integration testing strategy is still under development, and may be refined until EMI-1. The testing strategy for the file protocol support of EMI data elements is outlined at [R19]. The integration testing strategy is still under development, and may be refined until EMI-1. The expected availability of the test implementations is: 2011-01-31 The expected deployment within the EMI testbed is: 2011-01-31 6.3.9.2 Standard compliance testing The EMI data elements which are supporting GLUE 2.0, HTTPS, and the file protocol should be interoperable in regard to these standards. The standard compliant testing for these schemas and protocols in regard to data elements is still in evaluation. For the GLUE 2.0 compliance it is planned to create a test suite with using the GSTAT tool. INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 36 / 41

6.3.10 UNICORE data The UNICORE Data component stands for a flexible framework that is able to integrate various different storage implementations and concepts. The following technical objectives are addressed by this component also providing insights in the adopted test plans and the expected outcome of the testing activities. Constraints are reported where given. Technical Objectives Data 1 - All storage elements publishing initial GLUE 2.0 storage information. Data 5 - File Catalogue Access from UNICORE Test Plans Expected Outcome Constraints RC Date Integration: Testing the integration of GLUE 2.0 in StoRM. Interoperability/Standard Compliance: Testing compliance with other GLUE 2.0 enabled data components Integration: Testing the integration of the File Catalogue Access. The UNICORE data component is able to expose GLUE2 compliant pieces of information about storage resources. All affected EMI data components are compliant with GLUE 2.0 and will be interoperable in respect to this schema. A new UNICORE-Data component element will be developed that can access file catalogues GLUE2 support is experimental while GLUE1.3 remains production 2011-02- 28 n/a 2011-02- 28 6.3.10.1 Integration testing The testing strategy for the GLUE 2.0 support of EMI data elements is outlined at [R17]. The integration testing strategy is still under development, and may be refined until EMI-1. The testing of the File Catalogue Access will likely be performed just by unit tests which will be outlined in the components Validation and Certification Plan. The development of a unit test suite is still in development like the functionality itself. The expected availability of the test implementations is: 2011-01-31 The expected deployment within the EMI testbed is: 2011-01-31 6.3.10.2 Standard compliance testing No standard compliance testing scheduled for EMI-1. 6.4 OTHER SOFTWARE DEVELOPMENT ACTIVITIES 6.4.1 Risk Management This section describes any identified risks and the proposed corrective actions. Any risks associated with the software development in regard to Technical Objectives are described in the SDTP [R2]. Concerning integration tasks and testing the main risk is that required test suites are not in place at the deadline for EMI-1 testing phase that is the 28 th of February 2011. Therefore, JRA1 will continuously gathering and monitoring the testing activities in collaboration with the appropriate Product Teams. Especially, the Validation and Certification Plan of the affected components should be available INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 37 / 41

before the deadline exceeds. In case the test suites and plans are after all not available JRA1.7 will escalate the issue to JRA1 task leader who will address it to the technical area leaders. Another risk would be that the test suites and plans are available but the respective hard- and software resources at the EMI testbed are not yet in place. JRA1.7 will avoid that issue by identifying the needed resources for the integration tests until the end of February and checking their availability in the testbed. In general, installing and configuring the required resources shouldn t be a big obstacle, since the Product Teams have usually co-workers in their teams who are also responsible for the respective parts if the EMI testbed. A minor risk is a bad flow of information between Product Teams and JRA1. Also JRA1.7 depends on input about the current status of development, schedules, or arbitrary drawbacks in the software engineering process. JRA1.7 tries to avoid this issue by requesting information from the product teams in an early stage of EMI 1. Moreover, ARC has designated a dedicated person who is coordinating all tasks in EMI concerning integration and testing. INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 38 / 41

7 SCHEDULES AND ACTIVITY NETWORK This section provides a GANTT-based overview of the components which has been taken from the JRA1 EMI-1 Development and Test Plan [R10]. This chart shows software developments relevant for the EMI 1 release. INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 39 / 41

Figure 3 describes the sub-tasks of JRA1.7 and its relationships to other sub-tasks as well as other WP activities within the EMI project in an activity network. All these sub-tasks essentially collaborate with the JRA1.8 quality control task that in turn provides pieces of information about the test availability and relevant software quality control metrics. The collaboration with task JRA1.8 ensures that JRA1.7 is in-line with quality requirements monitored by the SA2 activity. The direct interaction of JRA1.7 with SA2 is ensured via the extensive use of the EMI testbed. Figure 3: Activity network INFSO-RI-261611 2010 Members of EMI collaboration PUBLIC 40 / 41