Euro-BioImaging Preparatory Phase II Project

Transcription

1 Euro-BioImaging Preparatory Phase II Project Testing of the basic framework of EuBI online user access portal, with the most central links and functions included, and improvements and corrections implemented based on the testing Project N Project Title Euro-BioImaging Preparatory Phase II Project Acronym EuBI PPII Associated Work Package WP5 Associated Task 5.1, 5.2, 5.3 and 5.4 Lead Beneficiary (short name) ABO Nature Report Dissemination Level Public Estimated Delivery Date (Grant Agreement, Annex I) 30/09/2017 Actual Delivery Date 23/12/2017 Task leader Pasi Kankaanpää (WP Manager) Contributors John Eriksson (WP5 Lead) Athresh Shigaval (IT planner) Funded by the Horizon 2020 Framework Program of the European Union Testing of the basic framework of EuBI online user access portal, with the most

2 Abstract Work Package 5 is developing the Web Access Portal (WAP), a dynamic web service that will be the virtual access point to all Euro-BioImaging (EuBI) research infrastructure services. The first public version of the WAP, a simpler interim version created to enable the EuBI Interim Operation, was launched in May The basic WAP workflow and structure have been thoroughly tested both before and during the EuBI Interim Operation, and especially the access workflow functionality has been fine-tuned in the process. The Interim Operation has been invaluable for gathering information and experience needed to develop the WAP to serve the upcoming EuBI ERIC. The most central links and functions are already in use in the current version of the interim WAP, including links to EuBI Node Candidates and BioSharing, and partial prototype integration of the EuBI Image Data Resource. However, many of the features are also under further development, and especially the main access workflow functionality is being re-built to be based on the ARIA portal of INSTRUCT, through a developer level collaboration agreement. The basic ARIA framework and the underlying database have been created, and initial testing conducted. Currently, the ARIA framework is being tested further, corrections being made, and new functionality for instance for user identification and authorization and training being added. The WAP development is a continuum, where different building blocks are developed partly separately with different schedules and technical solutions that also change over time. WP5 collaborates closely with WP3, WP6, WP7, WP8 and WP9, and the WAP is also developed to be compatible with other relevant European infrastructures. While the work of WP5 has partly changed from what was initially foreseen, it has progressed well, and work to create the final merged portal of the WAP and the EuBI website is progressing towards a launch in Table of Contents 1. Introduction Page 3 2. Testing of the basic framework for Interim Operation Page 4 3. Creating and testing the ARIA-based framework Page 4 4. Most central links and functions Page 9 5. Development continuum Page 12 Testing of the basic framework of EuBI online user access portal, with the most 2

3 1. Introduction The EuBI Preparatory Phase II (PP II) Work Package 5 (WP5) is developing the Web Access Portal (WAP), which is the virtual entry point to the Euro-BioImaging (EuBI) research infrastructure services. The WAP is designed to be a dynamic web service that includes a wide range of tools and serves as the gateway to e.g. imaging technology services, data services (developed by WP6), and training services (designed by WP7). Several factors have shaped the development of the WAP since the PP II project began. For instance, instead of developing the WAP through a long offline process and launching it only towards the end of the project, as initially planned, WP5 opted to develop the WAP partly publically from early on, and in a more dynamic fashion, with varying technical solutions. This approach enabled EuBI to start its Interim Operation already in mid-2016, and it allows the WAP services to be developed more interactively with the users and service providers that utilize them in their everyday life. The Interim Operation has also provided a valuable opportunity to test and tune the details of the EuBI access workflow, and quite a lot of work was needed to make the workflow functional in practise. Another significant factor affecting the WAP development was the thorough testing and evaluation of different technical solutions for WAP development, presented to the PP II Executive Project Management (EPM), which decided in December 2016 that EuBI will collaborate with the ARIA portal development team of Instruct ERIC (see D5.2). In order to benefit from synergies between the two infrastructures and develop more comprehensive services in a more costefficient manner, WP5 and Instruct negotiated a developer-level collaboration agreement in the end of However, practical work could only be started in the summer of 2017, because the contract signing was delayed at Instruct, and because WP5 had to wait for the release of ARIA version 2, which Instruct had scheduled for early 2017, but then postponed until late summer. This deliverable describes the testing and further development of the basic WAP framework for Interim Operation, and the most central links and functions of the interim WAP version currently in use. Since the technical framework of this interim WAP has been designed specifically for Interim Operation, this deliverable also includes a report on the construction and testing of the basic framework that will be in use after Interim Operation. This framework is largely based on an EuBI version of ARIA that WP5 is creating. Since it was not possible to start the work with ARIA until much later than scheduled, WP5 and EPM decided to postpone this deliverable so that initial testing of the ARIA-based framework could be included. Overall, the WAP is developed with strategy that enables accommodation of changes in service requirements, development needs and external conditions. This is important when developing the face of a dynamic infrastructure such as EuBI, which is still undergoing construction, with many practical and operational details still being defined. The WAP development strategy also allows for compatibility and collaboration with other research infrastructures, as exemplified by the ARIA collaboration, and collaboration with Elixir in developing user authentication and authorization solutions for EuBI. Testing of the basic framework of EuBI online user access portal, with the most 3

4 While WP5 now looks somewhat different from the plans of the initial PP II grant agreement, it has been progressing well with respect to its goal of providing advanced online services to EuBI. 2. Testing of the basic framework for Interim Operation The first public version of the EuBI WAP, the so-called interim Web Access Portal, was launched already in May Before this, all its central features were thoroughly tested, especially the user access application workflow, which was checked with isolated tests as well as numerous dummy applications that were run through the system together with the Hub Candidates Finland, EMBL and Italy. During this testing period, as well as after it, when the EuBI Interim Operation has already been ongoing and actual user applications coming in, the WAP online access functionality has been constantly tested and evaluated, and discussed in weekly conference calls among the Hub Candidates. Early on in the process it became evident that many details of the EUBI access workflow and the terminology used to describe it and the EuBI services, were not directly compatible with the online access procedures needed for the WAP and its database. Significant work was therefore done to adapt and tune the workflow, database and terminology, so that a functional experience could be provided to users, reviewers and EuBI staff, based on the EuBI access procedure developed and tested in PPI. The most thorough testing of the EuBI access workflow functionality, the interim WAP, and its basic framework has been the Interim Operation itself. At the time of writing this, 70 user applications have been submitted, coming from 23 countries in Europe, India, USA and Canada, and seeking access to 16 different imaging centers, in more than 67% of the cases across country borders. This use of the interim WAP and the feedback received from it have been invaluable in developing the WAP, and continue to be so, as the Interim Operation continues. 3. Creating and testing the ARIA-based framework Creating the ARIA-based EuBI WAP framework was started by setting up a separate instance of ARIA for EuBI. First, three high-reliability servers were set up in Turku for different versions of the development work. One of them was set up to run ARIA version 1, another would run ARIA version 2 somewhat later, and the third one would be used for constructing the eventual production level EuBI portal. Detailed work was then started to go through the ARIA source code of the different versions, map the functions of different parts of the code, and connect them to EuBI functionality on a conceptual level. This was rather time consuming, as there is no documentation on the ARIA code, and the code itself does not include many explanatory comments. Once the ARIA code was deciphered, work started on defining the basic user access workflow for EuBI. The Instruct workflow is quite different from the EuBI workflow. For instance, it is based on concepts such as platforms, services, centers and machines, none of which are directly applicable to the EuBI workflow, which is based on nodes and technologies. WP5 re-defined the entire ARIA workflow so that it fit the EuBI workflow established during Interim Operation, and also redefined the databases so that EuBI data could be stored in a similar manner as during Interim Operation. The workflow was also designed to accommodate the tripartite Hub structure of EuBI Testing of the basic framework of EuBI online user access portal, with the most 4

5 and the necessary reviewing and moderation steps, which are somewhat different from those of Instruct/ARIA. Various details of the workflow also had to be re-defined, compared to the worflow used during Interim Operation, because the basic ARIA framework is very different from the framework used on the interim WAP. Once the workflow was defined in detail, work was started to make the necessary changes to the ARIA source code. The Turku team hired additional staff to help in this process, which was rather substantial: more or less every part of the ARIA workflow and related functionality had to be modified, and since the ARIA version 2 was released in the middle of this work, several bugs also in the original ARIA version 2 had to be fixed. Daily online communication and video meetings every second week with the ARIA team greatly facilitated the work. Additional meetings with the ARIA team were arranged when needed, including a physical meeting in Amsterdam in July In addition to creating the ARIA-based EuBI workflow, the EuBI Hub Candidates together identified aspects of ARIA that should be changed for EuBI use. A primary aspect was the complexity of ARIA, which was seen as reducing usability. The Turku team therefore cleaned up the system, removing unnecessary items, streamlining much of the functionality and improving on various details related to usability. The screen captures below (Figures 1-4) illustrate the ARIAbased EuBI framework in its current developmental stage. Testing of the basic framework of EuBI online user access portal, with the most 5

6 Figure 1. The front page of the test-version of the ARIA-based EuBI framework, which has been embedded to a copy of the landing page of the interim WAP. The ARIA-based functionality appears as a colorful toolbar at the top of the page, and allows browsing for locations and technologies and logging in and out. Once logged in, additional functionality appears, allowing for instance submitting and managing user access proposals. Testing of the basic framework of EuBI online user access portal, with the most 6

7 Figure 2. The access proposal list, which shows ongoing applications and what their status is, allows viewing them, continuing with an unfinished proposal or starting a new proposal. Testing of the basic framework of EuBI online user access portal, with the most 7

8 Figure 3. The first step of submitting a new proposal is selecting (if known) the desired location(s) and technologie(s), as shown here. This is followed by a form where the user fills in the details of the proposal. Proposal progress is shown on the right. Figure 4. An elemental part of the ARIA framework is the Admin panel. Here, users have different possibilities, and depending on their role in the system they can for instance moderate proposals, conduct scientific or technical reviews, or provide post-access feedback. Testing of the basic framework of EuBI online user access portal, with the most 8

9 When the ARIA basic access workflow framework was successfully running, testing was started. The first stage of testing was done by the WP5 team, using a temporary user authentication and authorization system created solely for testing purposes. As the final user authentication will be based on actual personal identities (this will be described in D5.4), any user testing the system at that point can only have a single role at any given time, e.g. user, hub staff, reviewer, node etc. In contrast, the temporary testing authentication system allowed a single tester to perform several roles and therefore efficiently test the system with dummy applications. Several dummy applications were run through the system by three different testers, bugs and improvement suggestions collected, and fixes made. After the testing it was concluded that the fundamental EuBI physical user access workflow now worked in the ARIA framework. It should be noted that while this is the core functionality of the WAP being developed, a lot of other functionality (node content management, communication tools, training functionality etc.) is also under construction. Some of this will be based on the now functional basic workflow, and will be reported on in upcoming deliverables. As the WAP is constantly being developed and modified, testing it is naturally also a continuous process, rather than a single step. Following the testing described above, a second and third stage of testing by the Hub candidate partners Finland, EMBL and Italy is currently ongoing. This will be reported on in more detail in D5.5. Despite the fact that developing the EuBI ARIA framework is resource-consuming and access to the ARIA source code was significantly delayed, the collaboration agreement with Instruct has already enabled the development of more advanced features and usability than would have been possible with the same resources without ARIA. 4. Most central links and functions The version of the interim WAP currently in use has several features and links, and while many are still under development, the most central ones have been tested and implemented, and are in daily use. This section provides a short overview of these. Node and technology finder With this tool it is possible to browse EuBI technologies by country, by Node Candidate or by technology. A full PDF list is also available, for e.g. conducting personalized searches. Each Node Candidate has its own WAP sub-page, with descriptions of offered technologies, specialties and expertise, additional services, contact information and a link to the Node Candidate s own web page (if available). Application submission and review system This is one of the most central tools of the WAP, as it allows users to submit applications, which then proceed to independent scientific reviews and a technical review by the Node Candidate, with the process being moderated by the EuBI Hub. Re-submission or submission of additional requested information is also possible. Detailed instructions for users, scientific reviewers and technical reviewers are provided. Testing of the basic framework of EuBI online user access portal, with the most 9

10 Reporting and feedback system Tools are included for giving both mandatory and optional feedback for users, Node Candidates and scientific reviewer candidates. Users are required to give feedback twice, first after completed access and again later, when the project is completed e.g. in the form of a scientific publication. Node Candidates are required to give feedback after completed user access. The feedback system is used for quality control and further development of the WAP and EuBI services. The feedback received so far has been positive. Image Data Resource (IDR) The IDR, developed by WP6, has been partially integrated into the WAP. Several sample datasets, showcasing imaging technologies and research related to EuBI services, are accessible directly in the WAP (Figure 5). This prototype integration, done together with WP6, will be expanded upon in the future, with the goal that for instance sample data produced by all or most EuBI technologies would be available to users in this way. Importantly, for accessing these integrated data sets, users are not required to access the IDR interface (although links for this are provided for those who so prefer), as the prototype integration enables viewing and interacting with sample IDR image data directly in the WAP using javascript embedded viewers. EuBI has started to assemble a catalogue of other public image data resources as well, and this catalogue, hosted by BioSharing, is also linked to the WAP. New technology identification An important mission of EuBI is to offer the newest imaging technologies to its users and to shorten the time from the invention of a new technology to it being available to European researchers. WP8 has developed these procedures, and the WAP already features a new technology reporting tool, which can be used by technology developers, technology providers or EuBI users alike. Proposals received have been relayed to WP8 for further processing. So far, 30 proposals suggesting about 21 different technologies have already been received. Document gallery This feature lists all the online forms used in the interim WAP, to enable users to familiarize themselves with them beforehand. Guidelines for accessing different EuBI technology groups are also provided. Testing of the basic framework of EuBI online user access portal, with the most 10

11 Figure 5. The current WAP IDR interface. Database While not visible to the users of the WAP, all information in the portal is stored in an underlying database (Figure 6), initially built with M.S. SQL Server with relational dependencies, and currently being translated to MySQL for use with the ARIA-based functionality. For instance all information of Node Candidates, imaging technologies, submitted applications and their processing, and received feedback is stored in the database, in a total of 14 different interlinked tables, which are regularly updated and maintained. The database can be queried for reporting purposes, such as when different EuBI usage statistics are required. Testing of the basic framework of EuBI online user access portal, with the most 11

12 Figure 6. Screen capture of the EuBI WAP database. Euro-BioImaging Access Made Easy -video While the EuBI access workflow has been developed to be as simple as possible, it is still a multistep procedure that is new to the European life science research community. Therefore, the WP5 team produced an instructional video, Euro-BioImaging Access Made Easy, as an in-kind contribution for Interim Operation. The video is designed to facilitate ease of use in a way that modern users might expect, and it supplements other general information on the WAP home page: Helpdesk The WAP includes a Helpdesk run by the EuBI Hub Seat Candidate in collaboration with the other Hub Candidates and WP5. The Helpdesk assists users when needed, takes care of relaying communications, receives feedback regarding the development of the WAP, and keeps the EuBI database up to date, especially while some of the more automated procedures are still under development. Even though the interim WAP has already been operational for some time with the most central links and functions, most of these functions and links are under constant development, and will evolve further for instance to be based on the ARIA backbone described above. New features are also constantly being developed, and not all functions or links are included yet. 5. Development continuum Implementing improvements and corrections based on testing, experience and user feedback, is naturally a continuously ongoing process. An important part of this work so far has been finetuning the access workflow functionality based on testing results and working together with the Node Candidates to keep their information on the WAP updated. Next, ARIA-based functionality Testing of the basic framework of EuBI online user access portal, with the most 12

13 will be tested further, at the same time as development of new functionality, especially for training, is ongoing. As described in D5.1, the WAP is developed with an agile development strategy and building blocks that can be defined, developed and operated fairly independently of each other. This development strategy has become more pronounced, when the WAP development has become more dynamic than initially foreseen. The WAP development is effectively a continuum, where different building blocks and their features are developed with different schedules and different technical solutions, that also change and evolve as the work progresses. The nature of the WAP development has resulted in some changes in the schedule and content of the WP5 work and deliverables, but mostly these do not cause significant negative effects, and many of them are beneficial for the overall goals of the project. The main focus of WP5 is to enable EuBI operation through the WAP. Currently, a functional interim Web Access Portal has been running already for more than a year, and EuBI is operational. Recently, WP5 has been concentrating on creating the best possible tools and solutions for a long term EuBI WAP. This long term solution will merge together the current interim WAP, the current ARIA-based development version of WAP functionality, and a new EuBI web site being created by WP5 to replace the current website. The work will be done in collaboration with professional graphics designers and web site programmers, with the aim to create a single, seamless user experience. The integration work has already been started, and will be described in more detail in D5.4. The integrated site is estimated to be released in the spring 2018 or at the latest when the Euro- BioImaging ERIC is launched, and projected to contain functionality for each of the building blocks initially outlined for the WAP in D5.1. Of course, developing and updating the WAP will continue also after PP II and after the EuBI ERIC is established. Testing of the basic framework of EuBI online user access portal, with the most 13