Hellenic Food Authority (EFET) - Greece

Transcription

1 EXTERNAL SCIENTIFIC REPORT Implementation of Electronic Transmission of Chemical Occurrence Data 1 (CFP/EFSA/DATEX2011/01) Prepared by Dimitrios KASKARIKAS, Leonidas PALILIS, Dimitra KARDASSI ABSTRACT Hellenic Food Authority (EFET) - Greece In order to provide accurate data for risk assessment, the Hellenic Food Authority (EFET) has implemented and tested an electronic system for the transmission of chemical occurrence data according to EFSA standards. The system for the transmission of data was developed under the grant offered by EFSA and is now in function, after having been tested successfully. This project has enabled EFET to develop procedures and tools to automate data transfer to EFSA according to the specific Standard Sample Description SSD. The present report describes the procedures for adapting our system to the EFSA s requirements whereas the challenges encountered during implementation of the standard model and possible limitations or developments are also examined. KEY WORDS Chemical Occurrence, Transmission, Data, Residues, Contaminants, Pesticides, Food, XML, SOAP, EFSA, EFET DISCLAIMER The present document has been produced and adopted by the bodies identified above as author(s). In accordance with Article 36 of Regulation (EC) No 178/2002, this task has been carried out exclusively by the author(s) in the context of a grant agreement between the European Food Safety Authority and the author(s). The present document is published complying with the transparency principle to which the Authority is subject. It cannot be considered as an output adopted by the Authority. The European Food Safety Authority reserves its rights, view and position as regards the issues addressed and the conclusions reached in the present document, without prejudice to the rights 1 Question No EFSA-Q Any enquiries related to this output should be addressed to dcm@efsa.europa.eu Suggested citation: Dimitrios KASKARIKAS, Leonidas PALILIS, Dimitra KARDASSI, Implementation of Electronic Transmission of Chemical Occurrence Data. Supporting Publications 2013:EN-471. [16 pp.]. Available online: European Food Safety Authority, 2013

2 SUMMARY The Hellenic Food Authority (EFET) has expanded its Central Information Management System (OPS), according to EFSA specifications, in order to be able to electronically transmit data that are periodically requested though relevant EFSA data calls. This project was developed as an extension to our existing infrastructure and a system that extracts, transforms and transmits the final data through the Data Collection Framework (DCF) to the EFSA database has been implemented in order to respond to EFSA calls for data. Furthermore we have also implemented the necessary infrastructure to receive acknowledgment returns via web services (which are currently still under development) and as soon as it is available to us we will be able to complete our testing. It is also important to stress that during the process development a lot of critical points in our chemical data collection were successfully identified and improved enabling us to optimize the overall quality of the data collection. Supporting publications 2013:EN-471 2

3 TABLE OF CONTENTS Abstract... 1 Summary... 2 Table of contents... 3 Background... 4 Terms of reference... 4 Acknowledgments... 4 Introduction and Objectives MATERIALS AND METHODS DATABASE, SOFTWARE AND TOOLS METHOD OF IMPLEMENTATION Project Tracking and Debugging Mapping Analysis Database and GUI modifications on OPS Migration Tasks Data Selection Data Transmission RESULTS Description of implementation Data Collection XML - Output Web Service Implementation Calling EFSA Web Service Response Messages from EFSA Challenges During Implementation CONCLUSIONS / Recommendations References Supporting publications 2013:EN Regulation (EC) No 178/2002, this task has been carried out exclusively by the author(s) in the context of a grant agreement between the European Food Safety Authority and the author(s). The present document is published complying with the transparency principle to which the Authority is subject. It cannot be considered as an output adopted by the Authority. The European Food Safety Authority reserves its rights, view and position as regards the issues addressed and the conclusions reached in the present document, without prejudice to the rights

4 BACKGROUND Data collection is one of the most important tasks of the European Food Safety Authority (EFSA) since it is fundamental for performing its core task which is the risk assessment. For this purpose, EFSA has to collect and to analyze an increasing volume of data collected from various sources and providers: National Competent Authorities, local authorities, laboratories, universities, and other organizations. The amount of incoming data is increasingly high, and the effort required to homogenizing and processing them is increasing accordingly. Being able to quickly assemble data is crucial for risk assessment and for doing it more efficiently, EFSA proposed a common standard for recording and reporting data, regarding chemical occurrence in food and feed the SSD (described in the document Guidance on Standard Sample Description ). Following this standard EFSA has launched this Article 36 project as an extension of the previous Article 36 projects to include additional countries, with the aim to have the Standard Sample Description (SSD) supported by as many EU countries as possible. TERMS OF REFERENCE The following are the deliverables of this project as previously agreed: Document D1 Data standardization (CFP/EFSA/DATEX/2011/01 Hellenic Food Authority/ EFET) Successful Data transmissions to EFSA for 2011 data. Document D3 Final Report ACKNOWLEDGMENTS This grant was awarded by EFSA to: Hellenic Food Authority (EFET) Kifisias 124 & Iatridou 2, 11526, Athens, Greece Grant title: Electronic Transmission of Chemical Occurrence Data Grant number: CFP/EFSA/DATEX2011/01 Supporting publications 2013:EN Regulation (EC) No 178/2002, this task has been carried out exclusively by the author(s) in the context of a grant agreement between the European Food Safety Authority and the author(s). The present document is published complying with the transparency principle to which the Authority is subject. It cannot be considered as an output adopted by the Authority. The European Food Safety Authority reserves its rights, view and position as regards the issues addressed and the conclusions reached in the present document, without prejudice to the rights

5 INTRODUCTION AND OBJECTIVES INTRODUCTION Data collection is an important task of EFSA and a fundamental component of risk assessment (Articles 22 and 23 of regulation EC No 178/2002). EFSA requested the Dietary and Chemical Monitoring Unit (DCM) Unit to harmonize the collection of data for presence of harmful chemical substances in food, feed and water. For this purpose, a group of experts developed the Guidance on Standard Sample Description for Food and Feed and the Guidance on Data Exchange. In order to implement the electronic transmission of data directly from the data provider databases, EFSA decided to allocate financial support to the Member States willing to test and use this system contributing to collect, at European level, standardized, coherent and complete data. In Greece, EFET according to its legislation (Presidential Decree No 223/2010 Organisation of Hellenic Food Authority, article 10) and being assigned as Central Competent Authority has the responsibility of collecting, processing and analysing the data originating from laboratory controls in order to allow the characterisation and monitoring of risks which have a direct or indirect impact on food. In order to assist EFSA s mission and to improve the quality of its data, EFET decided to implement and test an electronic system for the transmission of food contaminant data available in our database system to EFSA, according to the above mentioned EFSA standards. OBJECTIVES Our main objective was to develop new functionalities to our, relatively newly developed, Central Information Management System (OPS) in a way that allows the extraction, transformation and final transmission of the data to EFSA in an automated way. This had to be produced in a manner that would affect the current user data-entry process as less as possible. To accomplish this, some extra fields were added to our existing system that were not recorded so far, and also a number of changes according to EFSA SSD were incorporated to our system, resulting in a lot of migration tasks. The last ones were also some of the biggest challenges we encountered during the implementation of the project. No matter the difficulties we faced, it was very much acknowledged that the common system of EFSA for data reporting will help Supporting publications 2013:EN Regulation (EC) No 178/2002, this task has been carried out exclusively by the author(s) in the context of a grant agreement between the European Food Safety Authority and the author(s). The present document is published complying with the transparency principle to which the Authority is subject. It cannot be considered as an output adopted by the Authority. The European Food Safety Authority reserves its rights, view and position as regards the issues addressed and the conclusions reached in the present document, without prejudice to the rights

6 collecting and maintenance of data in a uniform way. In this context, the grant proposed by EFSA was a good opportunity for initiating a project aiming at expanding our Information System towards EFSA standards. We consider that our report mighty assists and could be a helpful guidance for member states that are currently developing such infrastructure. 1. MATERIALS AND METHODS The selection of products used was based on the underlying infrastructure so that the end result is as homogeneous as possible with the minimum interruption of our current operations. More specifically we used the following: 1.1. DATABASE, SOFTWARE AND TOOLS Data base system (servers): Oracle 10g, including APEX Data selection tool (client/server): Oracle Business Intelligence, APEX Xml processor: xmlpad Webservice: XML File Reception Web Service (1.0) Mantis Bugtracker Hyper-V Server For security reasons specific information about hardware, operating systems, and application servers are not published but are available on request METHOD OF IMPLEMENTATION The project was divided in individual tasks that are summed below: Project Tracking and Debugging For tracking and debugging the components of the project we setup and used a local VM server running Mantis Bug tracker, which helped significantly in facing related problems we met during the process and gave us a total view of the whole process. Supporting publications 2013:EN Regulation (EC) No 178/2002, this task has been carried out exclusively by the author(s) in the context of a grant agreement between the European Food Safety Authority and the author(s). The present document is published complying with the transparency principle to which the Authority is subject. It cannot be considered as an output adopted by the Authority. The European Food Safety Authority reserves its rights, view and position as regards the issues addressed and the conclusions reached in the present document, without prejudice to the rights

7 Mapping Analysis One of the most basic steps of the project was to analyse the SSD, provided by EFSA, and determine which of the requested data-fields could be mapped directly to EFET data, which needed to be added and which could not be directly mapped and needed special handling. The positive thing regarding this process was that most of our fields originated from the SSD which we took into account when we developed OPS. The most common mismatches we faced, considered some food categories that were unique for Greece and finally matched against higher level categories in terms of reporting. The details about this process have already been reported in our previously submitted document D1-Data Standardisation Database and GUI modifications on OPS After determining the extra data needed we had to add extra fields in our database tables and also add corresponding fields at the data-entry web-forms. Next step was the translation of the SSD catalogue tables in Greek (some tables fully translated; others partially). The main tables that were translated were: SAMPSTR, SRCTYP, PRODPAC, PRODTR, LABACC, VALTYP MDSTAT, PARTYP, SAMPMD, RESEVAL, PARAM, ACTION, and FOODEX. After the completion of the above changes and extensive testing we also had to provide our users with a manual explaining all the new fields and final changes that were made. Furthermore the communication and problem solving was supported extensively through teleconferences, web conferences and telephone answers for any questions emerged Migration Tasks Since all tasks were performed on a live system where new data were entered every day we had to perform several migration tasks. Some were tasks regarding previous data that lacked the new fields and others were tasks to deal with secondary areas that were affected during development. One of those areas was our statistic reports that needed to include the new additions and be able to automatically adapt in future changes Data Selection For the data selection we created a web - search form (Figure 1) that allows users to narrow down the data for transmission using filters such as Parameter, Sampling Program Code, Date range and various others. After the data are selected they are exported in XML format, Supporting publications 2013:EN Regulation (EC) No 178/2002, this task has been carried out exclusively by the author(s) in the context of a grant agreement between the European Food Safety Authority and the author(s). The present document is published complying with the transparency principle to which the Authority is subject. It cannot be considered as an output adopted by the Authority. The European Food Safety Authority reserves its rights, view and position as regards the issues addressed and the conclusions reached in the present document, without prejudice to the rights

8 using the XMLDB feature of Oracle 10g, and are temporarily stored in a repository table that resides in our Database until they are selected for final transmission. Figure 1: Partial screenshot of the data-selection web-form Data Transmission The above xml file is stored in an intermediate repository in our DB and is available for transmission when a user selects it in combination with the destination OCC group. Before final transmission there is the option to see the total file entries in a viewable form (using Oracle BI). EFET s web service was developed according to EFSA s guidance on Data Exchange respective to EFSA s WSDL file. Supporting publications 2013:EN Regulation (EC) No 178/2002, this task has been carried out exclusively by the author(s) in the context of a grant agreement between the European Food Safety Authority and the author(s). The present document is published complying with the transparency principle to which the Authority is subject. It cannot be considered as an output adopted by the Authority. The European Food Safety Authority reserves its rights, view and position as regards the issues addressed and the conclusions reached in the present document, without prejudice to the rights

9 2. RESULTS 2.1. Description of implementation The main concepts that we had in mind during implementation were: Usage of our already implemented IT-Infrastructure (OPS) Minimum disruption of our users and EFET s current operations Dealing with non-efsa catalogues entries that are applicable for Greece Transparently migrating data that have already been entered and are lacking the extra data fields required by EFSA Creating a system as flexible as possible for future additions Creating a friendly interface for end users requiring minimum intervention from ITexperts Data Collection The data are entered into our system (OPS) across Greece by EFET staff (mainly food inspectors and sampling officers) although they are provided from different sources i.e. from official control laboratories that are not linked directly to EFET. In any case the collection of data is being performed currently in a centralised way and all the data related to a specific sample are collected and entered into our system by dedicated and assigned personnel at central level. Supporting publications 2013:EN-471 9

10 Figure 2: Screenshot of the main form of OPS web-interface. The data-entry is performed through a user-friendly web-interface available to all our users and all data are stored in our main Oracle 10g Database in real-time. During the entry, many basic validations take place on the web-interface forms to ensure all required information is present. After the initial collection, EFET staff responsible for the overview of the data, can select and analyse data with the help of Oracle Business Intelligence that makes it easier to view it and also produce some initial statistic results for internal use. Supporting publications 2013:EN

11 EFET Database Oracle 10g XML TRANSFORMATION AND FINAL EXPORT VALIDATION & TRANSMISSION TO EFSA EFSA DATA REPOSITORY DATASET SELECTION INTERFACE USER WEB INTERFACE Legend: Current OPS depicted inside the black rectangle and new additions in the green one Figure 3: Graphical Representation of Current Infrastructure and Data Flows including the new additions XML - Output User-selected data are exported on request in xml format using XMLDB feature of Oracle in combinations with custom developed code that automatically applies the provided XSD file to validate the data and properly structure it for transmission (add headers, meta-information etc.). Supporting publications 2013:EN

12 Web Service Implementation Calling EFSA Web Service The final Data Transmission is triggered by a user after selecting the data file to be sent and the Occurrence Group that is destined for. For this purpose a SOAP message including the xml data in BASE64 format is being sent to the web service of EFSA. The SOAP message is constructed and transmitted from inside Oracle DB using HTTP protocol Response Messages from EFSA Currently, the responses to data transmission to the EFSA system are submitted to a predefined account. The web service from EFET s side has already been implemented, according to EFSA s WSDL(Top-Down Implementation ), which will handle the response files and save them in EFET s DB, for viewing using the appropriate XSLT s. After the completion of the web-service we will be able to receive acknowledgment returns via web services and present them in an easier way for our end-users Challenges During Implementation As already mentioned, during the process there were many challenges to be faced and also many positive facts that helped improve the Agency s current workflows and internal processes. The following table (Table 1) sums up both benefits and difficulties. Table 1. Benefits and difficulties during implementation Benefits Improvement of the Data Quality of our Central Management System (OPS) Difficulties Since many new fields had to be added and some EFSA value lists are quite extensive we had many complaints from users that this made the data entry more time consuming since they were unfamiliar with many of them. It is also hard to convince them to do precise selections from large lists since in many cases they select parent categories and as a result we might lose some details that cannot be monitored by the system. It is foreseen that this drawback will be overcome when the SSD2 will be introduced into our system in which domains specific Supporting publications 2013:EN

13 lists are provided for. Moreover extensive migration tasks had to be performed manually for data entered previous years, lacking information that were mandatory for validation during transmission. Many errors occurred during our initial tests and had to be dealt, requiring expertpersonnel involvement. Embedding most of EFSA s SSD lists was a big step forward to better organize our data storage process. Many of the lists do not contain regionspecific entries such as: products (even some popular ones), parameters etc. When any of these catalogues gets updated by EFSA, new additions are not outlined and it is time consuming to identify them and append them to existing ones (especially in those containing hundreds of entries). We would very much welcome an automatic mail sent when the catalogues get updated. The effort for assembling the data required by specific calls for data is lower and the whole process is considerably quicker. Transmission process using SOAP is a widely used technology that can be re-used for future data exchange. High effort for maintaining mapping lists and handling requests to add country-specific entries that would appear as errors during transmission since they cannot be matched against any of the official lists. In cases of rejected data (especially of those of previous years) it is difficult to identify the errors without the help of specialized IT staff. Supporting publications 2013:EN

14 CONCLUSIONS / RECOMMENDATIONS CONCLUSIONS EFET successfully completed the transmission of four datasets for 2011 (OCC_GROUP1, OCC_GROUP2, OCC_GROUP3, OCC_GROUP4). The project in total helped EFET to further improve its currently available IT-infrastructure by providing a set of data validation guidelines that lead us to identify small (but important) system gaps such as data that were not properly recorded. Thus we can say that the outcome was an overall improvement of data quality. Nevertheless the maintenance and future changes of the system will be a new challenge that will require continuous and close cooperation with EFSA. RECOMMENDATIONS 1.) One of the main limitations of the project lies in the fact that results are coming from various sources (internal/external labs) and they are delivered in a variety of ways ( s, excel sheets, paper etc.). In many of the above cases there are data that are mandatory for transmission which are not always included and need to be handled by people doing the data entry. As it has already been mentioned, the time that can be spent on data entry is quite limited (especially for people who do a lot of their work out in the field or during major food incidents) so future considerations must be made in order to make these processes faster. Maybe re-examining the need of some fields being mandatory. 2.) Furthermore it is essential that EFSA should create a business-process to notify MS Authorities each time one of the SSD lists gets updated clearly indicating the new additions. Currently it is very difficult to trace new additions out of lists that contain hundreds of entries. Moreover future major structural changes in these lists (e.g. FOODEX 2) are possible to affect current infrastructures and require migration tasks that are not always taken into account. A good practice for such changes is to be applicable for the next physical year (from the year of announcement) so that Authorities could prepare their changes and apply them at the beginning of a year in order to avoid having mixed data that need migrating and also create confusion to end users. 3.) Finally, in cases of rejected data there is the need to receive error messages that are more explanatory and will help users identify possible problems without requiring assistance of specialized IT-staff. For the time being it is a time consuming process to identify them and also quite difficult to explain why, in some areas, the new system requires the same (if not more) man hours to do the same job. Supporting publications 2013:EN

15 REFERENCES EFSA-GDE PUB. EFSA-GDE XML DEF. EFSA-SSD. ORACLE XMLPAD MANTIS MICROSOFT Supporting publications 2013:EN

16 Glossary / Abbreviations ACK DCF EFET EFSA MRN MS OPS SM SOAP SSD WSDL XML XSD XSL XSLT Acknowledgment, Acknowledgment Message Data Collection Framework Hellenic Food Authority European Food Safety Authority Message Recognition Notification Member State of the European Union EFET - Central Information Management System Standard Data Model Simple Object Access Protocol Standard Sample Description Web Services Description Language Extensible Markup Language XML Schema Definition Extensible Stylesheet Language XSL Transformation Supporting publications 2013:EN