User manual for reporting officers and reporters for mapping Member State standard terminology to EFSA standard terminology

Transcription

1 TECHNICAL REPORT APPROVED: 09 March 2015 PUBLISHED: 13 March 2015 Abstract User manual for reporting officers and reporters for mapping Member State standard terminology to EFSA standard terminology European Food Safety Authority The European Food Safety Authority (EFSA) is tasked with coordinating the reporting of zoonoses, zoonotic agents, animal populations, antimicrobial resistance and food-borne outbreaks in the European Union under Directive 2003/99/EC, as well as analysing and summarising the data collected. For data collection purposes, EFSA created a simple Microsoft Office Excel-based mapping tool to allow Member States to manage the mapping of their standard terminology to that published by EFSA. This technical report is a user manual for data providers that describes how to map terms and generate extensible markup language (XML) for submission of 2014 data, according to Zoonoses data models. European Food Safety Authority, 2015 Key words: zoonoses data models, Excel, XML, XSD, DCF Requestor: European Food Safety Authority Question number: EFSA-Q Correspondence: zoonoses_support@efsa.europa.eu EFSA Supporting publication 2015:EN-775

2 Acknowledgements: EFSA wishes to thank the EFSA staff member Kenneth Mulligan for the preparation work on this technical output. Suggested citation: European Food Safety Authority, User manual for reporting officers and reporters for mapping Member State standard terminology to EFSA standard terminology. EFSA supporting publication 2015:EN pp. European Food Safety Authority, 2015 Reproduction is authorised provided the source is acknowledged. 2 EFSA Supporting publication 2015:EN-775

3 Summary This user manual provides guidance to data providers on the use of the Microsoft Office Excel-based mapping tool for the submission of 2014 data by Member States and other reporting countries. This tool was developed by the Evidence Management Unit (DATA) in order to help Member States to report electronic data for zoonoses, zoonotic agents, animal populations, antimicrobial resistance and food-borne outbreaks according to the zoonoses data models. The manual provides a general introduction to the mapping tool and covers all aspects related to the management and use of the mapping tool. In particular, detailed guidelines are provided for the mapping of standard terminology and the creation of a locally validated extensible markup language (XML) file for submission of data to EFSA via the Data Collection Framework (DCF). Instructions are also given on the updating of EFSA standard terminology catalogues, and how to incorporate these updates in the mapping tool. This manual is specifically aimed at guiding the mapping of Member State-specific standard terminology to EFSA standard terminology, thus creating a much needed centralised document on mapping of controlled terminology. 3 EFSA Supporting publication 2015:EN-775

4 Table of contents Abstract... 1 Summary Introduction Introducing the mapping tool How the coded sheet works General points on Excel worksheets Getting started by profiling your data Mapping your profiled terms Steps in the mapping process The pick list search functionality Refreshing and changing views on data The adding and removing of formula rows The refreshing of data Different views on the codified data Generating your XML Activating the XML Developer tab Exporting your XML Map table to an XML file Reverse mapping your data Adding the import XML Map schema for reverse mapping Map the import XML Map to the ZOO_FACT_(MODEL_NAME) Import an XML file into the ZOO_FACT_(MODEL_NAME) Rebuilding a broken coded file Summary tables Producing the summary table Manual mapping tool Adding rows with dropdowns in the ZOO_MAN worksheet Rebuilding rows with dropdowns in the ZOO_MAN worksheet References Abbreviations EFSA Supporting publication 2015:EN-775

5 1. Introduction Background and Terms of Reference as provided by EFSA 1 The Directive 2003/99/EC 2 lays down the European Union (EU) system for monitoring and reporting of information on zoonoses, which obligates the Member States to collect data on zoonoses, zoonotic agents, antimicrobial resistance (AMR) and food-borne outbreaks. The European Food Safety Authority is assigned the tasks of examining the data collected and preparing the EU Summary Reports (SR) in collaboration with European Centre for Disease Prevention and Control (ECDC). In 2013, based on the proposals issued by EFSA, the European Commission (EC) put forward and discussed with the MSs a new legislation on the harmonised monitoring of AMR in Salmonella, Campylobacter and indicator bacteria in food-producing animals and food derived thereof. The Commission implementing Decision 2013/652/EU 3 of 12 November 2013 establishes a list of combinations of bacterial agents, food-producing animal populations and food products and sets up priorities for the monitoring of AMR from a public health perspective. Based on the data reported each year, EFSA and ECDC will jointly produce an annual EUSR on zoonoses, zoonotic agents and food-borne outbreaks. Similarly, the two agencies will produce a EUSR on antimicrobial resistance. To support the Member States (MSs) in their reporting, the existing reporting manuals for zoonoses, antimicrobial resistance and food-borne outbreaks need to be updated to take into account the latest recommendations on reporting of antimicrobial resistance data and data on zoonoses and food-borne outbreaks. In addition, the manuals have to be revised due to the changed structure of the reporting tables in the web application and changes in the relevant EU legislation. EFSA runs a web-based reporting application for the annual reporting as well as the possibility of submitting data in extensible markup language (XML)/Microsoft Office Excel format via the Data Collection Framework (DCF). The amended web application has to be tested and new XML reporting schemas created before the start of the reporting period in April each year. This is supported by revised guidance documents. For quality improvement purposes, the EUSR will be submitted every third year to the Scientific panels of Biological Hazards and Animal Health and Welfare for their review and comment. In addition to these EUSR, EFSA will, in collaboration with ECDC and the European Medicines Agency (EMA), prepare, in the future, joint reports combining antimicrobial resistance and consumption data. These reports will be separate from the EUSR and are covered by another mandate (M ) 4. The BIOCONTAM and DATA units 5 are invited to: prepare and publish the EU Summary Reports on Zoonoses, Zoonotic agents and Food-borne Outbreaks in 2012, 2013 and 2014 in close collaboration with ECDC; prepare and publish the EU Summary Report on AMR in 2012, 2013 and 2014 in close collaboration with ECDC; revise the manual for reporting on zoonoses, zoonotic agents and AMR each year, and publish it as an EFSA technical report; revise the manual for reporting on food-borne outbreaks when appropriate, and publish it as an EFSA technical report; revise the user manual for the web reporting application each year and publish it as an EFSA technical report; 1 Available online: 2 Directive 2003/99/EC of the European Parliament and of the Council of 17 November 2003 on the monitoring of zoonoses and zoonotic agents, amending Council Decision 90/424/EEC and repealing Council Directive 92/117/EEC. OJ L 325, , p Commission Implementing Decision 2013/652/EU of 12 November 2013 on the monitoring and reporting of antimicrobial resistance in zoonotic and commensal bacteria. OJ L 303, , p The first joint report (ECDC/EFSA/EMA) on the integrated analysis of the consumption of antimicrobial agents and occurrence of antimicrobial resistance in bacteria from humans and food-producing animals was published on 30 January, 2015 (ECDC, EFSA and EMA, 2015). 5 On 1 January 2014, these tasks were moved from former BIOMO unit to the BIOCONTAM and DATA units. 5 EFSA Supporting publication 2015:EN-775

6 revise the guidelines (data dictionaries) for XML/Excel data reporting each year and publish them as an EFSA technical report. This report is closely linked to the sixth term of reference above: to revise the guidelines (data dictionaries) for XML/Excel data reporting each year and publish them as an EFSA technical report. In order to help Member States to report electronic data for zoonoses, the Evidence Management Unit (DATA) tool developed an Excel Mapping Tool according to the zoonoses data models and this manual provides guidance to data providers on the use of this mapping tool. 2. Introducing the mapping tool EFSA has published guidance in data submission using XML data transfer covering the reporting of isolate-based quantitative antimicrobial resistance data, as well as reporting of prevalence data on zoonoses and food-borne contaminants, food-borne outbreak data, animal population data, disease status data and text forms (EFSA, 2015d). For zoonoses data collection purposes, EFSA has created six data models for: zoonoses prevalence data (PRV), antimicrobial resistance isolate-based data (AMR), food-borne outbreak data (FBO), animal population data (POP), disease status data (DST) and text forms data (TXT). The main objective of the mapping tool is to provide a simple and useable platform for MSs to map their country-specific standard terminology to that published by EFSA and to enable the production of an XML file for the submission of sample- or aggregated-based zoonoses monitoring data via the DCF. Where the catalogues of each data model (AMR, PRV, FBO, POP, DST and TXT) are already inserted into the model data entry worksheet the mapping tool supports the manual mapping of terms, requiring only the selection of the correct term. Specific Excel mapping tools correspond to each of the six data models available. For each of the Excel mapping tools a manual and a dynamic mapping template has been created to encode MSs data to EFSA s standard terminology. The mapping tool is a solution for MSs who are currently unable to: implement the EFSA data models in a database. implement standard term mapping in a database. automatically generate XML. The Excel mapping tool could help MSs to: create a comprehensive mapping of MSs standard terms. build a future database. visualise data that is already coded in a readable text format (known as reverse mapping or decoding). improve the quality of data submitted to EFSA (no need for Excel, local validation). This mapping tool is useful not only for MSs reporting a small number of data, but also in understanding what is needed to report row-level data via the DCF according to Zoonoses data models. This user manual introduces the basic functionalities available in the mapping tool, such as: profiling your data; mapping profiled terms explicitly; providing lookup and encoding of explicitly mapped terms; summary tables of reported data; generating and validating XML via published Zoonoses data models XML schema definition (XSD). 6 EFSA Supporting publication 2015:EN-775

7 Figure 1: Default worksheets of the dynamic mapping tool The mapping tool is essentially an Excel workbook which has four default data processing worksheets. Outside of these four worksheets, each model contains all the catalogues needed for each data model. All the catalogue worksheets have ZOO_CAT at the beginning of or as the prefix to the pick list s name. It is in these catalogues worksheets that data providers will need to map their own specific terms to those required by EFSA. In Figure 1 the default worksheets are presented, numbered 1 to 4: Mapping_Options ZOO_FACT_(DATA MODEL_NAME) CODED DCF_FORMATTED_DATA Worksheet number (1) in Figure 1, the Mapping_Options worksheet, is where all the functionality data providers need to manage their mapping file is found. The features provided there are presented in each of the sections below. Worksheet number 2 in Figure 1, ZOO_FACT_(DATA MODEL_NAME) or ZOO_MAN_(DATA MODEL_NAME) is where data providers will insert their own data. This is the first step whereby data providers must fit their current data into the structure of the data model in question. This worksheet is the format into which reporting countries must organise their data for successful mapping and submission of data to EFSA. 7 EFSA Supporting publication 2015:EN-775

8 Figure 2: ZOO_FACT worksheet displays mandatory fields highlighted in red Each ZOO_FACT_(DATA MODEL_NAME) worksheet displays, highlighted in red in Figure 2, the mandatory fields that must be inserted into the model for a valid submission to the DCF. All other fields are coloured grey but this does not always mean that they are optional. Some grey fields are mandatory given certain reporting conditions; as stipulated in the reporting manuals and guidelines (EFSA, 2015 a, b, c, d). Figure 3: ZOO_FACT worksheet displays the definitions for each data element To further improve the tool s usability, definitions for each data element have been included in the ZOO_FACT_(DATA MODEL_NAME) worksheet. To observe the definition, data providers can simply place their mouse over the element name displayed in row one of the worksheet, and that element s definition will be displayed (Figure 3). By clicking on the element name, data providers will be transferred to that element s catalogue and can see the terms that are permissible and the mappings that have been made between the data porvider s data and EFSA s standard terminologies. 8 EFSA Supporting publication 2015:EN-775

9 Worksheet number (3) in Figure 1, CODED worksheet, reads the data inserted into the ZOO_FACT_(DATA MODEL_NAME) or ZOO_MAN_(DATA MODEL_NAME) worksheet and then searches for the correct mapped term in the various catalogue worksheets (this is presented in more detail in section 2.1). Figure 4: CODED worksheet Figure 4 shows that any terms that are not mapped are highlighted in red and display the text Not Mapped and the value of the term in the ZOO_FACT_(DATA MODEL_NAME). All terms have to be mapped correctly before data providers can produce a valid file to be submitted in the DCF. Worksheet number (4) in Figure 1, the last of the four default data processing/mapping worksheets, DCF_FORMATTED_DATA worksheet, is where completed coded data will be eventually exported from, either as XML via the published Zoonoses XSD schemas or in Excel format (as the DCF supports both Excel and XML formats). It is very important that the Excel workbook is saved in an Excel workbook (*.xls) format, as the DCF only supports Excel files of this type. In addition, if the workbook contains more than one worksheet when submitted to the DCF, it is only the first worksheet in the workbook that is read. Data providers must move the DCF_FORMATTED_DATA sheet to the first sheet in the series, before the worksheet Mapping_Options, if data providers wish to submit the Excel workbook to the DCF. Figure 5: DCF_FORMATTED_DATA worksheet 9 EFSA Supporting publication 2015:EN-775

10 As with the CODED worksheet in Figure 4, for DCF_FORMATTED_DATA all non-mapped terms are highlighted in red as also seen in Figure 5 number (1). Figure 5 number (2) shows that the DCF_FORMATTED_DATA worksheet formats the coded data, replacing all NULL (empty cells), to enable data providers to create a valid XML file. This can be seen clearly by comparing Figure 4 column totunitstested which contains the value NULL to column tounitstested in Figure How the coded sheet works Figure 6: CODED worksheet containing LookUpDicTerm function The worksheet CODED, see number (1) in Figure 6, contains the LookUpDicTerm function that will query the mapping that data providers will create in each of the catalogue worksheets. The function will return a mapped EFSA code. It is therefore important to make sure that the mappings created by data providers are correct. In Figure 6, highlighted by number (2), the value RF MCG can be seen. This is the code returned for a valid mapped term, found by the LookUpDicTerm function highlighted by the number (3) in Figure 6. The LookUpDicTerm expects as its first value the name of a catalogue to search in, in this case ZOO_CAT_PARAM_AMR, and a value. The value will be selected from the ZOO_FACT_AMR_ISOLATE_AST, column E row number 2 (the zoonosis data model element). The value contained in the cell E2 should be mapped to a catalogues term in the ZOO_CAT_PARAM_AMR catalogue. If the term is not mapped, the function returns NOT MAPPED. All NOT MAPPED values must be fixed before a valid submission file can be made. The number (4) in Figure 6 highlights the fact that ZOO_FACT_(DATA MODEL_NAME) elements that require an EFSA standard term (catalogue) are linked to their respective catalogue in the workbook. To view the standard terms catalogue, the data providers can simply click the link and the catalogue sheet will open. By default, the CODED worksheet has only one row pre-filled with formulas. If data providers want to extend the number of rows in the CODED worksheet to match the number of data entries (rows) in the worksheet ZOO_FACT_(DATA MODEL_NAME), data providers need to use the Mapping_Options Resize Coded Sheet Formulas button. For example, in a ZOO_FACT_(DATA MODEL_NAME) worksheet with 560 rows, only the first one (the default is one) will be visible in the CODED worksheet. Thus data providers will need to insert 560 in the text box beside the Resize Coded Sheet Formulas button in the Mapping_Options worksheet and then press the button Resize Coded Sheet Formulas (see below) EFSA Supporting publication 2015:EN-775

11 After data providers have performed the resizing of sheet formulas the number of rows in the CODED worksheet, with formulas will match the number 560 in the ZOO_FACT_(MODEL_NAME) worksheet. All 560 row values in the ZOO_FACT_(MODEL_NAME) worksheet will now be copied across and coded to the CODED worksheet. Each ZOO_FACT_(DATA MODEL_NAME) data model contains an up-to-date list of standard terms. These are the catalogues that are also available in the online Zoonoses Web-Application reporting tool. Each model will contain only pick list terms and catalogues that it needs to reference General points on Excel worksheets As each cell in the CODED worksheet contains a formula, the more data users put in, the slower the use of the Excel worksheet will become. It is recommended that data providers profile all of their data, mapping all their existing terms, and then create multiple copies of the Excel workbook. Data providers can then split the data into groups of, for example, to rows, thus keeping the Excel worksheets lighter and quicker to use. The Resize Coded Sheet Formulas button is specifically designed to help with this issue: it automatically adds and removes the formulas required by the CODED worksheet to map terms from the ZOO_FACT_(DATA MODEL_NAME). By using this feature, data providers can greatly reduce the size of the model file. Setting the CODED sheet value to 2 will reduce the size of the workbook by more than 70 %. Only when data providers need to check or actually produce a valid file for submission should they change the number of rows in the CODED worksheet to the same number of rows data providers entered into the ZOO_FACT_(DATA MODEL_NAME). 3. Getting started by profiling your data When data providers open any of the data models ZOO_FACT files, they must enable macros so that the custom function LookUpDicTerm can run. Figure 7 below shows you the sequence of steps to follow in doing this. Figure 7: Example of enable macros 11 EFSA Supporting publication 2015:EN-775

12 If data providers do not already have a list of all the terms that need to be mapped per data model to EFSA terms, one simple way to profile their data is to use the Profile Your Data button in the Mapping_Options worksheet (see below). To quickly profile the data, follow these steps: 1) Insert the data in the ZOO_FACT_(DATA MODEL_NAME) worksheet. 2) Click the Profile Your Data button, and watch as all unique terms in the data are collected and inserted into their catalogue worksheets under the MemberState_TERMS column. Figure 8 shows an example of the unique terms found in a dataset for the zoonosis data element. Figure 8: Example of collecting and inserting of all unique terms into specific catalogue The profile option selects and adds only unique and new terms to the catalogues; thus, if data providers add more new data points to the ZOO_FACT_(DATA MODEL_NAME) worksheet, data providers can continue to use the Profile Your Data button to check for new terms to be added to catalogues Mapping your profiled terms At this point data providers have profiled their data, and the unique terms that need to be mapped are inserted into their catalogue worksheets. The following sequence of steps will map these values to their corresponding EFSA standard term EFSA Supporting publication 2015:EN-775

13 All catalogues have the mapping sections, where mapping is performed. Figure 9 highlights the mapping sections in the ZOO_CAT_PARAM_AMR catalogue. A mapping section is linked to a specific standard term catalogue. The column MemberState_TERMS, highlighted by the number (1) in Figure 9, is where your reporting country-specific terms are inserted. The column EFSA_TERM, highlighted by the number (2) in Figure 9, is where the zoonoses standard term that will match the reporting country term is selected from the dropdown box. Figure 9: Example of mapping section in the ZOO_CAT_PARAM_AMR catalogue 3.2. Steps in the mapping process 1) Open the CODED worksheet in the workbook. The zoonosis element name is linked to the standard term from ZOO_CAT_PARAM catalogue. By using the hyperlinked text of the element zoonosis in the first column of the worksheet, data providers can move to the ZOO_CAT_PARAM_AMR catalogue. Once data providers have selected the standard terms from ZOO_CAT_PARAM_AMR catalogue, users should see their profiled terms under column A ( MemberState_TERMS ) (Figure 9). If there are no terms under column A ( MemberState_TERMS ), data providers will need to return to the Mapping_Options worksheet and select the Profile Your Data button. 2) Once data providers have their profiled terms, the mapping of their term with the term from the Zoonoses catalogues can be performed. In column B ( EFSA_TERM ), there is a dropdown list of all the standard terms in the ZOO_CAT_PARAM_AMR catalogue for the element Zoonosis (see the example in Figure 10). Data providers have to search through this list until they find a standard term that matches their reporting country-specific standard term. When data providers find a match, select it, and their standard term will be physically matched to EFSA s term. The LookUpDicTerm function in the CODED worksheet will return for every occurrence of data provider s standard term; in the example in Figure 10, Salmonella S. Abortusovis the code value is RF MCG. 3) The above steps should be repeated until all of terms are matched for all profiled elements EFSA Supporting publication 2015:EN-775

14 Figure 10: Example of mapping a national standard term with EFSA s term 3.3. The pick list search functionality If data providers have a large number of terms in a catalogue, like in ZOO_CAT_PARMA_AMR above, a search facility has been developed that will allow data providers to quickly search for terms. The search facility allows data providers to enter a maximum of three search text elements in the following format: [search text1]-[search text2]-[search text3] [search text1]-[search text2] [search text1] For example, if data providers want to search for the term Salmonella S. Enteritidis PT 24, the search string could be salmon-enteri-pt 24 or pt 24-salmon-enteri; the search function will return the first catalogue item that it finds that contains all three search strings in order of sequence. Each search string must be separated by the - character. The more specific the search string is, the greater the accuracy data providers will achieve in searching for their catalogue item. Figures 12 and 13 below demonstrate the search feature using the maximum three search strings as above. A match could also have been achieved using the search string enteri-pt 24, corresponding to [search text1]-[search text2] EFSA Supporting publication 2015:EN-775

15 Figure 11: Example of the pick list search functionality searching Figure 12: Example of the pick list search functionality finding 15 EFSA Supporting publication 2015:EN-775

16 4. Refreshing and changing views on data The Excel tool has turned off Excel s ability to automatically update cell formulas when a value linked to the formula has been changed to improve performance. In the CODED worksheet, all cells contain a formula to either look up/codify values entered in the ZOO_FACT_(MODEL_NAME) worksheet, or simply copy across values that have no pick list and therefore do not need to be coded. All this codifying and cell value copying affects performance; the more rows of data in the ZOO_FACT_(MODEL_NAME), the slower and larger the CODED worksheet and Excel workbook become. If we leave Excel to look after all these cell formulas, it will continuously recalculate them, making the use of a large file almost impossible. Instead we have switched off this recalculating and given the user the power to recalculate the formulas in the coded sheet when it is needed, thereby reducing the computer resources needed and speeding up the tool as a whole. We have also introduced a feature that allows the user to add and remove formula rows in the CODED worksheet. This feature may not seem useful, but with experience we have found that users sometimes need to either their data to other colleagues or simply upload their data to the DCF, and large files can take some time to upload and are virtually impossible to send via The adding and removing of formula rows The adding and removing of formula rows becomes important when problems of file size need to be resolved. By setting the number of formula rows to two (the minimum allowed), data providers can effectively reduce the size of their working file by more than 70 %. Data really needs to be coded only when data providers are producing a dataset which data providers wish to submit to EFSA s DCF; to do this data providers simply add as many rows to the CODED sheet as data providers have in their ZOO_FACT_(DATA MODEL_NAME) worksheet, select the resize coded sheet button (entering in the number of rows in the text box of course) and all the row values data providers have in the ZOO_FACT_( DATA MODEL_NAME) will be transferred and codified into the CODED worksheet The refreshing of data The refreshing of data is also very important. If data providers have spotted an error or missed mapping in their file they will want to fix it, and only when all their fixes have been made will data providers want to refresh the CODED worksheet with those changes. This is also necessary because we have switched off Excel s ability to recalculate which gives the user total control over the processing of their data. In the Mapping_Options worksheet, data providers will find the Refresh Coded Sheet Formulas button, and every time data providers click this button an update of their data in the CODED worksheet is performed Different views on the codified data The tool now also supports the concept of different views on the codified data by which a user can change the behaviour of the LookUpDicTerm display of coded terms. Normally in the coded sheet only the code connected to an EFSA term is displayed. However sometimes during the checking of the coded data it can be difficult to visually link a displayed code with its textual meaning. With this in 16 EFSA Supporting publication 2015:EN-775

17 mind, EFSA designed the mapping mode feature which allows the CODED worksheet to display both the coded term and its text together and also the user can also display their own term texts combined with the EFSA code and term text in the CODED worksheet. This feature allows users to make sure that the mappings that were made between the Member State term and its EFSA twin are correct. It is important to remember, that when all the data has been correctly coded and checked, the user must change the mapping mode back to the Normal Mapping Code only displayed. This is because only the EFSA codes are accepted in any submission of data to the DCF; Excel or XML files containing both the term text and the code together will be rejected. Figure 13: Functionalities of the Mapping Mode Among he options available via the Mapping Mode section in the Mapping_Options worksheet seen in Figure 13 above, data providers can see that, for normal mapping there is: 1) Normal Mapping Code only displayed 2) Normal Mapping Code/Term displayed together And under the Display mapped data there are: 1) EFSA Terms 2) Memberstate mapped terms These two features combined will provide us with the different views we mentioned above. For example, if we select 1) Normal Mapping Code only displayed, and 1) EFSA Terms the coded term is all that we will see in the CODED worksheet: If we choose zoonosis RF MCG 2) Normal Mapping Code/Term displayed together, and 1) EFSA Terms we get both the EFSA code and the term text: zoonosis RF MCG Salmonella - S. Montevideo 17 EFSA Supporting publication 2015:EN-775

18 If we really want to be sure the mapping we made was correct between the reporting country term and the EFSA term, we can also choose 2) Normal Mapping Code/Term displayed together, and 2) Memberstate mapped terms This will provide us the Member State term (in this example Montevideo), its mapped EFSA code and the term text: zoonosis Montevideo RF MCG Salmonella - S. Montevideo This level of display flexibility should greatly help to ease the use of the tool. When the user is ready to create and submit their data, the first example setting should be selected so that only the coded term is displayed and not the text associated with it namely: 1) Normal Mapping Code only displayed, and 1) EFSA Terms 5. Generating your XML With the release of Office Excel 2003, the importing and exporting of XML via a valid XSD has been supported. Under the Developer tab, an XML tool exists for the creation of what Excel calls XML Maps. When Zoonoses data models are published, an XML schema or XSD is also published that supports the strict structure and validation of any data model and, as such, can be used to generate from a CODED worksheet a valid XML file containing all of the coded values. The following sequence of steps (sections 5.1 and 5.2) will produce an exportable validated dataset in XML. If data providers do not want to submit XML-validated data, data providers can stop here at this point; data providers must only ensure that the DCF_FORMATTED_DATA worksheet is the first worksheet in the workbook. It is very important that the Excel workbook is saved in an Excel workbook (*.xls) format, as the DCF only supports Excel files of this type. In addition, if the workbook contains more than one worksheet when submitted to the DCF, it is only the first worksheet in the workbook that is used; this is why the DCF_FORMATTED_DATA sheet must be the first sheet in the series Activating the XML Developer tab The Developer tab allows data providers to import and create XML files; normally it is not immediately available as a default option and so it needs to be activated directly by data providers. Figure 14 below indicates the sequence of selections data providers need to make to activate the Developer tab EFSA Supporting publication 2015:EN-775

19 Figure 14: Steps in the activation of the Developer tab 5.2. Exporting XML Map table to an XML file To export your data, data providers first need to select the Developer tab, and then click the Export button. The Export button will bring up the Export XML File dialogue box. Here data providers can name and navigate to where data providers want to store your XML file. After naming your file, simply select the Export button to initiate the export procedure. During the export procedure, the XML file created will automatically be validated by the data model s XML XSD schema (see Figure 15). All Excel mapping tools will have this XSD already included for data validation. Figure 15: Exporting an XML file 19 EFSA Supporting publication 2015:EN-775

20 6. Reverse mapping your data The tool supports the concept of reverse mapping giving users the ability to import an already codified XML file into the tool and to decode it to view the values that it contains. This feature is really only for viewing already coded XML files, and, because the CODED sheet is working in reverse, the user will not be able to automatically generate a new XML file. The first step needed is to add the import XML schema to the ZOO_FACT_(DATA MODEL_NAME) worksheet. Once this has been done, the user can select the import feature of the Developer tab selecting the import button from the XML tool Adding the import XML Map schema for reverse mapping To import already coded XML data into your Excel worksheet and reverse map it (i.e. to decode the data), data providers will first have to prepare their ZOO_FACT_(MODEL_NAME) worksheet to accept XML data. To do this data providers first need to add an Excel table object to their ZOO_FACT_(MODEL_NAME). To add this Excel table object, data providers will need to go to the Mapping_Options worksheet and press the Add Import Table button; this will automatically add the required table. Next, data providers must go to the Developer tab and select the Source icon. The Source icon will activate the XML Source window; at the bottom of this window there is an XML Maps button which data providers need to select. Once the XML Maps button is selected, the XML Maps dialogue box will appear; data providers then need to select from the dropdown list (number 3 in Figure 16) the import name displayed. Figure 16: Adding the import XML Map schema for reverse mapping 6.2. Map the import XML Map to the ZOO_FACT_(MODEL_NAME) After data providers have selected their import XML Map schema for reverse mapping, data providers can see the structure of the import XML Map schema elements in the XML Source window (number EFSA Supporting publication 2015:EN-775

21 in Figure 17). The section of the XML Map that we are interested in is under the dataset branch. To map the XML Map to their data, data providers will need to left-click on the Result branch; with the mouse button still held down, data providers will then need to drag the Result branch across to the top-left cell (in this case resultcode), containing the first element column name in the ZOO_FACT_(DATA MODEL_NAME) worksheet. Let go of the mouse button when the top-left cell becomes active. In this way, data providers have successfully mapped their import XML Map schema to the elements in their data model. Figure 17: Mapping the import XML Map to the data model elements 6.3. Import an XML file into the ZOO_FACT_(DATA MODEL_NAME) To import an XML file into your ZOO_FACT_(DATA MODEL_NAME) worksheet, data providers should follow the steps outlined in Figure 18 below. Firstly, in the Developer tab, data providers need to select the Import icon from the XML menu (number 1 in Figure 18). Once data providers have selected this, the import XML dialogue box will appear; data providers need to select the XML file on their computer that they wish to import. Please note that the XML file must have been created by the same XSD Map schema that data providers just added to the ZOO_FACT_(MODEL_NAME). When data providers have found and selected the XML file, click the import button in the bottom right-hand corner of the import XML dialogue box (number 3 in Figure 18). A new dialogue box will appear asking data providers to choose the XML Map that data providers would like to use for the import; as well as the name would indicate to select the import XML map, followed by the ok button EFSA Supporting publication 2015:EN-775

22 Figure 18: Import an XML file into the data model elements Depending on the size of the XML file that data providers are importing, this procedure can take anything from a few seconds to about a minute to complete. Below, in Figure 19, data providers can see that the ZOO_FACT_(DATA MODEL_NAME) is populated with coded data; this is in fact the same data that data providers would normally have in the DCF FORMATTED DATA worksheet. It is needed to change the display in the CODED worksheet to reverse map these data, so that we can see the code and its EFSA term text displayed. Figure 19: Data elements are populated with coded data 22 EFSA Supporting publication 2015:EN-775

23 To change the CODED worksheet s display properties for reverse mapping of terms, data providers need to once again return to the Mapping_Options worksheet, and select the Reverse Mapping Code/Term displayed together option from the mapping mode dropdown list. Data providers also need to select EFSA Terms to display the standard term text that every EFSA code has. If the user has a file with reporting country-specific data already mapped, and they wish to see their own terms as well as the EFSA code and standard term text, they can choose the Memberstate mapped terms option. Figure 20: Settings of the Mapping_Option to Reverse Mapping Code/Term displayed together and EFSA Terms The displaying of the terms in the CODED sheet will apply the same rule as those outlined in section 4, Refreshing and changing views on data. For the standard EFSA terms view, the data in the CODED worksheet will be displayed as highlighted in Figure 21 below. As data providers can see, the term code and the standard term text are now displayed together, making the file readable again. Figure 21: Data elements are populated with terms and codes 7. Rebuilding a broken coded file Under normal conditions, the CODED worksheet is quite robust and will not break too easily. Some situations will arise, however, where formula references in the CODED worksheet will break. This is normally linked to cut/delete or paste operations performed in the ZOO_FACT_(MODEL_NAME) 23 EFSA Supporting publication 2015:EN-775

24 worksheets. When this happens it would be helpful to press a button and have all the broken formulas fixed. This is exactly what is implemented in the tool with the button called Rebuild Coded Sheet Formulas. The button can be found on the Mapping_Options worksheet, but hopefully data providers will not need to use it too often. 8. Summary tables The tool supports the summarising of entered data in the ZOO_FACT (MODEL_NAME) worksheets via an automatically generated Excel pivot table. For the 2014 reporting period, only the antimicrobial resistance isolate and prevalence data models are supported. The summary pivot table for antimicrobial isolate data will calculate and aggregate resistance to antimicrobials at the substance level, showing the aggregate counts in a very similar way to the aggregate quantitative antimicrobial tables in the online Zoonoses Web Application. This will allow reporters to check the correctness of the data directly in the tool, even before codification takes place. The same is also true for the prevalence data model tool, where the data are also presented in an aggregated summary table similar to the online Zoonoses Web Application Producing the summary table The addition and removal of the summary table for a ZOO_FACT (MODEL_NAME) is straightforward; data providers simply need to navigate to the tools mapping option page and select the Add Summary Table button. The tool will not create a new sheet called Summary; instead it will insert a new pivot table that will include all the data providers have entered into the ZOO_FACT (MODEL_NAME) worksheet. Once the pivot table summary is finished, it can take a few seconds to complete. Once it is complete data providers will have a fully functioning pivot on their data, allowing data providers to filter and search data for possible errors in aggregation. Figure 22 below is an example of a summarised prevalence data model EFSA Supporting publication 2015:EN-775

25 Figure 22: Example of a summarised prevalence data model 25 EFSA Supporting publication 2015:EN-775

26 When data providers have finished looking at summary data, it is considered a good idea to remove it as summary pivot tables are from both a file size perspective and a memory processing perspective very resource intensive. This is a similar issue to that for the number of formulae in the CODED worksheet, and as such we provide a button to remove the summary worksheet. The use need simply to select the Remove Summary Table button, and then select ok from the Are you sure dialogue box that will appear after data providers have pressed the button. 9. Manual mapping tool The manual tools support only the direct entry of EFSA coded standard terminology pick list terms. What this means is that no actual physical mapping of Member State terms to EFSA coded terms is performed; the user simply selects EFSA standard terms directly from catalogues embedded in the ZOO_MAN (MODEL_NAME) worksheet. The benefit of this approach is its simplicity; the drawbacks are the time needed to fill in a large number of data and the constant need to search for terms or select terms from the dropdown boxes in the data entry worksheet. Figure 23: Mapping_Options worksheets of the manual mapping tool The manual tool does not support all functionality of the dynamic mapping tool; this is because the LookUpDicTerm is not used in the CODED worksheet. The CODED worksheet in the manual model simple removes the text part of an EFSA standard coded term, leaving only the code; no dictionary 26 EFSA Supporting publication 2015:EN-775

27 lookup is needed because the values in the ZOO_MAN (MODEL_NAME) worksheet are already coded terms selected from a pick list. Figure 24 below shows how each term is picked from a dropdown list in the data entry worksheet. Figure 24: Example of selection of terms from a dropdown list in the data entry worksheet The manual tools data entry worksheet, ZOO_MAN (MODEL_NAME), supports the pick list search functionality outlined in section 3.3. Instead of selecting directly from the pick list dropdowns as shown in Figure 24, data providers can type in their pick list search terms Adding rows with dropdowns in the ZOO_MAN worksheet The adding of new rows that will contain all the catalogues required for mapping is very simple as it is linked to the Resize Coded Sheet Formulas button for increasing the number of formula rows in your CODED worksheet. Whatever number data providers put into the text box will be the exact number of rows that will be added with functioning dropdown pick lists. Alternatively, data providers could simply highlight the entire row and drag it down to do the same thing Rebuilding rows with dropdowns in the ZOO_MAN worksheet Section 8 covers the rebuilding of the CODED worksheet; for the manual tool, the process is exactly the same. Every time data providers want to rebuild either the CODED worksheet or the dropdowns in all of the columns of the ZOO_MAN (DATA MODEL_NAME) worksheet, select the Rebuild Coded Sheet Formulas button on the Mapping_Options worksheet. When data providers are rebuilding a worksheet because it has become broken or it is missing pick list dropdown boxes, enter the number of rows wanted to rebuild. By default only the first row on the CODED and ZOO_MAN worksheets are rebuilt. To rebuild all of the rows in the dataset, you will need to enter the number of rows in the Resize Coded Sheet Formulas text box, and then select the Rebuild Coded Worksheet Formulas button. For example, in Figure 25 below, if data providers were to press the Rebuild Coded Sheet Formulas, as currently displayed, only the first 230 row records would have their formulas and dropdown pick lists rebuilt. Figure 25: Example of rebuilding of CODED and or any of the dropdown pick lists 27 EFSA Supporting publication 2015:EN-775

28 References EFSA (European Food Safety Authority), 2015a. Manual for reporting on zoonoses and zoonotic agents, within the framework of Directive 2003/99/EC, and on some other pathogenic microbiological agents for information deriving from the year EFSA supporting publication 2015:EN pp. EFSA (European Food Safety Authority), 2015b. Manual for reporting on antimicrobial resistance within the framework of Directive 2003/99/EC and Decision 2013/652/EU for information deriving from the year EFSA supporting publication 2015:EN pp. EFSA (European Food Safety Authority), 2015c. Manual for reporting on food-borne outbreaks in accordance with Directive 2003/99/EC for information deriving from the year EFSA supporting publication 2015:EN pp. EFSA (European Food Safety Authority), 2015d. Data dictionaries guidelines for reporting data on zoonoses, antimicrobial resistance and food-borne outbreaks using the EFSA data models for the Data Collection Framework (DCF) to be used in 2015, for 2014 data. EFSA supporting publication 2015:EN pp EFSA Supporting publication 2015:EN-775

29 Abbreviations AMR DCF EC ECDC EFSA EMA EU MS XML XSD antimicrobial resistance Data Collection Framework European Commission European Centre for Disease Prevention and Control European Food Safety Authority European Medicines Agency European Union Member State of the European Union extensible markup language schema definition language Abbreviations used in codes of data elements that refer to a data model: AMR POP DST FBO PRV TXF AMR isolate based-data model animal population data model disease status data model food-borne outbreaks data model prevalence data model text forms data model 29 EFSA Supporting publication 2015:EN-775