Needs Assessment for a Web-Based IT Platform to Support the WHO Global Antimicrobial Resistance Surveillance System GLASS Prepared August 7, 2015 1
Table of Contents 1. Background and Scope of Work... 3 2. Needs assessment framework... 4 2.1. Enrolment of AMR surveillance programs in GLASS... 4 2.2. Creation of GLASS data submission files... 5 2.3. WHO GLASS Application... 5 2.4. Core functionality for GLASS application... 5 2.5. Some points to consider... 6 3. Data preparation and local validation... 8 3.1. AMR Statistics GLASS data file preparation and validation... 8 3.2. Patient and sample denominators... 8 3.3. Management and Evaluator Indicators... 8 4. Data submission and validation... 9 4.1. Data file import... 9 4.2. Data validation... 9 5. GLASS Database schema... 10 6. Reports... 11 6.1 Metrics... 11 6.2 Reports implementation... 11 2
1. Background and Scope of Work The draft global action plan 1 on antimicrobial resistance (AMR) was endorsed at the Sixty- Eight World Health Assembly, 21 May 2015. One of the strategic objectives of the global action plan is to strengthen the knowledge and evidence base through surveillance and research. Antimicrobial resistance surveillance is the cornerstone in identifying emerging threats and assessing burden of disease and provides the necessary information for action in support of the global action plan. An international consultation 2 was held in Sweden in December 2014, which convened representatives from 30 countries from all WHO regions, WHO Collaborating Centres, international networks and experts, who agreed on the need to develop local (healthcare facility), national, regional, and global collaborations to support surveillance of antimicrobial resistance in diseases relevant for human health. A priority objective requiring immediate action is establishment of a global AMR surveillance system to ensure that data obtained through the use of global standards are available to guide policies and monitor effectiveness of the response efforts. This document try s to address the needs of such a system, in particular: a) How to generate aggregated surveillance data at the national level in compliance with the Global Antimicrobial Resistance Surveillance System Manual for early implementation: surveillance of resistance in common human bacterial pathogens (August 2015); and b) determine the requirements of an integrated IT system to support: the submission, storage, display, analysis and reporting of data collected; and future revisions when needed to support new or additional features; The GLASS manual proposes a five-year road map for GLASS implementation shown in Table 1. Aspects specifically related to IT database and software development have been underlined. 1 Antimicrobial resistance, draft global action plan on antimicrobial resistance - http://apps.who.int/gb/e/e_eb136.html (document EB136/20) 2 http://www.folkhalsomyndigheten.se/amr-stockholm-2014/. 3
Table 1. Five-year road map for implementation of GLASS Year Targets 2015 Prepare manual, set up IT hub and plan support for implementation of GLASS. Establish a platform for international collaboration with WHO collaborating centres, national and regional networks and other laboratories and institutions to allow WHO to support countries in implementing GLASS. Initiate country enrolment. 2016 Start collection of baseline data on human antibacterial-resistant infections from WHO Member States. Report on progress in early implementation. Amend and consolidate the road map on the basis of results from early implementation. Target the participation of 15% of Member States. 2017 Consolidate baseline data collection on human antibacterial-resistant infections from WHO Member States. Increase the capacity of the platform to build relations with other AMR surveillance systems (e.g. in animal health, agriculture and use and consumption of antibiotics). Extend Member States participation to 20%. 2018 Report on the global and regional prevalence of AMR in human health. Extend Member States participation to 30%. Explore the feasibility of case-finding by surveillance of clinical syndromes at selected surveillance sites. 2019 Establish a web-based portal for sharing information and data from different sources. Extend Member States participation to 40%. Review lessons learnt from early implementation to inform further development of GLASS. 2. Needs assessment framework 2.1. Enrolment of AMR surveillance programs in GLASS Enrolment efforts will rely on effective communication and promotion by WHO of the public health value and technical feasibility of the GLASS initiative to relevant national and regional AMR surveillance networks. Any country capable of applying the GLASS surveillance methodology for compiling data nationally and locally (at surveillance sites)would be welcome to participate. It is a requirement that a national coordinating centre (NCC) should be set up to oversee the national AMR surveillance system, including the collection and aggregation of data from surveillance sites, to ensure that the system is functional. 4
2.2. Creation of GLASS data submission files It is recognized that WHONET is widely used. So for national programs which already utilize the WHONET software, enrolment into the GLASS data collection scheme should be reasonably straightforward. A new Export to GLASS data export feature would facilitate the convenient creation of the needed files. For programs which do not currently use WHONET but which wish to join GLASS, there are two strategies that could be considered: Begin to use WHONET as the primary data management system, especially for new programs that have not yet selected a data management strategy For programs that are satisfied with their current data management tools, a generic interface could be created to receive data in GLASS, in the prescribed format, using the output created in the local system. 2.3. WHO GLASS Application It is a requirement that GLASS start collecting surveillance data in the first quarter 2016. This means that a base GLASS system needs to be developed, tested and implemented by the end of 2015. The fundamental core of these milestones is development of a new tool for the robust, flexible collection, management, and reporting of data defined as routine surveillance in the GLASS manual. 2.4. Core functionality for GLASS application 1) Web-based solution 2) Friendly user interface (GUI) 3) Error correction and validation routines through the on-line system 4) User access and security levels 5) Standing data management 6) Ability to select from list of values 7) Generic interface (API) to load data from other systems 8) Support canned reports and adhoc queries in a user friendly manner 9) Back-up and recovery management 5
2.5. Some points to consider 1) The GLASS protocol stresses that nearly all data elements, including denominators, numbers of patients by specimen type and species, and antimicrobial resistance statistics be submitted stratified simultaneously by three variables: gender (male, female, unknown 3 values), infection origin (community or hospital origin or unknown 3 values), and age groups (<1, 1-4, 5-9, 10-14, 15-19,., 84+, unknown 20 values). As an example, rather than reporting that 90 patients in 2014 had blood isolates of Escherichia coli resistant to ciprofloxacin, the system must manage 180 (3 x 3 x 20) separate values for the various age group-gender-origin groupings that would add up to the total 90 patients. This large number of individual data points would require additional considerations: Data validation: how can one validate the accuracy or reasonableness of 198 individual data values? how can important mistakes in data entry (for example errors in dates of birth are common) or laboratory testing (for example, unconfirmed VRSA) be identified? Data editing: if mistakes or unlikely results are found, how can corrections be made to relevant subsets of the 198 data values? in some cases (especially for important questionable resistance findings), it may be worthwhile for laboratories to correct to the original, raw data files, while in other cases (incorrect ages) it would be more efficient to edit Data queries and reports: it will be rare that users will query results at the level of individual age group-gender-origin. The most common use of the data would require the 198 values to be aggregated to the total value. Consequently, it is a requirement the IT system permits the flexible display, dynamic aggregation, and configurable reporting (for example combining age groupings into higher level groupings) of the stratified GLASS data elements. 2) It is our expectation that the system be available to national and network coordinators, to submit data directly through a web interface with online data validation and feedback. This evolution would need to be considered in the initial design of the architecture. The expectation established above would require local automated feedback reports. Data validation could encompasses a number of issues including: Data quality check: are valid data values stored in the correct columns in a file with the correct structure? Microbiological plausibility: do findings suggest errors in data entry or laboratory performance, such as non-confirmed vancomycin-resistant S. aureus or carbapenemresistant Enterobacteriaceae in a country that has not yet witnessed any? Epidemiological biases: Are the biases in sample collection so extreme as to invalidate the value of the data submission? Laboratory testing biases: How significant are the issues of selective antimicrobial testing and selecting antimicrobial reporting? For example, if amikacin is only 6
tested in E. coli isolates resistant to a number of first-line agents, then the observed %R for amikacin will not be representative of all isolates processed. Data volume plausibility: does the data submission reflect the expected number of isolates in total and by organism, specimen type? Comparison with previous submissions: Are total numbers of isolates reasonably consistent with data from previous years? are %RIS statistics comparable over time? EARS-Net employs a five-level strategy for data validation would be a useful model to consider, and this is described in a later section. 3) Security management: Security of access to the database is a critical need, especially when the database would be accessible through on a public website. it will be critical to define various user roles with different levels of system access and functions.. 4) The system should permit more flexible and rich features for data queries and reports, including charting and map outputs. 5) In the five year road map of the implementation of GLASS, is the transfer of relevant, nonconfidential pages and features from the WHO GLASS internal application to public web pages for broad access to non-confidential data by the general public. A well-developed system with several years experience on which desirable query and report (tables, charts, maps, and export) features could be modeled would be the Interactive EARS-Net Database 3. It would be useful to know if the earlier implementation of a public web-based portal is feasible earlier than 2019 if the software tool developed for WHO staff is also web-based on WHO s Intranet. 6) Multiuser access control, i.e. features to handle multiple simultaneous users of the system is important as is transaction management. Transaction management refers to the ability of the system to roll-back data interaction such as record import or editing to a previous state. 7) Some potential future directions The GLASS timeline cited suggests a number of possible areas for future expansion of activities and collaborations not addressed in this document: AMR results from bacteria of food and animal origin Surveillance of antimicrobial use Case-based surveillance, as defined in the GLASS manual Healthcare associated infections (HAI) surveillance Multilingual support Ability to merge data files locally, e.g. from laboratories, hospitals, etc. 3 European Centre for Disease Prevention and Control. EARS-Net Interactive database, http://ecdc.europa.eu/en/healthtopics/antimicrobial_resistance/database/pages/dat abase.aspx 7
3. Data preparation and local validation On a periodic basis, GLASS participating members are requested to provide three types of data to the GLASS system: AMR Statistics Patient and sample denominators Management and Evaluation indicators 3.1. AMR Statistics GLASS data file preparation and validation The GLASS manual requests the determination of the number of bacterial isolates (first-isolate per calendar year) resistant, intermediate, and susceptible to requested antibiotics for eight pathogen groups stratified by sex, age group, and infection origin (hospital versus community). 3.2. Patient and sample denominators To better understand the epidemiological context of submitted data and to explore issues of diagnostic sampling rates, the GLASS protocol requests a number of denominator values to accompany AMR statistics submissions, notably: Population denominator: number of inhabitants covered by the network facilities. This would be obtained by data managers from official governmental or U.N. statistics offices Sample denominator data: number of patients by specimen type. The GLASS manual stresses that each of the above denominators should be stratified by sex, age group, and infection origin. 3.3. Management and Evaluator Indicators Annex 4 in the draft GLASS manual presents a number of Management and Evaluation (M&E) indicators for monitoring progress in a number of areas: Public health priorities targeted for surveillance: 3 indicators Surveillance structure: 6 indicators Core functions: 1 indicator Support functions (guidelines and training): 4 indicators Quality and outputs: 3 indicators Some of the indicators should be generated directly from the GLASS database, for example the number of countries reporting for each requested pathogen and specimen type. Others 8
would be requested through an annual survey or available through review of existing documents. The GLASS database should have the ability to generate relevant indicators from the database and to store responses to indicator surveys. Regarding the collection of indicator responses, this could be implemented as a feature within GLASS or alternatively other WHOrecommended survey tools could also be considered. 4. Data submission and validation 4.1. Data file import Once files with the appropriate data content and structure have been prepared locally, the next step would be for the country AMR surveillance staff to submit the needed AMR Statistics and Denominator files to the WHO GLASS system. All data submissions should be using GLASS-formatted files. At present, the GLASS protocol recommends the extensive stratification of most data elements by three variables: sex, age group, and infection origin. 4.2. Data validation Validation in the fully developed system could be based on the five-level model established and refined by EARS-Net: Step 1 Local data validation. Prior to submitting data to GLASS, data analysts should thoroughly review their own data for the different kinds of issues identified above. A much more thorough validation will be possible at this level than at higher levels since they have access to the full isolate-level data, preferably with quantitative susceptibility test measurements. Step 2 Data submission validation. Upon upload of EARS-Net data to TESSy, the system immediately checks to see whether the data file has the correct structure with valid data content. Errors or warnings will be issued if problems are identified. If the submitted file is valid, then the data submitter immediately receives an automatic feedback report similar in content to the EARS-Net Data Check Feedback Report including: number of isolates total and by species and organism, %RIS for priority organism-antimicrobial combinations, and alerts of priority unusual or unlikely findings such as VRSA. If this initial feedback is line with expectations, then the data submitter can approve the submission. Step 3 Results validation. The data submitter can then review the results more thoroughly for potential issues and biases: numbers of isolates and %RIS in line with expectations from previous years, identification of unexpected, unconfirmed 9
findings. This individual may choose to remove certain findings from the data submission. For example, if the file includes results for 1000 E. coli isolates of which only 50 have amikacin tested, then the data reviewer may wish to exclude amikacin from the final submission because of the likelihood of biases due to selecting testing practices. When the results are considered acceptable, the AMR program data submitter can approve this step. Step 4 Final approval by the country. When the results of all organisms and specimen types in the network s reporting plan have reached Step 3 approval, then a final review of the year s submission is performed by TESSy. When results are acceptable to the country coordinators, then the AMR program data submitter gives their final approval. Step 5 WHO GLASS validation. Once the data submissions have been finalized by the country coordinators, then the GLASS system can apply additional validation algorithms, in part repeating the steps taken above. In the case of warnings or alerts identified, WHO staff can request additional clarification from data submitters. In consultation with data submitters, GLASS coordinators may choose to delete suspect results, for example unconfirmed unlikely results (like VRSA) or significantly biased results (such as results for antibiotics only selectively tested against resistant isolates). 5. GLASS Database schema There are a number of reasonable strategies for implementation of the underlying database platform for GLASS, but the underlying data table and content needs will be relatively constant. Priority tables would include: Reference tables and code lists: These are relatively static lists defining the scope of the GLASS protocol including lists of countries, specimen types, organisms, antimicrobials, and susceptibility test reference method Antimicrobial susceptibility test statistics: This table stores the core organism and antimicrobial susceptibility test results needed for preparation of %RIS reports. It could be a valuable feature to store at least three versions of submitted data: 1) original data as submitted; 2) current live data reflecting manual edits and revisions; and 3) archival snapshots of filtered, validated data released in official reports. Denominators: The GLASS protocol requests a number of background denominator data on the patient population served and number of samples taken. Data submissions: These tables would track the history of data submissions and updates, including a record of who submitted and validated data. System users: This database will contain a list of all authorized users of the GLASS system with special consideration for defining user roles which will determine the level of system access and control of each user and user type. 10
6. Reports The ultimate value of the GLASS system will depend on the regular analysis and interpretation of submitted data and the dissemination of findings and conclusions to project stakeholders and collaborators through a variety of mechanisms. The requirement also includes the ability to interact with and provide feedback to National providers of data. 6.1 Metrics Section 3.2 defined a number of denominators (stratified by sex, age group, and infection origin) requested in the GLASS manual to be utilized in the calculation of a number of metrics. When provided with the stipulated numerators and denominators, the implemented GLASS system should be able to calculate the requested metric. Metric 1. Rate of patients sampled per specimen type per population covered. Example: Number of urinary cultures per 100,000 inhabitants Metric 2. Rate of patients with growth of non-susceptible bacteria per specimen type per priority species and antibiotic. Example: Number of samples with E.coli non-susceptible to fluoroquinolones, out of all blood-cultures Metric 3. Proportion of routinely sampled patients with growth of any bacteria per specimen type. Example: Number of patients with positive blood-cultures out of all blood cultures Metric 4. Proportion of samples with growth of non-susceptible bacteria of the species and antibiotic under surveillance per specimen type. Example: Proportion of E.coli nonsusceptible to fluoroquinolones out of all tested 6.2 Reports implementation Specifications should be created for report types: Internal reports for WHO staff members reflecting precisely the content of the database to be utilized in data validation Reports to validate data from and provide feedback to national providers of data Validated, filtered reports for general distribution in which unlikely or low-volume results are omitted. Automated filters may automatically exclude metrics based on too few or too low proportion isolates. (In EARS-Net, a minimum of 10 isolates are required to be included in the output this is much lower than the CLSIrecommended 30 isolates, but experience has shown that a number of low-resource or low-population countries would be omitted from analyses if a minimum of 30 isolates were required. 11
Interactive query and report features should be introduced, along with automated graph presentations and features for exporting and copying outputs to Excel, PowerPoint, and other desktop applications. Report features for denominator-based metrics should also be introduced. A valuable feature could include the preparation of automated maps, prepared in line with WHO policies on map presentations and recommended software. 12