Automated migration of CIOMS documents to a safety database via E2B data transfer

Transcription

1 Automated migration of CIOMS documents to a safety database via E2B data transfer Introduction During the last decade, companies have developed software solutions to collect clinical data, but there is no easy solution for existing documents, especially printed or scanned paper forms and reports. Advancements in scanning and automatic character recognition techniques (e.g. OCR) have facilitated the transformation from printed material to electronic documents, but it is still challenging to develop techniques to extract, classify and interpret data from scanned documents automatically. One good example of the difficulty is the parsing of CIOMS documents/forms used in pharmacovigilance. CIOMS documents are supposed to have a standardized lay out and well identified information blocks. The major problem is that the content of each block is free text. Therefore, using an OCR program to read scanned CIOMS documents will not be enough to obtain usable information. Some companies try to solve the problem by hiring a service provider for data entry in a low-wage country to re-enter data manually. Beside being quite expensive, this method has a major drawback: data entry personnel needs intensive training in order to understand the business comprehensively and might have difficulties in identifying misspelled information, inconsistencies and errors in the source documents. To address the problem, we have developed a methodology to analyze existing documents, define regions and fields to extract, and interpret collected data based on customer requirements. This method has been implemented and validated on large, real-life projects. Instead of trying a one-size-fits-all approach with a heavy-customized framework, we use a projectoriented strategy, which is more cost-effective and manageable. One major advantage of this method is that users are steadily involved in all project steps, from the functional specification definition to the validation/qualification process.

2 Process overview The final system we offer is able to: 1. Automatically extract raw data out of CIOMS forms (MS Word, PDF, scanned documents) by parsing the available information and using rules defined by the customer. 2. Report parse errors and allow for manual corrections and/or modifications. 3. Load data into a relational database for further processing. 4. Automatically recode products (safety database internal dictionary) and adverse events (MedDRA), according to user or regulatory agencies specifications 5. Give users from the business department a Web application to modify the parsed documents and make them E2B-compatible. 6. Automatically generate E2B documents (single or mass generation) according to E2B specification and customer requirements. 7. Generate additional reports as required by the business users. VOEGELIN system engineering AG Hauptstrasse 61 CH-4107 Ettingen / 11

3 Process description Extraction of raw data out of CIOMS documents We use a two-level parsing to extract raw data out of documents. The first parsing level is designed to identify valid blocks within documents. Advanced pattern-matching is used to cope with misspelled text. This is a recurring phase where patterns are tuned until all variations and deviations are successfully recognized and parsed. Supported input formats are MS Office or Open Office Word documents, PDF files, scanned documents as single files or wrapped in PDF documents. Other formats can be added on request. Special attention is given to non-text components such as check boxes. Once correctly parsed, blocks are loaded into a staging area for further processing. Picture 1: Automatically extracted raw data VOEGELIN system engineering AG Hauptstrasse 61 CH-4107 Ettingen / 11

4 Loading raw data into a relational database The second parsing level is the fine parsing step. Its goal is to analyze the content of the logical blocks to extract low-level information. This step uses data from the staging area to be able to refine parsing and extraction rules without having to repeat the document parsing step itself. Again, this is a recurring process where patterns are tuned until all usable information is extracted from the text blocks. Once successfully extracted, low-level data is stored in a custom data structure to easily generate reports, E2B documents or transfer files that will be imported into drug safety systems (Argus Safety, ARISg, etc.) or exchanged with health authorities worldwide. Data Mapping This step is of ten necessary to standardize data fields. This is almost mandatory for predefined fields (time units, quantity units, etc) that are supposed to match lookup tables in the target system or specification. This can be done using predefined or customer-defined tables. Simple mapping values or regular expressions can be used, as well as complex matching rules. Basic examples are the remapping of free-text country names to ISO 3166 codes, or the Duration Unit mapping in E2B documents (801=Year, 802=Month, etc.). Standard mapping rules such as E2B recoding are already implemented in the framework and can be used out-of -the-box. Online browsing and editing To allow users to control the whole process, a Web application (GUI) has been developed as part of the framework. This application allows users to visualize the output of the parsing processes and to apply necessary correction and/or recoding of data. All changes are tracked in order to meet validation requirements. The structure of the maintenance pages maps the structure of the CIOMS document to ease navigation. VOEGELIN system engineering AG Hauptstrasse 61 CH-4107 Ettingen / 11

5 Picture 2: Overview-screen showing all parsed CIOMS documents VOEGELIN system engineering AG Hauptstrasse 61 CH-4107 Ettingen / 11

6 Picture 3: Parsed data ready for clean-up and modification (section I II) VOEGELIN system engineering AG Hauptstrasse 61 CH-4107 Ettingen / 11

7 Picture 4: Parsed data ready for clean-up and modification (section III VI) VOEGELIN system engineering AG Hauptstrasse 61 CH-4107 Ettingen / 11

8 Picture 5: Parsed data ready for clean-up and modification (section VII XI) VOEGELIN system engineering AG Hauptstrasse 61 CH-4107 Ettingen / 11

9 Picture 6: Dynamic E2B message header data VOEGELIN system engineering AG Hauptstrasse 61 CH-4107 Ettingen / 11

10 Picture 7: Generated E2B data file Although a lot of work can be done automatically, there are always situations where it is easier to manually change data. This is f or example the case when the quality of scanned documents is so bad that the OCR program was unable to recognize parts of the documents. Using a GUI is much more user-friendly than editing text documents and can be done by end users without special training. The Web application is used to trigger reports or E2B file generation. In fact, once the basic parsing phase is complete, the iterative process of fine tuning data and generating output can be done by end users without IT support. VOEGELIN system engineering AG Hauptstrasse 61 CH-4107 Ettingen / 11

11 Reporting and Validation Using a central repository f or parsed documents makes reporting easy. Ad-hoc and/or validation reports can be generated to support the whole process. Picture 8: Qualification report All installation, configuration and execution tasks include validation and/or qualification documentation in compliance with governing regulatory requirements and industry standards. Conclusion This method has been successfully used to parse CIOMS documents and to load extracted data into an Argus Safety system using the standard E2B interface. Of course, this solution can be extended to any type of document formats, which have an identifiable structure (fixed layout or patterns) like e.g. Medwatch form or self developed forms. To discuss your specific needs and get a quote, please contact info@voesystem.ch VOEGELIN system engineering AG Hauptstrasse 61 CH-4107 Ettingen / 11