Dynamic Data Integration in Clinical Research

Transcription

1 Dynamic Data Integration in Clinical Research Judith R. Logan, MD, MS Department of Medical Informatics & Clinical Epidemiology Oregon Health & Science University Buenos Aires, June 16, 2011

2 in collaboration with Lois M. L. Delcambre, PhD James F. Terwilliger, PhD Scott Britell Vandana Kapoor Gabe Buckmaster with funding support from Collins Medical Trust National Institutes of Health, National Library of Medicine

3 The problem Data, data, data How do you we integrate these silos of data for research? *

4 The motivation

5 National Endoscopic Database ProVation Medical

6 Creating an analytical dataset From a single data source Data extraction Data transformation (units, categorization, etc) Loading into an analytical database From multiple data sources Repeat the same steps for each source then integrate the data Integrate or federate the databases first, then follow those steps

7 Our principles: The data analyst (the domain expert)should make the integration decisions. To do this, the data analyst should have an easy to understand data schema information about the context in which the data was collected tools to make integration easy Integration decisions should be dynamic, made at the time of the study and not just once

8 Overview of GUAVA and MultiClass SQL data sources SQL DBA dashboard channels G-trees Analyst dashboard Select records/field s Classification of data Analytical dataset File GUI-As-View (GUAVA) MultiClass

9 The G-tree An XML representation of the natural schema with contextual annotations The natural schema often mimics the user interface in which the data is collected one table per screen The contextual annotations are information about how the data is collected, like Was the data element required? What were the options for a menu or option group? What was the text on the screen?

10 The G-tree

11 - Patient - COLONOSCOPY The G-tree in a display + History + Physical Exam + Liver Disease - Indications - Evaluation of suspected malignancy (Group) Hereditary Nonpolyposis Colorectal Cancer Familial Adenomatous Polyposis Ulcerative Colitis Crohn s disease + History of colorectal cancer - History of adenomatous polyps History of breast or ovarian cancer Option group; Yes, No Age at diagnosis + Family history of adenomatous polyps + Family history of colorectal cancer + Abnormal XRay Average risk + Evaluation of Blood Loss (Group) + Therapeutic Procedure (Group)

12 G-tree as a query screen

13 Great! How do I get a G-tree? SQL Annotations

14 Overview of GUAVA and MultiClass SQL data sources SQL DBA dashboard channels G-trees Analyst dashboard Select records/field s Classification of data Analytical dataset File GUI-As-View (GUAVA) MultiClass

15 What the database expert (DBA) does The channel is a series of transformations defined by the DBA that change the database schema to the natural schema Vertical Split (VSplit) Vertical Merge (VMerge) Horizontal Split (HSplit) Horizontal Merge (HMerge) Pivot Unpivot Function Application (Apply) Renaming

16 What the database expert (DBA) does A query to the natural schema is transformed in the channel to a query that the database understands. The transformed query is then run against the database, returning data matching the natural schema The data itself is not transformed The analyst never has to know what the database looks like

17 Why have a channel? Employee table, generic schema ID Attribute Value 1 Name Joe 1 Gender male 1 Exp 1 Employee table natural schema ID Name Age Gender Exp 1 Joe male 1 2 Sue Name Sue 2 Age 31 2 Exp 4

18 O VSplit Operator Before: Patient Patient After: Patient_Details Patient_ID SSN FirstName LastName HomePhone CellPhone Patient_ID Patient_ID SSN HomePhone FirstName CellPhone LastName VSplit(Patient, Patient_Details, {SSN, FirstName, LastName}) Represents the decision to split some of the columns into their own table 18

19 O VMerge Operator Patient Before: Patient_Details After: Patient Patient_ID Patient_ID SSN HomePhone FirstName CellPhone LastName VMerge(Patient, Patient_Details) Patient_ID SSN FirstName LastName HomePhone CellPhone Represents the decision to merge two tables into a single table

20 O Pivot Operator Before: Patient Patient_ID Attribute Value Pivot(Patient, Patient_ID) After: Patient Patient_ID SSN FirstName LastName HomePhone CellPhone Represents the decision to transform a generic table into a traditional table.

21 O Unpivot Operator Before: Patient Patient_ID SSN FirstName LastName HomePhone CellPhone After: Patient Patient_ID Attribute Value Unpivot(Patient) Represents the decision to transform a table into (key, attribute, value) rows 21

22 Putting operators together to build a channel Physical DB DDL/DML O1 O2 O3 O4 On Queries Results Natural Schema

23 Overview of GUAVA and MultiClass SQL data sources SQL DBA dashboard channels G-trees Analyst dashboard Select records/field s Classification of data Analytical dataset File GUI-As-View (GUAVA) MultiClass

24 The Analyst Dashboard The data analyst selects the G-tree, which automatically links to the channel file and data source and is displayed as a tree which makes formulation of queries easy The data analyst designs a query with inclusion/exclusion criteria and a data dictionary Using the G-tree, the analyst maps the query elements to the data source For integrating data, the query elements are mapped to more than one data source The analyst runs the query to obtain an analytical dataset The analyst saves the query to be run again if needed

25 The Analyst Dashboard The data analyst designs a query with inclusion/exclusion criteria and a data dictionary The data analyst selects the G-tree, which automatically links to the channel file and data source and is displayed as a tree which makes formulation of queries easy Using the G-tree, the analyst maps the query elements to the data source For integrating data, the query elements are mapped to more than one data source The analyst runs the query to obtain an analytical dataset The analyst saves the query to be run again if needed

26 1. Define inclusion/exclusion criteria PolypIncidence.owl Study Inclusion criteria Exclusion criteria Data dictionary Data sources Mapping Results Recent Studies Filters allow you to limit the rows to be included in the result set. PolypIncidence.owl PolypRemoval2008.owl CecalIntubationRate.owl SedMedDocumentation.ow Name l Options Description Procs2008 Include colonoscopies performed in 2008 only AgeGT18 Include data on patients greater than age 18 only Exit Inclusion criteria Edit Save Delete Delete

27 2. Build the data dictionary PolypIncidence.owl Study Filters Inclusion criteria Data Exclusion criteria elements Data dictionary Data sources Mapping Results Recent Studies Data elements describe the columns to be included in the result set. PolypIncidence.owl PolypRemoval2008.owl CecalIntubationRate.owl Name Description Null? SedMedDocumentation.ow l Age Polyp Depth Options Age at time of procedure, may be calculated as date of procedure - date of birth Polyp found on examination Value Data Dictionary Exit Description Depth reached on examination Yes Value Description proximal terminal ileum to cecum Edit Delete distal cecum to rectum Save Delete Done

28 3. Select one or more data sources PolypIncidence.owl Study Inclusion criteria Exclusion criteria Data dictionary Data sources Recent Studies Select data sources to be used in the study. PolypIncidence.owl PolypRemoval2008.owl Open data source file: CecalIntubationRate.owl SedMedDocumentation.ow l Open data source file: Data Sources CORIv4.guv ProVation.guv Edit Edit Delete Delete Mapping Open data source file: Browse Results Add data source Options Exit Done

29 4. Map the elements to each data source PolypIncidence.owl CORIv4 Provider Patient dateofbirth gender Procedure Egd Colonoscopy procdate Procedure Recent Studies depthreached Finding Polyp Tumor Multiple polyps PolypIncidence.owl PolypRemoval2008.owl Depth = Proximal CecalIntubationRate.owl SedMedDocumentation.ow l Options depthreached Depth = Distal depthreached Mapping: Depth Data Element Exit in in cecum ascending colon hepatic flexure transverse colon splenic flexure descending colon sigmoid colon rectum cecum ascending colon hepatic flexure transverse colon splenic flexure descending colon sigmoid colon rectum

30 5. Run the query SQL data sources SQL channels GUAVA server G-trees Analyst dashboard Select records/field s Classification of data Analytical dataset File

31 What happens when the analyst clicks Run? The query is formulated using relational algebra and sent to the GUAVA server The GUAVA server applies the query to one end of the channel The query is transformed in the channel to an equivalent query in the appropriate database language (components in the channel are specific to the database system)

32 What happens when the analyst clicks Run? The transformed query is run against the database and retrieves all data, which are returned to the GUAVA server The data transformations (the mappings) are then applied The data sets are integrated The integrated data is sent to the data analyst who outputs it to a statistical package.

33 Saving your work This could be a lot of work. What if you want to run the study again? Or modify it a little? The study (the query) can be saved we have demonstrated the ability to save it as an Operational Data Model (ODM) file this file can be imported and rerun or modified

34 How have we used this? We have already seen the motivating example (CORI). Currently CORI has over 2 million records in the National Endoscopic Database and is beginning to receive ProVation data. The CORI data is from two versions We have demonstrated the ability to integrate data from a study on breast cancer, part captured electronically and part with scannable paper forms

35 SQL SQL File Our goal Integrate with statistical software Deploy in a grid computing environment such as the National Cancer Institute cagrid project

36 Publications Terwilliger JF, Delcambre LML, Logan J. The User Interface is the Conceptual Model. Proc 25 th International Conference on Conceptual Modeling (ER2006), Tucson, AZ. November 6-9, 2006, Logan JR, Terwilliger JF, Delcambre LML. Exploiting the User Interface for Tomorrow's Clinical Data Analysis. Journal on Information Technology in Healthcare, April 2008, 6(2): Reprinted from Today s Information for Tomorrow s Improvements 2007, an international conference addressing Information Technology and Communications in Health (ITCH 2007). Terwilliger JF, Delcambre LML, Logan JR. Querying Through a User Interface. Data and Knowledge Engineering, 2007;63: Logan JR, Britell S, Delcambre LML, Kapoor V, Buckmaster JG. Representing Multi- Database Study Schemas for Reusability. Proc AMIA Summits Trans Sci : PMC Terwilliger JF, LML Delcambre, JR Logan, D Maier, DW Archer, J Steinhauer, S Britell. Enabling the Revisitation of Fine-Grained, Clinical Information. Proc of the 1 st ACM International Health Informatics Symp (IHI '10). 2010;: Britell S, JG Buckmaster, JR Logan, LML Delcambre, V Kapoor. Providing Domain Analysts with Natural Schemas to Enable Clinical Research. Accepted for publication: Journal of Biomedical Informatics.