In-Memory Databases: Applications in Healthcare

Transcription

1 Dr. Matthieu-P. Schapranow Apr 21,

2 Frankfurter Allgemeine Zeitung Verlagsspezial Medizin zwischen Möglichkeiten und Erfolg 17. April

3 Important things first: Where do you find additional information? Online: Visit we.analyzegenomes.com for latest research results, tools, and news Offline: Read more about it, e.g. High-Performance In-Memory Genome Data Analysis: How In-Memory Database Technology Accelerates Personalized Medicine, In-Memory Data Management Research, Springer, ISBN: , 2014 In Person: Join us for German Biotechnology Days Apr 22-23, in Cologne, Germany 3

4 IT Challenges Distributed Heterogeneous Data Sources Human genome/biological data 600GB per full genome 15PB+ in databases of leading institutes Human proteome 160M data points (2.4GB) per sample >3TB raw proteome data in ProteomicsDB Hospital information systems Often more than 50GB PubMed database >23M articles Cancer patient records >160k records at NCT Prescription data 1.5B records from 10,000 doctors and 10M Patients (100 GB) Medical sensor data Scan of a single organ in 1s creates 10GB of raw data Clinical trials Currently more than 30k recruiting on ClinicalTrials.gov 4

5 Our Approach Analyze Genomes: Real-time Analysis of Big Medical Data Oncolyzer Clinical Trial Assessment Pathway Topology Analysis... Drug Response Analysis Cohort Analysis Medical Knowledge Cockpit Real-time Analysis Access Control, Data Protection Data Exchange, App Store Statistical Tools Fair Use Extensions for Life Sciences App-spanning User Profiles Genome Data Research Publications Genome Metadata Pipeline and Analysis Models Cellular Pathways Combined and Linked Data Drugs and Interactions In-Memory Database 5

6 The Setting Actors in Oncology Patients Individual anamnesis, family history, and background Require fast access to individualized therapy Clinicians Identify root and extent of disease using laboratory tests Evaluate therapy alternatives, adapt existing therapy Researchers Conduct laboratory work, e.g. analyze patient samples Create new research findings and come-up with treatment alternatives 6

7 Our Motivation Make Precision Medicine Come Routine in Real Life Motivation: Can we enable clinicians to take their therapy decisions: Incorporating all available specifics about each individual patient, Referencing latest lab results and worldwide medical knowledge, and Interactively during their ward round? 7

8 Cloud-based Services for Processing of DNA Data Control center for processing of raw DNA data, such as FASTQ, SAM, and VCF Personal user profile guarantees privacy of uploaded and processed data Supports reproducible research process by storing all relevant process parameters Standardized Modeling and runtime environment for analysis pipelines Implements prioritized data processing and fair use, e.g. per department or per institute Supports additional service, such as data annotations, billing, and sharing for all Analyze Genomes services Honored by the 2014 European Life Science Award 8

9 Real-time Processing of Event Data from Medical Sensors Processing of sensor data, e.g. from Intensive Care Units (ICUs) or wearable sensor devices (quantify self) Multi-modal real-time analysis to detect indicators for severe events, such as heart attacks or strokes t Comparison of waveform data with history of similar patients Incorporates machine-learning algorithms to detect severe events and to inform clinical personnel in time Successfully tested with 100 Hz event rate, i.e. sufficient for ICU use 9

10 Drug Safety Statistical Analysis of Drug Side Effects Data Combines confirmed side effect data from different data sources Interactive statistical analysis, e.g. apriori rules, to discover still unknown interactions Integrates personal prescription data and directly report side effects Work together with your doctor to prevent interaction with already prescribed drugs Unified access to international side effect data On-the-fly extension of database schema to add side effect databases 10

11 Real-time Assessment of Clinical Trial Candidates Switch from trial-centric to patient-centric clinical trials Real-time matching and clustering of patients and clinical trial inclusion/exclusion criteria No manual pre-screening of patients for months: In-memory technology enables interactive prescreening process Reassessment of already screened or already participating patient reduces recruitment costs Assessment of patients preconditions for clinical trials 11

12 Drug Response Analysis Data Sources and Matching Patient-specific Data Tumor-specific Data Experiment Data Metadata e.g. smoking status, tumor classification and age Genome Data e.g. raw DNA data and genetic variants Enable Interavtive Data Exploration And Analysis Experiment Results e.g. medication effectivity obtained from wet laboratory 12

13 Drug Response Analysis Interactive Data Exploration Drug response depends on individual genetic variants of tumors Challenge: Identification of relevant genetic variants and their impact on drug response is a ongoing research activity, e.g. Xenograft models Exploration of experiment results is timeconsuming and Excel-driven Interactive analysis of correlations between drugs and genetic variants In-memory technology enables interactive exploration of experiment data to leverage new scientific insights 13

14 Medical Knowledge Cockpit for Patients and Clinicians Search for affected genes in distributed and heterogeneous data sources Immediate exploration of relevant information, such as Gene descriptions, Molecular impact and related pathways, Scientific publications, and Suitable clinical trials. Unified access to structured and un-structured data sources Automatic clinical trial matching build on text analysis features No manual searching for hours or days: In-memory technology translates searching into interactive finding! 14

15 Medical Knowledge Cockpit for Patients and Clinicians Publications In-place preview of relevant data, such as publications and publication meta data Incorporating individual filter settings, e.g. additional search terms 15

16 Medical Knowledge Cockpit for Patients and Clinicians Latest Clinical Trials Personalized clinical trials, e.g. by incorporating patient specifics Classification of internal/external trials based on treating institute 16

17 Medical Knowledge Cockpit for Patients and Clinicians Pathway Topology Analysis Search in pathways is limited to is a certain element contained today Integrated >1,5k pathways from international sources, e.g. KEGG, HumanCyc, and WikiPathways, into HANA Implemented graph-based topology exploration and ranking based on patient specifics Unified access to multiple formerly disjoint data sources Pathway analysis of genetic variants with graph engine Enables interactive identification of possible dysfunctions affecting the course of a therapy before its start 17

18 Medical Knowledge Cockpit for Patients and Clinicians Search in Structured and Unstructured Medical Data Extended text analysis feature by medical terminology Genes (122, ,771 synonyms) Medical terms and categories (98,886 diseases, 47 categories) Pharmaceutical ingredients (7,099) T Unified access to structured and unstructured data sources Clinical trial matching using text analysis features Indexed clinicaltrials.gov database (145k trials/30,138 recruiting) Extracted, e.g., 320k genes, 161k ingredients, 30k periods Select studies based on multiple filters in less than 500ms 18

19 Medical Knowledge Cockpit for Patients and Clinicians Seamless Integration of Patient Specifics Google-like user interface for searching data Seamless integration of individual EMR data Search various sources for biomarkers, literature, and diseases 19

20 Our Methodology Design Thinking Methodology 20

21 Our Methodology Design Thinking Methodology Desirability Leveraging directed customer services Portfolio of integrated services for clinicians, researchers, and patients Include latest research results, e.g. most effective therapies Viability Enable personalized medicine also in far-off regions and developing countries Share data via the Internet to get feedback from word-wide experts (cost-saving) Combine research data (publications, annotations, genome data) from international databases in a single knowledge base Feasibility HiSeq 2500 enables high-coverage whole genome sequencing in 20h IMDB enables allele frequency determination of 12B records within <1s Detection of 1 relevant annotation out of 80M <1s Cloud-based data processing services reduce TCO 21

22 Our Technology In-Memory Database Technology + Combined column and row store Map/Reduce Single and multi-tenancy Insert only for time travel Real-time replication Working on integers Active/passive data store Minimal projections Group key Dynamic multithreading Bulk load of data Objectrelational mapping No aggregate tables Data partitioning Any attribute as index On-the-fly extensibility Analytics on historical data Multi-core/ parallelization Lightweight compression SQL P v t SQL interface on columns and rows x x + Reduction of software layers T disk Text retrieval and extraction engine No disk 22

23 In-Memory Data Management Overview Advances in Hardware A Multi-core architecture (6 x 12 core CPU per blade) Parallel scaling across blades 1 blade 50k USD = 1 enterprise class server 64 bit address space 4 TB in current server boards 4 MB/ms/core data throughput Cost-performance ratio rapidly declining Advances in Software P Row and Column Store Insert Only Compression Partitioning Parallelization Active & Passive Data Stores 23

24 In-Memory Database Technology Use Case: Analysis of Genomic Data Analysis of Genomic Data Alignment and Variant Calling Analysis of Annotations in World-wide DBs Bound To CPU Performance Memory Capacity Duration Hours Days Weeks HPI Minutes Real-time In-Memory Technology Multi-Core Partitioning & Compression 24

25 In-Memory Database Technology Hardware Characteristics at HPI FSOC Lab 1,000 core cluster at Hasso Plattner Institute with 25 TB main memory Consists of 25 nodes, each: 40 cores 1 TB main memory Intel Xeon E GHz 30 MB Cache 25

26 + Combined Column and Row Store Row stores are designed for operative workload, e.g. Create and maintain meta data for tests Access a complete record of a trial or test series Column stores are designed for analytical work, e.g. Evaluate the number of positive test results Identification of correlations or test candidates In-Memory approach: Combination of both stores Increased performance for analytical work Operative performance remains interactively 26

27 Insert-Only / Append-Only Traditional databases allow four data operations: INSERT, SELECT and DELETE, UPDATE Insert-only requires only INSERT, SELECT to maintain a complete history (bookkeeping systems) Insert-only enables time travelling, e.g. to Trace changes and reconstruct decisions Document complete history of changes, therapies, etc. Enable statistical observations 27

28 Lightweight Compression Main memory access is the new bottleneck Lightweight compression can reduce this bottleneck, i.e. Lossless Improved usage of data bus capacity Work directly on compressed data Attribute Vector Data Dictionary ValueId Value 1 Larynx 2 Lip 3 Rectum 4 Colon Table Typical RecId compression ValueId factor 5 of 10:1 Mama for enterprise software RecId Colon C C18.0 RecId Larynx In financial 2 C32.0 applications up Inverted to 50:1 Index C32.0 RecId Lip C C Colon C18.0 ValueId RecIdList RecId 4 4 C18.0 RecId Rectum C C20.0 RecId Rectum C Mama 6 C20.0 RecId 7 C ,6 RecId Colon C C ,4,8 8 C

29 Partitioning Horizontal Partitioning Cut long tables into shorter segments E.g. to group samples with same relevance Vertical Partitioning Split off columns to individual resources E.g. to separate personalized data from experiment data Partitioning is the basis for Parallel execution of database queries Implementation of data aging and data retention management 29

30 Multi-core and Parallelization Modern server systems consist of x CPUs, e.g. Each CPU consists of y CPU cores, e.g. 12 Consider each of the x*y CPU core as individual workers, e.g. 6x12 Each worker can perform one task at the same time in parallel Full table scan of database table w/ 1M entries results in 1/x*1/y search time when traversing in parallel Reduced response time No need for pre-aggregated totals and redundant data Improved usage of hardware Instant analysis of data 30

31 P Active and Passive Data Store Consider two categories of data stores: active and passive Active data are accessed frequently & updates are expected, e.g. Most recent experiment results, e.g. last two weeks Samples that have not been processed, yet Passive data are used for analytical & statistical purposes, e.g. Samples that were processed 5 years ago Meta data about seeds that are not longer produced Moving passive data on slower storages Reduces main memory demands Improves performance for active data 31

32 x x Reduction of Application Layers Layers are introduced to abstract from complexity Each layer offers complete functionality, e.g. meta data of samples Less layer result in Less code to maintain More specific code Reduced resource demands Improves performance of applications due to eliminating obsolete processing steps 32

33 What to take home? Test-drive it yourself: For patients Identify relevant clinical trials and medical experts Start most appropriate therapy as early as possible For clinicians Preventive diagnostics to identify risk patients early Indicate pharmacokinetic correlations Scan for similar patient cases, e.g. to evaluate therapy For researchers Enable real-time analysis of medical data and its assessment, e.g. assess pathways to identify impact of detected variants Combined free-text search in publications, diagnosis, and EMR data, i.e. structured and unstructured data 33

34 Keep in contact with us! Hasso Plattner Institute Enterprise Platform & Integration Concepts (EPIC) Program Manager E-Health Dr. Matthieu-P. Schapranow August-Bebel-Str Potsdam, Germany Dr. Matthieu-P. Schapranow 34