eveloping DataMed the current status

Transcription

1 eeloping DataMed the current status Hua Xu Core Deelopment Team (CDT) biocaddie AHM /8/17 Supported by the NIH grant 1U24 AI to the Uniersity of California, San Diego 1

2 Outline CDT Roles and Roadmap DataMed General Architecture Search engine Ne features Next steps 8/8/17 Supported by the NIH grant #1U24 AI to the Uniersity of California, San Diego 2

3 Roles of CDT Deelop a functional prototype of DDI DataMed Implement guidelines/suggestions by WGs Integrate systems/modules deeloped by pilot projects and supplements Engage end users into DataMed deelopment and ealuation 8/8/17 Supported by the NIH grant 1U24 AI to the Uniersity of California, San Diego 3

4 V0.1 DDI architecture Data ingestion Search function Feedback collection Data identifier Data indexing Dataset result display Terminology serer search engine Interface design Usability needs analysis Ranking algorithm Search function Architecture Wrap up of pilot projects RFA for pilot on Harester for DDI schema V0.2 CDT Roadmap Usability study phase I Ranking Metadata management V0.5 Personalized search Link dataset to external resources Search algorithm Metadata ingestion Import repositories Repository submission form Map metadata to DATs model 2.x NLP-based indexing/searching V1.0 Documentation Usability study phase II & III Data duplication issue Generation of benchmark datasets Terminology serer - indexing Visualization Personalized search Improe the tracking system Search/Ranking algorithms Similar datasets to be expanded Display of results Sort datasets Additional filters V2.0 V3.0 Pilot project integration Usability study Web API

5 Accomplishments Present DataMed V 1.5 No, 2016 V 2.0 Feb, 2017 V 3.0 Jul, 2017 Data ingestion Ingestion of 74 data repositories Implementation of DATS 2.2 metadata model Enhancement modules to indexing pipeline Search engine NLP serice Terminology serice ElasticSearch optimization Other functionalities Ealuation Benchmark data set / Data Retrieal challenge User surey 8/8/17 Supported by the NIH grant 1U24 AI to the Uniersity of California, San Diego 5

6 DataMed Architecture Ingestion Indexing Data Sources Repositories Online datasets Metadata Ingestion ElasticSearch User Interface Funding Agencies Publishers Data producers Terminology & NLP serer

7 Search engine architecture Model Vie Responsible for constructing ES query and retrieing search results. User interaction; Responsible for rendering of model. Controller Responsible for responding to user input, e.g.. generate corresponding search keyords, search fields, facets fields etc 8/8/17 Supported by the NIH grant 1U24 AI to the Uniersity of California, San Diego 7

8 Search engine orkflo Query Concept extraction NLP serer Synonym expansion Terminology serer Facets ElasticSearch Ranked results Search Refined Query 8/8/17 Supported by the NIH grant 1U24 AI to the Uniersity of California, San Diego 8

9 What s ne - NLP serer Goal leerage NLP approaches to extract biomedical concepts General biomedical concepts à MESH Fie specific types of entities: disease, chemical, gene, biological process, and cell line Uses Processing user entered queries Indexing metadata textual fields (e.g., description) Implementation mixture of existing tools and locally deeloped systems MetaMap Lite Machine learning based NER systems (e.g., based on CRF) Rule-based systems (e.g., dictionary lookup) 8/8/17 Supported by the NIH grant 1U24 AI to the Uniersity of California, San Diego 9

10 NLP ealuation Category Method Precision/recall Note Mesh term MetaMap Lite Gene CRF 89.95%/50.75% Trained on pubmed corpus, tested on dataset description Disease CRF 92.54%/88.89% Trained on pubmed corpus, tested on dataset description Drug CRF 91.08%/69.06% Trained on pubmed corpus, tested on dataset description Cell Line CRF Insufficient entities for ealuation Biological process Trained on corpus proided in Dictionary lookup 93.96% /76.8% QuickGo, tested on dataset description 8/8/17 Supported by the NIH grant 1U24 AI to the Uniersity of California, San Diego 10

11 NLP s effect on search No NLP With NLP expansion infap infndcg p@

12 What s ne - Terminology serer Goal leerage existing terminologies in the biomedical domain to facilitate search Uses Term expansion during indexing by the ingestion pipeline Query expansion for search engine Auto-completion for search engine Spelling correction for search engine 8/8/17 Supported by the NIH grant 1U24 AI to the Uniersity of California, San Diego 12

13 Terminology serer implementation Sources - Condition, Chemical, Gene, Procedure, Lab, Anatomy, Organism Terminology Source Concepts Relationships NCBI taxonomy 906,782 1,813,962 SNOMEDCT_US 315,904 5,414,108 MESH 254,883 2,947,524 FMA 83,282 1,105,781 GO 39,269 2,341,646 HGNC 38, ,996 TOTAL 1,725,407 7,578,017 Techniques Neo4j - graph Database SciGraph interface (RESTful API) for Neo4j Response time < 0.1 sec 8/8/17 Supported by the NIH grant 1U24 AI to the Uniersity of California, San Diego 13

14 What s ne - Ranking algorithm Current ranking algorithms ElasticSearch s implementation of probabilistic releance model called Okapi GM25 Citation count, such as GEO Ranking by citation counts (GEO), published time. Future implementation Ne releance ranking algorithms from the Dataset Retrieal challenge: Blind Releance Feedback algorithm and learning-to-rank algorithm 8/8/17 Supported by the NIH grant 1U24 AI to the Uniersity of California, San Diego 14

15 Highlights of DataMed 3.0 More repositories 74 Web API Fine grained User Tracking System Reporting of Broken Links Duplicated datasets Visualize data statistics (Diploid project) Schema.org markup

16 Dataset statistics 8/8/17 Supported by the NIH grant 1U24 AI to the Uniersity of California, San Diego 16

17 Web API API for DataMed - REQUEST API DataMed RESULT

18 User tracking system Search term 8/8/17 Supported by the NIH grant 1U24 AI to the Uniersity of California, San Diego 18

19 User tracking system List & rank order of results returned in the page 8/8/17 Supported by the NIH grant 1U24 AI to the Uniersity of California, San Diego 19

20 User tracking system Results & position in page clicked by user Scroll position 8/8/17 Supported by the NIH grant 1U24 AI to the Uniersity of California, San Diego 20

21 User tracking system - Uses Query logs for analyses Usability studies Information for most accessed data Information for ranking by user feedback 8/8/17 Supported by the NIH grant 1U24 AI to the Uniersity of California, San Diego 21

22 Broken links 8/8/17 Supported by the NIH grant 1U24 AI to the Uniersity of California, San Diego 22

23 Duplicated datasets 8/8/17 Supported by the NIH grant 1U24 AI to the Uniersity of California, San Diego 23

24 User statistics 8/8/17 Supported by the NIH grant 1U24 AI to the Uniersity of California, San Diego 24

25 Publications (10/2016-present) Papers Dong X, Zhang Y, Xu H. Search Datasets in Literature: A Case Study of GWAS, AMIA CRI/TBI Symposium, San Francisco, 2017 Cohen T, Roberts K, Gururaj AE, Chen X, Pornejati S, Hersh WR, Demner-Fushman D, Ohno- Machado L, Xu H. A Publicly Aailable Benchmark for Biomedical Dataset Retrieal: The Reference Standard for the 2016 biocaddie Dataset Retrieal Challenge. Database, 2017 (Accepted). Roberts K, Gururaj AE, Chen X, Pornejati S, Hersh WR, Demner-Fushman D, Ohno-Machado L, Cohen T, Xu H. Information Retrieal for Biomedical Datasets: The 2016 biocaddie Dataset Retrieal Challenge. Database, 2017 (Under Reie). Chen X, Liu R, Ozyurt B, Gururaj AE, Soysal E, Cohen T, Tiryaki F, Li Y, Zong N, Jiang M, Rogith D, Salimi M, Kim H, Rocca-Serra P, Gonzalez-Beltran A, Farcas C, Johnson T, Margolis R, Alter G, Sansone S-A, Fore I, Ohno-Machado L, Grethe J, Xu H, Bell E,. Building a search engine for finding biomedical datasets across repositories the DataMed system. JAMIA, 2017 (Under Reie) Dixit R, Rogith D, Narayana V, Salimi M, Gururaj AE, Ohno-Machado L, Xu H, Johnson T. User Needs Analysis and Usability Assessment of DataMed a Biomedical Data Discoery Index. JAMIA, 2017 (Under Reie) 8/8/17 Supported by the NIH grant 1U24 AI to the Uniersity of California, San Diego 25

26 Publications (10/2016-present) Abstracts/Presentations Systems Demo, AMIA Annual Symposium, 2016 Ingestion & Indexing Pipeline Abstract, AMIA Annual Symposium, 2016 DataMed Abstract, ICBO 2016 DataMed Abstract, BD2K AHM DataMed NLP Pipeline Abstract, AMIA Annual Symposium, Accepted biocaddie Dataset Retrieal Challenge Abstract, AMIA Annual Symposium, Accepted 8/8/17 Supported by the NIH grant 1U24 AI to the Uniersity of California, San Diego 26

27 Next Steps Data Ingestion Expand to 100+ data repositories, ith DATS 2.2 Deelop automated tools for data ingestion Search engine Ranking algorithms - implementation of pilot projects (Emory and UIUC) Deep search Common data elements (CDE) Other ne functionalities: Update Adanced Search Display most Accessed Datasets Word Cloud (search results isualization) Alerts to ne results from saed searches Ealuation and promotion User and performance ealuation 8/8/17 Supported by the NIH grant 1U24 AI to the Uniersity of California, San Diego 27

28 UCSD Claudiu Farcas Jeffrey Grethe Hyeoneui Kim Nansu Zong Stephen Trac Yueling Li Larry Lui Burak Ozyurt Oxford Alejandra Gonzalez-Beltran Philippe Rocca-Serra Susanna-Assunta Sansone NIH Ian Fore CDT members UTHealth Xiaoling Chen Firat Tiryakt Treor Cohen Anupama Gururaj Todd Johnson Deeakar Rogith Nina Salimi Ergin Soysal Cui Tao Hua Xu Ram Dixit Pilot projects team members Preious members Pratik Chaudhary Ruiling Liu Ron Margolis Saeid Pournejati Stephanie Ngyuen Supported by the NIH grant 1U24 AI to the Uniersity of California, San Diego 28

29 Thank you! For questions, please contact CDT at: 8/8/17 Supported by the NIH grant 1U24 AI to the Uniersity of California, San Diego 29

30 Preliminary ork on integrating CDE ith DataMed Hua Xu, PhD

31 Dimensions of DATS Dimension: meant to be used to report hat data points are about in a dataset, their nature, their units. Dimension must be typed biocaddie DATS Dimension as discussed ith Google schema.org team, ho added ariablemeasured property in schema.org to coer the notion. Application: List all measurement types performed in a clinical trial Purpose: allo indexing on measurement types / ariables / dimensions

32 Dimensions in DataMed As of May 2017, 66 repositories 1,375,801 datasets in DataMed. Collected dimension information for 4 repositories ImmPort (205/222) LINCS (286 /287) MPD (376/376) NeuroMorpho (50356/50356)

33 Mapping beteen Dimension and CDE NIH CDE Repositories 20,189 CDE across all initiaties, 19,985 unique CDE Examples: Smoking status Polysomnography blood oxygen saturation distribution percent alue In the past 7 days I felt comfortable ith others my age Thinking about your child's life, My child thinks his/her life has purpose. Dimension in DataMed 3083 unique dimension names from 4 repositories Examples: Blood Cell Count ith Differential Protein measurement Skin lesion inoled and skin lesion duration total horizontal distance traeled (immediately to 15 min post-injection), baseline Oerlap (exact matching) 6 only Smoking status, Medical history, Body eight, Age, Heart rate, Vital capacity

34 Mapping Dimension to CDE Dimension name axial length, right eye cardiac output hdl cholesterol left entricle ejection fraction serum itamin d concentration hole body bone mineral density (bmd) Obserations CDE axial length right eye cardiac output measurement hdl cholesterol alue echocardiogram left entricle ejection fraction measurement person serum itamin d leel number dual x-ray absorptiometry hole body bone mineral density alue Many datasets do not use CDE Some CDEs are represented as combination of seeral concepts, expressed as sentences Dimension and CDE may be related concepts at different granularity leels Synonyms and abbreiation may be inoled.

35 Query Backe Metadata nd - Dimensio n System architecture NLPbased CDE encodi ng serice Has CDE Ye identifi s er? DataMed n default o search algorithm Metadata enrichmen t ith CDE identifiers Search results Boosts applied to search CDE identifier in CDE enrichment field

36 Preliminary results Initial similarity algorithm based on Elasticsearch Examples: Dimension name axial length, left eye total cholesterol hdl cholesterol brain eight pulse rate left entricle diastolic olume non-hdl cholesterol Mapped to CDE axial length left eye total cholesterol alue hdl cholesterol alue brain eight measurement pulse rate measurement alue echocardiogram left entricle end diastolic olume measurement hdl cholesterol alue A lot body of of eight room at for dissection improement body eight at birth NLP CDE encoding algorithm can be applied to other applications here CDE recognition is needed.

37 Demo CDE enrichment using the algorithm for 4 repositories. ex.php hdl cholesterol ill mapped to "hdl cholesterol alue and suggest for a phrase search in dimension field. anced.php