Core Technology Development Team Meeting

Size: px

Start display at page:

Download "Core Technology Development Team Meeting"

Emmeline Shaw
5 years ago
Views:

1 Core Technology Development Team Meeting To hear the meeting, you must call in Toll-free phone number: Access Code: For international call in numbers, please visit:

2 Agenda v Updates regarding last meeting action items v biocaddie AHM slides v Indexing and metadata mapping plans v Brief progress reports from All Supported by the NIH grant 1U24 AI to the University of California, San Diego 2

3 Updates Action Items v Visit datamed.biocaddie.org and provide feedback for the new version, v0.2 via the prototype_issues repository in GitHub v Add user feedback from the BD2K AHM demo to the prototype_issues repository in GitHub v Prepare the biocaddie AHM slides for discussion during the next CDT meeting v Have the biocaddie posters from BD2K AHM available as flyers/handouts for the attendees Supported by the NIH grant 1U24 AI to the University of California, San Diego 3

4 DDI Core Technology Development Team

5 Introduction v Help users find accessible data By disease, data modality, demographics, etc. Engage these users and the data community v Assist data producers on how to publish data for maximal discoverability Tools to index, submit, visualize v Build a prototype that docks targeted pilot products RFAs for others to help build DataMed

Core Development Roadmap DDI architecture Setup website for searching for datasets Set up infrastructure for web portal Data ingestion Determine datasets Decide on scalable data/metadata input routes

identifier Implement Data identifier into the DDI Data indexing Set up indexing using metadata from WG 3.

6 Core Development Roadmap DDI architecture Setup website for searching for datasets Set up infrastructure for web portal Data ingestion Determine datasets Decide on scalable data/metadata input routes Metadata mapping Search function Implement the function for 3 repositories Feedback collection Github RFA for pilot on Harvester for DDI schema RFA announced Review, selection and award Data identifier Implement Data identifier into the DDI Data indexing Set up indexing using metadata from WG 3.0 Dataset result display Sort datasets Group metadata Terminology server Import ontology Integrate to Scigraph API Integrate autocomplete feature to prototype Interface design New interface for prototype v 0.2 Global statistics Wrap up of pilot projects PP 1.1 literature/dataset link: Advanced search for domain specific repository PP 2.1 Recommender System: Ranking results PP 2.2 isee/delve: Innovative visualization PP 3.2 PDB citation pipeline Usability study UI Analysis Ranking algorithm Results from PP 2.1 Search function Expand the function to 7 repositories Find similar datasets Search history Architecture Code refactoring September 2015 November 2015 Version 0.1 Version 0.2 We are here Usability study User study Track user s action Ranking algorithm Refine search results based on user s selection Report from WG 8 Data duplication problem Metadata management Personalized search Share/save search results User account Link dataset to external resources PubMed Grants Search algorithm Boolean/advanced search Data repository search function February 2016 Version 0.5 June 2016 Version 1.0

7 User Needs Analysis v Focused Semi-Structured Interviews 13 participants for about 30 mins in person or by phone v Proper guidance on data standardization and dataset linkage required v Visualization tool compatible unified agreeable formats that provide granular access to data decrease data format issues v Robust visualization tools and dashboard view of data enhances data visualization experience v Metadata challenges mitigated via standard ontologies and controlled vocabularies text mining user friendly tools to create metadata sufficient training

8 BioCADDIE prototype architecture Ingestion Indexing Repositories Metadata Ingestion ElasticSearch Data Sources Online datasets User Interface Funding Agencie s Publishers Data producers Terminology server

Data Indexing Pipeline Data Source 1. Configuration file developed by curator 2.

modules e.g. ID conversion, keyword extraction, data normalization 4.

9 Data Indexing Pipeline Data Source 1. Configuration file developed by curator 2. Extraction of metadata/data from data resource or dataset via ingestion module Cache information for further processing 3. Process metadata/data via a set of processing modules e.g. ID conversion, keyword extraction, data normalization 4. Mapping of metadata/data to metadata model(s) 5. Export to target endpoint(s)via export modules 6. Search via ElasticSearch APIs 12/1/15 Supported by the NIH grant #1-U24-AI to the University of California, San Diego 9

10 Data Indexing Pipeline Current Technologies 1. JSON based documents and services 2. mongodb (Apache 2 license) being used to manage cached dataset description documents 3. Processing pipeline components can take advantage of cloud deployment for scalability 4. Document processing coordinated via messaging queue (Apache ActiveMQ (Apache 2 license)) 5. ElasticSearch (Apache 2 license) being used as index endpoint Simple cloud deployment and management Sophisticated RESTful API Advanced index customization Full power of lucene and plug-ins 12/1/15 Supported by the NIH grant #1-U24-AI to the University of California, San Diego 10

11 Data Transformation 12/1/15 Supported by the NIH grant #1-U24-AI to the University of California, San Diego 11

12 Data Mapping 12/1/15 Supported by the NIH grant #1-U24-AI to the University of California, San Diego 12

13 ElasticSearch Basic Search 12/1/15 Supported by the NIH grant #xxxxxxxxx to the University of California, San Diego 13

$ElasticSearch via CURL curl -XGET 'https://data.biocaddie.org/biocaddie/pdb/_search' -d '{ "query" : { "match" : { "dataitem.keywords" : "GPCR" } },"size":100 }' > ES.$

14 ElasticSearch via CURL curl -XGET ' -d '{ "query" : { "match" : { "dataitem.keywords" : "GPCR" } },"size":100 }' > ES.out JSON data visualized via JSON Editor for the Mac wa/link?path=mac%2fjsoneditor 12/1/15 Supported by the NIH grant #1-U24-AI to the University of California, San Diego 14

Current Work with Data Ingest v Addition of more data repositories v Working with WG3 ensure consistency with metadata model Revisions of metadata model based on data currently

15 Current Work with Data Ingest v Addition of more data repositories v Working with WG3 ensure consistency with metadata model Revisions of metadata model based on data currently being ingested v Addition of enhancement modules Semantic Expansion Metadata Enhancement 12/1/15 Supported by the NIH grant #xxxxxxxxx to the University of California, San Diego 15

16 Ontology service roles v Indexing Process metadata v Searching Search expansion Brain tumor Glioblastoma, astrocytoma, Frontal lobe tumor, Suggestions (autocompletion) Fix typo (did you mean?) Facet management Search entity type: gene, disease,

17 Terminology services v Term/Phrase analyzer/normalizer Search and indexing Synonyms/hypo-hypernyms (term mapper) cancer à {carcinoma, sarcoma, } But not carcinoma à cancer Build metadata structure v Term mapper Semantic mappings Other Terminologies v Term processor Suggestions (autocompletion) Fix typo (did you mean?)

18 Terminology v Primary Disease (Condition) Drug (Chemical, Substance, Compound) Omic: Gene Protein v Auxillary Body Structure Procedure Laboratory Organism v Maybe Physical object

19 Sources v MeSH v SNOMED CT v NCBI v GO v HGNC v FMA Condition Chemical Omic Procedure Body Loc/Sys Organism v RXNORM v UBERON

20 Relationships v Is a, part of v Synonym v Sibling v MeSH à 10^6 v SNOMED CT à 10^6 v GO à 2x10^6 v NCI à 10^5 v HGNC à ~ 10^5 (synonyms)

21 Infrastructure v SciGraph Neo4j Graph database v ElasticSearch Term search

22 User Interface workflow Query Entry Entity Identification Expansion Query Execution biocaddi E backend Advanced filters Terminology server ElasticSearch Presentation Facets Organize results Visualization

23 MVC Structure Responsible for constructing ES query and retrieving search results. Model View User interaction; Responsible for rendering of model. Controller Responsible for responding to user input; Instructing the Model to respond to the user input. Generate corresponding ES index,type, search keyword, search fields, facets fields, filter fields. Integrate with terminology server.

24 User Interface v UI webpage address: datamed.biocaddie.org User name: biocaddie Password: biocaddie Supported by the NIH grant 1U24 AI to the University of California, San Diego 24

25 Pilot project integration PP Title Linking publications and underlying datasets using natural language processing Data Recommendation Using Machine Learning and Crowdsourcing Intelligent Search Expansion and Visualization of Datasets (isee-delve) Development of Citation and Data Access Metrics applied to RCSB Protein Data Bank and related Resources Integrated As As Specialized advanced search for GWAS datasets Ranking function based on citation metrics for GEO series data a) isee similarity metric in ElasticSearch b) DELVE implementation as exploratory search and visualization option. (i) for PDB (ii) for gene expression data Ranking function based on citation metrics (dataset mentions) for PDB data Supported by the NIH grant 1U24 AI to the University of California, San Diego 25

26 Refinement & Evaluation v v Agile approach: Iterative testing by users Feedback from users based on user requirement to inform development User participation facilitated by issue tracker software, GitHub: Biocaddie/prototype_issues ( System-centric evaluation (WG4): Evaluated in the context of pre-defined use cases. Queries and benchmark datasets being developed Evaluation using standard relevance metrics, such as precision, recall, mean average precision (MAP) and the F-measure. Supports the evaluation of improved search algorithms in future iterations. Supported by the NIH grant 1U24 AI to the University of California, San Diego 26

27 Evaluation & User-testing v User-centric evaluation: Collect user behavioral data to improve user interface design: Four key components of usability: 1. User Analysis 2. Function Analysis 3. Representation Analysis 4. Task Analysis Study representative sample of users interacting with the system while performing specific tasks. Analysis of verbal think-aloud protocols to capture and characterize the interaction between user and system in the context of a relevant task. Identify issues related to system usability, enhance understanding dataset search, and the knowledge and priorities brought to bear on it by domain experts. Supported by the NIH grant 1U24 AI to the University of California, San Diego 27

28 Timeline Y2 Q1 Sep. Nov., 2015 Y2 Q2 Dec Feb., 2016 Sep., 2015 Oct., 2015 Nov., 2015 Dec., 2015 Jan., 2016 Feb., 2016 Interface design Global sta)s)cs New interface for prototype v0.2 Improving interface based on feedback Searching algorithms Finder similar datasets Add Boolean search Add advanced search Add data repositories search Ranking algorithms Refine search results based on user's selec)on Report from WG 8 Dataset result display Sort datasets Group metadata Accessibility of dataset Summarize all returned results (isee- DELVE) Allow users to select mul)ple repositories Improve faceted browsering Personalized search Search history Share search results User account Save search results Link dataset to external resources PubMed Grants (via PubMed?) 28

29 Timeline Y2 Q1 Sep. Nov., 2015 Y2 Q2 Dec Feb., 2016 Sep., 2015 Oct., 2015 Nov., 2015 Dec., 2015 Jan., 2016 Feb., 2016 Data inges=on Data inges)on and indexing Metadata mapping Terminology server Import ontology Create UI broswer Integrate to Scigraph API Integrate autocomplete feature to core UI Integra=on of pilot projects Integrate PP 1.1 GWAS Finder Integrate PP 2.2 isee- DELVE for pdb Integrate PP 2.1 DataRank for GEO Explore gene expression data using PP 2.2 Integrate PP 3.2 for PDB Integrate PP 2.1 for other dataset Feedback collec=on Github Feedback form Documenta=on Source codes on github Tutorials Usability Study UI Analysis User Study Track user's ac)ons Data duplica=on problem Metadata management Architecture/Scalability Code refactoring Back up 29

30 Team members UCSD UTHealth Claudiu Farcas Muhammad Amith Jeffrey Grethe Xiaoling Chen Yueling Li Trevor Cohen Larry Lui Xiao Dong Burak Ozyurt Anupama Gururaj Min Jiang Oxford Todd Johnson Ruiling Liu Alejandra Gonzalez-Beltran Vidya Narayana Philippe Rocca-Serra Deevakar Rogith Susanna-Assunta Sansone Ergin Soysal Cui Tao NIH Hua Xu Yaoyun Zhang Ian Fore Ron Margolis Pilot projects team members Supported by the NIH grant 1U24 AI to the University of California, San Diego 30

31 Demo of prototype Supported by the NIH grant 1U24 AI to the University of California, San Diego 31

32 Indexing and metadata mapping plans v Next set of repositories to be indexed Timeline v Metadata mapping tools for repositories v Documentation for the workflow for mapping by the repositories ICPSR as a use case Supported by the NIH grant 1U24 AI to the University of California, San Diego 32

33 Ongoing work Task Status 1 Metadata Ingestion Import repositories PDB,GEO 2. LINCS 3. BioProject, ArrayExpress, GEMMA, dbgap 4. ICPSR Stable API details Ongoing Sample files 1.2 Metadata mapping Ongoing 1.3 Metadata management Ongoing 1.4 Indexing Ongoing 2 Terminology server 2.1 Develop terminology server 1) Imported terminologies (6) and validated them 2) Created UI-Browser for TS 3) Integration to Scigraph API 4) Create auto complete feature 09/01 10/09 Ongoing 10/ Integrate terminology server Ongoing Supported by the NIH grant 1U24 AI to the University of California, San Diego 33

34 Pilot project integration (Task 3) PP Presented to CDT / / / /01 As Integrated Specialized advanced search for GWAS datasets Ranking function based on citation metrics for GEO series data a) isee similarity metric in ElasticSearch b) DELVE implementation as exploratory search and visualization option. (i) for PDB (ii) for gene expression data Ranking function based on citation metrics (dataset mentions) for PDB data Completed On 09/22 10/21 9/01 Ongoing (12/31) Ongoing (11/30) Supported by the NIH grant 1U24 AI to the University of California, San Diego 34

35 Ongoing work Task Status 4 Interface Design 4.1 Global statistics Implemented 4.2 Design interface Ongoing 4.3 Implement new design Ongoing 4.4 Breadcrumb for website navigation Not started 4.5 Display most Accessed Datasets Not Started 5 Personalized search 5.1 Search history Implemented 5.2 Save search results Not Started 5.3 Share search results Not Started 5.4 User account - - Discussion Not Started 6 Searching/Ranking algorithms 6.1 Similar datasets Implemented 6.2 Data repositories search Ongoing 6.3 Boolean/advanced search Not Started 6.4 Refine search results based on user s selection Not Started Supported by the NIH grant 1U24 AI to the University of California, San Diego 35

36 Ongoing work Task Status 7 Display of results 7.1 Sort datasets Ongoing 7.2 What fields should be displayed? - Discussion 7.3 Browsing (grouping facets/metadata) Not started 7.4 Accessibility information Not started 8 Link to external resources 8.1 Pubmed Ongoing 8.2 Grants Ongoing 9 Feedback 9.1 GitHub Implemented 9.2 Feedback form Not Started 10 Documentation 10.1 Source code Not Started 10.2 Tutorials Not Started 11 Usability studies 11.1 UI Analysis Completed 11.2 User studies Not Started 12 Data Duplication issue Supported by the NIH grant 1U24 AI to the University of California, San Diego 36

37 Timeline Y2 Q1 Sep. Nov., 2015 Y2 Q2 Dec Feb., 2016 Sep., 2015 Oct., 2015 Nov., 2015 Dec., 2015 Jan., 2016 Feb., 2016 Interface design Global sta)s)cs New interface for prototype v0.2 Improving interface based on feedback Searching algorithms Finder similar datasets Add Boolean search Add advanced search Add data repositories search Ranking algorithms Refine search results based on user's selec)on Report from WG 8 Dataset result display Sort datasets Group metadata Accessibility of dataset Summarize all returned results (isee- DELVE) Allow users to select mul)ple repositories Improve faceted browsering Personalized search Search history Share search results User account Save search results Link dataset to external resources PubMed Grants (via PubMed?) 37

38 Timeline Y2 Q1 Sep. Nov., 2015 Y2 Q2 Dec Feb., 2016 Sep., 2015 Oct., 2015 Nov., 2015 Dec., 2015 Jan., 2016 Feb., 2016 Data inges=on Data inges)on and indexing Metadata mapping Terminology server Import ontology Create UI broswer Integrate to Scigraph API Integrate autocomplete feature to core UI Integra=on of pilot projects Integrate PP 1.1 GWAS Finder Integrate PP 2.2 isee- DELVE for pdb Integrate PP 2.1 DataRank for GEO Explore gene expression data using PP 2.2 Integrate PP 3.2 for PDB Integrate PP 2.1 for other dataset Feedback collec=on Github Feedback form Documenta=on Source codes on github Tutorials Usability Study UI Analysis User Study Track user's ac)ons Data duplica=on problem Metadata management Architecture/Scalability Code refactoring Back up 38

39 Other issues v Please deposit codes in GitHub. Please contact me at Anupama.E.Gururaj@uth.tmc.edu if you need access v Any other issues? v Thank You

Core Technology Development Team Meeting

Core Technology Development Team Meeting To hear the meeting, you must call in Toll-free phone number: 1-866-740-1260 Access Code: 2201876 For international call in numbers, please visit: https://www.readytalk.com/account-administration/international-numbers