Data Immersion : Providing Integrated Data to Infinity Scientists. Kevin Gilpin Principal Engineer Infinity Pharmaceuticals October 19, 2004

Transcription

1 Data Immersion : Providing Integrated Data to Infinity Scientists Kevin Gilpin Principal Engineer Infinity Pharmaceuticals October 19, 2004

2 Informatics at Infinity Understand the nature of the science at a company Work backwards from scientists data analysis needs to an understanding of what data needs to be captured Continuously re-evaluate and refactor : scientific priorities, system performance, utility of 3 rd party software, conciseness and effectiveness of the database, methods of data capture and delivery, integration of 3 rd party applications Make strategic investments in 3 rd party software Integrate, integrate, integrate Connect 3 rd party applications together so that scientists can make good decisions about which tools to use 2

3 Infinity Informatics Architecture Data Upload Applications Metadata JDBC Driver IPI Search Electronic Lab Notebook Template- Driven Reports Microsoft Office LIMS Many systems for writing data Some formalized schemas and some ad-hoc ones Fewer systems for reading data Metadata/JDBC layer provides integrated access to the federated data 3

4 Analysis of Validated Data : Overview August 2004 IPI Search : Unified searching across Chemistry and Biology 1½-way integration with Spotfire for all graphical visualization Better Search Spotfire integration than Spotfire Search Quick-and-dirty Excel export New needs Excel data is dead ; no hyperlinks links or smart tags Push data to scientists Latest results from LTS screens in a formatted report Increasing recognition of the importance of the details behind each data point Should be easily accessible visible from IPI Search or Spotfire Detailed experimental conditions, analytical QC data for a specific data point, information on chemical synthesis and building blocks, side-by-side comparison of titration curves from different samples, detailed accounting of experimental conditions for each data point 4

5 Analysis Overview, continued New needs Enhanced integration 2-way integration between IPI Search and Spotfire Data exported into Office documents (Excel, PowerPoint) should be live The software system should pull together relevant project data for a particular sample into a browsable and printable format Give scientists more control over how data is exposed Ability to annotate specific values, indicate the empirical quality of data points, publish and unpublish data Enhancements to computational capabilities Curve fitting, plotting, clustering, disparity analysis, SAR tables, common substructure analysis May be implemented as stored procedures, Scitegic pipelines, or J2EE services Available from Spotfire or any other client application 5

6 Basic Searching 6

7 IPI Search 2.0 : Plan for the future Availability and user acceptance of Search 1.0 User feedback Breathing room for design and development of a next generation architecture Step back from the database, build a more complete model of the scientific environment Think in terms of experimental results, structure properties, algorithms rather than tables and procedures Move well-understood requirements out of the code and into a declarative model Evolve from reasoning about tables and procedures to reasoning about scientific concepts 7

8 Data Model Overview Spotfire, Office, Scitegic, and many reporting tools fetch their data through either JDBC or a Microsoft data access technology (OleDB, ADO.NET) Therefore data is accessed using SQL according to a logical data model Not through a custom API or XML web services Physical data schema is hidden behind a logical model Analytical functions are modeled as database tables A custom database driver transforms incoming SQL queries and executes them against the physical database Query performance is >80% dependent on the underlying physical database Occasionally the transformation logic must be improved 8

9 Modeling : Entities and Relationships Identify and organize the parts of the database that are important for data analysis Group physical tables and views into logical Source Systems Compound Registry, Compound Management, HTS, LTS, AnalyticalQC, Library Production, etc Assign user-friendly names to columns SAUREUS_INHIB_DMSO_CM S. aureus Primary Inhibition Define data keys and their relationships Project Experiment Plate Well Lantern Synthesis Step Library Aliquot Structure Sample Replicates Result 9

10 Modeling : Functions A function is modeled as a table Arguments and return values are modeled as columns Required arguments must be bound to a value or values in the SQL statement Return values are generated by the function and written into the output columns The physical table is generally a global temporary table Each function invocation is assigned a data set id function similarity_search in Search String, Min, Max, Search Method = Tanimoto out Structure Id, Tanimoto Value gtmp_similarity_search data_set_id integer input_smiles varchar structure_id varchar output_smiles varchar similarity_value number 10

11 Modeling : Other Range and enumeration Min and max observed values of experimental results Domain of enumerated values such as Library Name, Experiment Protocol Data types Annotate values with domain-specific data types Id, URL, SMILES, Float Significant digits for experimental results Statistical annotations Replicate count, standard deviation Units of measure 11

12 Basic Searching Leverage the entity model to Name everything Join source systems to each other Automatically determine how to drill into a value Write declarative rules on entities that determine how they are filtered, queried, and displayed 12

13 Drill-Down to Individual Results 13

14 Analytical QC for a Sample 14

15 IPI Search Spotfire 15

16 IPI Search Excel Smart Tags recognize Compound Ids and provide a link to the home page for that compound 16

17 Entity Home Page This page is for a particular Sample within a Project IPI Search, Office, Spotfire will all link to these Entity Home pages 17

18 JDBC Driver Data model defines entities (Structure, Sample, LTS, AnalyticalQC) and properties of the entities (Structure.Canonical Smiles, LTS.Kinase 1 KI) JDBC driver exposes entities as tables, and properties as columns on those tables JDBC Driver can be plugged in to Information Interaction Designer to directly expose the data model through Spotfire Spotfire uses the entity and property names as defined in the data model, and fetches data through the JDBC driver Nomenclature and data values are guaranteed to be consistent with IPI Search and other client applications 18

19 Spotfire Data Source Definition Define Data Source in Information Interaction Designer DatabaseMetaData interface is used by Spotfire to populate the DataSource tables and columns 19

20 Spotfire Interaction Model Definition Define Information Interaction Model in Information Interaction Designer by defining Columns and Joins 20

21 Future Spotfire Integration Enhancements Richer communication between IPI Search and Spotfire Open a new IPI Search with columns selected in Spotfire IPI Search may listen for user data point selections in Spotfire Integrate IPI Search home pages into Spotfire pages Show project data for a selected data point Integrate saved searches between IPI Search and Spotfire A saved IPI Search is defined by a SQL query (which references the logical entities and their properties) Spotfire is using the same named entities and properties as its information model, so a saved IPI Search may be opened directly in Spotfire Use the power of the function modeling capabilities to provide computational capabilities in Spotfire [e.g. cluster on demand] As prioritized and driven by Infinity user community 21

22 Gaps Analysis : IPI Search and Spotfire What does IPI Search do that Spotfire doesn t do? Multi-column results ( = 0.36 [2] ) Numerical result, replicate count, standard deviation all treated as an integral unit Significant digits Enhanced display of tabular data Ability to plug in an HTML page into the main display Designed for more user interaction with the data model Adding/removing columns from the search Generalized model for applying filters Rich annotation of attributes Show me all the available results for project X 100% browser hosted Lower learning curve for naïve/occasional users Seamless integration with HTML pages (e.g. sample home page, external links) 22

23 Extra Slides 23

24 IPI Search Usage Statistics 24

25 Build vs Buy Build unless there is a 3 rd party solution that Buy Build Robustly meets our requirements Has well designed integration capabilities And 2 of the following Is cheap Is easy to deploy Has proven capabilities Spotfire, Oracle, elabnotebook, DayCart, Pipeline Pilot, Matlab LIMS Data capture for LTS, HTS, analytical QC, animal experiments Point-of-entry data search and analysis (IPI Search) 25