Genie Developing An Enterprise Biotechnology Data System Mark Forster June 2004

Transcription

1 Genie Developing An Enterprise Biotechnology Data System Mark Forster June 2004

2 Syngenta - a leading global agribusiness Global Agribusiness Market $40bn+ (2001) Crop Protection $26 bn Commercial Seeds $12-$14bn Ag Biotech $1bn Strong worldwide market presence Leader in crop protection Highly placed in commercial seeds Formed Nov 2000 Merger of Zeneca and Novartis agribusiness units.

3 Our research centers of excellence Syngenta Biotechnology Inc., Research Triangle Park, North Carolina Jealott's Hill Research Triangle Park Manchester Basel/Stein Jealott s Hill International Research Center, UK Syngenta Manchester, UK Basel and Stein, Switzerland

4 Genie customers and applications. Crop genetics research scientists primarily in the US, UK and also other parts of Europe. Several application areas and hence software modules. DNA sequencing (Commercial - Finch) DNA material LIMS Plant Experiment LIMS Plant Characterization / Assay data Greenhouse Seedstore/Inventory Cross module queries Also includes genetic data used for marker assisted plant breeding. Gene markers and Allele database Interface to seeds system Oracle to SQL/Server interface

5 Why Genie? Complicated picture of multiple point solutions & overlap Ex Novartis sequence mgmt DNA Analysis Crop Genetics Research Pipeline. Ex Zeneca 4D plant database Plant expt mgmt Legacy Oracle DB system Local Excel sheets Plant analysis Ex Zeneca 4D plant database Gene information MS Access db Plant Plant Biotechnology Biotechnology Research Research Pipeline Pipeline DNA Analysis Ex Zeneca 4D plant database MS Access db Ex Novartis sequence mgmt Material QA/QC Plant varieties Plant analysis Gene information Ex Zeneca 4D plant database Local Excel sheets Glasshousemgmt Gene information Field trial applications SQL/Server +VB

6 Integrated data system, global access, tailored interfaces. Rich client Application (eg Genome Viewer) DNA material Plant Plant interface Experiment Assay Wireless PDA Interface. Interface. (Greenhouse) Genie Oracle database (UK) Replicate Plant Plant Biotechnology Biotechnology Research Research Pipeline Pipeline Genie Oracle database (US) Rich client Application (eg Genome DNA material interface Plant Experiment Interface. Plant Assay Interface. Seeds / Breeding System Viewer)

7 User view of Genie 3-tier architecture 1) Data stored at UK and US sites 2) All queries run locally 3) Modified data replicated. 4) Users unaware of replication Constructs, Transformation, Characterisation, Greenhouse Web Interfaces User Excel workbooks Handheld data capture and queries via wireless networking (Wi-Fi) Resilient, high performance, user transparent solution. VNTI Suite Desktop tools Oracle DBs Chemical Discovery DB Genomic visualiser Molecule Files Seeds / Breeding data system (SQL/Server) Oracle 9i Database Server On Sun V880 Oracle 9i App Server On Sun V880

8 Overall Software/Hardware Architecture GUI: Almost all HTML web forms, Java Server Pages (JSPs) easy deployment. Vector NTI for construct PC desktop visualisation. In-house Java client for QTL/Genomic visualisation. Middle Tier: Java servlets running under OC4J container in Oracle 9i App Server (v 9.0.3), integrated HTTP Server on 8 CPU 32 GB RAM Sun V880 server Database Oracle 9i DB ( ) on second 8 CPU 32 GB Sun V880 server All sets of database schema in single replicated Oracle instance Most but not all schema replicated (eg Finch is not).

9 Oracle 9i Application Server Oracle 9i AS Architectural Components Communication Business Logic Presentation Caching Services Services Services Services Database cache Oracle BC4J Apache Jserver Web Cache Oracle Oracle 9i JVM/OC4J Oracle JSP HTTP PL/SQL Oracle PSP Server Oracle Forms Perl (Apache) Content Management Portal Services Business Intelligence Services Oracle Portal Services Oracle Internet Portal to go Oracle Reports File System (IFS) Oracle Discoverer

10 Genie software architecture Detailed middle tier Java coding methodology. Java code developed for each module, runs in Oracle OC4J container under 9iAS. Implemented using Java Server Pages / Servlets and Model View Controller (MVC) design pattern (Struts open source framework) View component - JSP pages and presentation layer Controller - implements core application logic Model component - deals with saving objects (persistence to DB) Data Pipeline Implemented as cross module controller function calls eg (Plant Expts module. to Greenhouse or Assay module) Implementation within modules can change but code interface changes must be coordinated between modules.

11 Wireless PDA access for Greenhouse Logistics User needs: Ruggedised PDAs to survive wet/hot conditions. Data integrity requires barcode scanning => avoid user input errors. Greenhouse based queries on plant project, construct and other information => instant DB access required. Provided by simple HTML pages and b Wi-fi. Benefits: Allows facile setting and tracking of sample location. Extend functionality in 2004/5 to cover smart RF ID tags. Improved sample tracking.

12 Greenhouse - Wireless access security features. (i) (ii) (iii) MAC address filtering only accept input from known devices Virtual private networks encrypt server/client traffic. Symbol PDAs, Pocket PC VPN client and Cisco VPN server.

13 Software Development Process Integrated development environment (IDE) Predominantly Oracle Jdeveloper 9i Some usage of Eclipse IDE, some of vi. Not identical but not an issue. Source code control Concurrent version system (CVS) used by all teams. Widely used internet standard, used by 10,000s developers globally. Supported by IDEs such as Jdeveloper, Eclipse. Source code build Jakarta / Apache Ant build tool running on Sun Solaris. De facto standard for Java development. Command line tool, but integrates seamlessly into various IDEs.

14 Additional features of software architecture Authentication Oracle Internet Directory (OID) LDAP server on Sun V480R server Oracle Roles and Privs stored in OID. Support for database, application and authorisation from one company. User IDs fed down from Active Directory (LDAP) when available. Java to RDBMS persistence Predominantly 9iAS Toplink (Webgain Inc now acquired by Oracle) Some use of simple JDBC layer and/or hibernate code (open source). Database replication Oracle Advanced replication (AR) eventual solution. Oracle Streams evaluated but memory leak issues at high throughput.

15 Third party software components. Sequencing logistics module Finch server commercial solution from Geospiza Inc. Apache web app server, Perl middle tier code. Oracle DB schema integrated into Genie schema. Workflow engine. DRALA Java workflow engine used in constructs module. BioJava. Free, open source framework for processing biological data

16 Characterization database schema concepts. Substance, Sample, Vessel model. IVN+ISN Chosen for compatability with Chemical data model. Plants, seeds treated as samples, pots, seed bags, etc are then vessels. Assay results storage, descriptors. Problem avoid thousands of closely related assay type (DB columns) Eg Assay = Copy Number Descriptor 1 name = Gene of interest Descriptor 1 value = Name of my gene Assay result => 2.1 (experimental value for copy number). Solution Store as row major data table. Synonymns, attributes. Allow alternate unique (synonym) or non unique (attribute) names eg plant pedigree, generation codes etc.

17 Genie reporting/querying methodology. Trial development Portal, Discoverer and PL/SQL Server Pages (PSPs). Portal Portlets within page from multiple databases or datasources. Incorporates J2EE components JSPs, Java Servlets, EJBs. Open standards (Integrate data from Microsoft IIS / Other web servers, servers) Third party portlets (eg Documentum repository, news, maps, etc.) Local Installation difficulties. Discoverer Oracle Discoverer, reports created over EUL (end business layer) or business layer. Very rich and flexible report functionality. Delivery via web clients (Viewer or Discover Plus) or PC client. Not easy to access row major table structure. JVM (Jinitiator) compatibility issues with Discoverer Plus. Not able to fully customise UIF to match other web pages.

18 Genie reporting/querying methodology. PL/SQL Server Pages (PSPs). PL/SQL skills widely available and/or quick to build up. Deploy directly to database (load_psp) avoids coupling to middle tier Java apps. High performance Flexibility to manipulate tables

19 Final Genie reporting/querying methodology - PSP

20 Key Lessons. User Requirements / Development Process Change Extreme Programming (XP) highly iterative process preferred to waterfall strict phased approach. Weekly meetings and requirements refinement. System meets user needs - but hard to close down input. HTML input and output as requirements definition. Hard to find very subtle bugs Java classloader incorrectly assigned for classes across multiple software modules consultancy valuable. Performance Need to stress test early on to identify performance problems. Data Migration Earlier is better.