Next Generation Science and Infrastructure Support

Transcription

1 Next Generation Science and Infrastructure Support James Lowey Director Network & Computing Systems TGEN

2 The Translational Genomics Research Institute (TGen) Non-profit Biomedical research institute Founded in 2002 in Phoenix AZ Bring Bio-Tech to Arizona Translational research Bench to Bedside

3 TGen (cont.) Commitment to technology Built #53 supercomputer in the world (top 500 list, Nov 2003) Build systems for large-scale genomic data processing Terabyte scale systems (2003)

4 Overview Computing in the Life Sciences space Bioinformatics In-silico research In 2003 the human Genome sequenced $3 billion 13 years Goal is $1,000 genome sequence

5 Microarray Technology Tool for analyzing gene expression Large amount of genes Survey a large number of genes quickly Data generation in 10s of Gigabytes

6 Required IT 1024 core supercomputer 1 TB temp file system 5-10 TB of storage 100 Mb networking

7 Rumblings of the Data Deluge Early 2006 started getting invited to scientific instrumentation vendors meetings.. New technology, Helicos, Illumina, ABI Terabytes of data per run, per instrument Get ready, because this is coming!!!

8 Next Generation Sequencing Evolution of sequencing Sequencing at unprecedented speed What it means to IT 100,000s of images 4-7 TB of data per run per sequencer 3-6 days to align sequences using 32 cores

9 First Stabs Recognize need for much larger scale storage 10 s of TB no longer adequate Multiple vendors Fibre Channel Direct attached storage NFS services

10 Challenges Backups Networking Management Expertise Cost

11 New Supercomputer NIH Grant - NCRR 1S10RR cores added to ASU HPCI cluster 4200 cores total 250 TB disk Lustre file system scratch space DDR Infiniband Tflop #82 in Nov 2008 Top500 list

12 Status Quo Purging data very difficult Keep everything forever Direct costs not felt Best Buy drives and other misconceptions

13 Second Phase Evolution NFS storage Cheap storage 8-10 TB file systems 72 hour fsck? One failure mode Constant management Data growth greater than anticipated

14 Second Phase Evolution (cont.)

15 Listening 5 year strategic roadmap Petascale computing Technology and solutions Total costs for research Much larger IT component than in the past

16 Requirements Single Name-space Moving data between storage points is very time consuming and expensive Data protection Failure modes > 10 TB unsure of recovery Highly available 24x7x365

17 Requirements (cont.) Permissions between disparate systems Windows Linux Mac Easy to manage Admins want to get out of storage management business Cost Effective Performance must be tolerable

18

19 Infrastructure

20 Infrastructure Support Networking Plumbing is key Time to move data is non-trivial! 6 TB /day Only fast as the slowest link! 1 Gb/E is the ENTRY point!!!

21 Petascale Backups? This is still an issue Time Money What is critical to backup? Answer is almost always everything In-line data compression Data exclusions Thinking of going to inclusive only model

22 Current Developments Single Namespace File system In-line compression for backups 10 Gb/E backbones, link to supercomputer Infiniband between supercomputer and downtown HQ

23 Things to do Tiered storage Increase performance Decrease costs for less frequently accessed data Increase data compression rate Local supercomputers Many core clustered computers closer to the sequencers Local storage for transient data storage during processing

24 Clouds Unanswered questions Clouds not yet capable of data ingest rates necessary Cost effective for small scale Cloud technologies such as Hadoop and map-reduce will transfer to private data centers

25 Next-Next Generation Scientific Data Increases Cost of storage Direct cost model Scalability and future-proofing Wired.com

26 Thank You Questions?

27