Research Related Activities

Transcription

1 Research Related Activities Lennart Johnsson

2 Research Infrastructure Research

3 Science and Engineering Research Infrastructure Observations Collaborators are increasingly chosen regardless of location Instruments are used remotely Interesting data is elsewhere Response Shared global infrastructures Networks (high-capacity, pervasive, customized) Software (grids) Collaboration environments

4 Global Lambda Integrated Facility Visualization courtesy of Bob Patterson, NCSA.

5 NLR Southern Route Preferred Alternative Wiltel Fiber in the West -- AT&T Fiber in the Southeast Seattle Sunnyvale Denver KC Chicago Clev Pitts New York Wash DC LA San Diego Phoenix Albuq. Tulsa Dallas Atlanta Raleigh El Paso - Las Cruces San Ant. Houston Pensacola Baton Rouge Jacksonville

6

7 NLR Ring: TMC-Rice-UH-AT&T-WilTel

8 Intercampus Ring: TMC-Rice-UH

9

10 λ-networks offers many new research challenges and offers the ability to enhance Quality of Service and establish customized (global) networks Shared, dynamic, inhomogeneous, unreliable environments pose many research challenges in software and algorithm design for correct and predictable results and efficient use

11 TLC 2 Computing, Storage and Visualization Facilities

12 High-demand large-scale applications A few examples

13 CS Retreat High-Energy Physics Truly heterogeneous system: People, languages, time zones Complex collaborative effort LHC Computing Grid (LCG) LCG prototype service ( ) ALICE Physics production OSU/OSC LBL/NERSC Houston Birmingham US-ATLAS DC1: Part of simulation; Pile-up; reconstruction EDG DC1: several tests (1st test in August02) NorduGrid DC1: full production Dubna NIKHEF RAL Nantes Saclay Krakow GSI Karlsruhe Budapest CERN Merida Lyon Padova IRB Torino Bologna Bari Cagliari Yerevan Catania Kolkata, India Capetown, ZA Grid in ATLAS DC1 (July 2002 April 2003)

14 Astronomy Impending Floods of Data The planned Large Synoptic Survey Telescope (LSST) will produce over 10 petabytes per year by 2008! All-sky survey every few days, so will have fine-grain time series for the first time

15 VLBI

16 E-MERLIN LOFAR LOFAR VLBI, 300 TB/day, 100 TFlops/s E-MERLIN (UK), Six Telescopes at 30 Gbps each LOFAR, regional (Holland) 20 Tbps European VLBI Network, 30 Gbps 2 Tbps

17 Houston BioGrid JEOL300 0-FEG Liquid He stage NSF support 500 Å No. of Particles Needed for 3-D Reconstruction Resolution B = 100 Å 2 B = 50 Å Å 4.5 Å 6,000 5,000,000 3, , Å Structure of the HSV-1 Capsid

18 Houston BioGrid EMAN Reconstruction Vitrification Robot EMAN Particle Selection Initial Power Spectrum 3D Model Analysis Micrographs 4-64 Mpixels, 16-bit (8 128 MB) /day per lab 10 1,000 particles per micrograph EMAN Database Archival Data Mining Management Classify Particles Reproject 3D Model Several TB/yr Project ,000+ micrographs 10,000 10,000,00 particles 10k 1,000k pixels/particle Align Average Deconvolute Build New 3D Model Up to hundreds of PFlops

19 Houston BioGrid Proteomics

20 National Digital Mammography Archive About 40 million mammograms/yr (USA) (estimates million) About 250,000 new breast cancer cases detected each year Over 10,000 units (analogue) Image size: 4kx6k, about 48 MB Images per patient: 4 Data set size per patient: about 200 Mbytes Data set per year: about 10 Pbytes Data set per unit, if digital: 1 Tbytes/yr, on average

21 The National Mammography Archive (NMA)

22 Telesurgery Avenue of the Americas, New York Strasbourg University Hospital Gallbladder surgery

23 Telemicroscopy UHVEM (Osaka, Japan) 1 st (Chicago) Tokyo XP STAR TAP TransPAC Globus vbns NCMIR (San Diego) (San Diego) SDSC CRL/MPT UHVEM (Osaka, Japan) 2 nd UCSD NCMIR (San Diego) Courtesy: Mark Ellisman

24 Severe Weather Prediction Data acquisition and integration from multiple platforms and instruments for quick exploitation Intra-project communications before, during, and after CAMEX campaigns Courtesy: Sara Graves, University of Alabama in Huntsville

25 Q: Why Do Some Severe Storms Produce Multiple (Cyclic) Tornadoes? Create Climatology Based Upon All Observations Data Assimilation The Process in LEAD Linked Environments for Atmospheric Discovery Data Cloud Data Mining Real Time Control of Radars Real-Time WRF Runs on Grid Only When Environment is Primed and Storms are Present Gridded Assimilated Data Sets 500 Storm Simulations 300 Tbytes Data and Metadata On-Demand Resource Scheduling My LEAD Virtual Public Space

26 Weather Detection Algorithms NETRAD/CLEAR Optimal Control Systems Signal Processing, Quality Control Engineering and Scientific Research: Sensor Hardware, Middleware, Storage, Communications, Computing The Distributed Collaborative Adaptive Sensing System NETRAD Data Commands NETRAD Data NETRAD Data Data QC & Assimilation Algorithms Hazard Detection Algorithms Optimal Control Systems Meta Data Creation, Real Time Data and Meta Data Distribution via Web, Storage/Archival NETRAD Data Model Output NETRAD Data Experimental Products Processed Data Algorithm Detections Experimental Products Experimental Numerical Weather Prediction Models Model Output Web Portal Experimental Products Graphical Products Products, Tools, Training Private Sector: Value-Added Products and Services, Decision Support Systems Oklahoma Climate Survey Products, Tools, Training Public/Front-Line Responders

27 March 28, 2000, Fort Worth Tornado Weather Courtesy Kelvin Droegemeier

28 In 1988 NEXRAD Was Becoming a Reality Courtesy Kelvin Droegemeier nsf

29 Environmental Monitoring

30 File Sharing performance 1.6 M downloads/day 150 M searches/day 10 TB data transfer/day 1-2 TB data transfer/day 100 servers servers

31 Estimated Bandwidth Needs Discipline Size of Archives Growth rate of Archives Bandwidth needs Life Sciences Mammography PB PB/yr ~ 10 Gbps Microscopy 100+ PB PB/yr 1 10 Gbps Other imaging 200+ PB 100+ PB/yr Gbps Major medical center 100 1,000PB ~100 Gbps Earth Sciences Weather 1 10 Gbps Climate 100+ PB PB/yr Gbps Environment ~ 10 Gbps High-Energy Physics 100+ PB PB/yr Gbps Astronomy 100+ PB PB/yr Gbps Telemedicine, Telescience 1-10 Gbps Collaboration 1 Gbps Remote steering 10 Gbps Remote Visualization 10 Gbps Computation 100 1,000 Gbps

32 Research Adaptive Software for HPC in Science and Engineering in shared (global) environments/grids Problem Solving Environments (in Grids) Network protocols and management

33 GrADS Grid Application Ken Kennedy Ruth Aydt Francine Berman Andrew Chien Keith Cooper Jack Dongarra Ian Foster Dennis Gannon Lennart Johnsson Carl Kesselman Dan Reed Richard Wolski Linda Torczon Development Software

34 Performance Tuning Methodology Input Parameters System specifics, User options UHFFT Code generator Input Parameters Size, dim., Library of FFT modules Initialization Select best plan Performance database Installation Execution Calculate one or more FFTs Run-time

35 VGrADS

36

37

38 SimDB