Engagement With Scientific Facilities

Transcription

1 Engagement With Scientific Facilities Eli Dart, Network Engineer ESnet Science Engagement Lawrence Berkeley National Laboratory Global Science Engagement Panel Internet2 Technology Exchange San Francisco, CA October 18, 2017

2 Outline The central role of the HPC center Data portals past, present, and future Value of cyberinfrastructure in Engagement 2

3 NCAR RDA Data Portal Let s say I have a nice compute allocation at the ALCF climate science Let s say I need some data from NCAR for my project Data sets (there are many more, but here are two): (1.5TB) (430GB) Download to ALCF (could also do NCSA or NERSC or OLCF) 3

4 HPC Centers Matter Computing centers are special Centers of excellence / expertise Data repositories Computing for simulation, data analysis Really though the people + cyberinfrastructure combo is what matters People who know how computers, networking, and storage work Enough resources to make things happen Computing facilities are anchors for many collaborations Common pattern: multi-institution team with access to one HPC center Shared data, analysis, simulation platform 4

5 HPC Centers Scale Up With Data My laptop isn t a big computer 4 CPU cores, 16GB RAM 1TB of storage (before installing the OS) Makes no sense for large-scale data processing HPC center is where large-scale work gets done Campus computing center National center (NSF, DOE, etc.) HPC centers should be a focus for large-scale data operations Also, making one HPC facility work well helps a lot of scientists 5

6 The Petascale Project Built on top of the Science DMZ model Effort to improve data transfer performance between the DOE ASCR HPC facilities at ANL, LBNL, and ORNL, and also NCSA. Multiple current and future science projects need to transfer data between HPC facilities Performance goal is 15 gigabits per second (equivalent to 1PB/week) Realize performance goal for routine Globus transfers without special tuning Reference data set is 4.4TB of cosmology simulation data 6

7 Cluster Performance HPC Facilities alcf#dtn_mira ALCF June 2017 L380 Data Set 22.9 Gbps 25.7 Gbps 27.2 Gbps 19.4 Gbps 23.0 Gbps nersc#dtn NERSC 20.6 Gbps 19.7 Gbps olcf#dtn_atlas OLCF 20.2 Gbps 15.1 Gbps 11.8 Gbps 21.2 Gbps ncsa#bluewaters NCSA 15.2 Gbps Data set: L380 Files: Directories: 211 Other files: 0 Total bytes: (4.4T bytes) Smallest file: 0 bytes (0 bytes) Largest file: bytes (11G bytes) Size distribution: 1-10 bytes: 7 files bytes: 1 files 100-1K bytes: 59 files 1K - 10K bytes: 3170 files 10K - 100K bytes: 1560 files 100K - 1M bytes: 2817 files 1M - 10M bytes: 3901 files 10M - 100M bytes: 3800 files 100M - 1G bytes: 2295 files 1G - 10G bytes: 1647 files 10G - 100G bytes: 3 files 7 ESnet Science Engagement (engage@es.net) , Energy Sciences Network

8 Science Data Portals Large repositories of scientific data Climate data Sky surveys (astronomy, cosmology) Many others Data search, browsing, access Many scientific data portals were designed 15+ years ago Single-web-server design Data browse/search, data access, user awareness all in a single system All the data goes through the portal server In many cases by design E.g. embargo before publication (enforce access control) 8

9 Legacy Portal Design Border Router perfsonar Firewall WAN Enterprise perfsonar Very difficult to improve performance without architectural change Software components all tangled together Difficult to put the whole portal in a Science DMZ because of security Even if you could put it in a DMZ, many components aren t scalable Can we do better? Browsing path Query path Data path Portal server applications: web server search database authentication data service Portal Server Filesystem (data store) 9

10 Next-Generation Portal Leverages Science DMZ Border Router perfsonar Firewall WAN Enterprise perfsonar Data Path Browsing path Query path perfsonar Science DMZ Switch/Router Portal server applications: web server search database authentication Portal Server Data Transfer Path Portal Query/Browse Path Filesystem (data store) API s (data access governed by portal) 10

11 Put The Data On Dedicated Infrastructure We have separated the data handling from the portal logic Portal is still its normal self, but enhanced Portal GUI, database, search, etc. all function as they did before Query returns pointers to data objects in the Science DMZ Portal is now freed from ties to the data servers (run it on Amazon if you want!) Data handling is separate, and scalable High-performance s in the Science DMZ Scale as much as you need to without modifying the portal software Outsource data handling to computing centers Computing centers are set up for large-scale data Let them handle the large-scale data, and let the portal do the orchestration of data placement 11

12 Data Portal Implications Portals hold a lot of valuable data Observations (sky surveys, satellite data, genomes, etc.) Many have been in place for years Most are inadequate to support large-scale analysis Legacy search/query interfaces Legacy access protocols/tools This is in the process of changing The technology exists to radically improve the utility of data portals What should the performance expectation be? HPC facilities can do 1PB/week if data portals could do this 12

13 NCAR RDA Performance to DOE HPC Facilities 1.5TB data set 1121 files NCAR RDA rda#datashare 13.9 Gbps 16.6 Gbps 11.9 Gbps nersc#dtn NERSC alcf#dtn_mira ALCF olcf#dtn_atlas OLCF 13

14 Central Cyberinfrastructure Is Key Many users are concentrated at major cyberinfrastructure sites HPC centers Gene sequencers Cryo-EM instruments Light sources, neutron sources, FELs Lots of leverage If you are engaging with a science team, it s very helpful to have access to stuff that works well Demonstration Best practice Design patterns Scientists can be your advocates as well as your customers They can ask for upgrades/improvements Now your goals are aligned everybody wins 14

15 In conclusion ESnet s vision: Scientific progress will be completely unconstrained by the physical location of instruments, people, computational resources, or data. 15

16 Links and Lists ESnet fasterdata knowledge base Science DMZ paper Science DMZ list Send mail to with subject "subscribe esnet-sciencedmz perfsonar Globus 16 ESnet Science Engagement , Energy Sciences Network

17 Thanks! Eli Dart Energy Sciences Network (ESnet) Lawrence Berkeley National Laboratory