Building a Global Data Federation for Climate Change Science The Earth System Grid (ESG) and International Partners

Transcription

1 Building a Global Data Federation for Climate Change Science The Earth System Grid (ESG) and International Partners 24th Forum ORAP Cite Scientifique; Lille, France March 26, 2009 Don Middleton National Center for Atmospheric Research Computational and Information Systems Lab On behalf of ESG and its Executive Team: Dean Williams - PCMDI, Ian Foster - ANL, Dave Bernholdt - ORNL 1

2 The National Center for Atmospheric Research Managed by the University Corporation for Atmospheric Research Mesa Campus; Boulder, Colorado, USA 2

3 CNN, on IPCC AR4 Report April

4 Modeling the Global Earth System Atmosphere Hydrosphere Cryosphere Biosphere 4

5 The Community Climate System Model (CCSM) 5

6 The Community Climate System Model (CCSM) 5

7 CMIP/IPCC Multimodel Archive 6

8 CMIP/IPCC Multimodel Archive 6

9 Towards Global Earth System Modeling 7

10 Towards Global Earth System Modeling CAM3 at T341 Resolution (about 35km) 7

11 Towards Global Earth System Modeling 1/10d POP Ocean Model CAM3 at T341 Resolution (about 35km) 7

12 Towards Global Earth System Modeling 1/10d POP Ocean Model MOZART Chem Model CAM3 at T341 Resolution (about 35km) 7

13 Living in a Complex Petascale Data Environment Supercomputers Observed Satellites Reanalysis and Models Data Data 8

14 Living in a Complex Petascale Data Environment A Distributed Science Enterprise Large-scale Distributed Resources Virtual Organizations Supercomputers Observed Satellites Reanalysis and Models Data Data 8

15 The Earth System Grid 9

16 The Earth System Grid NOAA/PMEL 9

17 The Earth System Grid 9

18 The Earth System Grid ESG provides access to 250TB of CCSM, IPCC,CSIM, CLM, POP, PCM data; the CCSM model, and analysis and visualization tools. 9

19 The Earth System Grid DOE SciDAC2 Center for Enabling Technologies Global Climate Grid ESG provides access to 250TB of CCSM, IPCC,CSIM, CLM, POP, PCM data; the CCSM model, and analysis and visualization tools. 9

20 The Earth System Grid DOE SciDAC2 Center for Enabling Technologies Science Gateway Global Climate Grid ESG provides access to 250TB of CCSM, IPCC,CSIM, CLM, POP, PCM data; the CCSM model, and analysis and visualization tools. 9

21 An Operational DataGrid for Climate Research 10

22 An Operational Archive for CMIP-3/IPCC Hosted at LLNL Program for Climate Model Diagnosis and Interpretation 11

23 Existing ESG Architecture 12

24 A Few Metrics 13

25 A Few Metrics Across all ESG portals, approximately 13,500 registered users Over 250TB of climate simulation data available, from models and centers around the world Over 600TB & 2 million files delivered Current rate of data delivery approximately 850GB/day More than just data: model source code, initialization datasets, analysis and visualization tools Approaching 500 scientific papers authored via analyses of ESG data hosted at DOE/PCMDI 13

26 January 14

27 Over 500 Sites Ingesting ESG Products Image courtesy Gary Strand, NCAR 15

28 What if, for a given problem, all the scientists in the world could analyze all of the data in the world? 16

29 Publications Using ESG/CMIP Data 17

30 Publications Using ESG/CMIP Data Data format and metadata convention standards were key 17

31 18

32 Science Gateway Framework Distributed, federated Support for discipline/ theme oriented gateways (skinning) Semantic-based search Browser and native application access 19

33 ESG-CET Architecture The ESG gateway is a site which supports portal services. End users interact with a portal to search and download data and data products. Users 20 20

34 User-centric digital identity e.g. Open, decentralized, free Estimated: 500M OpenID s, 30K sites accepting IBM, Microsoft, Google, Verisign, PayPal, FaceBook as corporate board members (BBC, Orange, SourceForge adoption) 21

35 Semantic Search 22

36 Portal-based Visualization and Analysis 23

37 ESG Testbed 24

38 ESG Testbed 24

39 ESG Testbed 24

40 ESG Testbed 24

41 ESG Testbed 24

42 ESG Testbed 24

43 25

44 Building Community The Global Organization for Earth System Science Portals June 2007 GO-ESSP Meeting in Paris, France 26

45 A Few ESG-CET Interactions 27

46 The Earth System Modeling Framework Courtesy Cecelia DeLuca & ESMF A high-performance, flexible framework for climate, weather, data-assimilation, and other Earth science endeavors ESMF provides tools for turning model codes into components with standard interfaces and standard drivers. ESMF provides data structures and common utilities that components use for routine services such as data communications, regridding, time management and message logging. 28

47 29

48 29

49 Bridging Across Modeling, Data, & Knowledge Curator is sponsored by the U.S. NSF and NASA (MAP) 30

50 31

51 32

52 METAFOR Common Metadata for Climate Modeling Digital Repositories 33

53 METAFOR An EU Framework Programme 7 Research Infrastructure Project Partners include Univ. of Reading (UK), IPSL (FR), BADC (UK), MPI (DE), UK Met Primary objective is to develop a Common Information Model (CIM) to describe climate data and the models that produce it Insure the wide adoption of the CIM 34

54 Dealing with Grids GridSpec Cube-sphere Yin-yang Grid Courtesy Randall, CSU Courtesy V. Balaji, NOAA/GFDL Courtesy Cassou, CERFACS 35

55 Closing Thoughts 36

56 Closing Thoughts Climate change science - and the next CMIP/ IPCC process - will offer formidable challenges for data management and access 36

57 Closing Thoughts Climate change science - and the next CMIP/ IPCC process - will offer formidable challenges for data management and access Volumes of data will be in the petascale range, and will be increasingly complex 36

58 Closing Thoughts Climate change science - and the next CMIP/ IPCC process - will offer formidable challenges for data management and access Volumes of data will be in the petascale range, and will be increasingly complex A distributed (and replicated) archive will be necessary 36

59 Closing Thoughts Climate change science - and the next CMIP/ IPCC process - will offer formidable challenges for data management and access Volumes of data will be in the petascale range, and will be increasingly complex A distributed (and replicated) archive will be necessary Metadata and knowledge are increasingly key 36

60 Closing Thoughts Climate change science - and the next CMIP/ IPCC process - will offer formidable challenges for data management and access Volumes of data will be in the petascale range, and will be increasingly complex A distributed (and replicated) archive will be necessary Metadata and knowledge are increasingly key International collaboration is required in order to meet our objectives 36

61 Merci Questions? 37