Data Integration and Analysis System (DIAS) as a platform for Asian Water Cycle Initiative (AWCI)

Size: px

Start display at page:

Download "Data Integration and Analysis System (DIAS) as a platform for Asian Water Cycle Initiative (AWCI)"

Corey Shepherd
5 years ago
Views:

1 Data Integration and Analysis System (DIAS) as a platform for Asian Water Cycle Initiative (AWCI) Akiyuki KAWASAKI and Toshio KOIKE The Earth Observation Data Integration & Fusion Research Initiative (EDITORIA), The University of Tokyo

2 2 Acknowledgement - Japanese Ministry of Education, Culture, Sports, Science and Technology (MEXT) for their financial support - Data providers including GEOSS community for DIAS project - EDITORIA Science team and DIAS R&D community for their support of DIAS project

3 3 Agenda - DIAS outline - AWCI Data Archive System - DIAS value - Applications and tools - In-situ (real-time) data - Data and motel integration - Summary

4 4 DIAS as an advanced e-infrastructure component. Global Earth Observation Satellite Imagery Simulations System WCRP CMIP3 Ocean Observation of Systems Land Observation MetBroker (GEOSS) Data Integration & Information Fusion Platform App. Layer User Apps. Common Utility Layer Visualizer(w display wall) Discovery Work Flow Assist Data Quality Manager Data Management Layer File System Layer Storage Layer User Apps. User Apps. User Apps. User Apps. Data Transformer Data Crawler ETL Data Mingrator Data Navigator Meta Data Manger Database management system Storage management PB scale logical file Power management Disk Arrays

5 DIAS: Structure S-8 CMIP5 GRENE-ei RECCA DIAS-P Climate Change Adaptation Hydroelectric power DIAS/CEOS Water Portal GEOSS/AWCI GEOSS/AfWCCI ASIAN Monsoon Year Joint Research Social Implementation Health International Contribution Climate ICT Experts Economy Field Specialists R&D Community Field Specialists Water Urban Biodiversity Agriculture Disaster Risk Reduction Data Archive ICT Experts Application Development Data Processing Search / Download Extra-large ICT Experts Base System Analysis High Speed Storage (26 PB) Server Network 5

6 6 Agenda - DIAS outline - AWCI Data Archive System - DIAS value - Applications and tools - In-situ (real-time) data - Data and motel integration - Summary

7 To archive hydro-meteorological dataset, including data loading, QC and metadata registration AWCI Data Archive 20-year Long-term precipitation data from 18 countries 18 River Basin, 280stations

8 Tool for in-situ data input and management (1) Data Uploading (3) Meta Data Registration (2) Quality Controlling (4) Data Downloading

9 (1) Data Uploading

10 (2) Quality Controlling G: Good I : Interpolated D: Dubious/Questionable B: Bad C: Abnormal value M: Missing U: Unchecked Flag Definitions Number of each Flags Flag Updated data

11 (3) Meta Data Registration Role of Metadata & Dataset Documents HTML PDF Data provider Manual input Publishing Document User Automatic extraction DIAS Core System Metadata Management XML API Metadata registration interface Storing to database Application Quality Control Metadata Search & browse Interface 11

12 (4) Data Downloading Click here to select all stations for downloading. Or choose to download data for a single station. 12

13 Effect of the System!

14 14 Agenda - DIAS outline - AWCI Data Archive System - DIAS value - Applications and tools - In-situ (real-time) data - Data and motel integration - Summary

15 15 DIAS Value Archived extra-large volume of observed and simulated data Real-time in-situ data Data and model integrator R&D community with domain scientists and IT experts

16 16 Agenda - DIAS outline - AWCI Data Archive System - DIAS value - Applications and tools - In-situ (real-time) data - Data and motel integration - Summary

17 35 Applications and tools < Data Dissemination Himawari-8 data GPV data C-Band Realtime Precipitation Data River Telemetry Data Utilization 1Climate Change 2Water resource management CMIP5 Data Analysis System Global Environmental Data Analysis Support Tone River Management support system DIAS/CEOS Water Portal 3Agriculture 4Biodiversity Simulation Model for RIce-Weather relations Ikimoni Particle Tracking Simulation System XRAIN data 17

18 18

19 Climate change impact analysis tool using CMIP 5 dataset (1.6PB) 19

20 A tool for Climate change impact analysis (Model selections and BIAS correction) using CMIP 5 (Coupled Model Intercomparison Project) data Ability of data integration among archived observed and simulated data with real-time data is one of DIAS Value 20

21 21 Agenda - DIAS outline - AWCI Data Archive System - DIAS value - Applications and tools - In-situ (real-time) data - Data and motel integration - Summary

22 MLIT Real-time data archiving on DIAS Live Camera River Telemetry /10 min. NOAA,GMS,MTSAT, MODIS,AMSR-E, GMS8 Satellite Data C-band Radar X-band Radar Live Camera Images Local government etc. /10min. /1 min. 250m mesh 14area /5 min. 1km mesh All area DIAS Core System /1 hour, 1300 points AMeDAS GSMaP JAXA /1 hour 0.1deg. mesh Global(60S-60N) /5 min. 91 points Tidal level Japan Coast Guard /1-3 hour hr. forecast 0.2~1deg. mesh Global, Japan area GPV JMA 22

X-band MP Radar - 250 m grid - Every 1 min.

23 archiving, analyzing and disseminating data and information with high velocity. X-band MP Radar m grid - Every 1 min. New Gestational Satellite km grid - Every 2.5 min. 500GB/day 500GB/day 23

24 24 24

25 25 Agenda - DIAS outline - AWCI Data Archive System - DIAS value - Applications and tools - In-situ (real-time) data - Data and motel integration - Summary

26 Weather Prediction Water Cycle Integrator (model) Models Seasonal Prediction Climate Prediction Satellite Vis. IR MW Gravity Rainfall Data Assimilation Water Cycle Model Rain fall Evaluation & Bias Correction River Flow Snow Evapotranspiration Soil Moisture Ground Water Val. River Flow Soil Moisture In-situ Data Urban Environment Conservation IWRM Agriculture Support Climate Change Adaptation River Basin Management socio-economic & recognition information Ground Water Urban Water Model Crop Model Index Organism Water Quality Ground Elevation Sewage Loading Unit Agricultural Info. Famer s Needs Biodiversity Data In-situ Data 26

$fraction,$ Longwave

27 27 Real-time Flood Forecast System: Real Time Hydrol. Model DIAS Data Archive Satellite, Radar Obs. weather QPF MLIT obs. Real-time system Basin water cont. river discharges real-time hydro. model Forcings Cloud fraction, Longwave radiation, Relative humidity, Shortwave radiation, Temperature, Wind speed real-time simulation of basins water content and discharges 流域内保水状態の時系列変化 : 土壌水分 Temporal change of basins water content 河川洪水流量 :H23 台風 15 号前橋 Simulated Flood on H23. Th15 obs. model

28 Trial simulation: River Water Level, Inundation Depth, and Sewerage Flow WEB-DHM model domain Without sewerage system 1D unsteady flow model section Urban sewerage domain inundation basin Conditions: peak flow at Ochiai, 1000m3/s precipitation,100 mm/h With sewerage system *timings of animation are not synched in this slide Sewerage flow visualization on the way (m) 28

29 29 Sawada, Koike, et.al., JGR (2016) 29

30 30 Agenda - DIAS outline - AWCI Data Archive System - DIAS value - Applications and tools - In-situ (real-time) data - Data and motel integration - Summary

31 31 DIAS Value Archived extra-large volume of observed and simulated data Real-time in-situ data Data and model integrator R&D community with domain scientists and IT experts

32 32 Need more collaboration! We would like to expand our collaboration with international experts, organizations, and partners: - To exchange knowledge and experience - To fill the gap between e-infrastructure and the society Transdisciplinary, especially commercial sector - To promote education and capacity development in e-infrastructure

33 Open access paper! 33

34 Please visit 34

Banglades Bhutan Cambodia. India Indonesia Japan Korea Lao. Malaysia. Mongolia. Myanmar

Banglades Bhutan Cambodia. India Indonesia Japan Korea Lao. Malaysia. Mongolia. Myanmar AWCI Integration & Archive ー Development of Loading and Quality-Control System ー Eiji Ikoma* Hiroko Kinutani*, Tetsu Ohta*, Katsunori Tamagawa* and Toshio Koike*/***,Masaru Kitsuregawa*/** The University