Use Hierarchical Storage and Analysis to Exploit Intrinsic Parallelism

Transcription

1 Use Hierarchical Storage and Analysis to Exploit Intrinsic Parallelism Charlie Zender1 Pedro Vicente1, Wenshan Wang1 1 Departments of Earth System Science and Computer Science, UC Irvine AGU Fall Meeting San Francisco, CA December 9-13, 2013 Seminar on Web

2 Model Evaluation Background 1. Evaluation of models by data (or other models) 2. Dataset-level operators simplify evaluation 3. Hierarchical groups are a powerful extension

3 Parallelism Already Exploited A. Replicate arithmetic over variables: T, p, q... (NCO uses OpenMP, also suitable for MPI). B. MPI & Collective I/O (PnetCDF, netcdf4 API): C. Compile scripts into (independent) Basic Blocks for batch scheduling (SWAMP, Wang & Zender)

4 What remains exploitable? Scientific method uses replication to build statistical significance and to correct mistakes. This translates into experimental ensembles of data. Larger ensembles facilitate more detailed questions and robust conclusions. Intrinsic Parallelism Often we pose the same questions of each ensemble, and thus of each member of each ensemble... Francis Bacon ( )

5 Flat File Group File (aka, a namespace) /var1, /var2,... /varn var1, var2,... varn group_name... varn

6 cesm ecmwf giss Contain different namespaces Inherit shared properties Advantageous because...datasets often use same variable, coordinate names

7 cesm run1 run2 Hierarchical Groups Nested groups can contain ensemble simulations, or measurement replications runj Multiple nested groups for Model Intercomparison Projects (CMIP5, PCMIP, AeroCom...)

8 giss cesm ecmwf run1 run1 run2 run2 run2 runj runj runj run1

9 What s New? nces: Ensemble Statistics + Runs 1,2,...J Runs J+1. Unlimited, ragged F Files E(f) Ensembles M(e,f) Members (ragged) V(e) Variables D(e,v) Dimensions (sharable)

10 Exploit Name Parallelism Non-Parallel: Exploit Parallelism: mdl=addfile(mdl.nc) ncdiff mdl.nc obs.nc out.nc Model mdl_var1=mdl->var1 mdl_var2=mdl->var2 obs=addfile(obs.nc) obs_var1=obs->var1 Obs. obs_var2=obs->var2 var1=mdl_var1-obs_var1 var2=mdl_var2-obs_var2 out=addfile(out.nc) Math out->var1=var1 out->var2=var2 Output

11 Exploit Namespace Parallelism Non-Parallel: obs=addfile(obs.nc) obs_var1=obs->var1 obs_var2=obs->var2 do j=1,8 mdl=addfile("mdl_"+j+".nc") mdl_var1=mdl->var1 mdl_var2=mdl->var2 var1(j)=mdl_var1-obs_var1 var2(j)=mdl_var2-obs_var2 end do out=addfile(out.nc) out->var1=var1 out->var2=var2 Exploit Parallelism: ncdiff mdl.nc obs.nc out.nc

12 Exploit Hierarchical Parallelism Non-Parallel: for i in 'cesm ecmwf giss'; do for j in ' '; do var1(i,j)=mdl_var1(i,j)-obs_var1 var2(i,j)=mdl_var2(i,j)-obs_var2... end do end do Exploit Parallelism: ncdiff cmip5.nc obs.nc out.nc Variable Parallelism: for i in 'cesm ecmwf giss'; do for j in ' '; do ncdiff $i_$j.nc obs.nc $i_$j_obs.nc done done

13 Exploit Intrinsic Parallelism Ensembles members (Run1, Run2, ) Or.HDF,.h5, DAP, SFTP... Ensembles means nces cmip_*.nc cmip_avg.nc ncdiff cmip_avg.nc obs.nc bias.nc Tropical hyperslab Area weight ncwa -w area -d latitude,-23.,23. \ -v tas bias.nc tropics.nc Weighted Averager Ts(t,x,y) Tropical-mean Ts biases of all CMIP5 models

14 Exploit Intrinsic Parallelism Disparate datasets in one suitcase : BCC-CSM CanAM CMCC CNRM CFS ACCESS CSIRO EC-EARTH GFDL FIO-ESM BNU-ESM CESM INM-CM4 IPSL-CM5 FGOAKS MIROC HadGEM2 MPI-ESM MRI-ESM GISS GEOS CCSM4 NorESM1 NICAM

15 Types of Intrinsic Parallelism math model.nc observ.nc out.nc ncdiff cmip5.nc modis.nc out.nc Type Name (e.g., multiple variables in flat file) Namespace (group namespaces, single level) Hierarchical Namespace (nested group hierarchy) Value/Challenge Simple, easy, netcdf3 Single ensemble Aggregator (ncecat) Multiple ensembles Aggregator (ncecat)

16 Exploit Intrinsic Parallelism math model.nc observ.nc out.nc ncdiff cmip5.nc modis.nc out.nc Hierarchical Storage and Analysis can exploit intrinsic parallelism of replication and other parallelisms. Use more Bacon in your analysis.