Coprocessors, multi-scale modeling, fluid models and global warming. Chris Hill, MIT

Size: px

Start display at page:

Download "Coprocessors, multi-scale modeling, fluid models and global warming. Chris Hill, MIT"

Cecil Walters
5 years ago
Views:

1 Coproessors, multi-sale modeling, luid models and global warming. Chris Hill, MIT

2 Outline Some motivation or high-resolution modeling o Earth oean system. the modeling hallenge An approah Sotware triks and tehniques Components Data strutures Status and plans

Why might we are about modeling global oean

3 Why might we are about modeling global oean at higher and higher resolution? Northern Hemisphere Mean Temp Reonstrution IPCC, 2013 Rising CO2, PPM, net inrease in radiative oring due to net o hanges in atmospheri hemial/partile omposition. Daily Mail, UK, 2012.

4 The Earth surae is 70% oean Oean is luid in theory (and in pratie) it an draw down and release heat on many time-sales, days, weeks, months, years, deades, enturies, millennia. C v OCEAN :C v AIR = 3000:1 (at sea-level, 300 o K). Variations in oean heat uptake an alter global mean surae temperature.

This presents a hallenge to how muh models an be used ab-initio to reason about ideas, test hypotheses about

5 Sales between two pitures are O(10m) proess O(10,000km). Deault approah, in large-sale models, is to model small-sale as diusive K grad(phi) term. This presents a hallenge to how muh models an be used ab-initio to reason about ideas, test hypotheses about dominant proesses et Haney et al, JMR, 2012 Oean heat/buoyany budget/exhanges. Heat/buoyany loss/uptake proesses have omplex smallsale omponents. Best estimate, rom somewhat sparse obs.

6 Outline Some motivation or high-resolution modeling o Earth oean system. the modeling hallenge An approah Sotware triks and tehniques Components Data strutures Status and plans

Multi-sale algorithm artoon (2-level) z y x iner mesh models oarser mesh model v T v T v T mesh ields oarse plus sub-grid sale variations z x Embed loal proess models in large sale model proesses

7 Multi-sale algorithm artoon (2-level) z y x iner mesh models oarser mesh model v T v T v T mesh ields oarse plus sub-grid sale variations z x Embed loal proess models in large sale model proesses more expliit, dierent physis, grid and timestep to large sale model Analogous to atmospheri superparam (e.g. Grabowski, 2001) Surprisingly analogous to oupled modeling in general. [ T ] T T,[ v ] ( v T ) v T ( v T ) [ T ] ore steps Ensure ine and oarse ields are onsistent mesh tendenies Coarse mesh tendenies

8 Parallel Formulation Sub-models an be expensive one has to solve a omplex sub-model at eah large-sale model grid ell. However, sub-models are independent omputations. So there is a lot o parallelism z y x oarse model ine sale models T ( v T ) [ T ] [ T ] T T,[ v ] ( v T ) v

9 Parallel Formulation Sub-models an be expensive one has to solve a omplex sub-model at eah large-sale model grid ell. However, sub-models are independent omputations. So there is a lot o parallelism z y x oarse model ine sale models T ( v T ) [ T ] [ T ] T T,[ v ] ( v T ) v

signiiantly improves representation o buoyany loss apturing

10 Proo o onept appliation to idealized, simpliied oean buoyany loss 2011 Q=800Wm -2 Q=800Wm -2 Embedded multi-sale model signiiantly improves representation o buoyany loss apturing vertial and temporal harateristis well. Fully domain deomposed inesale equations.

11 Some example dierenes Computational operation ount or multisale alg is 1/10 o ully resolved model and x10 o oarse alone model. Domain deomp introdues extra parallelism (in theory).

Next step experiments and relation to emerging o-proessor arhitetures. Coarse and ine an also run onurrently. Can we exploit this by adding extra o-pro hardware.

12 Next step experiments and relation to emerging o-proessor arhitetures. Coarse and ine an also run onurrently. Can we exploit this by adding extra o-pro hardware. z y x Coarser model ommuniates more broadly. z r model ommuniates more narrowly. (~1000 CPU ores) x Interation between oarse and ine ours through This an provide a avorable ompute to oproessor ommuniate balane a better way orward to improve model or a given transistor / power / dollar / spae budget. x ~16million (~100,000 o-pro ores)

13 Outline Some motivation or high-resolution modeling o Earth oean system. the modeling hallenge An approah Sotware triks and tehniques Components Data strutures Status and plans

14 Component Perspetive We use omponent based system, ESMF (Earth System Modeling Framework), to orhestrate distributed, onurrent exeution. Provides objet like high-level environment or abstrating mathematial building bloks. Overlaid on standard MPI parallelism or CPUs.

15 Implementation Generally appliable implementation not tied to a speii problem ast omputational ormulation as a oupled modeling sheme and borrow ESMF tehnology rom the oupled modeling ommunity e.g. Driver p x - pros {p } {p 1 } {p 2 } Coarse Model (,p ) Coarse to (,{p 1,p }) Coarse to (,{p 2,p }) Model (,p 1 ) Model (,p 2 ) to Coarse (,{p 1,p }) to Coarse (,{p 2,p }) oarse - ine

16 Implementation Generally appliable implementation not tied to a speii problem ast omputational ormulation as a oupled modeling sheme and borrow ESMF tehnology rom the oupled modeling ommunity e.g. Driver p x - pros {p } {p 1 } {p 2 } Coarse Model (,p ) Coarse to (,{p 1,p }) Coarse to (,{p 2,p }) Model (,p 1 ) Model (,p 2 ) to Coarse (,{p 1,p }) to Coarse (,{p 2,p }) oarse - ine

17 Implementation Generally appliable implementation not tied to a speii problem ast omputational ormulation as a oupled modeling sheme and borrow ESMF tehnology rom the oupled modeling ommunity e.g. Driver p x - pros {p } {p 1 } {p 2 } Coarse Model (,p ) Coarse to (,{p 1,p }) Coarse to (,{p 2,p }) Model (,p 1 ) Model (,p 2 ) to Coarse (,{p 1,p }) to Coarse (,{p 2,p }) oarse - ine

18 Co-proessor abstration Similar to high-level abstration reate abstrat data type handles. deine overloaded untions or mapping rom native CPU language (F90/95) to o-proessor handles and or basi, kernel operations (vetor, matrix, ustom solve et ) o-proessor kernels are hand written (CUDA or C)

19 Outline Some motivation or high-resolution modeling o Earth oean system. the modeling hallenge An approah Sotware triks and tehniques Components Data strutures Status and plans

MGHPCC. intelletual resoures that are also part o this ollaboration.

20 Testing and developing Targeting prodution environments at TACC, ORNL/NICS, Blue Waters, NASA Ames. Development leveraging loal ompute resoures at new ollaborative BU, Harvard, MIT, Northeastern, UMass aility MGHPCC. intelletual resoures that are also part o this ollaboration. these inlude a very broad set o interations that are working to evolve the HPC eosystem at all levels rom hardware/systems/arhitetures algorithms/appliations.

21 Note, by speial request - a shameless plug or some o our regional ativities! We need ways to test and evolve ideas easily and dynamially. Through these, and more, we are strengthening our regional ollaborative ativities signiiantly.

22 Summary/disussion Domain-deomp and overset mesh in a multi-sale paradigm may be a way to move orward aster in planetary sale modeling, v. monolithi resolution. Needs to be tested There are appealing potential synergies with oproessor developments around taking a hierarhial view. Interesting sotware hallenges around realizing in pratie Hierarhial algorithms have relevane or super-large HPC ativities in general. Physial aility network layouts are inreasingly hierarhial. Multiple sales in physis o problems o interest getting mathematis to ooperate remains hallenging.

Outline: Software Design

Outline: Software Design. Goals History of software design ideas Design priniples Design methods Life belt or leg iron? (Budgen) Copyright Nany Leveson, Sept. 1999 A Little History... At first, struggling