Sustainable DCI Operations scalable in size, time, and complexity

Transcription

1 Sustainable DCI Operations scalable in size, time, and complexity Bernhard Schott, DGSI Coordinator, Platform Computing Thijs Metsch, Senior Software Engineer, Platform Computing GridKa, , Karlsruhe, Germany

2 Sustainable DCI Operations Agenda The DGSI project Interoperation, Federation, Operation Implementation Technology Standardization in OGF & IEEE Rational for a federated DCI future

3 The DGSI Project DGSI D-Grid Scheduling Interoperability targets: #1: Interoperation of multiple technologies #2: Federation of Grids & Clouds #3: Operation, sustainable & scalable Scalable in size: interoperability of regions Scalable in time: interoperability of versions Scalable in complexity: interoperability of technologies and concepts

4 Interoperation of multiple technologies MSS* Meta Sched. 5 different types GridWay* Meta Sched. (WMS) C3WSS * Meta Sched. glite Globus Unicore glite Globus Unicore glite Globus Unicore Local Scheduler Local Scheduler Local Scheduler *examples

5 Federation Mode 1: Activity Delegation MSS Meta Scheduler DFP AD Gridway MetaSched DFP AD C3WSS Meta Scheduler Unicore Globus glite-ce Local Scheduler Local Scheduler Local Scheduler *DFP = DCI Federation Protocol

6 Federation Mode 2: Resource Delegation MSS Meta Scheduler DFP Gridway MetaSched DFP C3WSS Meta Scheduler Unicore Globus glite-ce Local Scheduler Local Scheduler Local Scheduler *DFP = DCI Federation Protocol

7 Federation of Grids & Clouds PetaFlop since 2008? Clouds CTRL RD DGS Desktop Grids CTRL DGS Combination of EDGeS bridge with DGSI Federation methods: AD Activity Delegation RD Resource Delegation Dynamically configured by negotiation, SLAs & policies. MS Service Grids MS CTRL=Cloud controller MS=Grid Meta-Scheduler DGS=Desktop Grid Server

8 DCI Operations: sustainable & scalable Operation, sustainable & scalable Scalable in size: interoperability of regions Scalable in time: interoperability of versions Scalable in complexity: interoperability of technologies and concepts

9 Operation, sustainable & scalable: interoperability of regions NE-Europe DFP Central-EU DFP SouthGrid MWxy MWxy MWxy Local Scheduler Local Scheduler Local Scheduler *DFP = DCI Federation Protocol

10 Operation, sustainable & scalable: interoperability of versions MS-vers.1.2 DFP MS-vers.2.3 DFP MS-vers.5.0 Globus 4.1 Globus 4.2 Globus 5.0 Local Scheduler Local Scheduler Local Scheduler *DFP = DCI Federation Protocol

11 Operation, sustainable & scalable: interoperability of versions MS-vers.1.2 DFP MS-vers.2.3 DFP MS-vers.5.0 glite 3 glite 4 glite 5 Local Scheduler Local Scheduler Local Scheduler *DFP = DCI Federation Protocol

12 Interoperability of concepts: ISF & DGSI CERN compute center HTC Computing HPC Computing Cluster MS-vers.1.2 HPC Platform ISF Shared computing infrastructure Cloud Federation Broker Instance Resource Delegation Service Globus 4.1 Grid DFP virtually moves resources Local Scheduler

13 Interoperability of concepts: ISF & DGSI CERN compute center HTC Computing HPC Computing Cluster MS-vers.1.2 Platform ISF Shared computing infrastructure Federation Broker Instance Resource Delegation Globus 4.1 DFP virtually moves resources Local Scheduler

14 Implementation Technology Activity Delegation Resource Delegation DGSI: Delegation methods 14

15 Implementation Technology Activity Delegation Resource Delegation DGSI: Delegation methods 15

16 The 4 reasons why Standards matter Or why Standards* rock! Thijs Metsch, Senior Software Engineer , GridKa School 2011, Karlsruhe *Industrial, Programming language, SDOs, 16 TORONTO 9/7/2011

17 PORTABILITY 17

18 Transport / Format Definition? 18

19 Interests Funding Program Interests Data / Methodology portability? Time 19

20 Technology??? Ruby JavaScript REST Python NoSQL Java SOAP C/C++ SQL 20

21 Compute power? 21 Source:

22 INTEROPERABILITY 22

23 DUH?? 23

24 IT ISN'T EASY 24

25 25

26 26

27 GLUE / BES / JSDL / 27

28 # of stds # of Standards to solve a Prb.???? Complexity/Simplicity of your problem 28

29 # of stds # of Standards to solve a Prb.???? Complexity/Simplicity of your problem 29

30 30

31 Interoperability is hard! Creating/Using a Standard is not necessary going to solve the problem Protipp: Do it right! How? Depends - there is no recipe (but learn from the past which standards and their processes are good which bad etc.) 31

32 32

33 33

34 INTEGRATION 34

35 Devices & location 35

36 (Some) Startups: plug it together? STARTUP HTTP 36

37 INNOVATION 37

38 INNOVATION 38

39 If technology thrills you 39

40 and you like working in groups 40

41 join a group and 41

42 create cool standards & technology. 42

43 Don t forget: share it... 43

44 and write it in you CV! 44

45 Interrest The time is now Time for Standards Time 45

46 Thank you 46