Networks & protocols research in Grid5000 DAS3

Transcription

1 1 Grid 5000 Networks & protocols research in Grid5000 DAS3 Date Pascale Vicat-Blanc Primet Senior Researcher at INRIA Leader of the RESO team LIP Laboratory UMR CNRS-INRIA-ENS-UCBL Ecole Normale Supérieure de Lyon France

2 2 Outline -RESO experiments in Grid Wizard gap problem studies - High speed transport protocols benchmarking - Long Distance MPI optimization - High speed WAN emulation - Grid traffic measurement & modeling - RESO goals in GRID5000-DAS3 - End to end bandwidth reservation optimisation - Flow scheduling & rate control - Internationals collaborations

3 3 Research Challenges : future networks Modeling, validation and optimization of network protocols and services for high performance computing & data grids Cross layer design: services / transport / network control/routing / lambdas Integrate performance&security (virtualization) Self-* architectures : self (adaptative metro, management, resource allocation, protocol composition, service deployment))

4 4 E2E performance problem Internet The high end applications are rarely able to take full advantage of the high-capacity (2.5Gbps, 10Gbps and upwards) networks installed today. 90% of the bulk TCP flows (transfers of at least 10MB of data) use less than 5 Mbits/sec -> End to end performance initiative => Wizard gap problem Slide from Matt Mathis- PSC

5 5 E2E performance problem Combination of many factors Problems related to the network Routing problems, leaky networks (networks with non-congestive packet loss), and excessive packet reordering Problems related to the TCP protocol TCP designed first and foremost to be robust and when congestion is detected, TCP accommodates at the expense of reduced performance. Problems related to the TCP configuration small buffer space or SACK improperly negotiated Problems related to the end system: to the processor and bus speed, to the NIC with its associated driver; Problems due to the applications small messages or pauses in the data flow

6 6 Grid5000 s 10G/s backbone RENATER-4 RENATER-4 9 Clusters with 256 to 1K CPUS => about 3000 CPUs 2,5 Gbit/s 2,5 Gbit/s Fibre noire Fibre noire CERN CERN 10Gb/s Dedicated lambdas Nodes have 1Gb/s interfaces Sophia

7 7 Is there a wizard gap problem in Grid5K? Novice: 1Gb/s measurement, with default kernel images => goodput in Mb/s Bo Gr Li Ly Na Or Re So To Bo Gr Li Ly Na Or Re So YES! To R <10%C

8 8 Is there a wizard gap problem in G5K? Expert: 1Gb/s measurement, with tuned kernel: goodput in Mb/s Bo Gr Li Ly Na Or Re So To Bo Gr Li Ly Na Or With tuned stacks Dynamically deployed Re So To YES! G> 80% of CT G < 9% of 10Gb/s

9 Long distance MPI optimisation 9

10 10 E2E performance problem Combination of many factors Problems related to the network Routing problems, leaky networks (networks with non-congestive packet loss), and excessive packet reordering Problems related to the TCP protocol TCP designed first and foremost to be robust and when congestion is detected, TCP accommodates at the expense of reduced performance. Problems related to the TCP configuration small buffer space or SACK improperly negotiated Problems related to the end system: to the processor and bus speed, to the NIC with its associated driver; Problems due to the applications small messages or pauses in the data flow

11 E2E performance Flows interaction problem The reality Two of TCP TCP Reno like sources congestion from independent control algorithm machines at high speed limited 490Mb/s - Bottleneck of 1Gb/s - RTT 100ms 11 Window-based 2 RENO flows (1Gb/S) rate contr - 100ms with => with rate controlol Packet pacing rate control

12 High Speed TCP variants evaluation 12 Protocol benchmarking for data transfers in high speed networks by using new TCP or UDP stacks is a key issue For this, we have developed a dedicated testbed within Grid5000 testbed We are defining a benchmark in collaboration with pfldnet community Explore the broad parameter space: RTTs, rates, protocol variants, cross traffic Individual flow behavior at 1Gb/s or at 10Gb/s Concurrent flows interactions at 1Gb/s or at 10Gb/s Parallel flows at 1Gb/s or at 10Gb/s (cf GridFTP)

13 Example of testbed setup 13 1 x 10GbE 11 x 1 GbE + 1 x 10GbE iperf iperf iperf 11 x 1 GbE 10 GbE 12 x PCs 12 x PCs iperf iperf iperfd Grid5000-DAS3path......

14 HS TCP : 1 to 12 flows 14

15 Results (throughput 100ms - empty net) 15

16 16 ewan: large scale gigabit WAN emulator Grids Virtualisation Gigabits links Gnet Boxes Non blocking multigigabit switch PC w. IXP NICs Virtualisation ISP P2P systems PC ISP PC Latency emulation -GNET -NistNet / netem Overlay routers - Network processor based - Linux routers ISP PC ISP PC

17 17 ewan & GRID5000 Has been validated on several G5K clusters ewan installer for this specific testbed Images have been created for different G5K machines & clusters Usage of real long distance links is possible

18 18 Grid metrology Global objective Develop a metrology activity on GRIDs infrastructures for a better understanding of the traffic in order to adapt existing protocols to this specific context Metrology Define sensible metrics. Measurement issues. Analysis difficulties Understanding Characterize the statistics, identify typical responses. Deduce the underlying mechanisms. Infer adequate models. Adapt protocols Better reactivity to the Grid specificities. Improve Quality of Service and security, subject to the network constraints and to the application(s) requirements.

19 19 Outline - RESO experiments in Grid Wizard gap problem studies - High speed transport protocols benchmarking - Long Distance MPI optimisation - High speed WAN emulation - Grid traffic measurement & modeling - RESO goals in GRID5000-DAS3 - End to end bandwidth reservation - Flow scheduling & rate control - Controlled performance & security : HIPerCAL - Internationals collaboration

20 Grid5000 <-> DAS3 20 RENATER Orsay RENATER Rennes RENATER Lille RENATER Nancy 10 Gbps 10 Gbps 10 Gbps 10 Gbps Paris Router RENATER Bordeaux Black fiber Dedicated Lambda Fully isolated traffic! 1Gbps 10 Gbps Lyon 1 Gbps RENATER Lyon RENATER 10 Gbps Grenoble 10 Gbps 10 Gbps RENATER RENATER Sophia Toulouse Black Fibers are rent by the network provider RENATER is enlighted by RENATER Next step? Lambdas on demand? Sharing Grid5000 & DAS3 Source: Cees de Laat (UvA)

21 21 E2E net resource reservation? Provide end-to-end Quality of Service to applications. This requires: Discovery and selection of resources Allocation of resources Advance reservation of resources Scalability, robustness issues! Supercomputer Workstation Workstation

22 22 Optimisation in hybrid packet/lambdas nets => Model: ex starplane => Simulator

23 23 Optimisation in hybrid packet/lambdas nets -Experiment with real equipements: - Explore Dragon software (G-MPLS) - Explore security issues (UvA token approach?) -Interaction with GLIF/OGF communities - GLIF control plane: common service definition - Participate to the definition of a standard for advance net resource reservation - Integrate GLIF community through this interco (cf les Cees )

24 Control plane drivers (Gigi Karmous-Edwards slide) 24 Visualization courtesy of Gigi Karmous-Edwards - MCNC.

25 25 Flow scheduling [Globecom06, HPDC06] Assumption: Core and LANs are overprovisionned Access links are potential bottlenecks Bulk traffic isolated from other traffic Scalable & easy to deploy solution is required Optimal flow scheduling

26 26 Flow control at very high speed INRIA RESO is studing a Grid Network Service for controlling Grid Bandwidth Sharing (GBS). Prototype this service in LINUX boxes (that may have Network Processor based NICs), in charge of running flow control distributed algorithms. Define the API for transfer request signaling Integrate an automatic rate control mechanism based on packet pacing + time window control (source & gateway) Deploy this solution in different locations of the testbed (G5K, DAS3?) Validate with HS transport protocols Test with applications & compare with other solutions

27 27 Contacts Opportunities in RESO team: M2-Internship->PhD; 2 x CDD 3Y; PostDoC 1Y First international IEEE GRIDNETS 2007 conference in LYON (France) october 2007