High performance Ethernet networking MEW24. High performance Ethernet networking

Transcription

1 MEW24

2 corporate overview The Only 21st Century Switching Vendor > 1,000,000 ports installed Focus on high performance datacenter networking #2 in 10G Ethernet ports in the Datacenter (fixed and modular)

3 Arista, run by Engineers Andy Bechtolsheim, Founder, Chairman, and CDO Founder of Sun Microsystems Founder of Granite Systems ( Cisco 4xxx) Initial investor in Google, Inc. Open Compute s board of directors David Cheriton, Founder, Chief Scientist Professor of CS at Stanford Founder of Granite Systems Founder of Kealia Initial investor in Google, Inc.

4

5 HPC thinking in a nutshell 10/40G Ethernet Transparent to any workload : deterministic any-to-any performance Must be open standards (L2, L3) based to avoid technology dead-end Must be simple to design, capacity plan, scale and troubleshoot Arista applies these principles at any size, big or small, with L2/L3 scale-out Must enable continuous innovation and pay as you grow scale

6 7500E - Most flexible 1/10/40/100GbE platform 12x100GbE 36x40GbE 48x10GbE + 2x100GbE Each 100G 3x40G or 12x10G Each 40G 4x10G 96 x 100G 288 x 40G 1152 x 10G

7 Arista Networks: The Best Data Center Portfolio 7050X & 7250X 7048T 48-port Data Center Class Gigabit Ethernet Switch 7050 S/T/Q 7150S & 7124FX 1/10G & 10/40G Data Center Switches 10G SFP+ / 10G-T Dense Virtualization 10GbE / 40GbE DC Ultra Low Latency 24,52,64-port SFP+ 1G-40GbE Switches Intelligent Application Switch Dense Low Latency 32 & 64-port QSFP+ 96xSFP+/8xQSFP+ Advanced Virtualization Scale-out Visibility 7300X 7500E High Density, Modular System supporting up to GbE GbE Lossless, High Density, Modular Switching System supporting up to 1152 Wirespeed 10GbE Ports Cloud Scale Leaf and Spine 10GbE-40GbE Spine GbE

8 Customer Use Case 40G Ethernet

9 HPC customer case 40G Ethernet storage 8 x 40G Ethernet Super-compute Replication Users 6PB Storage (GPFS) High density 10/40/100G Ethernet Compute Workstations 39.5 Gb/s utilisation per 40G Ethernet link, on all 8 simultaneously (=316Gbps) 30 GB/s GPFS aggregate throughput, with some disk drawers still unpopulated Low latency, large buffers. Highest performance without tuning on the network

10 Customer Use Case BP

11 World s largest supercomputer for commercial research 2.2 Petaflops 96,992 cores 11 PetaByte storage Pure Ethernet 10/40G Arista 7508E

12 Goodput vs Throughput : Deep Buffers «Hardware» buffers by Roger Ferrer Ibáñez

13 Buffer starvation Unfair drops - Oversubscribed networks with bursts > available bandwidth - Multiple nodes trying to read/write to one node (eg: Storage) - Lack of buffers means drops, which result in lower goodput 1 SRU (Server Request Unit) 2 3 Client Switch 4 Storage Servers Data Block

14 Deep Buffers Matter! Lossless networks How do they deal with incast problems?

15 Deep Buffers Matter! Lossless networks How do they deal with incast problems? Answer: Tell everyone to stop! In theory, allows small buffers In practice, this leads to tree saturation Causes blocking all the way to all sources

16 How much Buffer Memory do you need? Real World Data NS3 Network simulations match real-world data showing TCP incast issue - large # of TCP flows create microburst congestion Customer Real Buffer Utilization Observations Max Buffer Used per Port HPC Storage Cluster Medium 33 MB Animation Storage Filer (NFS) 6.2 MB Software vendor Engineering Build Servers (Perforce) 14.9 MB Online shopping Hadoop 2K servers Big Data 52.3 MB Educational Enterprise Data Center (Virtualization) 52.4 MB

17 x10G 4:1 3x10G 5.33: hosts x 10G hosts x 10G MB 1MB 48MB 1k TCP slow start/sec Deep Buffers Matter! 4:1 5.33:1 5.33:1 Zero! 0 Packets dropped per TeraGen

18 Goodhput Buffer Impact to High Performance Oversubscription Use Cases: Optimizing multi-speed: 10/40G Improving uplink contention in mixed speed networks High Density in core/spine (many-to-one, in-cast, fan-in)

19 Deep Buffers Matter! 20 node test with 10 flows per node (200 flows) Two tests: 4MB buffer 48MB buffer

20 Deep Buffers Matter! 20 node test with 10 flows per node (200 flows) Two tests: 4MB buffer 48MB buffer Results: complete fairness with large buffers Small buffers caused confusing flow transmission rates

21 Buffer consumed (MB) per port Buffer Utilisation per Port Live Large HPC Arista 7500E : 125MB Per 10G Trident ASIC 9MB per 64 ports th percentile

22 How much buffer are you using? Any visibility? What cause your retransmits?

23 Conclusion

24 Register for ipad-mini draw Visit the Arista booth