Architecting Data Center Networks in the era of Big Data and Cloud

Transcription

1 Architecting Data Center Networks in the era of Big Data and Cloud Spring Interop May 2012 VIDEO of this session:

2 Two approaches to DC Networking THE SAME OLD Centralized, Scale-up Layer 2 networks Monstrous chassis switches TRILL OpenFlow VEPA SPB Or a Different Approach Distributed, Scale-out Layer 3 fabrics Efficient fixed switches Open, industry standard protocols 2

3 Networks that suck for Cloud & Big Data Dist Core Network Topology Access VM PARTITIONED CAPACITY Capacity Topology Data center networks are in my way -James Hamilton, AWS 3

4 Networks that Don t suck for Cloud & Big Data Spine Network Topology Leaf VM UNIFORM CAPACITY Capacity Topology All points equidistant 4

5 Big Data TCP switch TCP switch switch switch switch switch switch Name Rack 1 Job Tracker Rack 2 Secondary NN Rack 3 Client Rack 4 TCP World Inverse Virtualization Workloads orchestrated like cattle L2 or L3 network. Does it matter? TCP Client Rack N 5

6 Basic requirements of Cloud (IaaS) switch switch switch switch switch switch Physical Network World FW VM VM LB VM VM Virtual Network Secure, Scalable Multi Tenancy Location independence On Demand virtual networks 6

7 Blend the Virtual and Physical Networks switch VLAN 20 switch VLAN 10 switch VM VM vswitch vswitch VM VM VM VM VM VM Host Host Tenant subnet = Network VLAN 7

8 Abstract the Virtual Network from Physical Segment ID 20 switch Segment ID 10 switch switch VM VM vswitch vswitch VM VM VM VM VM VM Host Host Network Virtualization Overlay Tenant subnet = Software VLAN 8

9 Scale-up centralized Layer 2 2-post Rooted Architecture Centralized L2/L3 L3 L2 L2/L3/ARP table scale? Scale w/ Bigger Boxes Precious Pets VLAN Provisioning? Broadcasts vswitch VM VM vswitch VM VM 9

10 Scale-out Layer 3 Leaf/Spine Fabric (16) (8) (2) (128) (64) (16) Server ports Mesh from Leaf to Spine Non-blocking Spine L3 L2 OSPF, ISIS, BGP, TRILL ToR w/ 16 uplinks (ECMP) ToR 128 port 2RU Spine 10

11 Uniform fabric for Cloud & Big Data (16) (8) Hadoop (2) Storage Access (128) (64) Name Client Rack 1 Client Rack 2 (16) Job Tracker Secondary NN Database Rack Server ports Block I/O NAS Object Rack N vswitch VM VM vswitch VM VM L3 L2 11 Global Brad Marketing Hedlund

12 Attaching Services & North/South (16) (8) (2) (128) (64) Name (16) Job Tracker Secondary NN LB LB Firewall Firewall x86 Gateways L3 L2 Client Rack 1 Client Rack N vswitch VM VM VM vswitch VM VM VM vswitch VM VM VM vswitch VM VM VM vswitch VM VM VM vswitch VM VM VM World 12 Global Brad Marketing Hedlund

13 Generic Logical Architecture 1 Overlay based L2 Physical/Static FW Fabric World DC router FW L3 NAT FW L3 NAT L2 L2 LB LB L3 VM VM L2 L2 VM VM VM Green Co. Big Data Orange Co. 13

14 Generic Logical Architecture 2 Overlay based L2 Virtual/Mobile FW Overlay Gateway Fabric Pub DMZ World DC router FW L3 NAT FW L3 NAT L2 L2 LB LB L3 VM VM L2 L2 VM VM VM Green Co. Big Data Orange Co. 14

15 Generic Logical Architecture 3 No Overlays TRILL based L2 Virtual/Mobile FW Fabric TRILL Pub DMZ World DC router FW L3 NAT FW L3 NAT L2 L2 LB LB L3 VM VM L2 L2 VM VM VM Green Co. Big Data Orange Co. 15

16 Density: Fixed vs. Chassis G per Line Rate (L3) Chassis Fixed

17 Power: Fixed vs. Chassis 18 Max Watts / Line Rate 10G (L3) Chassis Fixed

18 What are the Challenges? (128) (16) (8) (16) (2) Dell Fabric Manager Design Templates Validate deployment Automate fabric configuration Monitoring & Operations L3 L2 Deployment & Cabling Configuration & Policy Monitor & Troubleshoot Layer 2 (TRILL?) Design Best Practices Dell Fabric Manager 18 Global Brad Marketing Hedlund

19 Webinar: CLOS Fabrics Explained Wednesday, June 20, 2012 from 10:00 AM to 1:00 PM (ET) HOST CO-HOST Ivan Pepelnjak 19 Global Brad Marketing Hedlund

20 Three Stage Layer 3 Leaf/Spine Fabric (64) (8) (2) /26 (128) 0/0 (512) /26 /26 0/0 L3 24,576 Server ports top tiers Default ToR & Leaf L2 Leaf+ToR mesh groups ~8usec worst case 20

21 The case for 40G QSFP switch ports 10G 10G 10G 10G 10G 10G 10G 10G QSFP $1,800 VS SFP+ SFP+ SFP+ SFP+ $1K $1K $1K $1K 32 ToR 21 $230K $512K

22 Comparing Fabric efficiencies of Fixed vs. Chassis designs May 2012 Supplemental Section

23 Total Fabric Power: Chassis vs. Fixed KW 250 non-blocking Chassis Fixed Fabric size 23

24 Total Fabric RU: Chassis vs. Fixed RU 700 non-blocking Chassis Fixed Fabric size 24

25 8192 non-blocking Fabric (64) (8) (2) (128) 8192 non-blocking 384RU 153.6KW 8192 ISL 192 switches 25

26 8192 non-blocking Fabric (32) (256) 8192 non-blocking 608RU 216.3KW 8192 ISL 288 switches 26

27 4096 non-blocking Fabric (32) (2) (64) 4096 non-blocking 192RU 76.8KW 4096 ISL 96 switches 27

28 4096 non-blocking Fabric (16) (128) 4096 non-blocking 304RU 108.1KW 4096 ISL 144 switches 28

29 2048 non-blocking Fabric (16) (2) (32) 2048 non-blocking 96RU 38.4KW 512 ISL 48 switches 29

30 2048 non-blocking Fabric (8) (64) 2048 non-blocking 152RU 54KW 2048 ISL 72 switches 30

31 384 non-blocking Fabric (4) (2) (6) 384 non-blocking 20RU 8KW 96 ISL 10 switches 31

32 384 non-blocking Fabric (2) (12) 384 non-blocking 26RU 18KW 384 ISL 14 switches 32

33 256 non-blocking Fabric (2) (4) 256 non-blocking 12RU 4.8KW 64 ISL 6 switches 33

34 256 non-blocking Fabric (2) (8) 256 non-blocking 22RU 6.7KW 256 ISL 10 switches 34

35 Total Fabric Power: Chassis vs. Fixed KW 80 3:1 oversubscribed Chassis Fixed Fabric size 35

36 Total Fabric RU: Chassis vs. Fixed RU 250 3:1 oversubscribed Chassis Fixed Fabric size 36

37 3:1 oversubscribed Fabric (2) (16) 3:1 20RU 7.2KW 64 ISL 18 switches 37

38 3:1 non-blocking Fabric (2) (16) 256 non-blocking 30RU 7.5KW 256 ISL 18 switches 38

39 3:1 oversubscribed Fabric (4) (2) (32) 3:1 40RU 14.4KW 128 ISL 36 switches 39

40 3:1 oversubscribed Fabric (2) (32) 3:1 54RU 17KW 512 ISL 34 switches 40

41 3:1 oversubscribed Fabric (8) (2) (64) 3:1 80RU 28.8KW 1024 ISL 72 switches 41

42 3:1 oversubscribed Fabric (4) (64) 3:1 108RU 34KW 1024 ISL 68 switches 42

43 3:1 oversubscribed Fabric (16) (8) (2) (128) 3:1 160RU 57.6KW 2048 ISL 144 switches 43

44 3:1 oversubscribed Fabric (8) (128) 3:1 216RU 68KW 2048 ISL 136 switches 44

45 Total Fabric Power: Chassis vs. Fixed KW :1 oversubscribed - Massive Chassis Fixed Fabric size 45

46 Total Fabric RU: Chassis vs. Fixed RU :1 oversubscribed - Massive Chassis Fixed Fabric size 46

47 3:1 oversubscribed Fabric (32) (256) (64) (8) (2) 3:1 448RU 166.4KW 5120 ISL 352 switches 47

48 (16) 3:1 oversubscribed Fabric (256) 3:1 432RU 136.3KW 4096 ISL 272 switches 48

49 3:1 oversubscribed Fabric (64) (512) (128) (8) (2) 3:1 896RU 332.8KW 10,240 ISL 704 switches 49

50 3:1 oversubscribed Fabric (32) (512) (256) 3:1 1120RU 328.9KW 16,384 ISL 800 switches 50

51 (256) 3:1 oversubscribed Fabric (128) (1024) 3:1 1792RU 665.6KW ISL 1408 switches 51

52 (64) 3:1 oversubscribed Fabric (512) (1024) 3:1 2240RU 657.9KW 32,768 ISL 1600 switches 52

53 (512) 3:1 oversubscribed Fabric (256) (2048) 3:1 3584RU KW ISL 2816 switches 53

54 3:1 oversubscribed Fabric (128) (1024) (2048) 3:1 4480RU KW 65,536 ISL 3200 switches 54

55 55 The power to do more