DETOUR: A Large-Scale Non-Blocking Optical Data Center Fabric

Transcription

1 DETOUR: A Large-Scale Non-Blocking Optical Data Center Fabric Yi Dai 2 Jinzhen Bao 1,2, Dezun Dong 2, Baokang Zhao 2 1. PLA Academy of Military Science 2. National University of Defense Technology SCA2018, Sentosa, Singapore 1

2 Data Center Networks Various modern applications have proposed new requirements for data center networks High bisection bandwidth High Scalability High Flexibility 2

3 Optical Networking in Data Centers Electrical networking in data centers With high wiring complexity, power consumption and network building cost CLUE 3

4 Optical Networking in Data Centers Optical fabric is used to connect ToR switches Optical Fabric

5 Optical Networking in Data Centers Optical networking in data centers Optical Fabric Low cost Low power consumption Low wiring complexity High one-to-one bandwidth High-flexible topology 5

6 Prior Solutions Prior solutions suffer from the issues in simultaneously meet the goals of scalability, performance and flexibility c-through, Helios [SIGCOMM 10] OSA [NSDI 12] WaveCube [Infocom 15] 6

7 Prior Solutions By utilizing the broadcast-and-select optical switching mechanism, the optical fabric enables flexibility to support unicast and multicast Mordia [SIGCOMM 13] MegaSwitch [NSDI 17] OvS [Optics Express 15] 7

8 Prior Solutions Optical Solutions Scalability(Ports) Performance Flexibility c-through Helios [SIGCOMM 10] OSA [NSDI 12] Low(~320) High High WaveCube [Infocom 15] High(Unlimited) Low Low Mordia[SIGCOMM 13] MegaSwitch [NSDI 17] Low(~700) Low(~6000) High High OvS[Optics Express 15] High(100K+) Low High DETOUR High(69K+) High High 8

9 DETOUR Overview Scalability Optical Circuit Switches (OCSes) are physically connected in 2D- Torus topology Performance Enabling OCSes to optically forward signals between different dimensions Flexibility Broadcast-and-select optical switching mechanism SDN Controller OCS ToR Servers Rack 1 (1,1) Rack 2 (1,2) Rack 3 (1, 3) Rack n (1, n) Rack N (m, n) 9

10 DETOUR Data Plane 10

11 Optical components Multiplexer Wavelength Selective Switch Input 1 Input 2 Input 3 Input k Output 1 Input 1 Input 2 WSS DE Output 1 Output 2 Output 3 Input k Output 1 Input 1 Output k Demultiplexer 11

12 Mainly Optical components Multiplexer Wavelength Selective Switch Input 1 Input 2 Input 3 Input k Output 1 Input 1 Input 2 WSS DE Output 1 Output 2 Output 3 Input k Output 1 Input 1 Output k Demultiplexer 12

13 OCS Architecture W 1 West North E 1 W N/2 E N/2 W N/2+1 E N/2+1 N E N 3 N 1 N N/2 N N/2+1 N N EDFA (5:5) splitter 1 N+1 2 Passive Routing Fabric () Split optical signals, one part are sent to WSSes, another part are continue broadcasted N/2*1 South/North (5:5) splitter N/2*1 (1:N/2) splitter N WSS East DE Sender: k multiplexed wavelengths are South broadcasted across two dimensions S 1 S N/2 S N/2+1 S N Receiver: receive wavelengths from w other nodes, select wavelengths to local switch or forward to the other dimension P 1 P 2 P 3 P k 13

14 -Node Example RPF RPF Receive from w other nodes WSS WSS DE DE OCS 3 OCS EPS 3 EPS Each ToR sends on its own fiber WSS WSS Select k wavelengths from w*k wavelengths DE DE OCS 1 OCS 2 EPS 1 EPS 2 1

15 DETOUR Scalability RPF RPF WSS WSS DE OCS 3 OCS With current technology, w is up to 27 and k is up to 96, Thus EPS 3 DETOUR can scale to: EPS ToRs : w * w = 27 x 27 = 729 ToRs DE Ports : w x w x k = 27 x 27 x 96 69K+ Ports WSS WSS DE DE OCS 1 OCS 2 EPS 1 EPS 2 15

16 DETOUR Performance RPF RPF WSS WSS DE DE OCS 3 OCS EPS 3 EPS WSS WSS DE DE OCS 1 OCS 2 EPS 1 EPS 2 16

17 DETOUR Performance RPF RPF WSS WSS DE DE OCS 3 OCS EPS 3 EPS WSS WSS DE Support the establishment of directly connected optical OCS 1 OCS 2 links EPS between 1 arbitrary switches EPS 2 DE 17

18 DETOUR Flexibility RPF RPF WSS WSS DE DE OCS 3 OCS EPS 3 EPS WSS WSS DE DE OCS 1 OCS 2 EPS 1 EPS 2 18

19 DETOUR Flexibility RPF RPF WSS WSS DE DE OCS 3 OCS EPS 3 EPS WSS WSS DE DE OCS 1 OCS 2 Support multiple traffic patterns, such as unicast and multicast EPS 1 EPS 2 19

20 DETOUR Control Plane 20

21 Control Plane: Logically Centralized Optimize the network to better serve the traffic SDN Controller Topology and Link Capacities Routing Rack N (m, n) Rack 1 (1,1) Rack 2 (1,2) Rack 3 (1, 3) Rack n (1, n) 21

22 DETOUR Control Plane Hedera [NSDI 10] Maximum weight matching Shortest path routing 1. Estimate traffic demand between ToRs 2. Assign direct link to heavy communication ToR pairs SDN Controller 22

23 Topology Generation (a) Physically Topology (n = 9, k = ) Max-Min Fairness Bandwidth Allocation Hedera [NSDI 10] (b) ToR Bandwidth Demand Matrix (d) Wavelength Bipartite Graph Maximum weighted -matching Edmonds algorithm (c) Bandwidth Bipartite Graph 23

24 2 Edge-coloring of Bipartite Graph Conflict xy 5xy 5xy 9yx Wavelength Assignment Non-Conflict xy 5xy 9yx 9yx Wavelength Adjustment

25 Conflict Avoiding Algorithm There always exist two forwarding nodes between the source and destination nodes in different dimensions DST SRC 25

26 Conflict Avoiding Algorithm There always exist two forwarding nodes between the source and destination nodes in different dimensions For each match (color or wavelength), recursively adjust the forwarding nodes for conflict wavelengths until noconflicts Example DST SRC 26

27 Conflict Avoiding Algorithm There always exist two forwarding nodes between the source and destination nodes in different dimensions For each match (color or wavelength), recursively adjust the forwarding nodes for conflict wavelengths until noconflicts Example DST SRC 27

28 Conflict Avoiding Algorithm There always exist two forwarding nodes between the source and destination nodes in different dimensions For each match (color or wavelength), recursively adjust the forwarding nodes for conflict wavelengths until noconflicts Example DST SRC 28

29 Conflict Avoiding Algorithm There always exist two forwarding nodes between the source and destination nodes in different dimensions For each match (color or wavelength), recursively adjust the forwarding nodes for conflict wavelengths until noconflicts Example DST SRC 29

30 Conflict Avoiding Algorithm There always exist two forwarding nodes between the source and destination nodes in different dimensions For each match (color or wavelength), recursively adjust the forwarding nodes for conflict wavelengths until noconflicts Example DST SRC 30

31 Conflict Avoiding Algorithm There always exist two forwarding nodes between the source and destination nodes in different dimensions For each match (color or wavelength), recursively adjust the forwarding nodes for conflict wavelengths until noconflicts Example DST SRC 31

32 Conflict Avoiding Algorithm There always exist two forwarding nodes between the source and destination nodes in different dimensions For each match (color or wavelength), recursively adjust the forwarding nodes for conflict wavelengths until noconflicts Example DST SRC 32

33 DETOUR Control Plane Hedera [NSDI 10] Maximum weight matching Shortest path routing Seamless reconfiguration 1. Estimate traffic demand between ToRs 2. Assign direct link to heavy communication ToR pairs 3. Reconfigure optical links and route traffic SDN Controller 33

34 Seamless Reconfiguration 1. Set forwarding rules on ToR switches to mitigate flows to static network topology 2. Set optical connections 3. Set forwarding rules on ToR switches to mitigate flows to new network topology SDN Controller ToR Switch Optical Switch Southbound API 3

35 Simulation Settings A Event based flow-level simulator with a centralized controller Topology A 8*8 2D-Torus topology Each ToR switch has 18 ports, in which 10 ports connect with servers and 8 ports connect with OCSes Metric Flow completion time Energy consumption Traffic patterns Static traffic pattern (stride, random) Production cluster workload Compared solutions Jellyfish, OvS, Non-blocking Optical Switching Network 35

36 Simulation Results DETOUR achieves comparable average FCT and energy consumption with Non-Blocking optical switching network Compared with Jellyfish and OvS, DETOUR reduces 50% and 68% FCT, and reduces 1% and 8% energy consumption, respectively Average FCT Energy Consumption 36

37 Simulation Results Network throughput increases on each reconfiguration. Compared witch OvS, DETOUR increases a relative higher value after each reconfiguration 37

38 Conclusion Data plane Redesign the OCS to support optically forwarding between different dimensions Control plane Recursively wavelength conflict avoiding algorithm DETOUR: An optical design that supports scalability, high performance and flexibility to satisfy today s production workloads 38

39 Thanks! 39