TurboFlow: Information Rich Flow Record Generation on Commodity Switches

Transcription

1 Turbo: Information Rich Record Generation on Commodity Switches John Sonchack 1, Adam J. Aviv 2, Eric Keller 3, Jonathan M. Smith 1 1 University of Pennsylvania, 2 USNA, 3 University of Colorado

2 Introduction: Network Monitoring with Records record <srcip, dstip, srcport, dstport, arrivalts, avginterarrival, pktsdroppedct, queuelen, > 2

3 Introduction: Network Monitoring with Records Debugging Traffic Engineering Security record <srcip, dstip, srcport, dstport, arrivalts, avginterarrival, pktsdroppedct, queuelen, > 3

4 Introduction: Network Monitoring with Records Low throughput because packets dropped at switch 2! Debugging Traffic Engineering Security record <srcip, dstip, srcport, dstport, arrivalts, avginterarrival, pktsdroppedct, queuelen, > 4

5 Introduction: Network Monitoring with Records Congestion at Debugging switch 2! Traffic Engineering Security record <srcip, dstip, srcport, dstport, arrivalts, avginterarrival, pktsdroppedct, queuelen, > 5

6 Introduction: Network Monitoring with Records Congestion at Debugging switch 2! Traffic Engineering Security record <srcip, dstip, srcport, dstport, arrivalts, avginterarrival, pktsdroppedct, queuelen, > 6

7 Introduction: Network Monitoring with Records Hosts 2 and 4 are in a botnet! Debugging Traffic Engineering Security record <srcip, dstip, srcport, dstport, arrivalts, avginterarrival, pktsdroppedct, queuelen, > From botnet 7

8 Introduction: Network Monitoring with Records Debugging Traffic Engineering Security record <srcip, dstip, srcport, dstport, arrivalts, avginterarrival, pktsdroppedct, queuelen, > 3.2$Tb/s >$100$M$ packets/s >$10$M$ flows/s 8

9 Monitoring Switches: Prior Work Sampling Sampled Packets Records 9 Inaccurate

10 Monitoring Switches: Prior Work Sampling Sampled Packets Records Server Offloading Packets (or other records) Records Expensive Custom Hardware Offloading Packets (or other records) Records Restrictive 10 Inaccurate

11 Introduction: Turbo Main idea: Optimize instead of offload. Q : What can we get out of the programmable hardware in next-generation commodity switches? Programmable Forwarding Engines Onboard Microservers 11

12 Introduction: Turbo Main idea: Optimize instead of offload. Q : What can we get out of the programmable hardware in next-generation commodity switches? A : record generation for multi-terabit rate traffic without sampling or offloading. Programmable Forwarding Engines Onboard Microservers 12

13 Introduction: Turbo Turbo Microserver Records Record Generation Partial Records Pre-aggregation Programmable Forwarding Engine Packets 13

14 Outline Turbo Microserver Introduction Record Generation Architecture Evaluation Records Partial Records Programmable Forwarding Engine Conclusion Pre-aggregation Packets 14

15 Turbo Architecture Turbo Microserver Records Record Generation Partial Records Pre-aggregation Programmable Forwarding Engine Packets 15

16 Background: Programmable Forwarding Engines Switch CPU Forwarding Engine Match Action Stateful Variables 16

17 Background: Programmable Forwarding Engines Switch CPU Forwarding Engine Match Action Stateful Variables (IP 5-tuple) Packet Count Average Interarrival A -> B E -> G F -> G Update 3 1 ms Update 49 8 ms Update 3 42 ms 17

18 Background: Programmable Forwarding Engines Switch CPU Forwarding Engine Table Manager Rule installation rate: < 10 K / s Match Action Stateful Variables (IP 5-tuple) Packet Count Average Interarrival arrival 1 Tb/s: > 10,000 K / s A -> B E -> G F -> G Update 3 1 ms Update 49 8 ms Update 3 42 ms 18

19 Turbo Architecture: Using the FE Efficiently Switch CPU Table Manager Forwarding Engine 19

20 Turbo Architecture: Using Switch CPU the FE Efficiently Forwarding Engine Match Stateful Variables Current (IP 5-tuple) Packet Count Average Interarrival 20

21 Turbo Architecture: Using the FE Efficiently Switch CPU Table Manager Forwarding Engine Match Stateful Variables Key Hash 1 Current (IP 5-tuple) Packet Count Average Interarrival

22 Turbo Architecture: Using Switch CPU the FE Efficiently Tracked : Update Counters A->B Forwarding Engine HASH Match Key Hash Stateful Variables Current Key Packet Average (IP 5-tuple) Count Interarrival (IP 5-tuple) A A -> -> BB ms C -> -> D ms ms E E -> -> FF ms ms Z Z -> -> Q ms ms 22

23 Turbo Architecture: Using the FE Efficiently Switch CPU Record Aggregator Tracked : Update Counters A->B G->H Forwarding Engine HASH Match Key Hash Untracked : Replace colliding record, send it to CPU Tracked (IP 5-tuple) Stateful Variables Packet Count Average Interarrival A -> B 4 3 ms C -> D 49 8 ms E -> F 3 42 ms G -> H 1 Z -> Q: 9 10 ms 23

24 Turbo Design Turbo Microserver Records Record Generation Partial Records Pre-aggregation Programmable Forwarding Engine Packets 24

25 Turbo Architecture: Using the CPU Efficiently Count: 10 Count: 2 Key Count Count: 12 A->B 12 Partial Records Stats Dictionary Records 25

26 Turbo Architecture: Using the CPU Efficiently Optimization baseline (std:: unordered_map) Reduce Pointer Operations Vectorize Key Comparison Batch and Prefetch Performance Vs. Baseline X 3.79X 4.9X average of 146 cycles spent per partial flow record. Count: 10 Count: 2 Partial Records Key Count A->B 12 Stats Dictionary Count: 12 Records 26

27 Outline Turbo Microserver Introduction Optimized Record Generation Architecture Evaluation Records Partial Records Programmable Forwarding Engine Conclusion Pre-aggregation Packets 27

28 Implementation and Evaluation Implementations P4 Switch (3.2 Tb/s Barefoot Tofino) Benchmark Workloads 10 Gb/s Internet Router Traces (CAIDA 2015) P4 SmartNIC (40 Gb/s Netronome NFP) 144 Node Simulated Datacenter Cluster (YAPS simulator) 28

29 Implementation and Evaluation Implementations P4 Switch (3.2 Tb/s Barefoot Tofino) Benchmark Workloads 10 Gb/s Internet Router Traces (CAIDA 2015) P4 SmartNIC (40 Gb/s Netronome NFP) 144 Node Simulated Datacenter Cluster (YAPS simulator) 29

30 Required Average Throughput to Monitor 100 X 10 Gb/s Internet Links Partial Record per Second (Millions) No FE Aggregation FE Aggregation with 5 MB FE Memory ( 26%) 30

31 Required Average Throughput to Monitor 100 X 10 Gb/s Internet Links Partial aggregation using 5 MB of FE memory reduces workload by ~4X. Partial Record per Second (Millions) No FE Aggregation FE Aggregation with 5 MB FE Memory ( 26%) 31

32 Required Average Throughput to Monitor 100 X 10 Gb/s Internet Links Optimizations improve performance by ~5X. Partial Record per Second (Millions) 40 std::unordered map Fully Optimized Switch CPU Cores No FE Aggregation FE Aggregation with 5 MB FE Memory ( 26%) 32

33 Required Average Throughput to Monitor 100 X 10 Gb/s Internet Links FE pre-aggregation + optimizations = terabit rate workloads using 1 core and ~26% of FE memory. Partial Record per Second (Millions) 40 std::unordered map Fully Optimized Switch CPU Cores No FE Aggregation FE Aggregation with 5 MB FE Memory ( 26%) 33

34 Outline Introduction Turbo Design Implementation and Evaluation Conclusion 34

35 In the Paper More Evaluation Cost analysis Psuedocode More interesting flow features Pipeline layouts 35 Expected worst case analysis

36 Conclusion (and Thank You for Listening!) records are important for monitoring, but difficult to generate at the switch due to high traffic rates. Turbo is a flow record generator carefully optimized for next generation commodity switch hardware that scales to multi-terabit rate traffic without sampling. 36