How to Build a 100 Gbps DDoS Traffic Generator

Transcription

1 How to Build a 100 Gbps DDoS Traffic Generator DIY with a Single Commodity-off-the-shelf Server (COTS) Surasak Sanguanpong Surasak.S@ku.ac.th

2 DISCLAIMER THE FOLLOWING CONTENTS HAS BEEN APPROVED FOR APPROPIATE AUDIENCES THE PRESENTATION HAS BEEN RATED RESTRICTED NON TECHNICAL REQUIRES ACCOMPANYING OR MENTOR การดำเน นการใดจากต วอย างการบรรยายน ต องทำโดยไม รบกวนระบบคอมพ วเตอร อ นเพ อไม ให กระทบก บมาตรา ๑๐ ซ งอาจม ความผ ดตามมาตรา ๑๒ ตามท กำหนดโดย พรบ. ว าด วยการกระทำความผ ดเด ยวก บคอมพ วเตอร (ฉบ บท ๒) พ.ศ. ๒๕๖๐ USE AT YOUR OWN RISK

3 A sample of DDoS in Q Gbps Peak 190 Mpps Peak 18,300 Simultaneous HD TV channels

4 Why DDoS traffic generator? R&D Tool for : Network behavior Traffic Log, Traffic Analysis, Anti-DDoS Testing network middle boxes IDS, IPS, Firewall, Router Synthetic traffic but closed to realistic traffic

5 HW V.S. SW generator Items Dedicated Hardware Server with Software Precision High Moderate Latency Low Moderate Capability Full max rate Near max rate Cost High Economical

6 Goal: 100 Gb/s DDoS traffic generator Constraints: A single COTS server A single 100 GigE NIC (not 10x10 GigE)

7 Outline INTRO HW SW TEST BED Introduction DDoS understanding Ethernet revisiting HW and SW solution for 100 Gb/s generator Server and components Linux Networking Stack Open source SW generator Testbed and Performance results

8 Introduction: Understanding DDoS

9 DoS Single Source

10 DDoS The same traffic will be simulated

11 Broadly types of DDoS Volume Based Attacks To saturate the bandwidth of the attacked site Measured in bits per second (bps) Protocol Attacks To consumes actual target resources, or intermediate communication equipment (firewalls, load balancers, etc) Measured in packets per second (pps) Application Layer Attacks Low-and-slow attacks to crash targets Measured in requests per second (rps)

12 Introduction: Ethernet Update, Understanding Ethernet Wire Speed and Throughput Calculations

13 Evolution of Ethernet Capacity and speed requirements on data links keep increasing 40,000X in 34 yrs 40,100 Gb/s 25 Gb/s IEEE Std 802.3bs 200, 400 Gb/s Big Data, AI require more bandwidth 10 Mb/s Servers have begun to be capable of sustaining 100G to memory 100 Mb/s 1 Gb/s 10 Gb/s

14 Understanding Ethernet Wire speed Wire Speed refers to the hypothetical peak packet bitrate Q: What is the maximum packet per second (pps) that can be generated for a specific Ethernet speed?

15 The Frame sizes matter Two options for consideration: 1 Minimum Frame Size (Large number of frame per unit time) S S S S S S S 1 second 2 Maximum Frame Size (Small number of frame per unit time) L L L 1 second

16 Ethernet frame by frame delivery from 46 to (bytes) PA SFD DA SA Type Payload FCS IFG PA SFD S Mini Size (64 bytes) L Max Size (1518 bytes) Fields Size (bytes) Preamble+SFD 8 Dst Address 6 Src Address 6 Type 2 Payload 46 FCS 4 IFG 12 Total 84 Fields Size (bytes) Preamble+SFD 8 Dst Address 6 Src Address 6 Type 2 Payload 1,500 FCS 4 IFG 12 Total 1,538

17 GigE Maximum frame rate for 64 byte packets over 100 GigE link M = Speed/Size = 100x10 9 / 672 = 148,809,523 pps Maximum frame rate for 1518 byte packets over 100 GigE link M = Speed/Size = 100x10 9 / 12,304 = 8,127,438 pps Maximum throughput T = M*64*8 = Gbps Maximum throughput T = M*1518*8 = Gbps

18 100 GigE performance Max Frame speed Rate (Gb/s) #Frame (Min:64B) #Frame (Max:1518B) M 81 K M 812 K M 3.25 M M 8.12 M

19 Challenge for Packet Processing T 1 Inter Packet Arrival Time T 2 Incoming Packet 1 Lookup in Packet 1 Do Packet 1 Incoming Packet 2 Lookup in Packet 2 #Frame and Timing with 64 byte length Rate (Gb/s) #Frame (Million) Inter Packet Arriving Time (ns) Do Packet 2

20 Time/CPU budget in 100 Gbps With Mpps, the time budget for processing a single packet is: 1/(148.81x10 6 ) = 6.72 nanosecond Considering a server with 3 GHz CPU.. How many clock cycle does it require to handle minimum frame size of 100 Gb/s packet rate? 6.72x10-9 *3x10 9 ~ 20 clock cycles

21 Hardware Investigation To answer Hardware is it capable?

22 To Delivery 100 GigE 100 GbE 4 Crucial components: 1 CPU 2 Interconnection 100 GbE Memory Bus 4 3 PCI Bus

23 Hardware Capability 1 2 QPI 156 Gb/s 4 4 Channels DDR MZH upto 546 Gb/s PCIe 3.0 upto 40 lanes/sockets (252 Gb/s for x16) 3

24 Yes!, the hardware is capable.

25 OS Kernel & Network Stack Investigation To answer Software is it capable?

26 OS s obstacle Traditional OS network stacks is problematic Not design with this speed in mind Many features essential for networking filtering, connection tracking, memory management, VLANs, overlay, and process isolation Not scalable even many CPU cores these days

27 Overhead in Linux kernel Socket based system calls Linux Network Stack Walkthrough (2.4.20) Context switching and blocking I/O Data Copying from kernel to userspace Interrupts Handling High latency

28 How to solve this obstacle? Solution: Kernel Bypass

29 Conventional Stack V.S. Kernel bypass Let s bypass kernel and work directly with NICs Allows access to the hardware directly from applications Using a set of libraries for fast packet processing Reduces latency with more packets to be processed Handles packets within minimum number of CPU cycles But Provides only very basic set of functions (memory management, ring buffers, poll-mode drivers) Require reimplementation of others IP stack features Conventional (Sockets based) User Application Sockets Kernel TCP/IP Stack Network Driver Hardware Kernel Bypass (RDMA based) User Application Packets Library Kernel TCP/IP Stack Network Driver Hardware

30 Zero Copying (ZC) with RDMA Conventional (Sockets based) Kernel Bypass (RDMA based) User User Application Application App buffer Data copy Packet Libraries Shared buffer Kernel Sockets TCP/IP Stack Network Driver Sockets buffer Data copy Device buffer Data copy Kernel TCP/IP Stack Network Driver ZC with Remote Direct Memory Access Hardware Hardware

31 Scalable with multicores Application Application Application Kernel Packet Libraries Packet Libraries Packet Libraries Core 0 Core 1 Core 2 Core 3 NIC Tx0 Rx0 Tx1 Rx1

32 Fast (Userspace) Packet Processing DPDK Netmap PF Ring OS Linux, FreeBSD FreeBSD,Linux Linux License BSD BSD LGPL + paid Language C C C Use Case Appliances, NFV NFV, Router Packet Capture, IDS/IPS NIC vendors Several Intel Intel Supports Community Community Company Kernel bypass also known as Fast Packet Processing High-Performance Packet IO Data Plane Processing Acceleration Framework

33 DPDK Data Plane Development Kit A set of libraries and drivers for fast packet processing Main Libraries multicore framework huge page memory ring buffers poll-mode drivers Currently managed as an open-source project under the Linux Foundation

34 DPDK Architecture

35 DPDK in Linux Distros Available as part of several OS distributions ClearLinux

36 DPDK based Open Source Projects Virtual Multilayer Switch integrated into various cloud platform Carrier-grade, integrated, open source platform to accelerate Network Function Virtualization (NFV) SPDK Storage Performance Development Kit pktgen-dpdk libraries for writing high performance, scalable, usermode storage applications Software based traffic generator Packet-journey Linux router IO services framework for the network and storage software with Vector Packet Processing Linux scalable software routers, proved with 500k routes The Stateful Traffic Generator for L1-L7 Flexible Stateless/Stateful Traffic Generator for L4-L7

37 What can be built with DPDK? Switch/Router Stateless and stateful Firewall IDS/IPS Load balancer Traffic recorder Fast internet scanners Stateless packet generator Stateful, application-like flow generator IPsecVPN gateway Accelerated key-value DB Accelerated NAS

38 TRex DPDK based stateful/stateless traffic generator (L4-L7) Replay of real traffic (pcap), scalable to 10K parallel streams Supports about mpps per core, scalable with the number of cores Scale to 200 Gb/s for one COTS High scale benchmarks for stateful networking gear (Firewall/NAT/DPI) Generating high scale DDOS attacks High scale, flexible testing for switches Scale tests for huge numbers of clients/servers

39 TRex sample Traffic config file 255 clients talking to 255 servers root: ~/trex-core/scripts# cat cap2/dns.yaml - duration : 1.0 generator : distribution : "seq" clients_start : " " clients_end : " " servers_start : " " servers_end : " " clients_per_gb : 201 min_clients : 101 dual_port_mask : " " tcp_aging : 0 udp_aging : 0 cap_info : - name: cap2/dns.pcap cps : 10.0 ipg : rtt : w : 1

40 Testbed and Performance Measurements

41 40 Gb/s Traffic Generator Reports Pktgen-dpdk Linux-DPDK.pdf TRex Warp17 Where is the 100 Gb/s results?

42 Testbed Dell R430 Dell R430 HW: Dell R430 2xIntel Xeon E v GHz dual socket, 10-core 64 GB RAM (4x16 GB DDR MHz) 1.5 TB NL-SCSI DPDK based 100Gbps NICs Sender 100 GigE Receiver SW CentOS 7.3 Kernel 3.10 DPDK TRex 2.29

43 CMLI Output UDP packets generators Random Source IP Addresses

44 bytes

45 Ongoing R&D Project Porting Traffic Recorder HTTP Log and Flow Log Current testbed 30 Gb/s capability (4x10 Gb/s) ~60,000 flow/s ~10 Million active flows Support both IPv4 and IPv6 Development of Stateless DDoS mitigation Development of Traffic base IoT devices auto discovery and analysis

46 Summary COTS Server is capable for 100 GigE Data plane solution is a future for COTS based appliance Rising trend of SW based network appliances for high speed network

47 Thanks for your attention Q&A