DEVNET-1120 TRex Realistic Traffic Generator Hanoch Haim, Principal Engineer
Cisco Spark How Questions? Use Cisco Spark to communicate with the speaker after the session 1. Find this session in the Cisco Live Mobile App 2. Click Join the Discussion 3. Install Spark or go directly to the space 4. Enter messages/questions in the space cs.co/ciscolivebot#devnet-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Agenda Overview Stateless Stateful Advance Stateful 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
TRex Results Open Source Cisco Customers DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 5
TRex Usage Analytics monthly report (*) (*) ~1200 distinct returning users, (**) Users are distinct DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 6
TRex models of operation L7, DUT terminate TCP/SSL, flow based DUT inspect L7. does not change TCP. Flow based DUT L2/L3 Switch, packet based DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 7
What Problem is Being Solved? Networks include complex L4-7 features, such as Load Balancer, DPI/AVC, Firewall, NAT LB DPI/AVC Firewall, NAT Requires testing with stateful and realistic traffic mix DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 8
What Problem is Being Solved? Traffic generators for realistic traffic are Expensive ~$100-500K Not scalable for high rates Not flexible Implication Limited and late testing Different benchmarks and test methodologies Real life bottlenecks and design issues DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 9
What is TRex? Linux user-space application uses DPDK library Stateless: Stream based uses Scapy Stateful: flow based Generates, manipulates and amplifies based on templates of real, captured flows (W/O TCP stack) High performance: up to 200 Gb/sec Low cost: Standard server hardware Flexible and Open Software Virtualization Easy installation and deployment DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 10
DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 11
Stateless
Stateless High level functionality High scale ~10M-35MPPS/core Profile can support multiple streams, scalable to 20K parallel streams Interactive support GUI/TUI Statistic per port/ stream (e.g. latency/ Jitter) Python automation support Multi-user support Capture to Wireshark Scalable services using plugins (e.g. DHCP,IPv6) DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 13
Traffic Profile Example DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 14
Control plane High level DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 15
One stream with two directions DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 16
Python Automation example DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 17
Performance XL710 MPPS/Core link DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 18
Demo
Stateful
L7 inspection features Networks include complex L4-7 features, such as Load Balancer, DPI/AVC, Firewall, NAT LB DPI/AVC Firewall, NAT Requires testing with stateful and realistic traffic mix DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 21
Stateful Traffic Generation Model DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 22
High level software architecture DPDK/Multi-Threaded Scales linearly ~5MPPS/~20 Gb/sec per core Supports 1/10/40 Gb Intel NICs Share memo ry Intel NIC IF0 Client IF1 Server IF2 Client IF3 Server Flow-based Fast event scheduler Generates flow templates Can support 1K templates Scales up to 100K clients,1m servers CPU/Share memory CP thread 0 DP -thread 1 DP -thread 3 DP -thread 2 DP -thread 4 Rx thread 5 Flexible Client/server generation models Measures jitter/latency/flow order NAT translation/ipv6/tunnels User Interface Python API Benchmark automation GUI DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 23
Flow Generation Example of one flow with four packets FIF FIF FIF n Total PPS = CPS k flow_pkts k k=0 n Total CPS = CPS k k=0 Concurrent flows n = CPS k flow_duration k k=0 DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 24
DNS simple profile example DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 25
DNS output DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 26
HTTP & DNS DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 27
HTTP & DNS DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 28
Enterprise traffic profile Includes protocols with Control/Data dependency SIP RTSP DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 29
EMIX - YAML DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 30
EMIX DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 31
NAT/FW learning translation/randomization 1-Tx-SYN 1-Rx-SYN (learn translation per flow) 2-Tx-SYN-ACK 2-Rx-SYN-ACK (learn) 3-Tx-ACK 3-Rx-ACK (learn seq number randomization) DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 32
Client Clustering To simulate a big network Scale of the number of clients For Controller testing DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 33
Performance MPPS/Core @ 8M flows DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 34
Gbps extrapolation, average packet size 600B DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 35
Advanced Stateful
User space TCP stack Why? DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 37
User space TCP stack Why? DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 38
TRex ASTF features High scale TCP is the core component Can be tuned MSS/initwnd/delay-ack TCP is based on BSD with acceleration Interactive Accurate latency measurement usec Simulation of latency/jitter/drop in high rate OpenSSL integration L7 emulation layer Emulate application using micro-instructions Field engine DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 39
TRex ASTF features status High scale TCP is the core component Can be tuned MSS/initwnd/delay-ack TCP is based on BSD with acceleration Interactive Accurate latency measurement usec Simulation of latency/jitter/drop in high rate OpenSSL integration L7 emulation layer Emulate application using micro-instructions Field engine DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 40
L7 Emulation layer Client side Server side DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 41
HTTP simple profile DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 42
Client side pseudo code DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 43
Server side pseudo code DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 44
Profile with two template DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 45
Different IP pool for each template DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 46
Statistic DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 47
Client/Server only DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 48
Under the hood DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 49
TCP stack Flow Scale -TX 10M flows 320GByte 320M mbuf =10GB 0.01GB DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 50
TCP stack Flow Scale issue - RX 10M flows 320GByte 3-50GB for 1% drop rate 0.01GB DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 51
TCP stack Delay/Jitter/Drop simulation 100MPPS *100msec =10MPPS in Queue 100MPPS *100msec =10*16MPPS=0.16GB DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 52
TRex vs NGINX DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 53
Performance setup #2 DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 54
Performance numbers x80 faster x2000 less memory https://trex-tgn.cisco.com/trex/doc/trex_astf_vs_nginx.html DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 55
wrapup
TRex models of operation L7, DUT terminate TCP/SSL, flow based DUT inspect L7. does not change TCP. Flow based DUT L2/L3 Switch, packet based DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 57
DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 58
Cisco Spark How Questions? Use Cisco Spark to communicate with the speaker after the session 1. Find this session in the Cisco Live Mobile App 2. Click Join the Discussion 3. Install Spark or go directly to the space 4. Enter messages/questions in the space cs.co/ciscolivebot#devnet-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Please complete your Online Session Evaluations after each session Complete 4 Session Evaluations & the Overall Conference Evaluation (available from Thursday) to receive your Cisco Live T-shirt All surveys can be completed via the Cisco Live Mobile App or the Communication Stations Complete Your Online Session Evaluation Don t forget: Cisco Live sessions will be available for viewing on-demand after the event at www.ciscolive.com/global/on-demand-library/. 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Continue Your Education Stateless manual TRex documents Index GitHub DevNet zone DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 61
Continue Your Education Demos in the Cisco campus Walk-in Self-Paced Labs Tech Circle Meet the Engineer 1:1 meetings Related sessions DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 62
Thank you
Backup
Stateless Traffic profile
Simple Interleaving streams DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 67
Simple Interleaving streams -profile DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 68
Multi burst DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 69
Multi burst profile DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 70
Field Engine Flexible engine for changing packet fields Examples Change TOS 1-20 Range of client IPv4/IPv6 e.g. 10.0.0.1-10.0.0.254 Random packet size 64-9k Random destination IPv4/IPv6 Support any tunnel even not valid packet like QinQ/GRE/MPLS/Ipv6/UDP/Ipv4/HTTP DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 71
Field Engine, Syn attack DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 72
Pcap File Conversion to Streams DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 73
Per Stream Statistics DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 74
Per Stream Statistics - TUI DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 75
Per Stream Latency/Jitter Base on per stream stats hardware assist Software measures latency and jitter resolution is ~usec (not nsec) DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 76
Per Stream Statistics - TUI DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 77
Stateless Service mode
Service Mode DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 79
Service Mode Ping / ARP/DHCP Client IPv6 ND /Multicast setup Scan6 support Traffic Capturing Functional test New protocols multiplex framework DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 80
Service Mode Capture Monitoring Wireshark Pipe DEVNET-1120 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 81