MoonGen. A Scriptable High-Speed Packet Generator. Paul Emmerich. January 31st, 2016 FOSDEM Chair for Network Architectures and Services

Size: px
Start display at page:

Download "MoonGen. A Scriptable High-Speed Packet Generator. Paul Emmerich. January 31st, 2016 FOSDEM Chair for Network Architectures and Services"

Transcription

1 MoonGen A Scriptable High-Speed Packet Generator Paul Emmerich January 31st, 216 FOSDEM 216 Chair for Network Architectures and Services Department of Informatics Paul Emmerich MoonGen: A Scriptable High-Speed Packet Generator 1

2 Outline Hardware vs. Software Packet Generators Architecture of MoonGen Hardware Timestamping on Commodity NICs Precise Rate Control Example Measurements Paul Emmerich MoonGen: A Scriptable High-Speed Packet Generator 2

3 Chair for Network Architectures and Services Source: Paul Emmerich MoonGen: A Scriptable High-Speed Packet Generator 3

4 Challenges for software packet generators Hardware packet generators are Precise Accurate Fast Paul Emmerich MoonGen: A Scriptable High-Speed Packet Generator 4

5 Challenges for software packet generators Hardware packet generators are Precise Accurate Fast Software packet generators Run on cheap commodity hardware Flexible Paul Emmerich MoonGen: A Scriptable High-Speed Packet Generator 4

6 Challenges for software packet generators Hardware packet generators are Precise Accurate Fast Software packet generators Run on cheap commodity hardware Flexible Key challenges for software packet generators Rate control Timestamping Paul Emmerich MoonGen: A Scriptable High-Speed Packet Generator 4

7 Design goals Design goal of MoonGen Combine the advantages of both approaches while avoiding their disadvantages. Paul Emmerich MoonGen: A Scriptable High-Speed Packet Generator 5

8 Design goals Design goal of MoonGen Combine the advantages of both approaches while avoiding their disadvantages. Fast: DPDK for packet I/O, explicit multi-core support Flexible: Craft all packets in user-controlled Lua scripts Timestamping: Utilize hardware features found on modern commodity NICs Rate control: Hardware features and a novel software approach Paul Emmerich MoonGen: A Scriptable High-Speed Packet Generator 5

9 Architecture Userscript Lua VM Userscript master spawn Lua VM Lua VM Userscript Userscript slave config API data API MoonGen MoonGen Core config API data API DPDK HW NIC Q... Q n NIC Port Paul Emmerich MoonGen: A Scriptable High-Speed Packet Generator 6

10 Hardware timestamping NICs support PTP for precise clock synchronization PTP support requires hardware timestamping capabilities These can be (mis-)used for delay measurements Typical precision ±6.4 ns (Intel 1 GbE chips) ±32 ns (Intel GbE chips) Some restrictions Packets must be UDP or PTP L2 protocol Minimum UDP packet size is 84 bytes Paul Emmerich MoonGen: A Scriptable High-Speed Packet Generator 7

11 Software rate control in existing packet generators Loadgen p 5 NIC NIC DuT p 5 p 4 p 3 p 2 p 1 p Q memory Q NIC Wire Software tries to push single packets to the NIC Queues cannot be used, no batch processing NICs work with an asynchronous push-pull model This can lead to micro-bursts Unreliable, imprecise, and bad performance Paul Emmerich MoonGen: A Scriptable High-Speed Packet Generator 8

12 Hardware rate control Loadgen p 9 NIC HW rate control enabled NIC DuT p 9 p 8 p 7 p 6 p 5 p 4 p 3 p 2 p 1 p Q memory Q NIC Wire Modern NICs support rate control in hardware Limited to constant bit rate and bursty traffic Precision controlled by the hardware High performance as queues can be used, but inflexible Paul Emmerich MoonGen: A Scriptable High-Speed Packet Generator 9

13 Software rate control based on invalid packets Loadgen p 6 NIC HW rate control disabled NIC DuT p 6 p4 i p p3 i p i p p1 i p p i 5 p p 1 p Q memory Q NIC Wire Fill gaps with invalid packets p i (e.g. bad CRC) NIC in the DuT drops invalid packets without side-effects Combines advantages of both approaches Precision limited by byte rate (.8 ns per byte) and minimum packet size (33 byte) High performance & high precision Paul Emmerich MoonGen: A Scriptable High-Speed Packet Generator 1

14 Does it work? Test setup: forward packets with Open vswitch Measure the latency of the device under test CBR Hardware (Median) CBR Hardware (25/75th perc.) Latency [µs] Offered load [Mpps] Paul Emmerich MoonGen: A Scriptable High-Speed Packet Generator 11

15 Does it work? Compare both rate control approaches Maximum deviation: 2% Latency [µs] CBR Hardware (Median) CBR Hardware (25/75th perc.) CBR MoonGen (Median) CBR MoonGen(25/75th perc.) Offered load [Mpps] Paul Emmerich MoonGen: A Scriptable High-Speed Packet Generator 12

16 Does it matter? Compare CBR with Poisson traffic Different response from the device under test Latency [µs] CBR MoonGen (Median) CBR MoonGen (25/75th perc.) Poisson MoonGen (Median) Poisson MoonGen (25/75th perc.) Offered load [Mpps] Paul Emmerich MoonGen: A Scriptable High-Speed Packet Generator 13

17 Example: Linux NAPI Open vswitch on Linux Uniform distribution caused by interrupt throttling Probability [%] Latency [µs] Paul Emmerich MoonGen: A Scriptable High-Speed Packet Generator 14

18 Example: Linux Virtualization (VirtIO) Open vswitch forwarding through a VM Long tail distribution, typical for VMs Probability [%] Latency [µs] Paul Emmerich MoonGen: A Scriptable High-Speed Packet Generator 15

19 Example: Hardware Switch AS X 1/4 GbE OpenFlow switch Bimodal distribution caused by more input than output ports Some packets are forwarded directly (cut-through switch) Some packets are blocked by another flow and buffered Relative Probability [%] Latency [µs] Paul Emmerich MoonGen: A Scriptable High-Speed Packet Generator 16

20 Summary Speeds of 1 Gbit/s per CPU core (64 byte packets) Sub-microsecond precision and accuracy Execute user-defined script code for each packet Easy to use Paul Emmerich MoonGen: A Scriptable High-Speed Packet Generator 17

21 Q & A Try MoonGen yourself! Questions? Paul Emmerich MoonGen: A Scriptable High-Speed Packet Generator 18

22 [Backup slide] Performance I: Lua can be faster than C UDP packets from varying source IP addresses Packet rate [Mpps] CPU frequency [GHz] MoonGen Pktgen-DPDK Pktgen-DPDK needs a complicated main loop that covers all possibilites MoonGen can use a tight inner loop Paul Emmerich MoonGen: A Scriptable High-Speed Packet Generator 19

23 [Backup slide] Performance II: heavy workload and multi-core scaling Generate random UDP packets on 2 1 Gbit NICs 8 calls to Lua s standard math.random per packet CPUs artificially clocked down to 1.2 GHz Packet rate [Mpps] Number of 1.2 GHz CPU cores Paul Emmerich MoonGen: A Scriptable High-Speed Packet Generator 2

24 [Backup slide] Performance III: 4 GbE Generate random UDP packets on 2 1 Gbit NICs 8 calls to Lua s standard math.random per packet CPUs artificially clocked down to 1.2 GHz Rate [Gbit/s] core 2 cores 3 cores Packet size [byte] Paul Emmerich MoonGen: A Scriptable High-Speed Packet Generator 21

25 [Backup Slide] Rate control: 5 kpps 4 MoonGen (hardware) Probability [%] 2 2 Pktgen-DPDK zsend Inter-arrival time [µs] Paul Emmerich MoonGen: A Scriptable High-Speed Packet Generator 22

26 [Backup Slide] Rate control: 1, kpps 2 MoonGen (hardware) Probability [%] 2 4 Pktgen-DPDK zsend Inter-arrival time [µs] Paul Emmerich MoonGen: A Scriptable High-Speed Packet Generator 23

27 [Backup Slide] HW/SW rate control details Rate Software Bursts ±64 ns ±128 ns ±256 ns ±512 ns MoonGen.2% 49.9% 74.9% 99.8% 99.8% 5 kpps Pktgen-DPDK.1% 37.7% 72.3% 92% 94.5% zsend 28.6% 3.9% 5.4% 6.4% 13.8% MoonGen 1.2% 5.5% 52% 97% 1% 1 kpps Pktgen-DPDK 14.2% 36.7% 58% 7.6% 95.9% zsend 52% 4.6% 7.9% 24.2% 88.1% Paul Emmerich MoonGen: A Scriptable High-Speed Packet Generator 24

28 [Backup slide] Effects of bad rate control Interrupt rate of an Open vswitch packet forwarder 1 5 Interrupt rate [Hz] Load generated with MoonGen Load generated with zsend Offered load [Mpps] Micro-bursts confuse dynamic interrupt throttling This affects latency (cannot be measured with zsend) Paul Emmerich MoonGen: A Scriptable High-Speed Packet Generator 25

29 [Backup slide] Hardware timestamping precision and accuracy Measure latencies of cables of various length Calculate encoding time k and propagation speed v p 4 Time [ns] Latency Cable length [m] Result for fiber cable: k 311ns, v p =.72c ±.56c Paul Emmerich MoonGen: A Scriptable High-Speed Packet Generator 26

30 [Backup slide] Effects of invalid packets Median latency of an Open vswitch packet forwarder Packet rate controlled by hardware vs. invalid frames Deviation [%] Offered load [Mpps] Minor modifications to the DuT (e.g. an active SSH session) result in a deviation of up to 15% with the same rate control mechanism Paul Emmerich MoonGen: A Scriptable High-Speed Packet Generator 27

Precision and Accuracy of Packet Generators Who tests the testers?

Precision and Accuracy of Packet Generators Who tests the testers? Precision and Accuracy of Packet Generators Who tests the testers? Paul Emmerich, Sebastian Gallenmüller, Alexander Frank, Georg Carle emmericp@net.in.tum.de Technical University of Munich Chair of Network

More information

MoonGen: A Fast and Flexible Packet Generator

MoonGen: A Fast and Flexible Packet Generator MoonGen: A Fast and Flexible Packet Generator Paul Emmerich emmericp@net.in.tum.de Technical University of Munich Chair of Network Architectures and Services IETF-100, 16.11.2017 Research at net.in.tum

More information

Measuring High-Performance Packet Processing

Measuring High-Performance Packet Processing Chair of Network Architectures and Services Department of Informatics Technical University of Munich Measuring High-Performance Packet Processing Georg Carle, Sebastian Gallenmüller, Dominik Scholz, Florian

More information

Mind the Gap A Comparison of Software Packet Generators

Mind the Gap A Comparison of Software Packet Generators Mind the Gap A Comparison of Software Packet Generators Paul Emmerich Technical University of Munich Garching, Germany emmericp@net.in.tum.de Andrew W. Moore University of Cambridge Cambridge, United Kingdom

More information

Demystifying Network Cards

Demystifying Network Cards Demystifying Network Cards Paul Emmerich December 27, 2017 Chair of Network Architectures and Services About me PhD student at Researching performance of software packet processing systems Mostly working

More information

Fast packet processing in the cloud. Dániel Géhberger Ericsson Research

Fast packet processing in the cloud. Dániel Géhberger Ericsson Research Fast packet processing in the cloud Dániel Géhberger Ericsson Research Outline Motivation Service chains Hardware related topics, acceleration Virtualization basics Software performance and acceleration

More information

Tales of the Tail Hardware, OS, and Application-level Sources of Tail Latency

Tales of the Tail Hardware, OS, and Application-level Sources of Tail Latency Tales of the Tail Hardware, OS, and Application-level Sources of Tail Latency Jialin Li, Naveen Kr. Sharma, Dan R. K. Ports and Steven D. Gribble February 2, 2015 1 Introduction What is Tail Latency? What

More information

Open Source Traffic Analyzer

Open Source Traffic Analyzer Open Source Traffic Analyzer Daniel Turull June 2010 Outline 1 Introduction 2 Background study 3 Design 4 Implementation 5 Evaluation 6 Conclusions 7 Demo Outline 1 Introduction 2 Background study 3 Design

More information

Learning with Purpose

Learning with Purpose Network Measurement for 100Gbps Links Using Multicore Processors Xiaoban Wu, Dr. Peilong Li, Dr. Yongyi Ran, Prof. Yan Luo Department of Electrical and Computer Engineering University of Massachusetts

More information

Evolution of the netmap architecture

Evolution of the netmap architecture L < > T H local Evolution of the netmap architecture Evolution of the netmap architecture -- Page 1/21 Evolution of the netmap architecture Luigi Rizzo, Università di Pisa http://info.iet.unipi.it/~luigi/vale/

More information

The Power of Batching in the Click Modular Router

The Power of Batching in the Click Modular Router The Power of Batching in the Click Modular Router Joongi Kim, Seonggu Huh, Keon Jang, * KyoungSoo Park, Sue Moon Computer Science Dept., KAIST Microsoft Research Cambridge, UK * Electrical Engineering

More information

TLDK Overview. Transport Layer Development Kit Ray Kinsella February ray.kinsella [at] intel.com IRC: mortderire

TLDK Overview. Transport Layer Development Kit Ray Kinsella February ray.kinsella [at] intel.com IRC: mortderire TLDK Overview Transport Layer Development Kit Ray Kinsella February 2017 Email : ray.kinsella [at] intel.com IRC: mortderire Contributions from Keith Wiles & Konstantin Ananyev Legal Disclaimer General

More information

The.pdf version of this slide deck will have missing info, due to use of animations. The original.pptx deck is available here:

The.pdf version of this slide deck will have missing info, due to use of animations. The original.pptx deck is available here: The.pdf version of this slide deck will have missing info, due to use of animations. The original.pptx deck is available here: https://wiki.opnfv.org/download/attachments/10293193/vsperf-dataplane-perf-cap-bench.pptx?api=v2

More information

TLDK Overview. Transport Layer Development Kit Keith Wiles April Contributions from Ray Kinsella & Konstantin Ananyev

TLDK Overview. Transport Layer Development Kit Keith Wiles April Contributions from Ray Kinsella & Konstantin Ananyev TLDK Overview Transport Layer Development Kit Keith Wiles April 2017 Contributions from Ray Kinsella & Konstantin Ananyev Notices and Disclaimers Intel technologies features and benefits depend on system

More information

Disclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme

Disclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme NET1343BU NSX Performance Samuel Kommu #VMworld #NET1343BU Disclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no

More information

NFV performance benchmarking with OVS and Linux containers

NFV performance benchmarking with OVS and Linux containers Computer Science Tobias Rang NFV performance benchmarking with OVS and Linux containers Bachelor s Project NFV performance benchmarking with OVS and Linux containers 2017 The author(s) and Karlstad University

More information

A Look at Intel s Dataplane Development Kit

A Look at Intel s Dataplane Development Kit A Look at Intel s Dataplane Development Kit Dominik Scholz Chair for Network Architectures and Services Department for Computer Science Technische Universität München June 13, 2014 Dominik Scholz: A Look

More information

OpenFlow Software Switch & Intel DPDK. performance analysis

OpenFlow Software Switch & Intel DPDK. performance analysis OpenFlow Software Switch & Intel DPDK performance analysis Agenda Background Intel DPDK OpenFlow 1.3 implementation sketch Prototype design and setup Results Future work, optimization ideas OF 1.3 prototype

More information

New Approach to OVS Datapath Performance. Founder of CloudNetEngine Jun Xiao

New Approach to OVS Datapath Performance. Founder of CloudNetEngine Jun Xiao New Approach to OVS Datapath Performance Founder of CloudNetEngine Jun Xiao Agenda VM virtual network datapath evolvement Technical deep dive on a new OVS datapath Performance comparisons Q & A 2 VM virtual

More information

Measuring a 25 Gb/s and 40 Gb/s data plane

Measuring a 25 Gb/s and 40 Gb/s data plane Measuring a 25 Gb/s and 40 Gb/s data plane Christo Kleu Pervaze Akhtar 1 Contents Preliminaries Equipment Traffic generators Test topologies Host and VM configuration NUMA Architecture CPU allocation BIOS

More information

DeTail Reducing the Tail of Flow Completion Times in Datacenter Networks. David Zats, Tathagata Das, Prashanth Mohan, Dhruba Borthakur, Randy Katz

DeTail Reducing the Tail of Flow Completion Times in Datacenter Networks. David Zats, Tathagata Das, Prashanth Mohan, Dhruba Borthakur, Randy Katz DeTail Reducing the Tail of Flow Completion Times in Datacenter Networks David Zats, Tathagata Das, Prashanth Mohan, Dhruba Borthakur, Randy Katz 1 A Typical Facebook Page Modern pages have many components

More information

Distributed Systems Exam 1 Review Paul Krzyzanowski. Rutgers University. Fall 2016

Distributed Systems Exam 1 Review Paul Krzyzanowski. Rutgers University. Fall 2016 Distributed Systems 2015 Exam 1 Review Paul Krzyzanowski Rutgers University Fall 2016 1 Question 1 Why did the use of reference counting for remote objects prove to be impractical? Explain. It s not fault

More information

Rapid prototyping of DPDK applications with libmoon

Rapid prototyping of DPDK applications with libmoon Rapid prototyping of DPDK applications with libmoon Paul Emmerich emmericp@net.in.tum.de Technical University of Munich Chair of Network Architectures and Services DPDK Summit, 27.9.2017 About me PhD student

More information

DPDK Tunneling Offload RONY EFRAIM & YONGSEOK KOH MELLANOX

DPDK Tunneling Offload RONY EFRAIM & YONGSEOK KOH MELLANOX x DPDK Tunneling Offload RONY EFRAIM & YONGSEOK KOH MELLANOX Rony Efraim Introduction to DC w/ overlay network Modern data center (DC) use overly network like Virtual Extensible LAN (VXLAN) and GENEVE

More information

DESIGN AND IMPLEMENTATION OF AN AVIONICS FULL DUPLEX ETHERNET (A664) DATA ACQUISITION SYSTEM

DESIGN AND IMPLEMENTATION OF AN AVIONICS FULL DUPLEX ETHERNET (A664) DATA ACQUISITION SYSTEM DESIGN AND IMPLEMENTATION OF AN AVIONICS FULL DUPLEX ETHERNET (A664) DATA ACQUISITION SYSTEM Alberto Perez, Technical Manager, Test & Integration John Hildin, Director of Network s John Roach, Vice President

More information

Precision Time Protocol, and Sub-Microsecond Synchronization

Precision Time Protocol, and Sub-Microsecond Synchronization Linux Foundation End User Summit May 1, 2012 Precision Time Protocol, and Sub-Microsecond Synchronization Mike Kravetz IBM Linux Technology Center kravetz@us.ibm.com 2009 IBM Corporation Agenda Background/History

More information

Be Fast, Cheap and in Control with SwitchKV. Xiaozhou Li

Be Fast, Cheap and in Control with SwitchKV. Xiaozhou Li Be Fast, Cheap and in Control with SwitchKV Xiaozhou Li Goal: fast and cost-efficient key-value store Store, retrieve, manage key-value objects Get(key)/Put(key,value)/Delete(key) Target: cluster-level

More information

Implementation of Software-based EPON-OLT and Performance Evaluation

Implementation of Software-based EPON-OLT and Performance Evaluation This article has been accepted and published on J-STAGE in advance of copyediting. Content is final as presented. IEICE Communications Express, Vol.1, 1 6 Implementation of Software-based EPON-OLT and

More information

How to Measure the Killer Microsecond

How to Measure the Killer Microsecond How to Measure the Killer Microsecond Mia Primorac Edouard Bugnion Katerina Argyraki EPFL first.last@epfl.ch ABSTRACT Datacenter-networking research requires tools to both generate traffic and accurately

More information

vswitch Acceleration with Hardware Offloading CHEN ZHIHUI JUNE 2018

vswitch Acceleration with Hardware Offloading CHEN ZHIHUI JUNE 2018 x vswitch Acceleration with Hardware Offloading CHEN ZHIHUI JUNE 2018 Current Network Solution for Virtualization Control Plane Control Plane virtio virtio user space PF VF2 user space TAP1 SW Datapath

More information

A Userspace Packet Switch for Virtual Machines

A Userspace Packet Switch for Virtual Machines SHRINKING THE HYPERVISOR ONE SUBSYSTEM AT A TIME A Userspace Packet Switch for Virtual Machines Julian Stecklina OS Group, TU Dresden jsteckli@os.inf.tu-dresden.de VEE 2014, Salt Lake City 1 Motivation

More information

An Implementation of the Homa Transport Protocol in RAMCloud. Yilong Li, Behnam Montazeri, John Ousterhout

An Implementation of the Homa Transport Protocol in RAMCloud. Yilong Li, Behnam Montazeri, John Ousterhout An Implementation of the Homa Transport Protocol in RAMCloud Yilong Li, Behnam Montazeri, John Ousterhout Introduction Homa: receiver-driven low-latency transport protocol using network priorities HomaTransport

More information

Session based high bandwidth throughput testing

Session based high bandwidth throughput testing Universiteit van Amsterdam System and Network Engineering Research Project 2 Session based high bandwidth throughput testing Bram ter Borch bram.terborch@os3.nl 29 August 2017 Abstract To maximize and

More information

IX: A Protected Dataplane Operating System for High Throughput and Low Latency

IX: A Protected Dataplane Operating System for High Throughput and Low Latency IX: A Protected Dataplane Operating System for High Throughput and Low Latency Adam Belay et al. Proc. of the 11th USENIX Symp. on OSDI, pp. 49-65, 2014. Presented by Han Zhang & Zaina Hamid Challenges

More information

Technische Universität München. Comparison of Network Interface Controllers for Software Packet Processing

Technische Universität München. Comparison of Network Interface Controllers for Software Packet Processing Technische Universität München Department of Informatics Bachelor s Thesis in Informatics Comparison of Network Interface Controllers for Software Packet Processing Alexander P. Frank Technische Universität

More information

Got Loss? Get zovn! Daniel Crisan, Robert Birke, Gilles Cressier, Cyriel Minkenberg, and Mitch Gusat. ACM SIGCOMM 2013, August, Hong Kong, China

Got Loss? Get zovn! Daniel Crisan, Robert Birke, Gilles Cressier, Cyriel Minkenberg, and Mitch Gusat. ACM SIGCOMM 2013, August, Hong Kong, China Got Loss? Get zovn! Daniel Crisan, Robert Birke, Gilles Cressier, Cyriel Minkenberg, and Mitch Gusat ACM SIGCOMM 2013, 12-16 August, Hong Kong, China Virtualized Server 1 Application Performance in Virtualized

More information

Operating System: Chap13 I/O Systems. National Tsing-Hua University 2016, Fall Semester

Operating System: Chap13 I/O Systems. National Tsing-Hua University 2016, Fall Semester Operating System: Chap13 I/O Systems National Tsing-Hua University 2016, Fall Semester Outline Overview I/O Hardware I/O Methods Kernel I/O Subsystem Performance Application Interface Operating System

More information

Using Time Division Multiplexing to support Real-time Networking on Ethernet

Using Time Division Multiplexing to support Real-time Networking on Ethernet Using Time Division Multiplexing to support Real-time Networking on Ethernet Hariprasad Sampathkumar 25 th January 2005 Master s Thesis Defense Committee Dr. Douglas Niehaus, Chair Dr. Jeremiah James,

More information

Chair for Network Architectures and Services Prof. Carle Department of Computer Science Technische Universität München.

Chair for Network Architectures and Services Prof. Carle Department of Computer Science Technische Universität München. Chair for Network Architectures and Services Prof. Carle Department of Computer Science Technische Universität München Network Analysis 2b) Deterministic Modelling beyond Formal Logic A simple network

More information

Next Gen Virtual Switch. CloudNetEngine Founder & CTO Jun Xiao

Next Gen Virtual Switch. CloudNetEngine Founder & CTO Jun Xiao Next Gen Virtual Switch CloudNetEngine Founder & CTO Jun Xiao Agenda Thoughts on next generation virtual switch Technical deep dive on CloudNetEngine virtual switch Q & A 2 Major vswitches categorized

More information

Improve Performance of Kube-proxy and GTP-U using VPP

Improve Performance of Kube-proxy and GTP-U using VPP Improve Performance of Kube-proxy and GTP-U using VPP Hongjun Ni (hongjun.ni@intel.com) Danny Zhou (danny.zhou@intel.com) Johnson Li (johnson.li@intel.com) Network Platform Group, DCG, Intel Acknowledgement:

More information

Agilio CX 2x40GbE with OVS-TC

Agilio CX 2x40GbE with OVS-TC PERFORMANCE REPORT Agilio CX 2x4GbE with OVS-TC OVS-TC WITH AN AGILIO CX SMARTNIC CAN IMPROVE A SIMPLE L2 FORWARDING USE CASE AT LEAST 2X. WHEN SCALED TO REAL LIFE USE CASES WITH COMPLEX RULES TUNNELING

More information

Introduction to Ethernet Latency

Introduction to Ethernet Latency Introduction to Ethernet Latency An Explanation of Latency and Latency Measurement The primary difference in the various methods of latency measurement is the point in the software stack at which the latency

More information

CS 856 Latency in Communication Systems

CS 856 Latency in Communication Systems CS 856 Latency in Communication Systems Winter 2010 Latency Challenges CS 856, Winter 2010, Latency Challenges 1 Overview Sources of Latency low-level mechanisms services Application Requirements Latency

More information

IEEE 1588 PTP clock synchronization over a WAN backbone

IEEE 1588 PTP clock synchronization over a WAN backbone Whitepaper IEEE 1588 PTP clock synchronization over a WAN backbone A field study comparing PTP clock synchronization accuracy against GPS external time reference in a live production WAN environment Contents

More information

100 Gbps Open-Source Software Router? It's Here. Jim Thompson, CTO, Netgate

100 Gbps Open-Source Software Router? It's Here. Jim Thompson, CTO, Netgate 100 Gbps Open-Source Software Router? It's Here. Jim Thompson, CTO, Netgate @gonzopancho Agenda Edge Router Use Cases Need for Speed Cost, Flexibility, Control, Evolution The Engineering Challenge Solution

More information

IX: A Protected Dataplane Operating System for High Throughput and Low Latency

IX: A Protected Dataplane Operating System for High Throughput and Low Latency IX: A Protected Dataplane Operating System for High Throughput and Low Latency Belay, A. et al. Proc. of the 11th USENIX Symp. on OSDI, pp. 49-65, 2014. Reviewed by Chun-Yu and Xinghao Li Summary In this

More information

Software Routers: NetMap

Software Routers: NetMap Software Routers: NetMap Hakim Weatherspoon Assistant Professor, Dept of Computer Science CS 5413: High Performance Systems and Networking October 8, 2014 Slides from the NetMap: A Novel Framework for

More information

PowerPC on NetFPGA CSE 237B. Erik Rubow

PowerPC on NetFPGA CSE 237B. Erik Rubow PowerPC on NetFPGA CSE 237B Erik Rubow NetFPGA PCI card + FPGA + 4 GbE ports FPGA (Virtex II Pro) has 2 PowerPC hard cores Untapped resource within NetFPGA community Goals Evaluate performance of on chip

More information

Master Course Computer Networks IN2097

Master Course Computer Networks IN2097 Chair for Network Architectures and Services Prof. Carle Department for Computer Science TU München Master Course Computer Networks IN2097 Chapter 7 - Network Measurements Introduction Architecture & Mechanisms

More information

Improve VNF safety with Vhost-User/DPDK IOMMU support

Improve VNF safety with Vhost-User/DPDK IOMMU support Improve VNF safety with Vhost-User/DPDK IOMMU support No UIO anymore! Maxime Coquelin Software Engineer KVM Forum 2017 AGENDA Background Vhost-user device IOTLB implementation Benchmarks Future improvements

More information

Handout 4 Memory Hierarchy

Handout 4 Memory Hierarchy Handout 4 Memory Hierarchy Outline Memory hierarchy Locality Cache design Virtual address spaces Page table layout TLB design options (MMU Sub-system) Conclusion 2012/11/7 2 Since 1980, CPU has outpaced

More information

Enhanced Ethernet Switching Technology. Time Applications. Rui Santos 17 / 04 / 2009

Enhanced Ethernet Switching Technology. Time Applications. Rui Santos 17 / 04 / 2009 Enhanced Ethernet Switching Technology for Adaptive Hard Real- Time Applications Rui Santos (rsantos@ua.pt) 17 / 04 / 2009 Problem 2 Switched Ethernet became common in real-time communications Some interesting

More information

QuickSpecs. HP Z 10GbE Dual Port Module. Models

QuickSpecs. HP Z 10GbE Dual Port Module. Models Overview Models Part Number: 1Ql49AA Introduction The is a 10GBASE-T adapter utilizing the Intel X722 MAC and X557-AT2 PHY pairing to deliver full line-rate performance, utilizing CAT 6A UTP cabling (or

More information

Distributed Systems. 05. Clock Synchronization. Paul Krzyzanowski. Rutgers University. Fall 2017

Distributed Systems. 05. Clock Synchronization. Paul Krzyzanowski. Rutgers University. Fall 2017 Distributed Systems 05. Clock Synchronization Paul Krzyzanowski Rutgers University Fall 2017 2014-2017 Paul Krzyzanowski 1 Synchronization Synchronization covers interactions among distributed processes

More information

Accelerating Contrail vrouter

Accelerating Contrail vrouter WHITE PAPER Accelerating Contrail vrouter WHEN DEPLOYED WITH THE JUNIPER NETWORKS CONTRAIL CLOUD NETWORKING PLATFORM, THE NETRONOME AGILIO VROUTER SOLUTION DELIVERS ACCELERATED PERFORMANCE THAT ENABLES

More information

Interconnection Structures. Patrick Happ Raul Queiroz Feitosa

Interconnection Structures. Patrick Happ Raul Queiroz Feitosa Interconnection Structures Patrick Happ Raul Queiroz Feitosa Objective To present key issues that affect interconnection design. Interconnection Structures 2 Outline Introduction Computer Busses Bus Types

More information

Master Course Computer Networks IN2097

Master Course Computer Networks IN2097 Chair for Network Architectures and Services Prof. Carle Department for Computer Science TU München Master Course Computer Networks IN2097 Prof. Dr.-Ing. Georg Carle Christian Grothoff, Ph.D. Dr. Nils

More information

ELEC / COMP 177 Fall Some slides from Kurose and Ross, Computer Networking, 5 th Edition

ELEC / COMP 177 Fall Some slides from Kurose and Ross, Computer Networking, 5 th Edition ELEC / COMP 177 Fall 2011 Some slides from Kurose and Ross, Computer Networking, 5 th Edition Project #2 Due Thursday, Nov 10 th By midnight Homework #5 Due Thursday, Nov 17 th Later this semester: Homework

More information

Xilinx Answer QDMA Performance Report

Xilinx Answer QDMA Performance Report Xilinx Answer 71453 QDMA Performance Report Important Note: This downloadable PDF of an Answer Record is provided to enhance its usability and readability. It is important to note that Answer Records are

More information

Globally Synchronized time via Datacenter Networks

Globally Synchronized time via Datacenter Networks 1 Globally Synchronized time via Datacenter Networks Ki Suh Lee Cornell University Joint work with Han Wang, Vishal Shrivastav and Hakim Weatherspoon 2 Synchronized Clocks Fundamental for network and distributed

More information

DPDK Roadmap. Tim O Driscoll & Chris Wright Open Networking Summit 2017

DPDK Roadmap. Tim O Driscoll & Chris Wright Open Networking Summit 2017 DPDK Roadmap Tim O Driscoll & Chris Wright Open Networking Summit 2017 Agenda Overview: What is DPDK? What problems does it solve? Open source community and transition to Linux Foundation: Why is this

More information

QCN: Quantized Congestion Notification. Rong Pan, Balaji Prabhakar, Ashvin Laxmikantha

QCN: Quantized Congestion Notification. Rong Pan, Balaji Prabhakar, Ashvin Laxmikantha QCN: Quantized Congestion Notification Rong Pan, Balaji Prabhakar, Ashvin Laxmikantha Overview Description the QCN scheme Pseudocode available with Rong Pan: ropan@cisco.com Basic simulations Infinitely

More information

PE310G4TSF4I71 Quad Port SFP+ 10 Gigabit Ethernet PCI Express Time Stamp Server Adapter Intel Based

PE310G4TSF4I71 Quad Port SFP+ 10 Gigabit Ethernet PCI Express Time Stamp Server Adapter Intel Based PE310G4TSF4I71 Quad Port SFP+ 10 Gigabit Ethernet PCI Express Time Stamp Server Adapter Intel Based Product Description Silicom s 40 Gigabit Ethernet PCI Express Time Stamping server adapter is designed

More information

Optimizing Performance: Intel Network Adapters User Guide

Optimizing Performance: Intel Network Adapters User Guide Optimizing Performance: Intel Network Adapters User Guide Network Optimization Types When optimizing network adapter parameters (NIC), the user typically considers one of the following three conditions

More information

Low-Latency Datacenters. John Ousterhout Platform Lab Retreat May 29, 2015

Low-Latency Datacenters. John Ousterhout Platform Lab Retreat May 29, 2015 Low-Latency Datacenters John Ousterhout Platform Lab Retreat May 29, 2015 Datacenters: Scale and Latency Scale: 1M+ cores 1-10 PB memory 200 PB disk storage Latency: < 0.5 µs speed-of-light delay Most

More information

Systems. Roland Kammerer. 10. November Institute of Computer Engineering Vienna University of Technology. Communication Protocols for Embedded

Systems. Roland Kammerer. 10. November Institute of Computer Engineering Vienna University of Technology. Communication Protocols for Embedded Communication Roland Institute of Computer Engineering Vienna University of Technology 10. November 2010 Overview 1. Definition of a protocol 2. Protocol properties 3. Basic Principles 4. system communication

More information

Lecture 22: Buffering & Scheduling. CSE 123: Computer Networks Alex C. Snoeren

Lecture 22: Buffering & Scheduling. CSE 123: Computer Networks Alex C. Snoeren Lecture 22: Buffering & Scheduling CSE 123: Computer Networks Alex C. Snoeren Lecture 23 Overview Buffer Management FIFO RED Traffic Policing/Scheduling 2 Key Router Challenges Buffer management: which

More information

Speeding up Linux TCP/IP with a Fast Packet I/O Framework

Speeding up Linux TCP/IP with a Fast Packet I/O Framework Speeding up Linux TCP/IP with a Fast Packet I/O Framework Michio Honda Advanced Technology Group, NetApp michio@netapp.com With acknowledge to Kenichi Yasukata, Douglas Santry and Lars Eggert 1 Motivation

More information

A comparative analysis of Precision Time Protocol in native, virtual machines and container-based environments for consolidating automotive workloads

A comparative analysis of Precision Time Protocol in native, virtual machines and container-based environments for consolidating automotive workloads A comparative analysis of Precision Time Protocol in native, virtual machines and container-based environments for consolidating automotive workloads Speaker: Co-authors: Ong Boon Leong boon.leong.ong@intel.com

More information

Accelerating vrouter Contrail

Accelerating vrouter Contrail WHITE PAPER Accelerating vrouter Contrail A VIRTUAL ROUTER (VROUTER) IS A SOFTWARE ONLY IMPLEMENTATION OF LAYER 3 INTERNET PROTOCOL (IP) ROUTING. ROUTING FUNCTIONS THAT ARE TRADITIONALLY DELIVERED AS DEDICATED

More information

Operating Systems. Introduction & Overview. Outline for today s lecture. Administrivia. ITS 225: Operating Systems. Lecture 1

Operating Systems. Introduction & Overview. Outline for today s lecture. Administrivia. ITS 225: Operating Systems. Lecture 1 ITS 225: Operating Systems Operating Systems Lecture 1 Introduction & Overview Jan 15, 2004 Dr. Matthew Dailey Information Technology Program Sirindhorn International Institute of Technology Thammasat

More information

VALE: a switched ethernet for virtual machines

VALE: a switched ethernet for virtual machines L < > T H local VALE VALE -- Page 1/23 VALE: a switched ethernet for virtual machines Luigi Rizzo, Giuseppe Lettieri Università di Pisa http://info.iet.unipi.it/~luigi/vale/ Motivation Make sw packet processing

More information

PVPP: A Programmable Vector Packet Processor. Sean Choi, Xiang Long, Muhammad Shahbaz, Skip Booth, Andy Keep, John Marshall, Changhoon Kim

PVPP: A Programmable Vector Packet Processor. Sean Choi, Xiang Long, Muhammad Shahbaz, Skip Booth, Andy Keep, John Marshall, Changhoon Kim PVPP: A Programmable Vector Packet Processor Sean Choi, Xiang Long, Muhammad Shahbaz, Skip Booth, Andy Keep, John Marshall, Changhoon Kim Fixed Set of Protocols Fixed-Function Switch Chip TCP IPv4 IPv6

More information

Short answer (35 points)

Short answer (35 points) CPSC 360 Fall 2017 Exam 1 Version 2 Solutions (last updated 10/19/2017) This exam is closed book, closed notes, closed laptops. You are allowed to have one 8.5x11 sheet of paper with whatever you like

More information

Supporting Fine-Grained Network Functions through Intel DPDK

Supporting Fine-Grained Network Functions through Intel DPDK Supporting Fine-Grained Network Functions through Intel DPDK Ivano Cerrato, Mauro Annarumma, Fulvio Risso - Politecnico di Torino, Italy EWSDN 2014, September 1st 2014 This project is co-funded by the

More information

OPERATING SYSTEMS CS136

OPERATING SYSTEMS CS136 OPERATING SYSTEMS CS136 Jialiang LU Jialiang.lu@sjtu.edu.cn Based on Lecture Notes of Tanenbaum, Modern Operating Systems 3 e, 1 Chapter 5 INPUT/OUTPUT 2 Overview o OS controls I/O devices => o Issue commands,

More information

WHITE PAPER. Latency & Jitter WHITE PAPER OVERVIEW

WHITE PAPER. Latency & Jitter WHITE PAPER OVERVIEW Latency & Jitter In Networking Performance Evaluation OVERVIEW Latency and jitter are two key measurement parameters when evaluating and benchmarking the performance of a network, system or device. Different

More information

Be Fast, Cheap and in Control with SwitchKV Xiaozhou Li

Be Fast, Cheap and in Control with SwitchKV Xiaozhou Li Be Fast, Cheap and in Control with SwitchKV Xiaozhou Li Raghav Sethi Michael Kaminsky David G. Andersen Michael J. Freedman Goal: fast and cost-effective key-value store Target: cluster-level storage for

More information

The Link Layer II: Ethernet

The Link Layer II: Ethernet Monday Recap The Link Layer II: Ethernet q Link layer services q Principles for multiple access protocols q Categories of multiple access protocols CSC 249 March 24, 2017 1 2 Recap: Random Access Protocols

More information

DPDK Summit 2016 OpenContrail vrouter / DPDK Architecture. Raja Sivaramakrishnan, Distinguished Engineer Aniket Daptari, Sr.

DPDK Summit 2016 OpenContrail vrouter / DPDK Architecture. Raja Sivaramakrishnan, Distinguished Engineer Aniket Daptari, Sr. DPDK Summit 2016 OpenContrail vrouter / DPDK Architecture Raja Sivaramakrishnan, Distinguished Engineer Aniket Daptari, Sr. Product Manager CONTRAIL (MULTI-VENDOR) ARCHITECTURE ORCHESTRATOR Interoperates

More information

PacketShader: A GPU-Accelerated Software Router

PacketShader: A GPU-Accelerated Software Router PacketShader: A GPU-Accelerated Software Router Sangjin Han In collaboration with: Keon Jang, KyoungSoo Park, Sue Moon Advanced Networking Lab, CS, KAIST Networked and Distributed Computing Systems Lab,

More information

ADVA FSP 150 ProVMe. Performance and Functionality Test Report. Introduction. VNF Lifecycle Management

ADVA FSP 150 ProVMe. Performance and Functionality Test Report. Introduction. VNF Lifecycle Management ADVA FSP 150 ProVMe Performance and Functionality Test Report Introduction EANTC was commissioned by Intel under the Intel Network Builders program to perform independent tests of the ADVA FSP 150 ProVMe

More information

Backend for Software Data Planes

Backend for Software Data Planes The Case for a Flexible Low-Level Backend for Software Data Planes Sean Choi 1, Xiang Long 2, Muhammad Shahbaz 3, Skip Booth 4, Andy Keep 4, John Marshall 4, Changhoon Kim 5 1 2 3 4 5 Why software data

More information

Benchmarking of VPP. Arijit Pramanik RnD Project

Benchmarking of VPP. Arijit Pramanik RnD Project Benchmarking of VPP Arijit Pramanik RnD Project Abstract Vector Packet Processing Technology, currently released under FD.io is a framework for high-speed packet processing in user-space. VPP is capable

More information

Performance! (1/latency)! 1000! 100! 10! Capacity Access Time Cost. CPU Registers 100s Bytes <10s ns. Cache K Bytes ns 1-0.

Performance! (1/latency)! 1000! 100! 10! Capacity Access Time Cost. CPU Registers 100s Bytes <10s ns. Cache K Bytes ns 1-0. Since 1980, CPU has outpaced DRAM... EEL 5764: Graduate Computer Architecture Appendix C Hierarchy Review Ann Gordon-Ross Electrical and Computer Engineering University of Florida http://www.ann.ece.ufl.edu/

More information

Technology for Adaptive Hard. Rui Santos, UA

Technology for Adaptive Hard. Rui Santos, UA HaRTES Meeting Enhanced Ethernet Switching Technology for Adaptive Hard Real-Time Applications Rui Santos, rsantos@ua.pt, UA SUMMARY 2 MOTIVATION Switched Ethernet t became common in real-time communications

More information

CSCI-1680 Link Layer Wrap-Up Rodrigo Fonseca

CSCI-1680 Link Layer Wrap-Up Rodrigo Fonseca CSCI-1680 Link Layer Wrap-Up Rodrigo Fonseca Based partly on lecture notes by David Mazières, Phil Levis, John Jannotti Today: Link Layer (cont.) Framing Reliability Error correction Sliding window Medium

More information

Achieve Low Latency NFV with Openstack*

Achieve Low Latency NFV with Openstack* Achieve Low Latency NFV with Openstack* Yunhong Jiang Yunhong.Jiang@intel.com *Other names and brands may be claimed as the property of others. Agenda NFV and network latency Why network latency on NFV

More information

Getting Real Performance from a Virtualized CCAP

Getting Real Performance from a Virtualized CCAP Getting Real Performance from a Virtualized CCAP A Technical Paper prepared for SCTE/ISBE by Mark Szczesniak Software Architect Casa Systems, Inc. 100 Old River Road Andover, MA, 01810 978-688-6706 mark.szczesniak@casa-systems.com

More information

Design and Implementation of Virtual TAP for Software-Defined Networks

Design and Implementation of Virtual TAP for Software-Defined Networks Design and Implementation of Virtual TAP for Software-Defined Networks - Master Thesis Defense - Seyeon Jeong Supervisor: Prof. James Won-Ki Hong Dept. of CSE, DPNM Lab., POSTECH, Korea jsy0906@postech.ac.kr

More information

Module 6: INPUT - OUTPUT (I/O)

Module 6: INPUT - OUTPUT (I/O) Module 6: INPUT - OUTPUT (I/O) Introduction Computers communicate with the outside world via I/O devices Input devices supply computers with data to operate on E.g: Keyboard, Mouse, Voice recognition hardware,

More information

Lighting the Blue Touchpaper for UK e-science - Closing Conference of ESLEA Project The George Hotel, Edinburgh, UK March, 2007

Lighting the Blue Touchpaper for UK e-science - Closing Conference of ESLEA Project The George Hotel, Edinburgh, UK March, 2007 Working with 1 Gigabit Ethernet 1, The School of Physics and Astronomy, The University of Manchester, Manchester, M13 9PL UK E-mail: R.Hughes-Jones@manchester.ac.uk Stephen Kershaw The School of Physics

More information

Event Device Drivers. Release rc1

Event Device Drivers. Release rc1 Event Device Drivers Release 19.02.0-rc1 December 23, 2018 CONTENTS 1 NXP DPAA Eventdev Driver 2 1.1 Features........................................ 2 1.2 Supported DPAA SoCs................................

More information

Elastic Scaling of Stateful Network Functions

Elastic Scaling of Stateful Network Functions NSDI 2018 Elastic Scaling of Stateful Network Functions Shinae Woo *+, Justine Sherry *, Sangjin Han *, Sue Moon +, Sylvia Ratnasamy *, Scott Shenker * + KAIST, * UC Berkeley Elastic Scaling of NFs NFV

More information

Implementing Software Virtual Routers on Multi-core PCs using Click

Implementing Software Virtual Routers on Multi-core PCs using Click Implementing Software Virtual Routers on Multi-core PCs using Click Mickaël Hoerdt, Dept. of computer engineering Université catholique de Louvain la neuve mickael.hoerdt@uclouvain.be LANCASTER UNIVERSITY

More information

SmartNIC Programming Models

SmartNIC Programming Models SmartNIC Programming Models Johann Tönsing 206--09 206 Open-NFP Agenda SmartNIC hardware Pre-programmed vs. custom (C and/or P4) firmware Programming models / offload models Switching on NIC, with SR-IOV

More information

Today. Last Time. Motivation. CAN Bus. More about CAN. What is CAN?

Today. Last Time. Motivation. CAN Bus. More about CAN. What is CAN? Embedded networks Characteristics Requirements Simple embedded LANs Bit banged SPI I2C LIN Ethernet Last Time CAN Bus Intro Low-level stuff Frame types Arbitration Filtering Higher-level protocols Today

More information

Forwarding Architecture

Forwarding Architecture Forwarding Architecture Brighten Godfrey CS 538 February 14 2018 slides 2010-2018 by Brighten Godfrey unless otherwise noted Building a fast router Partridge: 50 Gb/sec router A fast IP router well, fast

More information

Data Path acceleration techniques in a NFV world

Data Path acceleration techniques in a NFV world Data Path acceleration techniques in a NFV world Mohanraj Venkatachalam, Purnendu Ghosh Abstract NFV is a revolutionary approach offering greater flexibility and scalability in the deployment of virtual

More information