A NetFlow/IPFIX implementation with OpenFlow José Suárez-Varela jsuarezv@ac.upc.edu Pere Barlet-Ros pbarlet@ac.upc.edu Valentín Carela-Español vcarela@talaia.io
Talaia is Catalan for Watchtower The Watchtower for your network Network Visibility Solution that collects and enriches network traffic metadata (flow-based) Worldwide Customer Base Decades of Networking Expertise We have clients from Australia to Silicon Valley, in 10 countries and 3 continents. Our customers range from terabit-per-second scale ISPs and tier-1 cloud providers to financial services companies. The company is builds upon decades of research in Network Visibility done primarily at UPC-BarcelonaTech. We continue to heavily invest in R&D, and focus on continuous improvement to bring innovations to market.
Network Operations Operational Insights Identify unwanted applications, congestion, top talkers, protocol usage, etc Drill Down Quickly cycle through visualizations to drill down into points of interest Agile Analysis A next-gen, snappy interface that offers results quickly, even at terabit-per-second scale networks
Network Security Attack Detection Artificial Intelligence automatically signals attacks or other anomalous behaviour Network Forensic Flow storage and easy retrieval, in-app analysis and exporting Regulation Compliance Pain-free compliance with Government regulation regarding Metadata Retention
Strategic Network Planning Top Talkers Quickly find and identify network hogs, bandwidth abuse, or network-intensive apps Capacity Planning Understand how the network is utilized to plan network upgrades with the right info Peering Decisions See what Autonomous Systems you exchange data with as input for peering decisions
Motivation SDN-Polygraph: Cloud-based Monitoring Service for Software Defined Networks H2020 SME Instrument Phase 2 (nº726763) Migration of Talaia s network visibility solution to the SDN paradigm To provide not only visibility features but also actuation features (e.g., attack mitigation, traffic shaping) SDN-based Features Non SDN-based Features Distributed and Automatized Infrastructure for Large Scale Service SDN Acquisition Traditional Acquisition
Motivation SDN Acquisition Current input for Talaia is NetFlow/IPFIX. Can we get this output from SDN-based devices? Not all of them (e.g., HP E3500, NEC IP8800) Find a NetFlow/IPFIX implementation for SDN-based devices Collaboration between UPC BarcelonaTech and Talaia Networks
Proposal Solution based on OpenFlow It is one of the most extended protocols in SDN Each entry has packets and bytes counters X Different versions of OpenFlow and different implementations X OpenFlow/SDN scalability limitations - Number of entries in switches is limited - The controller is a critical point and it is prone to become a bottleneck Flow sampling-based solutions depending on OpenFlow constraints
Flow sampling-based solution 2 Phases 1) Add sampling rules proactively Decoupling monitoring function from other network functions Flow table #0 Flow table #1 2) Add flow (5-tuple) monitoring rules reactively Sampling and monitoring module Linked flow tables (OpenFlow 1.1+) Other module
IP/Port-based Approaches Initial flow rules for sampling: (Proactively installed) Flow table #0 Flow table #1 IP-based 1 rule with a mask fixing the last m and n bits of the source and destination IPs Sampled flows Go to table 1 Sampling flow rule(s) Output to controller Other module Port-based m ans n rules with source or destination port matches where: Go to table 1 Default flow rule Go to table 1 Pair of ports
Hash-based Approach Output to controller Initial flow rules for sampling: (Proactively installed) 1) Create new group table and buckets Group table Round robin n buckets with only one that apply the action output to controller where: Flow table #0 Sampled flows Go to table 1 Bucket #0 Sample flow n buckets or 1 bucket with weight n = 1 / sampling rate Buckets with weights 2 buckets Sampling flow rule (IP packets) Default flow rule (non-ip packets) Go to group Go to table 1 Go to table 1 Flow table #1 Other module Drop - Bucket #0 (weight 1) Output to controller - Bucket #1 (weight = 1/sampling rate - 1) Drop 2) Create flow rule to link table #0 with group table and table #1 for IP packets
Flow-sampling based solutions Port-based vs IP-based sampling IP-based sampling needs to add less flow rules to sample the traffic IP-based -> 1 rule (with wildcards in the IP field) Port-based -> O(sampling rate) rules Hash-based sampling Round robin -> O(1/sampling rate) rules Buckets with weights -> 2 rules It is required that hardware implements an appropriate load balancing algorithm
Results Evaluation scenario CAIDA Dataset: 10 Gbps link in Chicago Feb 16 2,353,413 flows of TCP and UDP traffic An OpenDayLight controller, an Open vswitch, and a VM injecting traffic Accuracy by the sampling rate applied
Summary OpenFlow-based solution to provide NetFlow/IPFIX-like output IP/Port-based solutions Hash-based solution Suitable for current off-the-shelf OpenFlow devices Only needs OpenFlow v1.1 Transparent -> Monitoring system decoupled from others Scalable -> Flow sampling
A NetFlow/IPFIX implementation with OpenFlow José Suárez-Varela jsuarezv@ac.upc.edu Pere Barlet-Ros pbarlet@ac.upc.edu Valentín Carela-Español vcarela@talaia.io