Flow-based Traffic Visibility Operations, Performance, Security Pavel Minařík, Chief Technology Officer
What is Flow Data? Modern method for network monitoring flow measurement Cisco standard NetFlow v5/v9, IETF standard IPFIX Focused on L3/L4 information and volumetric parameters Real network traffic to flow statistics reduction ratio 500:1 Flow data
Flow-Enabled Devices Network equipment (routers/switches) Traditional capability known for many years Firewalls, UTMs, load balancers, hypervisors Ongoing initiative of majority of vendors Packet brokers and matrix switches Convenient option
Flow vs. Packet Analysis Flow data Packet analysis Strong aspects Works in high-speed networks Resistant to encrypted traffic Visibility and reporting Network behavior analysis Full network traffic Enough details for troubleshooting Supports forensic analysis Signature based detection Weak aspects No application layer data Sometimes not enough details Sampling (routers, switches) Useless for encrypted traffic Usually too much details Very resource consuming Solution? Take advantage of strong aspects in one solution Versatile and flexible Probes for visibility into all network layers Flowmon long-term strategy
Probes (by Flowmon Networks) Versatile and flexible network appliances Monitoring ports convert packets to flows Un-sampled export in NetFlow v5/v9 or IPFIX Wire-speed, L2-L7 visibility, PCAPs when needed L2 MAC VLAN MPLS GRE tunnel OVT L3/L4 Standard items NPM metrics RTT, SRT, TTL, SYN size, ASN Geolocation L7 NBAR2 HTTP DNS DHCP SMB/CIFS VoIP (SIP) Email
Flow Gathering Schemes Pros Probe on a SPAN port Probe on a TAP Flows from switch/router Accuracy Performance L2/L3/L4/L7 visibility Same as on a SPAN All packets captured Separates RX and TX Already available No additional HW Traffic on interfaces Cons Facts May reach capacity limit No interface number Fits most customers Limited SPANs number Additional HW 2 monitoring ports Usually inaccurate Visibility L3/L4 Performance impact Always test before use Use Enterprise networks ISP uplinks, DCs Branch offices (MPLS, )
Is Flow Data Really That Helpful? Myth 1: Flow is sampled and highly inaccurate. This is true for sflow and NetFlow Lite For NetFlow/IPFIX this depends on flow source Probes and new network equipment do just fine Myth 2: Flow is limited to L3/L4 visibility. This is the original design but today s flow data come with L2 and L7 extensions (usually using IPFIX) Myth 3: You need continuous packet capture. Flows with L7 visibility + on-demand or triggered packet capture is cost efficient option
Network Performance Monitoring & Diagnostics Provides visibility eyes into the network traffic Reduces mean-time to resolve, builds up efficiency Reduces downtimes and network operational costs Gartner: 80% of operational issues can be analyzed and solved by flow monitoring. Recommendation: Implement NetFlow/IPFIX to allow better measurement of user experience.
It s easy to start. You already have NetFlow sources with routers, switches, firewalls, etc. It s only a matter of collection and interpretation of this data with Flowmon Collector. This is all you need to fix 80% of network issues.
Dashboards are brilliant for reporting and optimization. Drill-down capability is what you need for problem solving. NetFlow from network equipment has limitations. Use Probes to drill deeper than ever to solve 95% of network problems.
Use Case Monitoring of Cloud Applications
Migration to the cloud, in its various forms, creates a fundamental shift in network traffic that traditional network performance monitoring tools fail to cover. I&O leaders must consider cloud-centric monitoring technologies to fill visibility gaps. Flow monitoring vendors that cater to hybrid IT environments include Flowmon Networks. Source: Network Performance Monitoring Tools Leave Gaps in Cloud Monitoring, Gartner Report G00301635, by Sanjit Ganguli, published 27 th May 2016
CLOUD APPs PERFORMANCE NPM metrics (RTT, SRT, Jitter) In time visualizations per application Get quick insight, understand deviations Time axis on the right side of traffic chart Selection of current view/application
Cloud CRM Performance
NPMD and Security Volumetric DDoS detection Anomaly detection Incident reporting
Neil MacDonald, VP Distinguished Analyst Gartner Security & Risk Management Summit, London 2015
Flow-Based Anomaly Detection Network as a sensor concept (and enforcer) blogs.cisco.com/enterprise/the-network-as-a-security-sensor-and-enforcer Bridges the gap left by signature-based security Key technology for incident response Designed for multi 10G environment Volumetric DDoS Network Behavior Analysis Statistical analysis Volumetric DDoS detection Advanced data analysis algorithms Detection of non-volumetric anomalies
Use Case: Enterprise Security NBAD: On-demand Triggered Packet Capture
Flowmon ADS Flowmon Anomaly Detection Principles Machine Learning Adaptive Baselining Heuristics Behavior Patterns Reputation Databases
Traffic overview, anomalies detected
Attacker is looking for potential victims And starts SSH attack That turns out to be successful
Few minutes after that breached device starts to communicate with botnet C&C
Data exfiltration (ICMP anomaly traffic with payload present)
PCAP available, what is the ICMP payload?
Linux /etc/passwd file with user accounts and hash of passwords
Network Against Threats Flow monitoring including L7 Network Behavior Analysis Full packet capture Triggered by detection
Use Case: DDoS Protection Volumetric DDoS Detection Traffic Redirection and Mitigation Control
Backbone DDoS protection Backbone protection is specific High number of up-links, network perimeter is wide Massive throughputs dozens or hundreds of Gbps In-line solution is out of question! flow export 1. Flow collection 2. DDoS detection 3. Routing control 4. Mitigation orchestration Detection based on flow analysis and out-of-path mitigation Simple and cost-efficient solution for backbones Prevents volumetric attacks to reach enterprise networks
Attack Detection For each segment, a set of baselines is learned from real traffic Attack is detected if the current traffic exceeds defined threshold Baseline is learned for: TCP traffic with specific flags UDP traffic ICMP traffic
Attack Reporting Start/end time Attack target Type and status Traffic volumes during attack/peace time Attack targets (top 10 dst IPs, source subnets, L4 protocols, TCP flags combinations )
Response to Attack Alerting E-mail, Syslog, SNMP trap Routing diversion PBR (Policy Based Routing) BGP (Border Gateway Protocol) BGP Flowspec RTBH (Remotely-Triggered Black Hole) User-defined scripting Automatic mitigation With on premise mitigation devices With cloud scrubbing
For internet service provides, data centers and large enterprises Third-party mitigation equipment deployed out of path Mitigation through infrastructure itself (BGP Flowspec) For enterprises On premise detection and mitigation through cloud scrubbing DDoS Mitigation Scenarios
Dynamic Protection Policy Deployment incl. Baselines and attack characteristics Anomaly Detection Mitigation Enforcement Traffic Diversion via BGP Route Injection Scrubbing center Flow Data Collection Learning Baselines Attack Attack path Clean path Protected Object 1 e.g. Data Center, Organization, Service etc Internet Service Provider Core Protected Object 2
Anomaly Detection Mitigation Enforcement Sending specific Route advertisement via BGP FlowSpec Dynamic signature: Dst IP: 1.1.1.1/32 Dst Port: 135 Protocol IP: 17 (UDP) Discard Flow Data Collection Learning Baselines Attack Protected Object 1 e.g. Data Center, Organization, Service etc Internet Service Provider Core Protected Object 2 Dropped traffic for Dst IP: 1.1.1.1/32 Dst Port: 135 Protocol IP: 17 (UDP)
Traffic Diversion via BGP Route Injection and Propagation of the Change Complex Traffic Scrubbing HTTP/UDP Attack Alerting and Incident Characteristics 2. Rerouted Attack Cloud Scrubbing Center 3. Cleaned Traffic Anomaly Detection & Mitigation Enforcement Flow Data Collection Learning Baselines GRE tunnel Flow Internet 1. Original Attack Customer network
A real customer case study ČD Telematika 2 nd Largest Czech Backbone Provider
ČD Telematika and its Core Infrastructure CORE CONNECTIVITY 4x10Gb/s NIX 3x10 Gb/s global transit 4x10Gb/s Google CONNECTED NETWORKS 180+ CONTRACTED TRAFFIC 90 Gb/s REAL PEAK TRAFFIC 50 Gb/s in 10Gb/s out
ČDT AntiDDoS Service Operated since July 2015 Attack detection and re-routing with Flowmon DDoS Defender 40 Scrubbing center for traffic cleaning is Radware DefensePro, 10Gbps of legitimate traffic + 12Gbps of attack Protection against volumetric attacks
ČDT-ANTIDDoS: Deployment scheme upstream upstream peering aggregation access upstream Protected Objects e.g. Data Center, Organization, Service etc REST API Access networks upstream upstream upstream
Sample attack handled From internet to the scrubbing center From the scrubbing center to the customer
About Flowmon Networks many tasks, single solution
Customer references is an international vendor devoted to innovative network traffic & performance & security monitoring 700+ customers 30+ countries First 100G probes in the world Strong R&D background European origin
Technology partner of premium vendors The only vendor recognized in both NetFlow related Gartner reports network visibility & security MAGIC QUADRANT
Flowmon Architecture Flow export from already deployed devices Flow data export + L7 monitoring Flow data collection, reporting, analysis Flowmon modules for advanced flow data analysis
Thank you Flowmon Networks, a.s. U Vodarny 2965/2 619 00 Brno, Czech Republic www.flowmon.com