Flow Measurement For IT, Security and IoT/ICS Pavel Minařík, Chief Technology Officer EMITEC, Swiss Test and Measurement Day 20 th April 2018
What is Flow Data? Modern method for network monitoring flow measurement Cisco standard NetFlow v5/v9, IETF standard IPFIX Focused on L3/L4 information and volumetric parameters Real network traffic to flow statistics reduction ratio 500:1 Flow data
Flow Monitoring Principle Flow Export Start Duration Proto Src IP:Port Dst IP:Port Packets Bytes 9:35:24.8 0.1 0 TCP 192.168.1.1:10111 -> 10.10.10.10:80 12 40 80 9:35:25.0 0.9 0.7 0.3 0.5 0 TCP 10.10.10.10:80 -> 192.168.1.1:10111 54 21 3 1231 862 156 40 362
Flow-Enabled Devices Network equipment (routers/switches) Traditional capability known for many years Firewalls, UTMs, load balancers, hypervisors Ongoing initiative of majority of vendors Packet brokers and matrix switches Convenient option
Myth: Flow data do not provide sufficient level of detail when it comes to network troubleshooting or forensics. Full packet traces are absolute must to investigate on network issues and cyber crime.
Flow vs. Packet Analysis Flow data Packet analysis Strong aspects Works in high-speed networks Resistant to encrypted traffic Visibility and reporting Network behavior analysis Full network traffic Enough details for troubleshooting Supports forensic analysis Signature based detection Weak aspects No application layer data Sometimes not enough details Sampling (routers, switches) Useless for encrypted traffic Usually too much details Very resource consuming Solution? Take advantage of strong aspects in one solution Versatile and flexible Probes for visibility into all network layers Flowmon long-term strategy
Flowmon Probes Versatile and flexible network appliances Monitoring ports convert packets to flows Un-sampled export in NetFlow v5/v9 or IPFIX Wire-speed, L2-L7 visibility, tunnel decapsulation, PCAPs when needed L2 MAC VLAN MPLS SPB GRE/ERSPAN OVT L3/L4 Standard items VxLAN NPM metrics RTT, SRT, TTL, SYN size, ASN Geolocation L7 NBAR2 HTTP DNS DHCP SMB/CIFS VoIP (SIP) Email
Use Case: Network Investigation Using Flow Data with L7 Visibility
There is a lot of IPs hosting multiple domains, even thousands of them. How can I identify which domain was really accessed from client device? Probe DNS visibility.
Investigation on User Activity Traffic of Interest Internal IP address 192.168.222.65 External IP address 217.11.249.139 Timeframe 2018-04-07 10:00-2018-04-07 11:00 FTP data transfer Which domain was target of data transfer? What we do? Find the corresponding FTP connection in the network traffic Look for DNS query that precedes this connection Analyze extended flow data including L7 DNS information
Investigation on User Activity FTP data upload from 192.168.222.65 DNS traffic identifying target domain modularis.cz
Use Case: Network Performance Monitoring On-Premise or Cloud, Both Works the Same
Migration to the cloud, in its various forms, creates a fundamental shift in network traffic that traditional network performance monitoring tools fail to cover. I&O leaders must consider cloud-centric monitoring technologies to fill visibility gaps. Flow monitoring vendors that cater to hybrid IT environments include Flowmon Networks. Source: Network Performance Monitoring Tools Leave Gaps in Cloud Monitoring, Gartner Report G00301635, by Sanjit Ganguli, published 27 th May 2016
What We Need To Monitor Cloud Applications? Identify individual cloud applications Detailed visibility into HTTP, both traffic directions (request, response) Information included in both traffic directions Visibility into HTTP & HTTPS (SNI)
What We Need To Monitor Performance? Client TCP handshake Client request Server response Syn Ack Req Probe Server Syn, Ack Ack Data Data Data Data RTT SRT Delay Round Trip Time delay introduced by network Server Response Time delay introduced by server/application Delay (min, max, avg, deviation) delays between packets Jitter (min, max, avg, deviation) variance of delays between packets
CLOUD APPs PERFORMANCE NPM metrics (RTT, SRT, Jitter) In time visualizations per profile / channel Get quick insight, understand deviations New Y axis on the right side of traffic chart Selection of current view
Drill Down to Individual IPs
Use Case: Enterprise Security Network Behavior Analysis and Triggered Capture
Flow-Based Anomaly Detection Network as a sensor concept (and enforcer) blogs.cisco.com/enterprise/the-network-as-a-security-sensor-and-enforcer Bridges the gap left by signature-based security Key technology for incident response Designed for multi 10G environment Volumetric DDoS Network Behavior Analysis Statistical analysis Volumetric DDoS detection Advanced data analysis algorithms Detection of non-volumetric anomalies
Flowmon ADS Flowmon Anomaly Detection Principles Machine Learning Adaptive Baselining Heuristics Behavior Patterns Reputation Databases
Traffic overview, anomalies detected
Attacker is looking for potential victims And starts SSH attack That turns out to be successful
Few minutes after that breached device starts to communicate with botnet C&C
Data exfiltration (ICMP anomaly traffic with payload present)
PCAP available, what is the ICMP payload?
Linux /etc/passwd file with user accounts and hash of passwords
Network Against Threats Flow monitoring including L7 Network Behavior Analysis Full packet capture Triggered by detection
Flowmon Labs IoT/ICS Monitoring
IEC 60870-5-104 Protocol used for telecontrol (supervisory control and data acquisition) in electrical engineering and power system automation applications. Application layer (L7) Transport layer (L4) Network layer (L3) Link layer (L2) Physical layer (L1) Application Service Data Units Application Protocol Control Information Selection of TCP/IP Protocol Suite (RFC 2200) Physical transfer medium Application visibility Traditional flow data Flow records L2 Src MAC, Dst MAC L3/L4 Src IP, Src Port, Dst IP, Dst Port, Protocol L7 APCI, ASDU
Flow export (IPFIX) IEC 60870-5-104 Monitoring Scenario Master Slave Port mirror Flowmon Probe Time L2 L3/L4 L7 Volume Flow start Duration Src MAC address Dst MAC address Src IP address Src port Protocol Dst IP address Dst port APCI type Information object type Number of objects Cause of transmission Originator address ASDU address file Packtes Bytes Analysis Visualization Reporting Anomaly Detection Performance Troubleshooting
CoAP: Web Services for Constrained Networks Home use IoT devices Testbed at Hallym University, Korea CoAP traffic from sensors is mirrored to Flowmon Probe Probe exports extended flow data Message ID CoAP Code CoAP Token URI
CoAP: Web Services for Constrained Networks Flowmon Probe
About Flowmon Networks many tasks, single solution
Customer references is an international vendor devoted to innovative network traffic & performance & security monitoring 700+ customers 30+ countries First 100G probes in the world Strong R&D background European origin
Flowmon Architecture Flow export from already deployed devices Flow data export + L7 monitoring Flow data collection, reporting, analysis Flowmon modules for advanced flow data analysis
Thank you Performance monitoring, visibility and security with a single solution Pavel Minařík, Chief Technology Officer pavel.minarik@flowmon.com, +420 733 713 703 Flowmon Networks, a.s. Sochorova 3232/34 619 00 Brno, Czech Republic www.flowmon.com