Network Security Monitoring with Flow Data

Network Security Monitoring with Flow Data

IT Monitoring in Enterprises NPMD (Network Performance Monitoring & Diagnostics) SNMP basics Flow data for advanced analysis and troubleshooting Packet capture for specialties What about security? Different technology Different tools Different vendors

NPMD and Security Volumetric DDoS detection Anomaly detection Incident reporting

Neil MacDonald, VP Distinguished Analyst Gartner Security & Risk Management Summit, London 2015

What is Flow Data? Modern method for network monitoring flow measurement Cisco standard NetFlow v5/v9, IETF standard IPFIX Focused on L3/L4 information and volumetric parameters Real network traffic to flow statistics reduction ratio 500:1 Flow data

Flow Monitoring Principle Flow Export Start Duration Proto Src IP:Port Dst IP:Port Packets Bytes 9:35:24.8 0.1 0 TCP 192.168.1.1:10111 -> 10.10.10.10:80 12 40 80 9:35:25.0 0.9 0.7 0.3 0.5 0 TCP 10.10.10.10:80 -> 192.168.1.1:10111 54 21 3 1231 862 156 40 362

Flow Gathering Schemes Pros Probe on a SPAN port Probe on a TAP Flows from switch/router Accuracy Performance L2/L3/L4/L7 visibility Same as on a SPAN All packets captured Separates RX and TX Already available No additional HW Traffic on interfaces Cons Facts May reach capacity limit No interface number Fits most customers Limited SPANs number Additional HW 2 monitoring ports Usually inaccurate Visibility L3/L4 Performance impact Always test before use Use Enterprise networks ISP uplinks, DCs Branch offices (MPLS, )

Flow-Enabled Devices Network equipment (routers/switched) Traditional capability known for many years Firewalls, UTMs, load balancers, hypervisors Ongoing initiative of majority of vendors Packet brokers and matrix switches Convenient option

Flow-Based Traffic Analysis Network as a sensor concept (and enforcer) blogs.cisco.com/enterprise/the-network-as-a-security-sensor-and-enforcer Bridges the gap left by signature-based security Key technology for incident response Designed for multi 10G environment DDoS Anomaly detection Statistical analysis Volumetric DDoS detection Advanced data analysis algorithms Detection of non-volumetric anomalies

How L3/L4 Data Helps Security? Myth 1: Flow is sampled and highly inaccurate. This is true for sflow and NetFlow Lite For NetFlow/IPFIX this depends on flow source Probes and new network equipment do just fine Myth 2: Flow is limited to L3/L4 visibility. This is the original design but today s flow data come with L2 and L7 extensions (usually using IPFIX) Myth 3: You need continuous packet capture. Flows with L7 visibility + on-demand or triggered packet capture is cost efficient option

Flow vs. Packet Analysis Flow data Packet analysis Strong aspects Works in high-speed networks Resistant to encrypted traffic Visibility and reporting Network behavior analysis Full network traffic Enough details for troubleshooting Supports forensic analysis Signature based detection Weak aspects No application layer data Sometimes not enough details Sampling (routers, switches) Useless for encrypted traffic Usually too much details Very resource consuming Solution? Take advantage of strong aspects in one solution Versatile and flexible Probes for visibility into all network layers Flowmon long-term strategy

Probes (by Flowmon Networks) Versatile and flexible network appliances Monitoring ports convert packets to flows Un-sampled export in NetFlow v5/v9 or IPFIX Wire-speed, L2-L7 visibility, PCAPs when needed L2 MAC VLAN MPLS GRE tunnel OVT L3/L4 Standard items NPM metrics RTT, SRT, TTL, SYN size, ASN Geolocation L7 NBAR2 HTTP DNS DHCP SMB/CIFS VoIP (SIP)

Use Case: Enterprise Security NBAD: On-demand Triggered Packet Capture

Fighting Advanced Threats Network visibility is essential component of new protection strategies against advanced attacks.

Flowmon ADS Flowmon ADS Principles Machine Learning Adaptive Baselining Heuristics Behavior Patterns Reputation Databases

Traffic Analysis (Using Flows) Bridges the gap left by endpoint and perimeter security solutions Behavior based Anomaly Detection (NBA) Detection of security and operational issues Attacks on network services, network reconnaissance Infected devices and botnet C&C communication Anomalies of network protocols (DNS, DHCP, ) P2P traffic, TOR, on-line messengers, DDoS attacks and vulnerable services Configuration issues

SIEM Integration Event exporting (syslog based) NetFlow IPFIX Collection and Behavior Analysis Flowmon Collector & ADS Network Traffic Monitoring Syslog SNMP Event Collection and Correlation SIEM system Links Flowmon <-> Log Management Special vendor relationships IBM QRadar (whitepaper, integration SW package) ArcSight native support through CEF

Traffic overview, anomalies detected

Attacker activity (port scan, SSH authentication attack)

Victim of the attack, source of anomalies

Attacker is looking for potential victims And starts SSH attack That turns out to be successful

Few minutes after that breached device starts to communicate with botnet C&C

Botnet identification using Flowmon Threat Intelligence

Flow data on L2/L3/L4

Including L7 visibility

Full packet capture and packet trace (PCAP file)

Analysis of PCAP file with botnet C&C communication in Wireshark

Data exfiltration command via ICMP

Command to discover RDP servers

ICMP anomaly traffic with payload present

PCAP available, what is the ICMP payload?

Linux /etc/passwd file with user accounts and hash of passwords

Looking for Windows servers with RDP Attack against RDP services

Network Against Threats Flow monitoring including L7 Network Behavior Analysis Full packet capture Triggered by detection

Use Case: DDoS Protection Volumetric DDoS Detection Traffic Redirection and Mitigation Control

Enterprise Protection Strategy Enterprise perimeter scheme Limited number of uplinks and capacity DMZ Internet CPE In-line DDoS mitigation appliance All-in-one detection & mitigation out of the box Volumetric + application (L3/L4/L7) attacks coverage Up to the uplink capacity! LAN

Backbone Protection Strategy Backbone perimeter specifics Multiple peering points routers & uplinks Large transport capacity tens of gigabits easily In-line protection is close to impossible! flow export 1. Flow collection 2. DDoS detection 3. Routing control 4. Mitigation control Flow-based detection and out-of-path mitigation Easy and cost efficient to deploy in backbone/isp Prevents volumetric DDoS to reach enterprise perimeter

Flow-Based DDoS Protection Define customers = protected segments Usually by network subnets (simple) Configure rules for DDoS detection Multiple types of baselines per protected segment Set alerts Notify about attacks (humans & systems) Configure traffic diversion = changes in routing Divert traffic for mitigation of DDoS attack Configure mitigation control = scrubbing Integration with scrubbing equipment or services

Attack Detection For each segment, a set of baselines is learned from monitored traffic. The attack is detected if the current traffic exceeds defined threshold. Baseline is learned for: TCP traffic with specific flags UDP traffic ICMP traffic

Attack Reporting Start/end time Attack target Type and status Traffic volumes during attack/peace time Attack targets (top 10 dst IPs, source subnets, L4 protocols, TCP flags combinations )

Response to Attack Alerting E-mail, Syslog, SNMP trap Routing diversion PBR (Policy Based Routing) BGP (Border Gateway Protocol) BGP Flowspec RTBH (Remotely-Triggered Black Hole) User-defined scripting Automatic mitigation With out-of-band mitigation devices With services of Scrubbing centers

DDoS Protection Scenario 1 Out-of-path Mitigation

Out-of-Path Mitigation Anomaly Detection Mitigation Enforcement Dynamic Protection Policy Deployment incl. baselines and attack characteristics Traffic Diversion via BGP Route Injection Scrubbing center Flow Data Collection Learning Baselines Attack Attack path Clean path Protected Object 1 e.g. Data Center, Organization, Service etc Internet Service Provider Core Protected Object 2

DDoS Protection Scenario 2 Mitigation with BGP Flowspec

BGP Flowspec Requires dynamic signature of the attack Provides specific action to take with network traffic BGP Flowspec rules are based on Destination Prefix Source Prefix IP Protocol Destination port ICMP type ICMP code

BGP Flowspec Rule BGP Flowspec rules are proposed based on dynamic attack signature Manual or automatic trigger is available Default action can be modified Rule is pushed to routers via BGP session

BGP Flowspec Scenario Anomaly Detection Mitigation Enforcement Sending specific Route advertisement via BGP FlowSpec Dynamic signature: Dst IP: 1.1.1.1/32 Dst Port: 135 Protocol IP: 17 (UDP) Discard Flow Data Collection Learning Baselines Attack Protected Object 1 e.g. Data Center, Organization, Service etc Internet Service Provider Core Protected Object 2 Dropped traffic for Dst IP: 1.1.1.1/32 Dst Port: 135 Protocol IP: 17 (UDP)

Flowmon Networks

Customer references is an international vendor devoted to innovative network traffic & performance & security monitoring 700+ customers 30+ countries First 100G probes in the world Strong R&D background European origin

Technology partner of premium vendors The only vendor recognized in both NetFlow related Gartner reports network visibility & security MAGIC QUADRANT

Flowmon Portfolio Network Visibility IT Operations Security Network Performance Monitoring and Diagnostics Application Performance Monitoring Network Behavior Analysis NPMD APM NBA DDoS Detection & Mitigation

Flowmon Architecture Flow export from already deployed devices Flow data export + L7 monitoring Flow data collection, reporting, analysis Flowmon modules for advanced flow data analysis

Flowmon Architecture Flowmon Probes & Collectors Flowmon extension modules Flowmon Anomaly Detection Flowmon DDoS Defender Application Performance Monitoring Flowmon Traffic Recorder Network Visibility Troubleshooting IPFIX/NetFlow export Flowmon Collector Network Security Anomaly Detection Application Performance Monitoring DDoS Protection

User Perspective Next Generation Network Monitoring (NetFlow/IPFIX) Full network traffic visibility Close to real-time and historical data for LAN & WAN & Internet communications Network operation & connectivity cost optimization Effective troubleshooting Next Generation Network Security (NBA, NBAD) Bridges the gap left by endpoint and perimeter and signature based security solutions Behavior-based Anomaly Detection Detection of polymorphic malwares, zero days attacks, suspicious data transfers, behavior changes and various operational and configuration issues

User Perspective Full Packet Capture On-demand packet capture for troubleshooting and forensic analysis producing PCAP files Traffic capture capabilities on 1G/10G/40G Distributed architecture Application Performance Monitoring Agent-less monitoring of all user transactions No influence on target application Designed for HTTP/HTTPS and SQL applications DDoS Protection Flow-based detection of volumetric attacks Universal deployment scenarios (stand-alone, integrated, with scrubbing center) Traffic diversion and control of mitigation process

Summary Make Use of Flow Data

Levels of Visibility SNMP monitoring Amount of transferred data, number of packet, insufficient Flow monitoring (based on IP flows) Traffic structure visibility, anomaly detection and reporting Packet analysis For forensics and to deal with specific issues Basic monitoring Flowmon

Using Flow Data for Security Keep in mind that there is no silver bullet Security is balanced combination of technology, people, processes Flow data & Probes can help you with Moving the infrastructure monitoring into next level Traffic visibility, engineering and troubleshooting Performance reporting and analysis Bridging the gap left by signature-based products Detection and mitigation control of volumetric DDoS Incident response and on-demand full packet capture

Thank you Flowmon Networks, a.s. U Vodarny 2965/2 619 00 Brno, Czech Republic www.flowmon.com