Advanced Attack Response and Mitigation
Agenda Overview of cloud DDoS detection and mitigation which features geographically diverse scrubbing and high velocity auto-mitigation capabilities. - Overview - Architecture & Deployment - Trends - Auto-mitigation Approach
Overview
CRS-8 CRS-1 CRS-8 CRS-1 CRS-1 CRS-8 CRS- 8 CRS-1 CRS-8 CRS-1 CRS-1 CRS-8 CRS-8 CRS-1 CRS-8 CRS-8 CRS-8 CRS-8 CRS-1 CRS-8 CRS-1 CRS-8 9 Total CRANs INTERNET CRAN Seattle L HFC CONTENT PROVIDERS TELCOs Portland L Eugene L San Jose L 4 Reith L Tionesta Sacramento L Bakersfield Los Angles L 4 Boise L Edison Odgen L Denver QWEST Tucson Stratford L Omaha L Centerview L St. Louis L Chicago L Omaha L Denver L Indianapolis L Rocky Ford Louisville Nashville L Little Rock 4 4 4 Detroit L Cleveland L Pittsburg L Raleigh 5 4 Atlanta L Charlotte Toronto L Montreal L Philadelphia L CRAN McLean L New York L Boston L NODE HFC Commercial CRS-8 Dallas L MSOs Santa Teresa L CRAN 1 Orlando Miami L NODE HFC MDUs HFC Residential Third Party National / Global Metro Access CP
Considerations Do we need this? Are we under attack? Why don t we ask them to stop? More harm than good? Who s responsible? Ambiguity around response Do we have capability to take specific action for impacted customers? What types of actions should be taken (with and without authorization)? How do we strike the balance between risk mitigation and availability? Overall DDoS needs to be evaluated Historically, such events or traffic simply got blocked or shut down Today, such events get detected or escalated; mitigation not easy Slightly easier on the Residential side (tolerance for service outage)
Solution Detection Architecture Heavily reliant on Netflow for traffic analysis Leverage flow replication to redistribute Netflow to appropriate tools for analysis Monitor DDoS Host Detection for the entire footprint X-services and beyond (Xfinity) Mitigation Architecture Auto vs. Manual Real-Time Blackhole (RTBH) to drop DDoS by either source or destination Sinkhole - cloud based DDoS solution: BGP off-ramping for surgical mitigation. It also includes packet sniffing and analysis Ability to mitigate DDoS attacks for any customers residing on Comcast networks
Architecture
Peer Peer CR/PE CR/PE CMC Comcast Backbone AS79 CH Multi-hop ebgp CR/PE CR/PE CR/PE Multi-hop ebgp BHS01 AR AR AR BHS0 Region Sub-AS 65xxx+1 Region Sub-AS 65xxx+n-1 Region Sub-AS 65xxx+n
Backbone (55 routers/1156 interfaces) 819 Sampled All IBONE routers Service Delivery (80 routers/40k interfaces) 819 Sampled All AR routers National Data Center (15 routers/k interfaces) 18 Sampled Layer 7 application visibility (flow sensor) Enterprise (471 routers/4k interfaces) 18 Sampled Layer 7 application visibility with (flow sensor)
UI Controller Platform HA Scrubbing 18 x Collector Appliances 8 x Flow Sensors 10 Router Capacity 16 of 10 routers 74,050 of 50,000 interfaces 607 of 1000 MOs.M BGP routes of 610M IPv6 capable *PI for High Availability
1 1 San Jose, CA Scrubbing NYC, NY Atlanta, GA Chicago, IL CRAN P E Customer Los Angeles, CA Ashburn, VA CRAN Customer P E Threat Center Threat Center Core P E CRAN Customer Core CRAN Threat Center Detection Core Threat Center P E Core Comcast Backbone Core Customer CRAN Threat Center P E Core Threat Center P E CRAN Customer Customer
Trends
Attack Protocol Distribution Majority is Volumetric or Flood Attacks Large botnets or spoofed IP to generate a lot of traffic bps or pps 99% is UDP-based floods from spoofed IP take advantage of connection less UDP protocol Take out the infrastructure capacity routers, switches, servers, links
Common DDoS Attacks Universal Plug and Play (SSDP 1900) Network Time Protocol (NTP 1) Simple Network Management Protocol (SNMP 161) Chargen (19) ICMP Flood (0)
Common DDoS Attacks HTTP over UDP (80, 44, 8080, 8081, etc) DNS (5) Xbox (074) UDP Fragmentation Based Attacks (0) TCP Syn Flood (very small)
ASN Attack Detail
Attack Size Large SSDP (UDP 1900) attacks over 100G observed in May 014 Large DNS attack closed to 00G observed in March 015
Typical Month September Attacks: 10,000 or more Attacks >100Gbps: 97, a 746% increase in this reporting period Cases Mitigated: 8,151 - over 0+ Trillion Packets Scrubbed Commericial Cases: 44% of mitigated attacks were for Commercial (BCS) users
Global Benchmarks Comcast attack size distribution very different to world-wide Much higher percentage of events over 1Gb/sec o 1.1% v s 48.8% in Q1 o 15.% v s 50.1% in Q Much higher proportions of events over 10Gb/sec o o 1.76% v s 5.56% in Q1 0.9% v s 4.49% in Q World 014 Q Size Break-Out, BPS <500Mbps >500Mbps<1Gbps >1<Gbps ><5Gbps Comcast 014 Q Size Break-Out, BPS <500Mbps >500Mbps<1Gbps >1<Gbps ><5Gbps
Auto-Mitigation
Anomaly Types Misuse Anomalies Traffic of a certain type directed towards an individual hosts that exceeds what should normally be seen on a network Profiled Anomalies Customized detection event tailored to specific conditions within Fingerprints.0 Fingerprints received via ATF, FSA, or traffic that match a user specified signature
Alert Generation 1. Detection Real time discovery of deviant traffic Traffic that deviates from acceptable Internet use (Misuse) Traffic exceeding normal levels for a resource (Profiled) Traffic that matches user specified threat patterns (Fingerprint). Classification Sets a level of importance to detected anomalies Misuse Based on static thresholds Profiled Based on auto classification or administrator configured high severity traffic rates Helps determine what anomalies to give precedence Three classification levels: High severity (Red) Medium Severity (Orange) Low Severity (Green) Once an alert has been detected and classified, its severity can only go up
Looking Ahead Looking at VRF as an option for onramp/reinjection the traffic IPv4/IPv6 transparent routing and mitigation Using VRF where you can, avoid GRE if you could Ease the pain of the high maintenance in GRE Possible leverage BGP dynamic route leaking feature or MPLS VPN to import routes from global/default (dirty) VRF routing table to non-default (clean) VRF routing table
Lessons Learned Know your monitoring Trust your tools and telemetry Test often Not all vendors are created equal Track everything and kill what you need to