Large-Scale Geolocation for NetFlow Pavel Čeleda, Petr Velan, Martin Rábek Rick Hofstede, Aiko Pras {celeda velan xrabek1}@ics.muni.cz, {r.j.hofstede a.pras}@utwente.nl IFIP/IEEE IM 2013, 27-31 May 2013, Ghent, Belgium
Part I Introduction Pavel Čeleda Large-Scale Geolocation for NetFlow 2 / 22
Motivation and R&D Goals I : SURFmap - a Network Monitoring Tool Based on the Google Maps API. Pavel Čeleda Large-Scale Geolocation for NetFlow 3 / 22
Motivation and R&D Goals II How flow-based geolocation can be performed in a large-scale? exporter-based approach, collector-based approach. How can we benefit from geolocation data in flow records? traffic engineering, traffic profiling, anomaly detection. Pavel Čeleda Large-Scale Geolocation for NetFlow 4 / 22
Part II Architecture Pavel Čeleda Large-Scale Geolocation for NetFlow 5 / 22
Exporter-Based Geolocation Packets Input Flow cache Export NetFlow v9 Pavel Čeleda Large-Scale Geolocation for NetFlow 6 / 22
Exporter-Based Geolocation Packets Input Flow cache Export NetFlow v9 Flows Geolocated flows GeoPlugin Pavel Čeleda Large-Scale Geolocation for NetFlow 6 / 22
Exporter-Based Geolocation Packets Input Flow cache Export NetFlow v9 Flows Geolocated flows GeoPlugin exporter filter plugin for IP address geolocation, NetFlow v9 template mapping GEO data to AS fields SRC_AS=*SRC_GEO, DST_AS=*DST_GEO, AS mapping transparent to any flow collector. Pavel Čeleda Large-Scale Geolocation for NetFlow 6 / 22
MaxMind GeoLite Country Database MaxMind GeoLite free off-line country database, C-API for IPv4/IPv6 geolocation. Queries/s (x 10 6 ) 16 14 12 10 8 6 4 Standard Memory cache Check cache MMAP cache 2 0 IPv4 IPv6 : IPv4/IPv6 geolocation database performance. Pavel Čeleda Large-Scale Geolocation for NetFlow 7 / 22
Collector-Based Geolocation Data collection NetFlow nfcapd v5, v9 Geolocation patch for NFDUMP and NfSen toolset, native geolocation support for any NetFlow v5/v9, IPFIX data. Pavel Čeleda Large-Scale Geolocation for NetFlow 8 / 22
Collector-Based Geolocation Data collection NetFlow nfcapd Storage v5, v9 Geolocation patch for NFDUMP and NfSen toolset, native geolocation support for any NetFlow v5/v9, IPFIX data. Pavel Čeleda Large-Scale Geolocation for NetFlow 8 / 22
Collector-Based Geolocation Data collection NetFlow nfcapd Storage v5, v9 Geolocation Data processing Top-N stats nfdump Aggregation Filtering Raw data nfprofile NfSen Web UI (profiles) patch for NFDUMP and NfSen toolset, native geolocation support for any NetFlow v5/v9, IPFIX data. Pavel Čeleda Large-Scale Geolocation for NetFlow 8 / 22
NFDUMP Database Extension #15 Country Code Flow Record: Flags = 0x06 Unsampled size = 80 first = 1348387461 [2012-09-23 10:04:21] last = 1348387462 [2012-09-23 10:04:22] msec_first = 890 msec_last = 100 src addr = 23.63.79.144 dst addr = 147.251.170.165 src port = 80 dst port = 57046 tcp flags = 0x1a.AP.S. proto = 6 (in)packets = 4 (in)bytes = 936 input = 5 src as = 20940 dst as = 2852 in src mac = 00:0e:38:5e:30:c0 out dst mac = 00:1e:be:8b:26:c0 src ctry = 840... ISO 3166-1 country code - US dst ctry = 203... ISO 3166-1 country code - CZ Pavel Čeleda Large-Scale Geolocation for NetFlow 9 / 22
NFDUMP Flow Listing a) numeric code %scc %dcc 194.228.29.173:0 147.251.48.205:3.13 147.251.210.106:51885 69.171.227.59:443 151.40.40.243:15833 147.251.79.246:49159 157.55.235.165:40040 147.251.215.10:49464 147.251.170.77:59408 89.79.20.120:18973 b) alpha-2 code %sccan %dccan 194.228.29.173:0 147.251.48.205:3.13 147.251.210.106:51885 69.171.227.59:443 151.40.40.243:15833 147.251.79.246:49159 157.55.235.165:40040 147.251.215.10:49464 147.251.170.77:59408 89.79.20.120:18973 Usage example nfdump -M /data/nfsen/profiles-data/live/p3000:p3001 \ -r 2012/09/23/nfcapd.201209231005 \ -o fmt:%pr %sap -> %dap %sccan %dccan -m -c 20 Pavel Čeleda Large-Scale Geolocation for NetFlow 10 / 22
NFDUMP Geofiltering Geofiltering country filter syntax is similar to other NFDUMP filters syntax : ctry [comp] <num>, country can be compared to a list (red-black tree) of country codes, syntax : ctry in [ <ctrylist> ], filters are often used for traffic profilling in NfSen. Usage example nfdump -M /data/nfsen/profiles-data/live/p3000:p3001 \ -r 2012/09/23/nfcapd.201209232035 -c 5 \ src ctry 203 and not dst ctry in [ 203 840 166 ] Pavel Čeleda Large-Scale Geolocation for NetFlow 11 / 22
NfSen Geoprofiling : Screenshot of collector-based geolocation prototype. Pavel Čeleda Large-Scale Geolocation for NetFlow 12 / 22
Part III Use Case I Traffic Profiling Pavel Čeleda Large-Scale Geolocation for NetFlow 13 / 22
Geolocated and Non-geolocated ICMP Traffic I 150 (1) IN 100 Packets/s 50 0-50 -100 (2) (3) (4) In Out OUT -150 00:00 02:00 04:00 06:00 08:00 10:00 12:00 : ICMP traffic. Pavel Čeleda Large-Scale Geolocation for NetFlow 14 / 22
Geolocated and Non-geolocated ICMP Traffic II 150 (1) IN 100 Packets/s 50 0-50 -100 (2) (3) (4) UA US Other CZ OUT -150 00:00 02:00 04:00 06:00 08:00 10:00 12:00 : Geolocated ICMP traffic. Pavel Čeleda Large-Scale Geolocation for NetFlow 15 / 22
Distribution of HTTPS Traffic over Countries I 150 IN 100 50 Flows/s 0-50 -100-150 -200 US CZ Other OUT : HTTPS flows/s. Pavel Čeleda Large-Scale Geolocation for NetFlow 16 / 22
Part IV Use Case II Anomaly Detection Pavel Čeleda Large-Scale Geolocation for NetFlow 17 / 22
Bad Neighboring Countries 300 250 All countries China 200 Flows/s 150 100 50 0 00:00 06:00 12:00 18:00 00:00 : Incoming TCP SYN-only flows. Pavel Čeleda Large-Scale Geolocation for NetFlow 18 / 22
UDP DoS Attack 2000 0 IN -2000 Packets/s -4000-6000 -8000-10000 -12000 DNS In/Out US DNS In/Out OUT 18:00 19:00 20:00 21:00 22:00 23:00 00:00 : UDP DoS attack from infected Linux machine. Pavel Čeleda Large-Scale Geolocation for NetFlow 19 / 22
Part V Conclusion Pavel Čeleda Large-Scale Geolocation for NetFlow 20 / 22
Conclusion Summary country-level information in flow data, native geolocation support for NfSen/NFDUMP, pilot geo-prototype deployment at MU CESNET link. Future Work IPFIX-compliant prototype for exporter-based geolocation, ipfixcol AS and GEO support implementation, AS + GEO data for traffic profiling and anomaly detection. Pavel Čeleda Large-Scale Geolocation for NetFlow 21 / 22
Thank You For Your Attention! Large-Scale Geolocation for NetFlow P. Čeleda, P. Velan, M. Rábek {celeda velan rabek}@ics.muni.cz R. Hofstede, A. Pras {r.j.hofstede a.pras}@utwente.nl Geolocation Toolset http://www.muni.cz/research/publications/1090804 Pavel Čeleda Large-Scale Geolocation for NetFlow 22 / 22