sflow Elisa Jasinska

Transcription

1 sflow Elisa Jasinska

2 Agenda What is sflow? What is AMS-IX? AMS-IX requirements Existing software solutions Performance issues Software used at AMS-IX Privacy Results

3 What is sflow? Monitoring networks Cisco IOS - NetFlow Sampling mechanism, not touching every packet Applicable to high speed networks (>= 1GE)

4 What is sflow? sflow datagrams sent via UDP Datagram format standard defined in RFC 3176 Implemented on a wide range of devices (Foundry, Force10, Extreme...)

5 What is sflow? Not everything is sampled information Two different types provided by the datagram format: Flow samples Counter samples

6 What is sflow? Flow samples Whole captured packet (L2-L7) Defined sampling rate (eg. one out of 8192) Counter samples Interface counters (octets/packets/errors) Polling interval (eg. 30 sec.)

7 What is sflow? sflow Datagram 32 bit int sflow version (2 4 5) int IP version of the Agent / Switch (1=v4 2=v6) Agent IP address (v4=4byte v6=16byte) 4-16 byte int datagram sequence number int switch uptime in ms int n samples in datagram n samples Elisa Jasinska - Copyright (c) AMS-IX B.V.

8 What is sflow? Sampledata 32 bit int data format sample data (20 bit enterprise & 12 bit format) (standard enterprise 0, formats 1,2,3,4) int sample length byte sample data

9 What is sflow? enterprise = 0, format = 1 Flow Sample int sample sequence number int source id type (0=ifIndex 1=smonVlanDataSource 2=entPhysicalEntry) int source id index value int sampling rate int sample pool (total number of packets that could have been sampled) int drops (packets dropped due to a lack of resources) int input (SNMP ifindex of input interface, 0 if not known) int output (SNMP ifindex of output interface, 0 if not known) broadcast or multicast are handled as follows: the first bit indicates multiple destinations, the lower order bits number of interfaces int n * flow records flow record

10 What is sflow? enterprise = 0, format = 2 Counter Sample int sample sequence number int source id type (0=ifIndex 1=smonVlanDataSource 2=entPhysicalEntry) int source id index value int n * counter records counter record

11 What is sflow? What do you need? Hardware supporting sflow Central server to collect the data Software to analyze the received data

12 What is sflow? Switch Traffic In/Out Collector Sampling rate eg. 1 out of 8192 sflow datagrams UDP Up to 10 samples per sflow datagram

13 What is AMS-IX? Non-profit Internet exchange Based in Amsterdam 4 independent co-location facilities Operates only on Layer 2

14 What is AMS-IX? Interconnects parties to exchange IP traffic Internet Service Providers International carriers Mobile operators Content providers VoIP providers

15 What is AMS-IX? 252 members 419 ports Traffic peaks up to 199 Gb/s

16 What is AMS-IX?

17 What is AMS-IX?

18 What is AMS-IX?

19 What is AMS-IX? Foundry Networks hardware BigIron 15000, MG8, RX8, RX16 Interfaces 10 Mbit, 100 Mbit, 1 Gbit, 10Gbit Redundant topology Using Glimmerglass photonic switches and VSRP

20 What is AMS-IX? NIKHEF SARA Telecity Global Switch psw-nik-03 psw-sar-02 psw-tel-03 psw-nik-02 psw-sar-01 psw-tel-02 psw-glo-01 stub-nik-05 RX16 stub-nik-06 RX16 stub-sar-01 MG8 stub-sar-02 MG8 stub-tel-02 MG8 stub-tel-03 MG8 stub-glo-01 RX8 stub-glo-02 RX8 core-nik-04 RX16 core-tel-03 RX16 edge BI15K edge BI15K Customer <= 1 GE edge BI15K edge BI15K Customer <= 1 GE edge BI15K edge BI15K Customer <= 1 GE edge BI15K Customer <= 1 GE

21 What is AMS-IX? Statistics Total traffic statistics Interface counter (octets/pacets/errors) for each member Polled via SNMP MRTG... no sflow so far...

22 AMS-IX requirements Use flow samples to... Provide member-to-member traffic information See growth (or lack) of of IPv6 Due to high throughput a very efficient system is required

23 Existing software solutions Free software: InMon sflowtool Pmacct sflow2mysql Commercial: InMon Traffic Sentinel

24 Existing software solutions Issues with existing software Saves each sample to DB No caching or preprocessing possible Graphing with RRDtool overhead due to data export to RRD same data saved twice

25 Performance issues Traffic up to 188 Gb/s (30 Mpps) ca samples per second Cannot store each sample in a DB

26 Software used at AMS-IX Written in PERL Easy to understand Good integration with RRDtool Due to PERL's re-use architecture (modules) lots of subtasks have already been programmed Largest common denominator of a language understood at the AMS-IX NOC

27 Net::sFlow Decodes sflow datagrams Supports sflow version 2/4 and 5 Single (exportable) function, decode() Available on CPAN

28 sflow daemon Based on module Net::sFlow Receives UDP datagrams Analyzes the information Stores data to RRD files

29 Performance Results CPU usage while decoding sflow datagrams Growing linearly with amount of packets / samples

30 Performance Results

31 Performance Results I/O performance while writing data Currently: Writing ca RRD files in 8 seconds High load tests: Writing RRD files in 27 seconds Max. at AMS-IX conversations

32 Performance Results Foundry hardware BigIron 15k ASIC (Application-specific integrated circuit) Switch CPU not affected MG8 & RX* Blade CPU affected

33 Performance Results

34 Software used at AMS-IX Analysis Ether type graph percentage of IPv4, IPv6, ARP and other Total IPv6 traffic graph in bps and pps Member-2-Member analysis in bps and pps

35 Privacy Statistical analysis Samples not saved after decoding Decoding only up to L2 (ethernet) More data not decoded by the software

36 Results Ether Type - monthly - %

37 Results Total Multicast Traffic - weekly - bps

38 Results Total Broadcast Traffic - weekly - pps

39 Results Total IPv6 traffic - weekly - bps

40 Results Total IPv6 traffic - monthly - bps

41 Results Member-2-Member traffic

42 Results Member-2-Member traffic

43 Results Member-2-Member traffic

44 Future plans Use counter samples... Separate interfaces Aggregated links Backbone links Core network Replace current MRTG solution

45 Questions? Comments / Requests / Ideas: - elisa.jasinska@ams-ix.net