Using Visual Motifs to Classify Encrypted Traffic

Transcription

1 Using Visual Motifs to Classify Encrypted Traffic VizSEC'06 - November 3, 2006 Charles V Wright Fabian Monrose Gerald M Masson Johns Hopkins University Information Security Institute

2 Traffic Classification: Why? To detect intrusions or malware Is your mail server hosting a phishing website? (Are you sure?) To detect misuse by legitimate users File sharing Chat, Instant Messaging

3 Traffic Classification: Why? Port Numbers are not reliable They can be changed at will by the end hosts Increased use of cryptography precludes inspection of packet payloads Good: Hackers can't get our passwords. Bad: Network admins have less info to work with

4 Traffic Classification: How? Manually? tcpdump output? Ethereal/Wireshark?

5 Traffic Classification: How? Manually? No. tcpdump output? Ethereal/Wireshark? Machine Learning Text classification [ZP00] [MP05] [Dre06] [Ma06] Decision Trees [EBR03] Naïve Bayes [MZ05] Hidden Markov Models [WMM04] [WMM]

6 Traffic Classification: How? Manually? No. tcpdump output? Ethereal/Wireshark? Machine Learning [ZP00] [EBR03] [WMM04] [MP05] [MZ05] [Dre06] [Ma06] [WMM] Visually Look for distinctive visual motifs in the patterns produced by packets on the wire

7 Core observation of this work: Application protocols behave differently and thus look different from each other on the wire.

8 Core observation of this work: Application protocols behave differently and thus look different from each other on the wire. Even when encrypted using SSL or TLS.

9 Application to Traffic Classification We can use these differences to distinguish between common application protocols in the traffic that we see on our networks Quickly and Easily Without port numbers Without packet payloads

10 What does a TCP connection look like? from server from client Example: HTTP

11 What does a TCP connection look like? from client HTTP Request TCP 3-way Handshake from server Data Transfer from Server to Client Example: HTTP

12 What does a TCP connection look like? from client TCP 3-way Handshake Data Transfer from Client to Server TCP FIN from server SMTP Handshaking (EHLO, RCPT TO, etc.) SMTP GOODBYE Example: SMTP

13 Viewing many similar TCP connections at once from client n = 1 from server Example: HTTP

14 Viewing many similar TCP connections at once from client n = 2 from server Example: HTTP

15 Viewing many similar TCP connections at once from client n = 3 from server Example: HTTP

16 Viewing many similar TCP connections at once from client n = 50 Yuck! from server Example: HTTP

17 Viewing many similar TCP connections at once - heat maps from client dark spots - very few packets from server bright spots - lots of packets Example: HTTP

18 Viewing many similar TCP connections at once heat maps HTTP requests from client TCP handshake ACKs from client from server HTTP response Example: HTTP Data from server

19 Classifying traffic with heat maps and visual motifs HTTP SMTP AIM HTTP

20 Classifying traffic with heat maps and visual motifs HTTP SMTP AIM SSH

21 Does this look like HTTP?

22 Or more like SMTP?

23 Limitations The previous graphs illustrate time-dependent properties of the application protocols They also cover a very short time span Long-lived, free-form protocols like SSH may be better characterized by taking a different view of the data

24 Steady-State Properties We assume these don't change over the life of the connection Look at individual packets (unigrams) How big is the packet? How long since the previous packet? Look at pairs of consecutive packets (bigrams)

25 Unigram Frequencies: HTTP from server from client

26 Unigram Frequencies HTTP SMTP AIM SSH

27 Bigram Frequencies HTTP SMTP AIM SSH

28 Bigram Frequencies: HTTP from server from client from server from client

29 Bigram Frequencies: SMTP from server from client from server from client

30 Bigram Frequencies: AIM from server from client from server from client

31 Bigram Frequencies: SSH from server from client from server from client

32 Bigrams in 3D

33 Future Work Work is in progress to build an interactive GUI application for analyzing packet traces Open Source release planned for later this academic year We're also exploring ways to integrate Machine Learning with Visualization more effectively

34 Acknowledgments Many thanks to the developers of Numerical Python and the Python matplotlib package Thanks also to the Statistics Group at GMU and to Pang et al. at LBNL for providing access to their packet traces

35 Questions? Thanks!

36 References [Dre06] H. Dreger, A. Feldmann, M. Mai, V. Paxson, and R. Sommer. Dynamic Application-Layer Protocol Analysis for Network Intrusion Detection. USENIX Security [EBR03] J. Early, C. Brodley and C. Rosenberg. Behavioral Authentication of Server Flows. ACSAC [Ma06] J. Ma, K. Levchenko, C. Kreibich, S. Savage, and G.M. Voelker. Unexpected Means of Protocol Inference. IMC [MP05] A. Moore and D. Papagiannaki. Toward the Accurate Identification of Network Applications. PAM [MZ05] A. Moore and D. Zuev. Internet Traffic Classification Using Bayesian Analysis Techniques. ACM SIGMETRICS, June [WMM04] C. Wright, F. Monrose, and G.M. Masson. HMM Profiles for Network Traffic Classification (Extended Abstract). VizSEC/DMSEC [WMM] C.V. Wright, F. Monrose, and G.M. Masson. On Inferring Application Protocol Behaviors in Encrypted Network Traffic. JMLR Special Topic on Computer Security. (to appear) [ZP00] Y. Zhang and V. Paxson. Detecting Back Doors. USENIX Security 2000.