{ } Embedded Montreal Meetup. Char *lecture = Lecture 2 Deep Packet Inspection in Userspace ;

Size: px

Start display at page:

Download "{ } Embedded Montreal Meetup. Char *lecture = Lecture 2 Deep Packet Inspection in Userspace ;"

Dustin Richardson
5 years ago
Views:

1 { } Embedded Montreal Meetup October 17 th, 2014 Ron Brash Embedded Developer Ron.brash@gmail.com Char *lecture = Lecture 2 Deep Packet Inspection in Userspace ;

2 Objectives Quick introduction Refresher on bytes and sign (order) Brief overview on protocol layers Reasons for DPI and firewall differences Reading packet headers Packet processing methods Example DPI application and methodology Tools and performance

3 Who Am I? I am an embedded developer that : Develops on OpenWRT and non-x86 architectures Writes DPI applications on embedded platforms I have a Bachelors in tech and currently working towards a MsCompSci My ron.brash@gmail.com

4 The Basics

5 A Quick Refresher What is a byte? Is 8 bits Represented as a decimal value OR hexadecimal Can be signed or unsigned Both negative and positive, or only positive Int vs u_int Can big or little Endian Byte order is a u_int8 with a value can be: 0x10 or 0x01

6 Network Protocols In the non-osi network model there are 4 layers Application Transport Internet Network Access

7 Network Access Layer (1) Also known as the Ethernet layer Foundation for most protocols Contains: MAC addressing information VLAN information Information identifying the following protocol

8 Internet Layer (2) Builds upon the network layer Determines the method of addressing Contains IP addressing information Following protocol information Enables routing and more Called the IP layer IPv4 or IPv6

9 Transport Layer (3) Builds upon the Internet layer Consists of protocols such as ICMP, TCP, UDP May have port information or useful fields: ACK, sequence numbers (useful for tracking connections and data)

10 Application Layer (4) The most important layer This is where you would do DPI Example applications layer protocols: HTTP, DNS, FTP Most firewalls don t inspect these layers If they do, they are usually Enterprise and used for: Anti-virus inspection, spam filtering, policy enforcement

11 Firewalls Typically all modern firewalls are state tracking (even consumer) Only inspect the first three layers Called Layer 3 Firewalls Doesn t inspect application layer traffic

12 What Is DPI and Why Is It Needed? Deep packet inspection (DPI) Allows a firewall to inspect actual application protocols and data Better granularity for policy enforcement Restrict data access and specification checking

13 Extending Firewalls Through DPI Two methods for development DPI user space applications Kernel modules Requires extensive knowledge of the protocol Performance and latency are now important Inspection requires better hardware Adds latency due to processing You could use Snort in IPS mode or a web-proxy. But that s not why we are here!

14 Reading Packet Headers

15 What Is A Frame? A frame is "the unit of transmission in a link layer protocol, and consists of a link-layer header followed by a packet." In other words, a frame's contents look like: 1. Ethernet Header (is 14 bytes in size) 2. IP Header (usually and is 20 bytes in size) 3. Protocol Header (usually ICMP,UDP or TCP) 4. Payload(optional)

16 The Ethernet header Is the first header of the potential three There are other types of headers or protocols If we ignore the 8 bits that are in the preamble then there are just 14 bytes in the header. 6 bytes for the destination MAC address 6 bytes for the source MAC address 4 bytes for the Type. If this is an IP packet, the type will be 0x0800

17 Ethernet Header Diagram

18 IP Header Diagram The second header in the frame

19 UDP Header The UDP header is the second header of the potential three in the frame

20 TCP Header The TCP header is another potential header of the three in the frame.

21 ICMP Header The ICMP header is another potential header of the three in the frame

22 Reviewing A Packet s Hex Values Each of those double groupings of letters/numbers equal 1 byte. If you look at the highlighted part of the image then 00 is 1 BYTE! And now you remember that 1 Byte is 8 bits and that each of these header specifications is 32 bits across.

23 Reviewing A Packet s Hex Values Here is the same marked packet dump 0x06 is TCP The start of the source IP address - is at bytes 13-16: C0 A and the destination IP address is at bytes 17-20: C7 3B 96 2A. To convert these bytes into an IP address, you must convert each of the 4 bytes to decimal C0 =192, A8 = 168, 02 = 2 and 65 = 101. As a result the source IP address is

24 Reviewing Continued Without any padding or extra data, the TCP header is 20 bytes long. You could guesstimate by the number of bytes and guess the potential packet header specification The first 4 bytes is the Source Port, the next 4 bytes is the Destination Port To get these values just convert the hexadecimal to decimal. The TCP flags requires a bit more work From the start of the TCP header, count 13 bytes and the 14th bytes is the flags field! And the flags in hex are 0x50 11 Next convert the hexadecimal to binary to find which bits have been set - each flag is one bit. 0x5011 is equal to We aren't concerned with the first byte or 8 bits, so looking at the last byte, we can see that the 4th and 8th bit are set which alternatively is ACK and FIN.

25 DPI In Userspace

26 Why Userspace Vs. Kernel? Advantages of Kernel DPI: Better performance Disadvantages: Errors can cause system crashes (OOPS) GPL infections Advantages of Userspace: Easier to develop/debug Can be non-gpl (LGPL or proprietary) Disadvantages: Less performance Added latency (kernel to user and back)

27 Userspace Options C libraries libnetfilter-queue (GPL) Libmnl (LGPL) Python and perl nfqueue bindings Netmap Packets are sent from iptables/ip6tables using the NFQUEUE target Ebtables requires patching for kernel and userspace Ethernet header is not passed without patching

28 Packet Flow From Xtables Packet is matched in xtables Sent to userspace using Netfilter sockets Application processes packet Returns verdict to kernel Packet is dropped or allowed

29 Honorable Mentions Other methods to receive packets in user space: Proxies using raw sockets Adds latency Allows you to track/store a whole stream Logging and then forwarding using crafted packets Slow Libpcap to monitor applications Read only and crafting would still be required Slow and not in-line!

30 Development Process

31 System/Dev Requirements One or more NICs Good test network/setup For example, real world systems vs. Pcaps Wireshark/tcpdump Compilers and dev libraries Queuing libraries Libraries such as glib for data structures Xtables extensions and support NFQUEUE in kernel and userspace

32 The Development Process 1. System setup 2. Setup test network 3. Review network traffic with Wireshark Write a filter if needed 4. Review RFCs and protocol specifications Data flow diagrams and planning 5. Write application and test

33 Application Design Decisions Scenarios that should be accounted for: Application layer fragmentation Sanity checking fields Real-world edge-cases Processing latency Legacy protocol versions Startup conditions and port hopping

34 Choosing Data Structures Real-world vs. theoretical performance may differ for data structures Bench mark and experiment Simple is usually better than complex Different arches may alter performance Compilers, libraries and flags may affect results!

35 Sample Application Using FTP

36 Application For FTP FTP in a nutshell A simplistic, but relatively non-trival application Requires two ports One for the Control Channel One for the Data Channel Two modes, active and passive Clear text (ASCII) No encryption or security Other than plain text password authentication

37 Before Coding & Development Research potential issues Review RFCs, wikipedia, and wireshark captures Create application requirements Determine and write configuration schema XML for example Determine options for sanity checking Specification verification and validation Application specific actions Drop and generate alert for example

38 RFC Research Example After reviewing the FTP RFC: Different login/connection use case Request/response format With a special case of a response to a response Two different methods of network communication Active vs. Passive Allow anonymous logins

39 RFC Commands Notice that Connection Estab does NOT have a command! Everything else has a command and a response Logout has a third case Command->response-> response

40 RFC Communication Modes

41 Example 10k Requirements 1. Requests shall only be allowed from a connected session 2. Allowed responses shall only be allowed following a request 3. Connections must be tracked and stored in a ring-buffer 4. Shall allow possibility and tracking of a third succeeding message Request -> response -> succeeding response

42 Example Configuration XML Schema <ftpsettings name="ftpsettings"> <allowanonymousconns>true</allowanonymousconns> <allowpassiveconns>true</allowpassiveconns> <allowactiveconns>true</allowactiveconns> </ftpsettings> <connectionestablishment name="connectionestablishment"> <replycodes> <replycode>220</replycode> <replycode>421</replycode> <replycode>120</replycode> <succeedingcodes> <replycode>220</replycode> <succeedingcodes> <replycode>220</replycode> <replycode>421</replycode> </replycodes> </connectionestablishment> <ftpprotocol name="ftpprotocol"> <commandcodes> <commandcode>user</commandcode> <replycodes> <replycode>230</replycode>

43 Wireshark Passive Port hop information Active

44 Setting Up Libmnl and Example App Install libmnl Clone GIT repository./configure; make; sudo make install Install IPtables NFQUEUE Install xxxx iptables-dev & libnfnetlink-queue Verify by running: iptables A FORWARD p all j NFQUEUE queue-num 21 Lsmod grep ip

45 The Basic Application There is a bare-bones example in libmnl/examples/netfilter

46 Retrieving Packet Data and Length If your callback has been called (queue_cb()) mnl_attr_parse(nlh, sizeof(struct nfgenmsg), parse_attr_cb, tb); if (tb[nfqa_packet_hdr]) { ph = mnl_attr_get_payload(tb[nfqa_packet_hdr]); id = ntohl(ph->packet_id); } /** Retrieve payload size */ pkt_size = mnl_attr_get_payload_len(tb[nfqa_payload]); /** If there is a payload (tb[nfqa_payload]), lets setup the pointers!*/ if (tb[nfqa_payload]) { unsigned char *pkt = mnl_attr_get_payload(tb[nfqa_payload]);

47 Get Em Protocol Structures!!! Always ask yourself when using pointers: Are you bounds checking? Is it NULL? There is no Ethernet header in NFQUEUE (by default) */ ip_ptr = (struct iphdr *)((u_int8_t *) pkt); u_int8_t ip_len = (ip_ptr->ihl * 4); tcp_ptr = (struct tcphdr *)((u_int8_t *) ip_ptr + ip_len); /** If we have data, set the pointers */ if (tcp_ptr->doff!= 0) { payload_ptr = (u_int8_t *) tcp_ptr + (tcp_ptr->doff * sizeof(u_int32_t)); payload_len = pkt_size - (ip_len + tcp_ptr->doff * 4); }

48 Process Packet

(without lengths) Solution: Evaluate for special cases

49 Multi-parts Without Spec There is more than a simple request and response system: Multi-part responses (without lengths) Solution: Evaluate for special cases and if request/response is true; insert and track subsequent connections.

50 Determining Requests/Responses A request is <=4 alphabetical chars May not be in all of the same case Usually followed by a space (0x20) A response is a numeric 3 digit value May be followed by a space or CRLF (0x0d0a)

51 Sanity Checking Data Should be: User-configurable Realistic (some things vs everything) Sometimes flexible Verify protocol specification adherence

52 Application Actions Ultimately, your application will need to: Drop or allow packets Generate human readable events (messages) * May send responses to close a connection or prevent retransmits * Potentially custom exception messages TCP or stream Resets ;)

53 Application Code Please see companion ftp-application code Iptable functionality is missing IE. Port hopping and rules for the data channel Checking for response from correct side of connection not implemented IE. A server s response is actually from a response

54 Tools and Performance

55 Where is App Performance Lost? In your application: Intensive sanity checking (strings/regexs) Data structures tracking connections Or requests/responses Extraneous memsets and memcpys Byte misalignment Generating alerts and crafting packets Stitching frames together Storing streams for store and forward

56 Where is App Performance Lost? In the Linux Kernel: Data copy from kernel to user space Receiving the verdict from the kernel NF Bridge code Xtables code (iptables etc ) Ebtables (ewwww) Skbuff Driver code

57 Enhancing Performance Improving performance options: Custom kernels & profiling MMAP netlink I/O (3.10+) Reducing messages from kernel Better quality NICs and drivers CPU fanout and pinning Writing DPI in a logical manner Parameter tunning (kernel, network stack)

58 Enhancing Performance Continued May also be architecture specific Specialized microcode/assembly Rewriting functionality to gain that last 10% Understanding the compiler to write efficient code Ex. PPC vs. ARM vs. X86 Parallel programming Synchronizing data and conntracking will be hard Response before Request for example!

59 Useful Tools Not just limited to performance: Cachegrind & valgrind suites Address Sanitizer Gprof/oprofile Tcpreplay/tcpdump Wireshark Bit twist Scapy Reverse engineering may help you optimize your code EX. Sometimes GCC likes to do memcpys when all it has to do is move a pointer

60 If (!not_enough_perf) If all else fails: Move DPI into kernel and be infected And or improve xtables/queueing in kernel Faster and better hardware Reduce functionality FPGAs and TCAMs Expensive and have their own challenges

61 Summary Research your protocol and edge-cases Successful app requires knowledge! Expect odd non-conforming behaviors Set realistic performance expectations Performance is mediocre without kernel improvement Write your application for others and future Modular design for edge cases Catch errors before they happen! Profile your code! Open source vs. proprietary Be aware of your industry

62 Resources Reading Hex Dumps Without Wireshark NFQUEUE example

63 QUESTIONS?

Some of the slides borrowed from the book Computer Security: A Hands on Approach by Wenliang Du. Firewalls. Chester Rebeiro IIT Madras

Some of the slides borrowed from the book Computer Security: A Hands on Approach by Wenliang Du Firewalls Chester Rebeiro IIT Madras Firewall Block unauthorized traffic flowing from one network to another