websnort Documentation

Transcription

1 websnort Documentation Release 0.8 Steve Henderson Jul 04, 2018

2

3 Contents 1 Features 3 2 Contents 5 3 Issues 15 Python Module Index 17 i

4 ii

5 Websnort is an Open Source web service for analysing pcap files with intrusion detection systems such as snort and suricata. It allows multiple configurations of IDS setups and rulesets to be defined for running against submitted samples. Its primary use case is for analysing short network captures from sandboxes and honeypots but can be used in any scenario where there is a need to scan pcap samples. This guide will explain how to deploy websnort in different environments and example configurations. Source code for websnort is hosted on GitHub. Any bug reports or feature requests can be made using GitHub s issues system. Contents 1

6 2 Contents

7 CHAPTER 1 Features Support for Suricata and Snort Easy to extend support for other intrusion detection systems Parallel execution of multiple configurations and rulesets Simple Web API for integrating with other systems 3

8 4 Chapter 1. Features

9 CHAPTER 2 Contents 2.1 Installation Dependencies websnort relies on a user already having one or more functioning IDS installations on their deployment host. For help with installing snort please follow their documentation. For help with installing suricata please follow their documentation. For Linux operating systems, packaged versions of these applications may already be available in your system software repository/library Install with Pip The simplest way to install is using the pip package install utility. This will ensure all python dependencies are downloaded/installed appropriately: pip install websnort It is recommended to use virtualenv to keep third-party packages isolated from system python packages. However, if installing system wide you will need to run pip as root/sudo Run from Source The latest code can be run directly by cloning the GitHub repository: git clone 5

10 2.1.4 Configuration The default config for websnort is setup to interface with a snort deployment on Ubuntu/Debian, using the ruleset referenced by /etc/snort/snort.conf. To customise the setup you can override the websnort config file by creating a new config file (in order of loading precedence): ~/.websnort/websnort.conf /etc/websnort/websnort.conf Look at the example config files provided in websnort/conf for other common configurations. The config file format is as follows: [websnort] # Comma-separated list of config sections/instances to run ids = snort [snort] # python ids module name/type to use module = snort # name to give the ruleset in results ruleset = community # path to snort binary, will search path if not absolute path = snort # snort rules config file location config = /etc/snort/snort.conf # any additional command line args to include extra_args = Inbuilt Webserver websnort uses the python bottle framework to provide its web interface. This provides the ability to run a simple webserver from the command-line. usage: websnort [-h] [-H HOST] [-p PORT] optional arguments: -h, --help show this help message and exit -H HOST, --host HOST Web server Host address to bind to -p PORT, --port PORT Web server Port to bind to By default the webserver will bind to all network interfaces and run on port To run on a different port number: websnort -p 8000 You will need to ensure the user that you are running the webserver as, has the appropriate permissions to run snort/suricata from the command-line and can read any applicable config files Python WSGI websnort also provides an entrypoint for interfacing with other webservers that support python WSGI. An example httpd config for apache could look something like the following: 6 Chapter 2. Contents

11 <VirtualHost *:80> ServerName ServerAlias example.com ServerAdmin WSGIDaemonProcess example.com processes=3 threads=1 display-name=%group} WSGIProcessGroup example.com WSGIScriptAlias / /usr/lib/python/site-packages/websnort/web.py <Directory /usr/lib/python/site-packages/websnort> Order allow,deny Allow from all </Directory> </VirtualHost> See QuickConfigurationGuide for more information on setting up modwsgi with Apache. 2.2 Usage Interactive Use a web browser and navigate to the address the websnort webserver is listening on. In these examples, the default URL is used. Simply select a pcap file for analysis and choose Submit Usage 7

12 If the server s configuration is correct, in a few seconds you should see the results of any alerts returned. 8 Chapter 2. Contents

13 Tabs are presented for each IDS configuration that was executed. A summary count of any produced alerts is visible in the tabs labels Web API The same functionality is exposed via a json web api. Full details of the api can be found by navigating to http: //localhost:8080/api. To submit a pcap file for analysis, perform a HTTP multipart form POST to For example, using curl: $ curl -i --form file=@zeus-sample.pcap HTTP/ Continue HTTP/ OK Content-Length: 5830 Content-Type: application/json Date: Mon, 08 Jun :18:42 GMT "status": "Success", "errors": [], "apiversion": "0.5", "filename": "zeus-sample.pcap", (continues on next page) 2.2. Usage 9

14 "start": " T13:18: ", "filesize": , "duration": , "analyses": [ "status": "Success", "name": "snort_vrt", "alerts": [ "source": " :80", "classtype": "Misc activity", "protocol": "TCP", "sid": 2925, "timestamp": " T16:17: ", "message": "INFO web bug 0x0 gif attempt", "destination": " :1089", "revision": 3 }, "source": " :80", "classtype": "Misc activity", "protocol": "TCP", "sid": 2925, "timestamp": " T16:17: ", "message": "INFO web bug 0x0 gif attempt", "destination": " :1087", "revision": 3 }, "source": " :80", "classtype": "Misc activity", "protocol": "TCP", "sid": 2925, "timestamp": " T16:17: ", "message": "INFO web bug 0x0 gif attempt", "destination": " :1091", "revision": 3 }, "source": " :80", "classtype": "Misc activity", "protocol": "TCP", "sid": 2925, "timestamp": " T16:17: ", "message": "INFO web bug 0x0 gif attempt", "destination": " :1098", "revision": 3 }, "source": " :80", "classtype": "Misc activity", "protocol": "TCP", "sid": 2925, "timestamp": " T16:17: ", "message": "INFO web bug 0x0 gif attempt", "destination": " :1087", "revision": 3 (continued from previous page) (continues on next page) 10 Chapter 2. Contents

15 ", }, Download", }, }, "source": " :80", "classtype": "Misc activity", "protocol": "TCP", "sid": 2925, "timestamp": " T16:17: ", "message": "INFO web bug 0x0 gif attempt", "destination": " :1087", "revision": 3 "source": " :80", "classtype": "Misc activity", "protocol": "TCP", "sid": 2925, "timestamp": " T16:17: ", "message": "INFO web bug 0x0 gif attempt", "destination": " :1119", "revision": 3 } ], "module": "snort", "version": " GRE (Build 47)", "ruleset": "Sourcefire VRT", "duration": (continued from previous page) "status": "Success", "name": "suricata_et", "alerts": [ "source": " :1081", "classtype": "A Network Trojan was detected", "protocol": "TCP", "sid": , "timestamp": " T16:16: ", "message": "ET CURRENT_EVENTS Zbot Generic URI/Header Struct.bin }, }, "destination": " :80", "revision": 4 "source": " :1081", "classtype": "A Network Trojan was detected", "protocol": "TCP", "sid": , "timestamp": " T16:16: ", "message": "ET TROJAN PRG/wnspoem/Zeus InfoStealer Trojan Config "destination": " :80", "revision": 11 "source": " :1120", "classtype": "A Network Trojan was detected", "protocol": "TCP", (continues on next page) 2.2. Usage 11

16 } ", Download", }, (continued from previous page) "sid": , "timestamp": " T16:17: ", "message": "ET CURRENT_EVENTS Zbot Generic URI/Header Struct.bin "destination": " :80", "revision": 4 "source": " :1120", "classtype": "A Network Trojan was detected", "protocol": "TCP", "sid": , "timestamp": " T16:17: ", "message": "ET TROJAN PRG/wnspoem/Zeus InfoStealer Trojan Config "destination": " :80", "revision": 11 } ], "module": "suricata", "version": "1.4.7 RELEASE", "ruleset": "Emerging Threats", "duration": } ], "md5": "266c1cabfae4c66dc05443eaeaa054e0" 2.3 Troubleshooting Why do I get a permission denied error from websnort? You need to ensure that snort/suricata can run as the same user running the web application. In particular check that all config files are readable by the web user. On Ubuntu recent packages of snort deploy /etc/snort/snort.conf as root readable only. If this is the case try: sudo chmod a+r /etc/snort/snort.conf It is also worth testing without using the web application, by attempting to run the snort/suricata command-line as the web user, manually on the command-line to verify it produces the expected results. For example: snort -r /tmp/test.pcap -c /etc/snort/snort.conf -A console -l /tmp Why doesn t websnort show the alerts I expect? If you expect the pcap you are submitting to generate alerts and it doesn t, verify that the IDS generates the expected alerts from the command-line as the webapp user. For example: 12 Chapter 2. Contents

17 snort -r /tmp/test.pcap -c /etc/snort/snort.conf -A console -l /tmp If this is not working you may want to disable checksum validation for the IDS, especially if the pcaps were generated from a virtual network/sandbox or replay tool. For example, in /etc/snort/snort.conf add: validate_checksums off Or in /etc/suricata/suricata.yaml change: stream: memcap: 128mb checksum-validation: no If your pcaps have some unusual VLAN tagging and you are running Suricata, you may want to try disabling VLAN tracking in the sessionisation. For example in /etc/suricata/suricata.yaml change: vlan: use-for-tracking: false Websnort still doesn t work what should I do? If you have read through the relevant sections of the documentation but are still having problems, please raise an issue on the project s issue tracker and someone may be able to assist. 2.4 Development If you are interested in contributing to the project please read through the following sections Websnort API The codebase is quite simple with the execution logic defined in websnort.runner: websnort.runner.is_pcap(pcap) Simple test for pcap magic bytes in supplied file. Parameters pcap File path to Pcap file to check Returns True if content is pcap (magic bytes present), otherwise False. websnort.runner.run(pcap) Runs all configured IDS instances against the supplied pcap. Parameters pcap File path to pcap file to analyse Returns Dict with details and results of run/s Bottle App And the web handling, bottle routes defined in websnort.web: 2.4. Development 13

18 websnort.web.home() Main page, displays a submit file form. websnort.web.api_submit() Blocking POST handler for file submission. Runs snort on supplied file and returns results as json text. websnort.web.submit_and_render() Blocking POST handler for file submission. Runs snort on supplied file and returns results as rendered html. IDS Plugins Interfacing with other IDS systems is possible by implementing a new plugin. The plugin can either be statically registered in websnort.plugins.registry or hooked in at install time by defining the correct setuptools entrypoint in your project. See websnort.runner.idsrunner for expected class API. class websnort.plugins.idsrunner(conf ) run(pcap) Run the IDS over the supplied pcap. Parameters pcap File path to Pcap for analysis. Returns A tuple of version, alerts list Pull Requests If you wish to contribute a bug fix or feature, please open a pull request on the GitHub project page for discussion/review. While not strictly enforced, the code-style should follow python PEP8 standard Licensing All contributions to the project are to be made under the terms of the GNU Public License v3. Copyright of any contributions remain the property of the original authors. If there are significant community contributions to the project we will look at updating the copyright headers of the project to make it clear that the project copyright and ownership is that of all said community developers Issues If you have encountered a problem or need help in some aspect of the project you are probably not alone. Please raise an issue in the issue tracker on the project s GitHub page so other users can benefit from the answers too. 14 Chapter 2. Contents

19 CHAPTER 3 Issues If you encounter problems with websnort, please refer to the Troubleshooting section of the documentation. 15

20 16 Chapter 3. Issues

21 Python Module Index w websnort.plugins, 14 websnort.runner, 13 websnort.web, 13 17

22 18 Python Module Index

23 Index A api_submit() (in module websnort.web), 14 H home() (in module websnort.web), 13 I IDSRunner (class in websnort.plugins), 14 is_pcap() (in module websnort.runner), 13 R run() (in module websnort.runner), 13 run() (websnort.plugins.idsrunner method), 14 S submit_and_render() (in module websnort.web), 14 W websnort.plugins (module), 14 websnort.runner (module), 13 websnort.web (module), 13 19