Linux Systems Security Logging and Network Monitoring NETS1028 Fall 2016
Monitoring Monitoring can take many forms, from passive periodic inspection to realtime intrusion detection For this unit, we will consider it in the form where we analyze logs produced by various systems and services to identify concerns and possible solutions
Logging 3 main alternatives exist for Linux message logging: syslog syslog-ng rsyslog The default configurations for all of them log messages to files in /var/log Programs send messages to the logging service by sending them to the /dev/log socket file, syslog daemons read messages from this socket in real time Configuration files allow messages to be directed a number of places, including to other log servers
syslog The syslog project started in 1980 to provide a unified method for daemons to save messages for later examination syslog uses the concept of labelling messages with a source and priority (i.e. facility.level) and then writing them to text files in / var/log syslog runs over UDP for speed and simplicity syslogd is still around and uses port 514, included in Apple products and others Configured in /etc/syslog.conf, it supports the logger command to manually send messages
syslog-ng A project started in 1998 to offer an enhanced system logging service Supports basic logging like syslog, but adds a number of new capabilities including reliable message delivery, transport security using TLS, database support for message stores, message parsing/filtering/rewriting/ classification A number of the most interesting features of syslog-ng are only available in the paid version (see balabit.com)
rsyslog Created in 2004 to address the shortcomings of the original syslog and to create an alternative to syslog-ng because the most interesting tech in syslog-ng is not available in the OSS version and is unlikely to be folded back into the main branch even if mods are submitted to balabit rsyslog can use standard syslog configuration files for backwards compatibility Enhancements to syslog found in rsyslog include improved timestamping, reliable transport using TCP and TLS, database message store support, RELP/BEEP support, message buffering, and systemd logging
rsyslog Configuration /etc/rsyslog.conf is the main config file and sets global parameters /etc/rsyslog.d/*.conf are additional service specific configuration files (e.g. ufw, postfix) There are man pages and rsyslog.com has many sample configs The config file language is not friendly to humans
rsyslog Tour Examine the rsyslog.conf and rsyslog.d/*.conf config files Compare the kernel ring buffer (e.g. dmesg) to the kernel messages log file Can you figure out your sshd access history from the log files?
Logwatch Logwatch parses log files and extracts summary reports, based on config files in /etc/logwatch/conf/ and /usr/share/logwatch The default is text format and output on the terminal Commonly implemented as automated daily reports sending email summaries Shows software changes, hardware changes, user changes, sudo usage, services access, kernel errors, storage usage, and whatever else you add config files for Lots of default config files are in /usr/share/logwatch/ default.conf
Logwatch Example Install logwatch and make the cache directory which the install script doesn't make apt update ; apt install logwatch ; mkdir /var/cache/logwatch Make an override config file for any of the default ones you want to modify cp /usr/share/logwatch/default.conf/logwatch.conf /etc/logwatch/conf/ Logwatch has a number of useful options logwatch --range all logwatch --range 'since last week' logwatch --logfile secure --logfile http --range all --detail high Logwatch can be added to cron easily, some package builds create /etc/ cron.daily/00logwatch automatically for you which is a script instead of just a command line to run at specific times echo "59 23 * * * logwatch -range 'since yesterday' --format html --output mail" crontab - crontab -l
LogAnalyzer GPL'd realtime webserver-based log analysis tools Download the package from http:// loganalyzer.adiscon.com/downloads/ and extract it Install instructions are in the file named INSTALL, mostly just needs to have the files copied to a web server document store
LogAnalyzer Lab Install apache2 (i.e. apt install apache2) Download the loganalyzer package Follow the INSTALL instructions to set up the loganalyzer tools using a file source from /var/log
Loganalyzer on Ubuntu 14.04 with MySQL Example Install apache2 php mysql-server rsyslog-mysql php-mysql php-gd loganalyzer (use dbconfig-common when asked), this is loganalyzer 3.6.6, one release back and slightly broken if you install it on something newer than 14.04 If the rsyslog-mysql install didn't do it for you, run the mysql command to create the Syslog database using the SQL from https://github.com/rsyslog/rsyslog/raw/master/plugins/ ommysql/createdb.sql Restart the rsyslog service to start using the mysql database you just created Rename the existing config.php in /etc/loganalyzer and create a new empty one owned by www-data so loganalyzer will self-configure Open http://yourip/loganalyzer and run through the configure steps using the user database in Step 3, database name Syslog, database user root, with the password you set up when you installed mysql, check Require user to be logged in Choose mysql native source in step 7, and set database Syslog, table name SystemEvents, and set the user login and password
Example Syslog Appliance Download from http://www.syslogappliance.de/en/ and open in VMWare Login as root/appliance on the console and answer a few trivial questions, skip the upgrade choices, there is some doubt as to whether this is being maintained Remove the extra user "appliance" Point your browser at the VM IP and you're good to go! Reconfigure your VM resources if you want to log non-trivial message volumes