15-Minute Linux DFIR Triage. Dr. Phil Polstra Bloomsburg University of Pennsylvania

Transcription

1 15-Minute Linux DFIR Triage Dr. Phil Polstra Bloomsburg University of Pennsylvania

2 What is this talk about? Determining with some certainty if you have been hacked In a matter of minutes With minimal disturbance to the subject system Automating the process with shell scripting

3 Why should you care? Someone calls you about a suspected breach You need to need to figure out if they were hacked Quickly so as to avoid further harm to your client Without destroying evidence Without taking down a critical machine

4 Roadmap Opening a case Talking to the users Mounting known-good binaries Minimizing disturbance to the system Collecting Data Automation with scripting Next steps if there is a breach

5 Opening a Case Decide on case name (date?) Create a case folder on your laptop Start making entries in your notebook Bound notebook with numbered pages Easy to carry Hard to insert/remove pages No batteries required

6 First Talk to the Users They know the situation better than you Might be able to tell a false alarm before digging in Why did you call me? What they suspect No internal experts or policy to use outsider? Why do you think there was an incident? Everything they know about the subject system

7 USB Response Drive Contains known-good binaries Minimum /bin, /sbin, /lib for same architecture Might also grab /usr/sbin, /usr/bin, /usr/lib Must be on an ext2, ext3, or ext4 partition Could contain a bootable Linux on another partition This partition will probably be FAT Should be first partition See Chapter 1

8 Mounting Known-Good Binaries Insert response drive Exec your bash binary Set path to your binaries (and only your binaries) Set LD_LIBRARY_PATH Run all shell scripts as bash <script> Don't use she-bang (#!) in scripts!

9 Demo: Mounting Binaries

10 Minimize Disturbance to System You will always change the system a little Goal is to Minimize memory footprint Never write to subject media Two basic options Save everything to your USB response drive Send it over the network

11 Sending data over the network Better than USB drive due to caching Use netcat Create a listener for log information on forensics workstation Send log information from client Also create a listener for interesting files on forensics workstation Spawn a new listener when files are sent

12 Setting Up Log Listener netcat -k -l 9999 >> case-log.txt (-k) keep alive (-l) listen (>>) append From subject {command} netcat {forensic ws IP} 9999 Let's use shell scripting to automate this Shell not Python because we want to minimize memory footprint

13 Automating the Log Listener usage () { #if the directory doesn't exist create it echo "usage: $0 <case number>" if [! -d $1 ] ; then echo "Simple script to create case folder and start listeners" mkdir $1 fi exit 1 } # create the log listener `nc -k -l 4444 >> $1/log.txt` & if [ $# -lt 1 ] ; then usage echo "Started log listener for case $1 on $(date)" nc localhost 4444 else echo "Starting case $1" # start the file listener fi `./start-file-listener.sh $1` &

14 Automating the Log Client-Part 1 usage () { if [ $# -gt 2 ] ; then echo "usage: $0 <IP> [log port] [fn port] [ft port]" exit 1 } export RFPORT=$3 else export RFPORT=5555 # did you specify a file? if [ $# -lt 1 ] ; then usage fi fi if [ $# -gt 3 ] ; then export RFTPORT=$4 export RHOST=$1 if [ $# -gt 1 ] ; then export RPORT=$2 else export RFTPORT=5556 else export RPORT=4444 fi fi

15 Automating the Log Client Part 2 # defaults primarily for testing [ -z "$RHOST" ] && { export RHOST=localhost; } [ -z "$RPORT" ] && { export RPORT=4444; } usage () { echo "usage: $0 <command or script>" echo "Simple script to send a log entry to listener" exit 1 } # did you specify a command? if [ $# -lt 1 ] ; then usage else echo -e "++++Sending log for $@ at $(date) ++++\n $($@) \n----end----\n" nc $RHOST $RPORT fi

16 Automating Sending Files Listener on forensics workstation listens for file name When a new file name is received Create a new listener to receive file Redirect file to one with correct name Also log in the main case log (optional) On the client side File name is sent After brief pause send file to data listener port

17 Automating the File Listener usage () { while true echo "usage: $0 <case name>" do echo "Simple script to start a file listener" exit 1 filename=$(nc -l 5555) } nc -l 5556 > $1/$(basename $filename) # did you specify a file? done if [ $# -lt 1 ] ; then usage fi

18 Automating the File Client # defaults primarily for testing # did you specify a file? [ -z "$RHOST" ] && { export RHOST=localhost; } if [ $# -lt 1 ] ; then usage [ -z "$RPORT" ] && { export RPORT=4444; } [ -z "$RFPORT" ] && { export RFPORT=5555; } fi [ -z "$RFTPORT" ] && { export RFTPORT=5556; } #log it usage () { echo "usage: $0 <filename>" echo "Simple script to send a file to listener" exit 1 } echo "Attempting to send file $1 at $(date)" nc $RHOST $RPORT #send name echo $(basename $1) nc $RHOST $RFPORT #give it time sleep 5 nc $RHOST $RFTPORT < $1

19 Cleaning Up # close the case and clean up the listeners echo "Shutting down listeners at $(date) at user request" nc localhost 4444 killall start-case.sh killall start-file-listener.sh killall nc

20 Collecting Data Date (date) Clock skew on subject Time zone on subject Kernel version (uname -a) Needed for memory analysis Might be useful for researching vulnerabilities

21 Collecting Data (continued) Network interfaces (ifconfig -a) Any new interfaces? Strange addresses assigned? Network connections (netstat -anp) Connects to suspicious Internet addresses? Strange localhost connections? Suspicious ports? Programs on wrong ports (i.e malware on port 80)

22 Collecting Data (continued) Open files (lsof -V) What programs are using certain files/ports Might fail if malware installed Running processes (ps -ef and/or ps -aux) Things run as root that shouldn't be No login accounts logged in and running things Might fail if malware installed

23 Collecting Data (continued) Routing info (netstat -rn and route) Routed through new interface New gateways Conflicting results = malware Failure to run = malware

24 Collecting Data (continued) Mounted filesystems (mount and df) Things mounted that shouldn't be (especially tempfs) Mount options that have changed Filesystem filling up Disagreement = malware Kernel modules (lsmod) New device drivers Modules that have changed

25 Collecting Data (continued) Who is logged in now (w) System accounts that shouldn't be allowed to login Who has been logging in (last) System accounts that shouldn't be allowed to login Accounts that don't normally use this machine Failed logins (lastb) Repeated failures then success = password cracked

26 Collecting Data (continued) User login info (send /etc/passwd and /etc/shadow) Newly created login System accounts with shells and home directories Accounts with ID 0 Accounts with passwords that shouldn't be there

27 Putting It Together with a Script usage () { # now collect some info! echo "usage: $0 [listening host]" send-log.sh date echo "Simple script to send a log entry to listener" send-log.sh uname -a exit 1 send-log.sh ifconfig -a } send-log.sh netstat -anp send-log.sh lsof -V # did you specify a listener IP? if [ $# -gt 1 ] [ "$1" == "--help" ] ; then usage fi send-log.sh ps -ef send-log.sh netstat -rn send-log.sh route send-log.sh lsmod send-log.sh df send-log.sh mount # did you specify a listener IP? send-log.sh w if [ "$1"!= "" ] ; then send-log.sh last source setup-client.sh $1 fi send-log.sh lastb send-log.sh cat /etc/passwd send-log.sh cat /etc/shadow

28 Running the Initial Scan

29 Have I been hacked?

30 Who is Johnn? /etc/passwd

31 Why do these accounts have passwords? /etc/shadow

32 Who's been logging in? Results from last

33 Who failed to login? Results from lastb

34 Looks Like They Were Hacked Now What?

35 Live Analysis Use techniques described to Grab file metadata for key directories (/sbin, /bin, etc.) Grab users' command history Get system log files Get hashes of suspicious files Dump RAM Must compile LiME (off subject machine!) Risky can cause a crash

36 Dead Analysis Unless the machine absolutely cannot be taken offline it is strongly recommended that you shut it down and get a filesystem image If you cannot shutdown the machine You can still get a filesystem image with dcfldd You probably cannot use this evidence in court

37 More on Dead Analysis Filesystem analysis is much more mature and powerful than memory analysis The Linux support in Volatility is somewhat lacking Relatively new addition to a new tool Seems to fall down a lot with late 3.x and 4.x kernels None of the investigators I've talked to could come up with a case where evidence existed only in memory

38 Finding Out More Heard there was a new book out (1 kg+ of knowledge) Resources on Harass me on

39 Questions?