Evolutionary Kernel Fuzzing Black Hat USA 2017 Richard Johnson

Transcription

1 Evolutionary Kernel Fuzzing Black Hat USA 2017 Richard Johnson

2 Evolutionary Kernel Fuzzing Richard Johnson Black Hat USA 2017

3 whoami Richard Johnson Research Cisco Talos Vulndev Third party vulnerability research Microsoft Apple Oracle Adobe Google IBM, HP, Intel Security tool development Fuzzers, Crash Triage Mitigation development Special Contributor Andrea Allievi, Microsoft

4 Introduction High performance tracing and fuzzing since High Performance Fuzzing Input selection Engine design AFL-DYNINST Windows fork() 2015 Go Speed Tracer Guided Fuzzing Binary translation Hardware tracing

5 Introduction High performance tracing and fuzzing since Harnessing Intel Processor Trace for Vulnerability Discovery Intel Processor Trace internals Usermode fuzzing with Intel Processor Trace Persistent mode fuzzing native Windows binaries In June 2016 we opensourced Windows driver for Intel Processor Trace

6 Introduction Today we will bring this knowledge to the context of fuzzing the Windows kernel See for previous slides and talk videos

7 Introduction Agenda Evolutionary Fuzzing Kernel Code Coverage Linux Kernel Fuzzing Windows Kernel Fuzzing Goals Understand the benefits of guided fuzzing Understand coverage collection techniques for kernels Identify critical Windows Kernel attack surface Learn how to apply state of the art fuzzing to kernels

8 Introduction Kernels are a critical attack surface Modern mitigations utilize isolation and sandboxing Weaponized exploits include kernel attacks Pwn2own Leaked government warez Kernel vulndev is still in its infancy Room for improvment on fuzzing tech

9 Introduction Application Sandboxing IE sandbox IE Protected Mode Chrome sandbox Adobe Reader sandbox etc Windows Isolation / Sandboxing Driver Signature Verification Patchguard / Kernel Patch Protection AppContainers ProcessMitigationPolicy etc

10 Introduction Prior Windows Kernel vulndev by the following people Ilja van Sprundel Mateusz Jurczyk Jesse Hertz Tim Newsham Nils Georgi Geshev James Loureio Peter Hlavaty Daniel King Marco Grassi Nikita Tarakanov

11 Evolutionary Fuzzing

12 Evolutionary Fuzzing History 2006: Sidewinder Sparks & Cunningham 2007: Evolutionary Fuzzing System Jared Demott 2007: Bunny the Fuzzer Michal Zalewski 2013: American Fuzzy Lop Michal Zalewski 2014: Nightmare/BCCF Joxean Koret 2015: Honggfuzz Robert Swiecki 2015: covfuzz Atte Kettunen 2016 : Choronzon Zisis Sialveras / Nikos Naziridis

13 Evolutionary Fuzzing Incrementally better mutational dumb fuzzing Trace while fuzzing and provide feedback signal Evolutionary algorithms Assess fitness of current input Manage a pool of possible inputs

14 Evolutionary Fuzzing Required Fast tracing engine Block granularity code coverage Fast logging Memory resident coverage map **Not a list of every basic block Fast evolutionary algorithm Minimum of global population map Maximum pool diversity

15 Desired Portable Easy to use Helper tools Grammar detection Evolutionary Fuzzing

16 Evolutionary Fuzzing AFL delivers the most complete package Lets review!

17 Amercian Fuzzy Lop Michal Zalewski 2013 Delivered the first performant opensource evolutionary fuzzer Features Variety of traditional mutation strategies Block coverage via compile time instrumentation Simplified approach to genetic algorithm Edge transitions are encoded as tuple and tracked in a bloom filter Includes coverage and frequency Uses portable* Posix API for shared memory, process creation

18 Amercian Fuzzy Lop Contributions Tracks edge transitions Not just block entry Global coverage map Generation tracking Fork server Reduce target initialization Persistent fuzzing Builds corpus of unique inputs reusable in other workflows

19 Trace Logging Each block gets a unique ID Amercian Fuzzy Lop Traversed edges are indexed into a byte map (bloom filter) Create a hash from the src and dst block IDs Increment map for each time an edge is traversed Each trace is easily comparable to the entire session history

20 Okay, so lets take a fuzzer that targets userland programs with source code and make it work for closed source Windows kernel targets! But first how about Windows binaries

21 WinAFL Ivan Fratric First performant windows evolutionary fuzzer Features Its American Fuzzy Lop! For Windows! Windows API port for memory and process creation DynamoRIO based code coverage Filter based on module Block and Edge tracing modes Persistent execution mode

22 WinAFL-IntelPT Richard Johnson First hardware assisted guided fuzzer for Windows First public guided fuzzer for Windows kernel Features Intel Processor Trace based coverage engine Online disassembly engine to decode Intel PT trace Filter based on module Edge tracing mode Persistent execution mode Kernel tracing mode

23 Kernel Code Coverage

24 Kernel Code Coverage Kernel code coverage can be elusive to obtain Opensource code can be instrumented by compilers Binary code must use runtime instrumentation, static rewriting, or hardware engines

25 Kernel Code Coverage Existing tools and approaches Source GCC gcc --coverage AFL adds hooks into the.s intermediate files Clang clang -fprofile-instr-generate -fcoverage-mapping afl-clang-fast uses a compiler pass

26 Kernel Code Coverage Existing tools and approaches Binary QEMU Hook Tiny Code Generator (TCG) translates IR to native ISA BOCHS Seems to work for j00ru syzygy Statically rewrite PE32 binaries with AFL Requires symbols Requires additional dev to make WinAFL kernel aware

27 Kernel Code Coverage Intel / AMD CPUs Branch Trace Store Per-kernel-thread hardware trace Use in combination with Last Branch Record to get edge transition Supported passthrough by some hypervisors

28 Kernel Code Coverage Intel / AMD CPUs Branch Trace Store New opensource software recently released for Windows BTS

29 Kernel Code Coverage Intel CPUs Intel Processor Trace Introduced in Broadwell / Skylake

30 Kernel Code Coverage Intel CPUs Intel Processor Trace Performance Low over-head (15% CPU perf hit for recording) Logs directly to physical memory Bypass TLB and eliminating cache pollution Minimal log format One bit per conditional branch Only indirect branches log dest address Additional overhead to decode trace, requires disassembly See Harnessing Intel Processor Trace for Vuln Discovery for deep dive

31 Kernel Code Coverage Intel CPUs Intel Processor Trace Sparse binary packet format Complex format - decode with Intel s opensource libipt library!

32 Kernel Code Coverage We have contributed two opensource projects to harness Intel Processor Trace! Get the code! WindowsPtDriver Implements Intel Processor Trace support for Windows PtCov Intel Processor Trace Library Userland API for interacting with the kernel mode driver Easily turn any existing file fuzzer into coverage driven fuzzer

33 Kernel Code Coverage PtCov Intel Processor Trace Library typedef struct _PtCovConfig { int cpu_number; DWORD trace_buffer_size; DWORD trace_mode; char *trace_modules[4]; // trace up to four module names char **cov_map; // optional user supplied buffer for afl coverage map int cov_map_size; char *ptdump_path; // optional path for saving intel ptdump file to disk } PtCovConfig; PTSTATUS ptcov_init(); PTSTATUS ptcov_init_trace(ptcovconfig *ptcov_config, PtCovCtx *ptcov_ctx);

34 Kernel Code Coverage PtCov Intel Processor Trace Library PTSTATUS ptcov_set_cpu_number(ptcovctx ptcov_ctx, int cpu_number); PTSTATUS ptcov_set_cpu_affinity(ptcovctx ptcov_ctx, KAFFINITY cpu_affinity); PTSTATUS ptcov_set_process_handle(ptcovctx ptcov_ctx, HANDLE process_handle); PTSTATUS ptcov_get_process_handle(ptcovctx ptcov_ctx, HANDLE *process_handle); PTSTATUS ptcov_get_free_processor(ptcovctx ptcov_ctx, int *processor_number); PTSTATUS ptcov_add_target_module(ptcovctx ptcov_ctx, char *module_name); PTSTATUS ptcov_add_target_driver(ptcovctx ptcov_ctx, char *driver_name); PTSTATUS ptcov_trace_process(ptcovctx ptcov_ctx, HANDLE process_handle); PTSTATUS ptcov_trace_driver(ptcovctx ptcov_ctx);

35 Kernel Code Coverage PtCov Intel Processor Trace Library PTSTATUS ptcov_start_trace(ptcovctx ptcov_ctx); PTSTATUS ptcov_pause_trace(ptcovctx ptcov_ctx); PTSTATUS ptcov_resume_trace(ptcovctx ptcov_ctx); PTSTATUS ptcov_clear_trace(ptcovctx ptcov_ctx); PTSTATUS ptcov_end_trace(ptcovctx ptcov_ctx); PTSTATUS ptcov_get_afl_map(ptcovctx ptcov_ctx, char **map);

36 Kernel Code Coverage Other methods Single step / branch step (BTF) Int 0x1 enabled on each instruction to singlestep DbgCtrl MSR flag to interrupt only on branch PMU Sampling Can be forced to interrupt on each branch Asynchronous but slow Works everywhere (including ARM) Dynamic binary translation Attempts with PIN for drivers, not public

37 Demo Windows Kernel Code Coverage

38 Linux Kernel Fuzzing

39 Linux Kernel Fuzzing Trinity Built into the Linux kernel tree Type aware via templates Not coverage driven #include "sanitise.h" struct syscallentry syscall_shmat = {.name = "shmat",.num_args = 3,.arg1name = "shmid",.arg2name = "shmaddr",.arg2type = ARG_ADDRESS,.arg3name = "shmflg", }; Jones has considered feedback-guided fuzzing for Trinity in the past, but found the coverage tools that were available at the time to be too slow.

40 Linux Kernel Fuzzing Syzkaller Coverage driven system call fuzzing Uses built in GCC port of ASAN coverage gcc -fsanitize-coverage=trace-pc Exposes coverage via /sys/kernel/debug/kcov Template driven for system call fuzzing Relies heavily on KASAN to catch bugs write(fd fd, buf buffer[in], count len[buf]) pwrite64(fd fd, buf buffer[in], count len[buf], pos fileoff) writev(fd fd, vec ptr[in, array[iovec_in]], vlen len[vec]) pwritev(fd fd, vec ptr[in, array[iovec_in]], vlen len[vec], off fileoff) lseek(fd fd, offset fileoff, whence flags[seek_whence])

41 Syzkaller Good support tooling WebUI for monitoring Good logging Repro minimizer Linux Kernel Fuzzing

42 Syzkaller Very effective, but.. Complicated to get setup properly Complex workflow Not easily retargetable Linux Kernel Fuzzing

43 Linux Kernel Fuzzing TriforceAFL 2016 Tim Newsham & Jesse Hertz (NCC Group) AFL compatible QEMU based coverage fuzzer Added fork server to QEMU post-boot Added a great serialization technique for APIs Allows to fuzz APIs via a file format

44 Linux Kernel Fuzzing TriforceAFL 2016 Tim Newsham & Jesse Hertz (NCC Group) Extends QEMU trace support in AFL to target kernel COW fork() of QEMU after boot for performance Extends native ISA with custom hypercalls (aflcall) startforkserver getwork startwork endwork

45 Linux Kernel Fuzzing TriforceAFL 2016 Tim Newsham & Jesse Hertz (NCC Group) Uses syscall templates / shapes Serializes system calls into files to fuzz with AFL Supports sequences of system calls Syscall shapes buffer, len, int fd, buffer fd, buffer, buffer, int fd, buffer, buffer, int, int fd, buffer, int etc Syscall types Int Buffer BufferLength FileContents FileName FileTableNumber

46 Demo TriforceAFL

47 Windows Kernel Fuzzing

48 Kernel Attack Surface Kernels attack surface includes any untrusted input Userland System calls, file parsers, software interrupts Devices Network, USB, Firewire, etc Two categories: structured input or APIs

49 Windows Kernel Attack Surface System Calls ntoskrnl.sys Windows system services ~465 system calls win32k.sys Kernel mode Graphics Display Interface support ~1216 system calls

50 Windows Kernel Attack Surface win32k.sys File Parsers Fonts TTF, OTF, FON Images BMP, JPEG, CUR, ANI, ICO Metafiles EMF, WMF

51 Windows Kernel Attack Surface Other attack surface Graphics drivers Audio drivers Network drivers Print drivers See other publications for deeper dives into attack surface

52 Windows Kernel Fuzzing Legacy ioctlfuzzer Dimitry Oleksander (cr4sh) Misc Syscall fuzzers Misc file format fuzzers Techniques Random syscall arguments or ioctl input Hooking and interception (ioctlfuzzer) Dumb or structured file fuzzing

53 Windows Kernel Fuzzing KernelFuzzer 2016 James Loureiro and Georgi Geshev Windows system API fuzzer Techniques Type aware API fuzzing Manual definition of generators per-type Pre-generated HANDLE tables Outputs C code for each testcase to repro after crash

54 Windows Kernel Fuzzing KernelFuzzer 2016 James Loureiro and Georgi Geshev Windows system API fuzzer Observations Type aware API fuzzing is effective Manual definition of generators is tedious Can benefit from TriforceAfl style API sequence generation

55 Windows Kernel Fuzzing GOOD NEWS! API fuzzing has a type-aware strategy and tools BAD NEWS! IOCTLs and Graphics drivers are opaque blobs Sounds like we need for evolutionary fuzzing!

56 Windows Graphics Driver Fuzzing

57 Windows Graphics Driver Fuzzing Windows Graphics Hierarchy Gdi32.dll -> Dxgkrnl.sys -> HW driver Interesting Direct3D functions D3DKMTEscape D3DKMTRender D3DKMTCreateAllocation D3DKMTCreateContext etc

58 Windows Graphics Driver Fuzzing D3DKMTEscape NTSTATUS D3DKMTEscape( _In_ const D3DKMT_ESCAPE *pdata ); Entry point for internal graphics functionality Each driver implements a proprietary format for *pdata A few header fields and command data This is a perfect target for evolutionary file format style fuzzing

59 Windows Graphics Driver Fuzzing D3DKMTEscape NTSTATUS D3DKMTEscape( _In_ const D3DKMT_ESCAPE *pdata ); Entry point for internal graphics functionality Each driver implements a proprietary format for *pdata A few header fields and command data This is a perfect target for evolutionary file format style fuzzing

60 Windows Graphics Driver Fuzzing Search for usage of D3DKMTEscape: "C:\Program Files\Git\bin\bash.exe" export output="/tmp/dumpbin.txt" rm $output for i in `find. -type d` ; \ do echo $i ; dumpbin -imports "$i/*.exe" ; dumpbin "$i/*.dll" ; \ done tee $output export srch="dump D3DKMT" egrep $srch $output grep -B2 D3D Dump of file./displayswitch.exe B1 D3DKMTNetDispStopMiracastDisplayDevice AD D3DKMTNetDispQueryMiracastDisplayDeviceSupport Dump of file./igfxcuiservice.exe 65 D3DKMTCloseAdapter A7 D3DKMTOpenAdapterFromDeviceName 81 D3DKMTEscape Dump of file./proximityuxhost.exe AD D3DKMTNetDispQueryMiracastDisplayDeviceSupport

61 Windows Graphics Driver Fuzzing Search for usage of D3DKMTEscape: windbg> bp dxgkrnl!dxgkescape ".echo DxgkEscape; kb 50; g;" 3: kd> kb 30 # RetAddr : Call Site 00 fffff803`7800c413 : win32kbase!ntgdiddddiescape ffe`fc4644e4 : nt!kisystemservicecopyend+0x ffe`f8b69e68 : win32u!ntgdiddddiescape+0x ffe`ebb595f7 : d3d11!ndxgi::cdevice::escapecb+0x ` : igd10iumd64!openadapter10_2+0x64a7b7

62 Windows Graphics Driver Fuzzing Search for usage of D3DKMTEscape: windbg> bp dxgkrnl!dxgkescape "kb 50; g;" 00 fffff013`640870b9 : dxgkrnl!dxgkescape 01 fffff803`7800c413 : win32kbase!ntgdiddddiescape+0x ffe`fc4644e4 : nt!kisystemservicecopyend+0x ffe`f8b69e68 : win32u!ntgdiddddiescape+0x ffe`eb8cbc0a : d3d11!ndxgi::cdevice::escapecb+0x a0`7218e808 : 0x00007ffe`eb8cbc0a `3d9a5108 : 0x000000a0`7218e a0`7218e8a8 : 0x `3d9a ffe`f8b13c2c : 0x000000a0`7218e8a ffe`f8be28eb : d3d11!ndxgi::cdevice::driversupportsoverlays+0x9c 0a 00007ffe`f8bad13e : d3d11!ndxgi::cdevice::getinternalmultiplaneoverlaycaps+0xff 0b 00007ffe`fa232c2f : d3d11!dxrt11::direct3ddevice::release+0xcb8e 0c 00007ffe`fa2152ef : dxgi!atl::ccomobject<cdxgilightweightdevice>::release+0x135ef 0d 00007ffe`fa : dxgi!cdxgioutput::getmultiplaneoverlaycaps+0x9f 0e 00007ffe`f96214a3 : dxgi!cdxgiswapchain::getmultiplaneoverlaycaps+0x54 0f `41c71070 : 0x00007ffe`f96214a3

63 Windows Graphics Driver Fuzzing Intel HD Graphics Driver igdkmd64.sys 7.5 MB graphics driver This won t end well

64 Windows Graphics Driver Fuzzing TALOS (Piotr Bania) Intel HD Graphics Windows Kernel Driver (igdkmd64) RCE Vulnerability igdkmd64!hybdriverentry+1485b0 fffff801`61fd0920 ff call qword ptr [rax+250h] fffff801`61fb33b1 : igdkmd64!hybdriverentry+0x1485b0 fffff801`61ee4166 : igdkmd64!hybdriverentry+0x12b041 fffff801`61edfa4a : igdkmd64!hybdriverentry+0x5bdf6 fffff801`61ed5b1f : igdkmd64!hybdriverentry+0x576da fffff801`61edc798 : igdkmd64!hybdriverentry+0x4d7af fffff801`61ed51b5 : igdkmd64!hybdriverentry+0x54428 fffff801`61e48613 : igdkmd64!hybdriverentry+0x4ce45 fffff801`61e48507 : igdkmd64+0x26613 fffff801`60d1ea34 : igdkmd64+0x26507 fffff801`60ceffef : dxgkrnl!dxgadapter::ddiescape+0x48 fffff960`002c563b : dxgkrnl!dxgkescape+0x54f fffff800`ac5d41b3 : win32k!ntgdiddddiescape+0x `770574aa : nt!kisystemservicecopyend+0x ` : 0x770574aa

65 Windows Graphics Driver Fuzzing NVIDIA Graphics Driver nvlddmkm.sys ~800 graphics handling functions This also won t end well

66 Windows Graphics Driver Fuzzing TALOS (Piotr Bania) Nvidia Windows Kernel Mode Driver ZwSetValueKey Denial Of Service nt!memcpy+0xa0: fffff801`b0bcfc20 f30f6f040a movdqu xmm0,xmmword ptr [rdx+rcx] ds:ffffd000`26a45ff8=?? ffffd000`26a44408 fffff801`b0bde42c : nt!kebugcheckex ffffd000`26a44808 fffff801`b0f26473 : nt!memcpy+0xa0 ffffd000`26a44810 fffff801`b0fbcd18 : nt!cmpsetvaluedatanew+0x157 ffffd000`26a44860 fffff801`b0f0f588 : nt!?? ::NNGAKEGL::`string'+0x27928 ffffd000`26a448d0 fffff801`b0e3a977 : nt!cmsetvaluekey+0x784 ffffd000`26a449e0 fffff801`b0bcebb3 : nt!ntsetvaluekey+0x55f ffffd000`26a44bb0 fffff801`b0bc7020 : nt!kisystemservicecopyend+0x13 ffffd000`26a44db8 fffff801`4175a51a : nt!kiservicelinkage ffffd000`26a44dc0 fffff801`4175a051 : nvlddmkm+0xb751a ffffd000`26a44f70 fffff801`41f44769 : nvlddmkm+0xc0faf ffffd000`26a44fb0 fffff801`41f39e24 : nvlddmkm!nvdumpconfig+0x1253a1 ffffd000`26a45580 fffff801`413604f8 : nvlddmkm!nvdumpconfig+0xdc075 ffffd000`26a45650 fffff801`413c5b4e : dxgkrnl!dxgadapter::ddiescape+0x48 ffffd000`26a45680 fffff960`002d41d3 : dxgkrnl!dxgkescape+0x802 ffffd000`26a45ab0 fffff801`b0bcebb3 : win32k!ntgdiddddiescape+0x53

67 Demo winafl-intelpt vs idgkmd64.sys

68

69

70

71

72 Conclusions Kernels expose a massive amount of attack surface Hardware tracing enables code coverage for tricky targets Coverage guided kernel fuzzing is new and promising Get the code! - Windows PT Driver available since Jan 2017 WinAFL-IntelPT available today PtCov library available next week

73