Tolerating Malicious Drivers in Linux. Silas Boyd-Wickizer and Nickolai Zeldovich

Size: px

Start display at page:

Download "Tolerating Malicious Drivers in Linux. Silas Boyd-Wickizer and Nickolai Zeldovich"

Simon Harrell
6 years ago
Views:

1 XXX

2 Tolerating Malicious Drivers in Linux Silas Boyd-Wickizer and Nickolai Zeldovich

3 How could a device driver be malicious? Today's device drivers are highly privileged Write kernel memory, allocate memory,... Drivers are complex; developers write buggy code Result: Attackers exploit vulnerabilities

4 How could a device driver be malicious? Today's device drivers are highly privileged Write kernel memory, allocate memory,... Drivers are complex; developers write buggy code Result: Attackers exploit vulnerabilities

5 How could a device driver be malicious? Today's device drivers are highly privileged Write kernel memory, allocate memory,... Drivers are complex; developers write buggy code Result: Attackers exploit vulnerabilities

6 Current approach -space drivers in μkernels (Minix, L4,...) Write device driver in new language (Termite) Handle common faults (Nooks, microdrivers,...)

7 Goal Secure, efficient, & unmodified drivers on Linux

8 Previous user-space drivers μkernel Hardware Ethernet driver Network stack Application core

9 Previous user-space drivers Confine driver in a process μkernel Hardware Ethernet driver Network stack Application core

10 Previous user-space drivers Confine driver in a process μkernel Hardware Ethernet driver Network stack Application core General purpose syscall API to configure device

11 Previous user-space drivers Confine driver in a process μkernel Hardware Ethernet driver Network stack Application Confine device with IO virtualization HW. core General purpose syscall API to configure device

12 Previous user-space drivers Confine driver in a process μkernel IPC network driver API E.g. tx_packet Hardware Ethernet driver Network stack Application Confine device with IO virtualization HW. core General purpose syscall API to configure device

13 Current Linux driver architecture Hardware Application Ethernet driver Network stack RT netdevice

14 Current Linux driver architecture Hardware runtime (e.g. kmalloc) Application Ethernet driver Network stack RT netdevice

15 Current Linux driver architecture Hardware runtime (e.g. kmalloc) Network driver API (e.g. tx_packet) Application Ethernet driver Network stack RT netdevice

16 Linux user-space driver problem RT and driver APIs won't work for untrusted drivers in a different AS Ethernet driver Hardware Application Network stack RT netdevice

17 SUD's approach Ethernet driver Hardware Application Network stack RT netdevice

18 SUD's approach SUD UML handles calls to kernel RT Ethernet driver Hardware SUD UML Application Network stack RT netdevice

19 SUD's approach SUD UML handles calls to kernel RT Proxy driver and SUD UML allow reuse of existing driver APIs Ethernet driver Hardware SUD UML Application Ethernet proxy driver Network stack RT netdevice

20 SUD's approach SUD UML handles calls to kernel RT Proxy driver and SUD UML allow reuse of existing driver APIs Ethernet driver Hardware SUD UML Application Network driver API Ethernet proxy driver Network stack RT netdevice

21 SUD's approach SUD UML handles calls to kernel RT Proxy driver and SUD UML allow reuse of existing driver APIs Ethernet driver Hardware SUD RPC API SUD UML Application Network driver API Ethernet proxy driver Network stack RT netdevice

22 SUD's approach SUD UML handles calls to kernel RT Proxy driver and SUD UML allow reuse of Network driver API existing driver APIs Ethernet driver Hardware SUD RPC API SUD UML Application Network driver API Ethernet proxy driver Network stack RT netdevice

23 SUD's results Tolerate malicious device drivers Proxy drivers small (~500 LOC) One proxy driver per device class Few kernel modifications (~50 LOC) Unmodified drivers (6 test drivers) High performance, low overhead No need for new OS or language

24 Security challenge: prevent attacks Problem: driver must perform privileged operations Memory access, driver API, DMA, interrupts, Attacks from driver code: Direct system attacks: memory corruption,... Driver API attacks: invalid return value, deadlock,... Attacks from device: DMA to DRAM, peer-to-peer attacks, interrupt storms

25 Practical challenges High performance, low overhead Challenge: interact with hardware and kernel at high rate, kernel-user switch expensive E.g. Ethernet driver ~100k times a second Reuse existing drivers and kernel Challenge: drivers assume fully-privileged kernel env. Challenge: kernel driver API complex, non-uniform

26 SUD overview Hardware Driver SUD UML Application HW access module Proxy driver core

27 SUD overview Hardware Driver SUD UML Application HW access module Proxy driver core

28 Linux driver APIs Linux defines a driver API for each device class Driver and kernel functions and variables

29 Example: wireless driver API Linux defines a driver API for each device class Driver and kernel functions and variables struct wireless_ops { int (*tx)(struct sk_buff*); int (*configure_filter)(int);... }; struct wireless_hw { int conf; int flags... };

30 Example: wireless driver API Linux defines a driver API for each device class Driver and kernel functions and variables struct wireless_ops { int (*tx)(struct sk_buff*); int (*configure_filter)(int);... }; struct wireless_hw { int conf; int flags... }; Proxy drivers and SUD-UML convert API to RPCs

31 Example: wireless driver API Linux defines a driver API for each device class Driver and kernel functions and variables struct wireless_ops { int (*tx)(struct sk_buff*); int (*configure_filter)(int);... }; struct wireless_hw { int conf; int flags... }; Proxy drivers and SUD-UML convert API to RPCs

32 Example: wireless driver API Linux defines a driver API for each device class Driver and kernel functions and variables struct wireless_ops { }; int (*tx)(struct sk_buff*); int (*configure_filter)(int);... struct wireless_hw { }; int conf; int flags... Called in a nonpreemptable context Proxy drivers and SUD-UML convert API to RPCs

33 Example: wireless driver API Linux defines a driver API for each device class Driver and kernel functions and variables struct wireless_ops { }; int (*tx)(struct sk_buff*); int (*configure_filter)(int);... struct wireless_hw { }; int conf; int flags... Driver API variable Called in a nonpreemptable context Proxy drivers and SUD-UML convert API to RPCs

34 driver in SUD Basic driver API SUD RPC API driver API Non-preemptable function: implement in proxy Driver API variable: shadow variables

35 Example 1: transmit a packet Hardware driver SUD UML Web browser proxy driver core

36 Example 1: transmit a packet Socket write Hardware driver SUD UML Web browser proxy driver core

37 Example 1: transmit a packet Hardware driver SUD UML Web browser wireless_ops.tx proxy driver core

38 Example 1: transmit a packet TX packet RPC Hardware driver SUD UML Web browser proxy driver core

39 Example 1: transmit a packet wireless_ops.tx Hardware driver SUD UML Web browser proxy driver core

40 Example 1: transmit a packet DMA TX Hardware driver SUD UML Web browser proxy driver core

41 Example 2: non-preemptable callback Problem: unable to switch to user-space Hardware driver SUD UML Web browser proxy driver core

42 Example 2: non-preemptable callback Problem: unable to switch to user-space Hardware driver SUD UML Web browseracquires a spin lock proxy driver core

43 Example 2: non-preemptable callback Problem: unable to switch to user-space Hardware Web driver browser SUD UML wireless_ops.configure_filter proxy driver core

44 Example 2: non-preemptable callback Problem: unable to switch to user-space Filter RPC Hardware driver SUD UML Web browser proxy driver core

45 Example 2: non-preemptable callback Problem: unable to switch to user-space Hardware driver SUD UML Web browser proxy driver core

46 Example 2: non-preemptable callback Problem: unable to switch to user-space Solution: implement directly in proxy driver Hardware driver SUD UML Web browser proxy driver core

47 Example 2: non-preemptable callback Problem: unable to switch to user-space Solution: implement directly in proxy driver Hardware driver SUD UML Register RX Web packet types browser proxy driver core

48 Example 2: non-preemptable callback Problem: unable to switch to user-space Solution: implement directly in proxy driver Hardware driver SUD UML Web browseracquires a spin lock proxy driver core

49 Example 2: non-preemptable callback Problem: unable to switch to user-space Solution: implement directly in proxy driver Hardware driver SUD UML Web browser wireless_ops.configure_filter proxy driver core

50 Example 2: non-preemptable callback Problem: unable to switch to user-space Solution: implement directly in proxy driver Hardware driver SUD UML Web Return RX browser packet types proxy driver core

51 Example 3: driver API variables Problem: user-space can't access API variables Hardware driver SUD UML Web browser proxy driver core wireless_hw

52 Example 3: driver API variables Problem: user-space can't access API variables Hardware driver SUD UML Web browser proxy driver Driver API variable core wireless_hw

53 Example 3: driver API variables Problem: user-space can't access API variables Hardware driver SUD UML Web browser Writes to wireless_hw proxy driver core wireless_hw

54 Example 3: driver API variables Problem: user-space can't access API variables Hardware driver SUD UML Web browser proxy driver core wireless_hw

55 Example 3: driver API variables Problem: user-space can't access API variables Solution: allocate a shadow copy and synchronize before and after RPCs Hardware driver SUD UML Web browser proxy driver core wireless_hw

56 Example 3: driver API variables Problem: user-space can't access API variables Solution: allocate a shadow copy and synchronize before and after RPCs Shadow variable Hardware driver SUD UML wireless_hw Web browser proxy driver core wireless_hw

57 Example 3: driver API variables Problem: user-space can't access API variables Solution: allocate a shadow copy and synchronize before and after RPCs Hardware driver SUD UML wireless_hw Web browser Writes to wireless_hw proxy driver core wireless_hw

58 Example 3: driver API variables Problem: user-space can't access API variables Solution: allocate a shadow copy and synchronize before and after RPCs Synchronize before driver Hardware SUD UML sending RPC wireless_hw Web browser proxy driver core wireless_hw

59 Example 3: driver API variables Problem: user-space can't access API variables Solution: allocate a shadow copy and synchronize before and after RPCs Hardware Send RPC driver SUD UML Web browser wireless_hw proxy driver core wireless_hw

60 Example 3: driver API variables Problem: user-space can't access API variables Reads updates from Solution: allocate a shadow shadow copy variableand synchronize before and after RPCs Hardware driver SUD UML wireless_hw Web browser proxy driver core wireless_hw

61 SUD overview Hardware Driver SUD UML Application HW access module Proxy driver core

62 SUD overview Hardware Driver SUD UML Application HW access module Proxy driver core

63 Attacks from hardware CPU Memory interconnect PCI bus DRAM

64 Attacks from hardware Driver configures the device to execute attacks CPU Memory interconnect PCI bus DRAM

65 Attacks from hardware Driver configures the device to execute attacks DMA to DRAM CPU Memory interconnect PCI bus DRAM

66 Attacks from hardware Driver configures the device to execute attacks DMA to DRAM Peer-to-peer messages CPU Memory interconnect PCI bus DRAM

67 Attacks from hardware Driver configures the device to execute attacks DMA to DRAM Peer-to-peer messages Interrupt storms CPU Memory interconnect PCI bus DRAM

68 Attacks from hardware Driver configures the device to execute attacks DMA to DRAM Peer-to-peer messages Interrupt storms HW access module prevents attacks Interposes on driver-device communication Uses IO virtualization to provide direct device access

69 IO virtualization hardware APIC interconnect CPU MSI Memory interconnect IOMMU PCI express switch DRAM

70 IO virtualization hardware Use IOMMU to map DMA buffer pools Prevents DMA to DRAM attacks APIC interconnect CPU MSI Memory interconnect IOMMU PCI express switch DRAM

71 IO virtualization hardware Use PCI ACS to prevent peer-to-peer messaging Prevents peer-to-peer attacks APIC interconnect CPU MSI Memory interconnect IOMMU PCI express switch DRAM

72 IO virtualization hardware Use MSI to mask interrupts Prevents interrupt storms APIC interconnect CPU MSI Memory interconnect IOMMU PCI express switch DRAM

73 Interrupt handlers in Linux MSI Driver IRQ core

74 Interrupt handlers in Linux MSI Driver IRQ core

75 Interrupt handlers in Linux Driver called with IRQs disabled (non-preemptable) MSI Driver IRQ core

76 Interrupt handlers in Linux calls driver interrupt handler Driver clears interrupt flag MSI Driver IRQ core

77 Interrupt handlers with SUD MSI Driver SUD UML HW access module IRQ core

78 Interrupt handlers with SUD calls HW access module interrupt handler HW access module masks interrupt with MSI MSI Driver SUD UML HW access module IRQ core

79 Interrupt handlers with SUD calls HW access module interrupt handler HW access module masks interrupt with MSI MSI Driver SUD UML HW access module IRQ core

80 Interrupt handlers with SUD calls HW access module interrupt handler HW access module masks interrupt with MSI Asynchronous RPC to driver MSI Driver SUD UML HW access module IRQ core

81 Interrupt handlers with SUD calls HW access module interrupt handler HW access module masks interrupt with MSI Asynchronous RPC to driver Driver clears interrupt MSI Driver SUD UML HW access module IRQ core

82 Interrupt handlers with SUD HW access module masks interrupt with MSI Asynchronous RPC to driver Driver clears interrupt HW access module unmasks MSI MSI Driver SUD UML HW access module IRQ core

83 SUD overview Hardware Driver SUD UML Application HW access module Proxy driver core

84 Prototype of SUD Trusted code Lines of code PCI access module 2800 Ethernet proxy driver 300 proxy driver 600 Audio proxy driver 550 Untrusted code Lines of code -mode runtime 5000 Drivers ,000 (each) Supports all Ethernet, wireless, USB, audio drivers Tested: e1000e, ne2k-pci, iwlagn, snd_hda_intel, ehci_hcd, uhci_hcd,...

85 Prototype of SUD Trusted code Lines of code PCI access module 2800 Ethernet proxy driver 300 proxy driver 600 Audio proxy driver 550 Untrusted code Lines of code -mode runtime 5000 Drivers ,000 (each) Supports all Ethernet, wireless, USB, audio drivers Tested: e1000e, ne2k-pci, iwlagn, snd_hda_intel, ehci_hcd, uhci_hcd,...

86 Prototype of SUD Trusted code Lines of code PCI access module 2800 Ethernet proxy driver 300 proxy driver 600 Audio proxy driver 550 Untrusted code Lines of code -mode runtime 5000 Drivers ,000 (each) Supports all Ethernet, wireless, USB, audio drivers Tested: e1000e, ne2k-pci, iwlagn, snd_hda_intel, ehci_hcd, uhci_hcd,...

87 Performance For most devices, does not matter Printers, cameras, Stress-test: e1000e gigabit network card Requires high throughput Requires low latency Many device driver interactions Test machine: 1.4GHz dual core Thinkpad

88 Performance questions? What performance does SUD get? Network throughput, latency How much does it cost? CPU cycles

89 SUD achieves same device performance 1 Throughput relative to Linux Linux Sud 0 TCP UDP TX UDP RX UDP latency Normalized throughput relative to Linux TCP: streaming (950 Mbps in both cases) UDP: one-byte-data packets

90 CPU cost is low CPU utilization Linux Sud 0 TCP UDP TX UDP RX UDP latency SUD overhead: user-kernel switch, TLB misses Overheads not significant for many workloads (packets larger than min. packet size)

91 Future directions Explore hierarchical untrusted device drivers PCI bus SATA controller SATA disk Explore giving apps direct hardware access Safe HW access for network analyzer, X server, Performance analysis and optimizations SUD specific device drivers, super pages,...

92 Related work Mircokernels (Minix, L4,...) Simple drivers, driver API designed for user-space Nooks, microdrivers Handles common bugs, many changes to kernel Languages (e.g. Termite), source code analysis Complimentary to user-space drivers No need for new OS or language

93 Summary Driver bugs lead to system crashes or exploits SUD protects Linux from malicious drivers using proxy drivers and IO virtualization HW Runs unmodified Linux device drivers High performance, low overheads Few modifications to Linux kernel

95 Security evaluation Manually constructed potential attacks Memory corruption, arbitrary upcall responses, not responding at all, arbitrary DMA,... Relied on security heavily during development SUD caught all bugs in user-mode driver framework No crashes / reboots required to develop drivers Ideal, but not done: red-team evaluation?

Tolerating Malicious Device Drivers in Linux

Tolerating Malicious Device Drivers in Linux The MIT Faculty has made this article openly available. Please share how this access benefits you. Your story matters. Citation As Published Publisher Boyd-Wickizer,