SafeBricks: Shielding Network Functions in the Cloud Rishabh Poddar, Chang Lan, Raluca Ada Popa, Sylvia Ratnasamy UC Berkeley
Network Functions (NFs) in the cloud Clients 2 Enterprise Destination
Network Functions (NFs) in the cloud Clients 3 Enterprise Destination
Network Functions (NFs) in the cloud NF NF providers providers Clients 4 Enterprise Destination
Problem: Security NF NF providers providers Email Clients 5 Enterprise Destination
Problem: Security 1 Need to protect traffic from the cloud provider NF NF providers providers Hackers / curious employees Email Clients 6 Enterprise Destination
Problem: Security 2 Need to protect traffic from the Exfiltration NF NF providers providers Email Clients 7 Enterprise Destination
Problem: Security 3 Need to protect NF code and rulesets from client enterprise and cloud NF NF providers providers Email Clients 8 Enterprise Destination
Cryptographic solutions do not suffice NF NF providers providers Email Clients 9 Enterprise Destination
Cryptographic solutions do not suffice 1 Standard encryption: e.g. end-to-end TLS NF NF providers providers Email Clients 10 Enterprise Destination
Cryptographic solutions do not suffice 1 Standard encryption: e.g. end-to-end TLS Functionality: Doesn t allow any computation on encrypted payload NF NF providers providers? Email Clients 11 Enterprise Destination
Cryptographic solutions do not suffice 1 Standard encryption: e.g. end-to-end TLS Functionality: Doesn t allow any computation on encrypted payload Security: Unencrypted fields (e.g. IP headers) still leak information NF NF providers providers? Email Clients 12 Enterprise Destination
Cryptographic solutions do not suffice 2 Specialized encryption: e.g. BlindBox, Embark [Sherry et al. (SIGCOMM 15)] [Lan et al. (NSDI 16)] 13
Cryptographic solutions do not suffice 2 Specialized encryption: e.g. BlindBox, Embark Too limited in functionality! Header-based comparisons Regular expressions Keyword matching Cross-flow analysis Statistical computations 14
How to achieve full functionality and our security goals simultaneously? 15
SafeBricks 1 Protects traffic from the cloud provider 2 Protects traffic from the 3 Protects NF source code and rulesets from client enterprise and cloud 16
SafeBricks Hardware enclaves + language-based isolation 1 Protects traffic from the cloud provider 2 Protects traffic from the 3 Protects NF source code and rulesets from client enterprise and cloud 17
Background: Hardware enclaves (e.g. Intel SGX) Secure region of memory (enclaves) protected by hardware Application (untrusted) Enclave (trusted) Operating System (untrusted) 18
Background: Hardware enclaves (e.g. Intel SGX) Secure region of memory (enclaves) protected by hardware Application (untrusted) Secret data Trusted code Enclave (trusted) Operating System (untrusted) 19
Background: Hardware enclaves (e.g. Intel SGX) Secure region of memory (enclaves) protected by hardware Application (untrusted) Secret data Trusted code Client Enclave (trusted) Remote attestation by clients Operating System (untrusted) 20
Background: Hardware enclaves (e.g. Intel SGX) Secure region of memory (enclaves) protected by hardware Application (untrusted) Secret data Trusted code Client Enclave (trusted) Remote attestation by clients Remotely verify enclave contents Operating System (untrusted) 21
Background: Hardware enclaves (e.g. Intel SGX) Secure region of memory (enclaves) protected by hardware Application (untrusted) Secret data Trusted code Client Enclave (trusted) Remote attestation by clients Remotely verify enclave contents Operating System (untrusted) Establish a secure channel with enclave 22
Background: NetBricks [Panda et al. (OSDI 16)] Framework for developing arbitrary NFs Programming abstractions NetBricks Scheduler State abstractions I/O interface DPDK Poll for I/O NICs 23
Background: NetBricks [Panda et al. (OSDI 16)] Framework for developing arbitrary NFs MapReduce like programming abstractions (operators) for packet processing Programming abstractions State abstractions NetBricks Scheduler I/O interface DPDK Poll for I/O NICs 24
Background: NetBricks [Panda et al. (OSDI 16)] Framework for developing arbitrary NFs MapReduce like programming abstractions (operators) for packet processing NFs represented as a directed graph with operators as nodes Programming abstractions State abstractions NetBricks I/O interface DPDK Scheduler Poll for I/O NICs 25
Background: NetBricks Written in Rust Fast, safe, zero-copy semantics NetBricks NF 1 NF 1 Isolates NFs deployed in a chain while running them in the same address space NICs 26
SafeBricks 1 Protects traffic from the cloud provider 2 Protects traffic from the 3 Protects NF source code and rulesets from client enterprise and cloud 27
SafeBricks 1 Protects traffic from the cloud provider 2 Protects traffic from the 3 Protects NF source code and rulesets from client enterprise and cloud 28
Outsourcing NFs using hardware enclaves Cloud (untrusted) Enclave OS (untrusted) Clients Gateway Destination 29 Enterprise
Outsourcing NFs using hardware enclaves Cloud (untrusted) NF Enclave OS (untrusted) Clients Gateway Destination 30 Enterprise
Outsourcing NFs using hardware enclaves Cloud (untrusted) NF Enclave OS (untrusted) Clients Gateway Destination 31 Enterprise
Outsourcing NFs using hardware enclaves Cloud (untrusted) IPSec NF Enclave IPSec OS (untrusted) Clients Gateway Destination 32 Enterprise
Outsourcing NFs using hardware enclaves Cloud (untrusted) IPSec NF Enclave IPSec TLS OS (untrusted) Clients Interception proxy Gateway Destination 33 Enterprise
Outsourcing NFs using hardware enclaves Cloud (untrusted) IPSec NF Enclave IPSec TLS OS (untrusted) Clients TLS Gateway Destination 34 Enterprise
Outsourcing NFs using hardware enclaves Cloud (untrusted) IPSec NF Enclave IPSec TLS IPSec OS (untrusted) Clients TLS Gateway Destination 35 Enterprise
Outsourcing NFs using hardware enclaves Cloud (untrusted) IPSec NF Enclave IPSec TLS IPSec OS (untrusted) Clients TLS Gateway Packet headers also encrypted Destination 36 Enterprise
Outsourcing NFs using hardware enclaves Cloud (untrusted) IPSec NF Enclave IPSec TLS IPSec OS (untrusted) Clients TLS Gateway TLS Destination 37 Enterprise
Outsourcing NFs using hardware enclaves Cloud (untrusted) IPSec NF Enclave IPSec TLS OS (untrusted) Clients TLS 38 Enterprise Gateway SafeBricks also supports direct delivery of traffic Destination
Outsourcing NFs using hardware enclaves Cloud (untrusted) IPSec NF Enclave Can use general purpose frameworks, e.g. Haven, Scone IPSec TLS IPSec OS (untrusted) Clients TLS Gateway TLS Destination 39 Enterprise
Challenges 1 Small trusted computing base (TCB) enclave should contain minimal amount of code 40
Challenges 1 Small trusted computing base (TCB) enclave should contain minimal amount of code 2 High performance Transitioning into / out of enclaves is expensive! 41
Challenges 1 Small trusted computing base (TCB) enclave should contain minimal amount of code 2 High performance Transitioning into / out of enclaves is expensive! 3 Illegal enclave instructions SGX does not support system calls or instructions that may lead to a VMEXIT 42
Challenges 1 Small trusted computing base (TCB) enclave should contain minimal amount of code 2 High performance Transitioning into / out of enclaves is expensive! 3 Illegal enclave instructions SGX does not support system calls or instructions that lead to a VMEXIT 43
1 NetBricks Programming abstractions Scheduler State abstractions I/O interface DPDK Poll for I/O NICs 44
1 Enclave Programming abstractions Scheduler Maximal TCB: NetBricks State abstractions stack entirely within enclave I/O interface DPDK Poll for I/O NICs 45
1 Enclave Programming abstractions Scheduler State abstractions Minimal TCB: Only securitycritical components within enclave I/O interface One enclave transition per node per packet batch DPDK Poll for I/O NICs 46
1 Enclave Programming abstractions Scheduler Intermediate TCB State abstractions One enclave transition per packet batch I/O interface DPDK Poll for I/O NICs 47
1 Programming abstractions Scheduler SafeBricks enclave (trusted) State abstractions Partitioned NetBricks Glue code (trusted) framework; glue code connects trusted and SafeBricks host (untrusted) Glue code (untrusted) I/O interface DPDK untrusted code Poll for I/O 48 NICs
1 SafeBricks enclave (trusted) Programming abstractions State abstractions Scheduler Two new operators for packet transfer to/from enclave: toenclave and tohost Partitioned NetBricks Glue code (trusted) framework; glue code connects trusted and SafeBricks host (untrusted) Glue code (untrusted) I/O interface DPDK untrusted code Poll for I/O 49 NICs
Challenges 1 Small trusted computing base (TCB) enclave should contain minimal amount of code 2 High performance Transitioning into / out of enclaves is expensive! 3 Illegal enclave instructions SGX does not support system calls or instructions that lead to a VMEXIT 50
2 SafeBricks enclave NF toenclave tohost One enclave transition per packet batch SafeBricks host NICs 51
2 SafeBricks enclave NF toenclave tohost Shared queues in non-enclave Enclave I/O heap recv send Separate enclave and host threads Access queues without exiting Host I/O SafeBricks host enclave zero enclave transitions 52 NICs
Challenges 1 Small trusted computing base (TCB) enclave should contain minimal amount of code 2 High performance Transitioning into / out of enclaves is expensive! 3 Illegal enclave instructions SGX does not support system calls or instructions that lead to a VMEXIT 53
3 Observation: NFs in general do not require support for system calls / instructions that lead to VMEXITs 54
3 Observation: NFs in general do not require support for system calls / instructions that lead to VMEXITs, except: Logging Timestamps (using rdtsc) 55
3 Observation: NFs in general do not require support for system calls / instructions that lead to VMEXITs, except: Logging Timestamps (using rdtsc) SafeBricks designs custom solutions for these operations without enclave transitions 56
SafeBricks 1 Protects traffic from the cloud provider 2 Protects traffic from the 3 Protects NF source code and rulesets from client enterprise and cloud 57
Problem: Malicious NFs within enclaves! Malicious NFs inside the enclave can exfiltrate or tamper with packets! 58
Problem: Malicious NFs within enclaves! Malicious NFs inside the enclave can exfiltrate or tamper with packets! Observation: NFs typically need access only to specific packet fields E.g. Firewall needs read-only access to TCP/IP headers E.g. NAT needs both read-write access to headers but not to packet payload 59
Problem: Malicious NFs within enclaves! Malicious NFs inside the enclave can exfiltrate or tamper with packets! Observation: NFs typically need access only to specific packet fields E.g. Firewall needs read-only access to TCP/IP headers E.g. NAT needs both read-write access to headers but IP not addresses; to packet payload TCP ports; HTTP payload 60
Problem: Malicious NFs within enclaves! Malicious NFs inside the enclave can exfiltrate or tamper with packets! Observation: NFs typically need access only to specific packet fields E.g. Firewall needs read-only access to TCP/IP headers E.g. NAT needs both read-write access to headers but not to packet payload SafeBricks enforces least privilege across NFs within the enclave 61
Least privilege enforcement Run NFs within the same enclave Firewall NAT toenclave tohost Host 62
Least privilege enforcement Run NFs within the same enclave Firewall NAT toenclave tohost Host 63
Least privilege enforcement Run NFs within the same enclave Firewall NAT Stitch NFs together interspersed with an operator ( wlist ) that embeds a vector of permissions in packets two bits per packet field wlist wlist toenclave tohost Host 64
Least privilege enforcement Enforce permissions by mediating access to packets using Rust s ownership model SafeBricks Controller 65
Least privilege enforcement Enforce permissions by mediating access to packets using Rust s ownership model Packet buffer Controller module holds ownership of packet buffers SafeBricks Controller 66
Least privilege enforcement Enforce permissions by mediating access to packets using Rust s ownership model Packet buffer Controller module holds ownership of packet buffers SafeBricks Controller NFs borrow references to packet fields from the Controller, which checks permissions vector in packet Firewall NAT 67
Least privilege enforcement Enforce permissions by mediating access to packets using Rust s ownership model Packet buffer Controller module holds ownership of packet buffers SafeBricks Controller NFs borrow references to packet fields from the Controller, which checks permissions vector in packet Firewall NAT Returns an immutable reference for readonly access, and a mutable reference for write access 68
Assumption: Trusted compilation of NFs Least privilege guarantees only hold if NFs are built using a compiler that prohibits unsafe operations! 69
Assumption: Trusted compilation of NFs Least privilege guarantees only hold if NFs are built using a compiler that prohibits unsafe operations! E.g. Check array bounds, no pointer arithmetic, no unsafe type casts 70
Assumption: Trusted compilation of NFs Least privilege guarantees only hold if NFs are built using a compiler that prohibits unsafe operations! NF NF providers providers Possible solution: Client obtains NF source codes from providers and assembles them NF code + rulesets locally Client Enterprise 71
Assumption: Trusted compilation of NFs Least privilege guarantees only hold if NFs are built using a compiler that prohibits unsafe operations! NF NF providers providers Possible solution: Client obtains NF source codes from providers and assembles them NF code + rulesets locally Problem: This violates the confidentiality of NF source code! Enterprise 72
SafeBricks 1 Protects traffic from the cloud provider 2 Protects traffic from the 3 Protects NF source code and rulesets from client enterprise and cloud 73
Assembling NFs Key idea: Build NFs within a special meta -enclave in the cloud using an agreed upon compiler 74
Assembling NFs Key idea: Build NFs within a special meta -enclave in the cloud using an agreed upon compiler Both client and can verify the agreed upon compiler using remote attestation 75
Assembling NFs Assembly enclave Loader Compiler 76 Enterprise
Assembling NFs Assembly enclave Loader Compiler Remote attestation 77 Enterprise
Assembling NFs Assembly enclave Loader Compiler Remote attestation 78 Enterprise
Assembling NFs Assembly enclave Loader Compiler Remote attestation 79 Enterprise
Assembling NFs Assembly enclave NF code + rulesets Loader Compiler 80 Enterprise
Assembling NFs Assembly enclave Loader Compiler NF code + rulesets Config 81 Enterprise
Assembling NFs Assembly enclave Loader Compiler NF code + rulesets Config 82 Enterprise Placement of NFs, least privilege policies per NF
Assembling NFs Assembly enclave Loader Compiler NF code + rulesets Config 83 Enterprise
Assembling NFs Assembly enclave Loader Compiler Deployment enclave 84 Enterprise
SafeBricks 1 Protects traffic from the cloud provider 2 Protects traffic from the 3 Protects NF source code and rulesets from client enterprise and cloud 85
86 Performance
Throughput decline across NFs 40% Firewall NAT Throughput decline 30% 20% 10% Load Balancer DPI 0% 64 256 512 1024 Packet size (bytes) ~0 15% overhead across applications for different packet sizes 87
DPI performance with increasing no. of rules 100% 8MB 64MB 94MB Throughput decline 75% 50% 25% 0% 0 5000 10000 15000 20000 25000 Number of rules 88 Overhead spikes when NF working set exceeds enclave memory
DPI performance with increasing no. of rules 100% 8MB 64MB 94MB Throughput decline 75% 50% 25% Not a fundamental limitation 0% 0 5000 10000 15000 20000 25000 Number of rules 89 Overhead spikes when NF working set exceeds enclave memory
Summary rishabhp@eecs.berkeley.edu SafeBricks uses a combination of hardware enclaves and language-based isolation to: Protect client traffic from the cloud provider Enforce least privilege across NFs Protect the confidentiality of NF code and rulesets Modest overhead across a range of applications 90