Nomad: Mitigating Arbitrary Cloud Side Channels via Provider-Assisted Migration

Transcription

1 Nomad: Mitigating Arbitrary Cloud Side Channels via Provider-Assisted Migration Soo-Jin Moon, Vyas Sekar Michael K. Reiter

2 Context: Infrastructure-as-a-Service Clouds Client API Cloud Controller Machine Machine Machine

3 Information Leakage via Co-Residency Co-Residency Cloud Controller Machine

4 Information Leakage via Co-Residency Co-Residency Cloud Controller Shared resources Machine Shared Resources

5 Information Leakage via Co-Residency Co-Residency Cloud Controller Shared resources Side Channels Machine Shared Resources

6 Information Leakage via Co-Residency Co-Residency Cloud Controller Shared resources Side Channels Machine Shared Resources Information Leakage

7 Limitations of Current Defenses 1. Requires significant/detailed upgrades OS OS e.g., Noise injection Hypervisor Hardware e.g., Deterministic execution e.g., New cache design 2. Attack-specific Proposed defense includes but not limited to: Y. Zhang et al., CCS2013; T. Kim et al., USENIXSec 2012; F. Liu and R. Lee, Micro 2014

8 Limitations of Current Defenses 1. Requires significant/detailed upgrades OS OS e.g., Noise injection Hypervisor Hardware e.g., Deterministic execution e.g., New cache design 2. Attack-specific What about future side-channel attacks? Proposed defense includes but not limited to: Y. Zhang et al., CCS2013; T. Kim et al., USENIXSec 2012; F. Liu and R. Lee, Micro 2014

9 Ideal Properties 1) General 2) Immediately deployable

10 Ideal Properties 1) General 2) Immediately deployable Single-tenancy?

11 Ideal Properties 1) General 2) Immediately deployable Single-tenancy?

12 Nomad Ideas 1) General 2) Immediately deployable

13 Nomad Ideas 1) General Tackle root-cause Minimize co-residency 2) Immediately deployable

14 Nomad Ideas 1) General Tackle root-cause Minimize co-residency 2) Immediately deployable Migration

15 Nomad Vision: Migration-as-a-Service Provider-assisted Cloud Controller Machine Machine Machine

16 Nomad Vision: Migration-as-a-Service Provider-assisted Cloud Controller Move s { } Machine Machine Machine

17 Nomad Vision: Migration-as-a-Service Opt-in Service Cloud Provider Service offering Opt-in? Clients Provider-assisted Cloud Controller Move s { } Machine Machine Machine

18 Nomad Practical Challenges Logic Characterize information leakage due to co-residency Cloud Controller Machine Machine Machine

19 Nomad Practical Challenges Logic Characterize information leakage due to co-residency Scalable Design e.g., can Amazon EC2 run this? Cloud Controller Machine Machine Machine

20 Nomad Practical Challenges Logic Characterize information leakage due to co-residency Cloud Controller Scalable Design e.g., can Amazon EC2 run this? Practical Impact (cloud) Minimal modifications? Machine Machine Machine

21 Nomad Practical Challenges Logic Characterize information leakage due to co-residency Cloud Controller Scalable Design e.g., can Amazon EC2 run this? Practical Impact (cloud) Minimal modifications? Machine Machine Machine Practical Impact (applications) 1) Advancement of migration techniques 2) Many cloud workloads with in-built resilience to migration

22 1. Idea General side-channel defense via migration Our Work

23 Our Work 1. Idea General side-channel defense via migration 2. Logic Characterize information leakage due to co-residency

24 Our Work 1. Idea General side-channel defense via migration 2. Logic Characterize information leakage due to co-residency 3. Scalable Design Scalable migration strategy that can handle large cloud deployments

25 Our Work 1. Idea General side-channel defense via migration 2. Logic Characterize information leakage due to co-residency 3. Scalable Design Scalable migration strategy that can handle large cloud deployments 4. Practical Impact Practical OpenStack implementation with minimal modifications

26 Our Work 1. Idea General side-channel defense via migration 2. Logic Characterize information leakage due to co-residency 3. Scalable Design Scalable migration strategy that can handle large cloud deployments 4. Practical Impact Practical OpenStack implementation with minimal modifications

27 Threat Model Objective: Extract secrets via co-residency Can use any kind of resource Can launch/terminate s at will s of a given client can collaborate

28 Threat Model Objective: Extract secrets via co-residency Can use any kind of resource Can launch/terminate s at will s of a given client can collaborate Cannot control placement No info. sharing across distinct clients

29 Threat Model Objective: Extract secrets via co-residency Can use any kind of resource Can launch/terminate s at will s of a given client can collaborate Cannot control placement No info. sharing across distinct clients? Provider Don t know which other clients are malicious

30 Information Leakage (InfoLeak) Model Clients InfoLeak?

31 Information Leakage (InfoLeak) Model Clients InfoLeak? Replicated? (R or NR) -level view B1 B2 R

32 Information Leakage (InfoLeak) Model Clients InfoLeak? Replicated? (R or NR) -level view B1 B1 B2 B2 R NR

33 Information Leakage (InfoLeak) Model Clients InfoLeak? Replicated? (R or NR) Collaborating? (C or NC) -level view B1 B1 B2 B2 R NR C R1 R2

34 Information Leakage (InfoLeak) Model Clients InfoLeak? Replicated? (R or NR) Collaborating? (C or NC) -level view B1 B1 B2 B2 R NR C NC R1 R1 R2 R2

35 Information Leakage (InfoLeak) Model NR Replicated? R NC Collaborating? C <NR,NC> Least InfoLeak <NR,C> <R,NC> Most InfoLeak <R,C>

36 Our Work 1. Idea General side-channel defense via migration 2. Logic Characterize information leakage due to co-residency 3. Scalable Design Scalable migration strategy that can handle large cloud deployments 4. Practical Impact Practical OpenStack implementation with minimal modifications

37 System Overview Cloud Controller Move s { } Machine Machine Machine

38 System Overview Cloud Provider Deployment model (e.g., <NR,NC>) Opt-in? Clients Cloud Controller Move s { } Machine Machine Machine

39 Operational Timeline 1 epoch = D time units Time (epoch) Sliding Window of epochs Run placement algorithm every epoch

40 Operational Timeline 1 epoch = D time units Time (epoch) Sliding Window of epochs Run placement algorithm every epoch Side-channel Parameters: K: Information leakage rate (i.e., bits per time unit) P: secret length (i.e., bits)

41 Operational Timeline 1 epoch = D time units Time (epoch) Sliding Window of epochs Run placement algorithm every epoch Extracted secret (bits) if two s are co-resident for epochs Provider chooses D and to AT LEAST satisfy: D * * K < P

42 Placement Algorithm Deployment Model (e.g.,<nr,nc>) Recent Placements Client Workloads & Constraints Placement Algorithm Placement

43 Placement Algorithm Deployment Model (e.g.,<nr,nc>) Recent Placements Client Workloads & Constraints Placement Algorithm Goal (per epoch): Minimize a global sum of a clientpair InfoLeak across past epochs i.e.,! InfoLeak * * -([t, t]) *,*7 subject to a fixed migration budget Placement

44 Placement Algorithm Deployment Model (e.g.,<nr,nc>) Recent Placements Client Workloads & Constraints F(Deployment Model) Placement Algorithm Goal (per epoch): Minimize a global sum of a clientpair InfoLeak across past epochs i.e.,! InfoLeak * * -([t, t]) *,*7 subject to a fixed migration budget Placement

45 Placement Algorithm Deployment Model (e.g.,<nr,nc>) Recent Placements Client Workloads & Constraints F(Deployment Model) Placement Algorithm Placement Goal (per epoch): Minimize a global sum of a clientpair InfoLeak across past epochs i.e.,! InfoLeak * * -([t, t]) *,*7 subject to a fixed migration budget F(Network Capacity)

46 Challenge: Scalability Inputs Should handle tens of thousands of servers Placement Algorithm Placement

47 Challenge: Scalability Inputs Should handle tens of thousands of servers Placement Algorithm ILP (Integer Linear Programming) For 40 machines, D > 1 day Placement

48 Challenge: Scalability Inputs Should handle tens of thousands of servers Placement Algorithm ILP (Integer Linear Programming) For 40 machines, D > 1 day Placement

49 Challenge: Scalability Inputs Should handle tens of thousands of servers Placement Algorithm ILP (Integer Linear Programming) For 40 machines, D > 1 day Basic Greedy For 400 machines, D > 1 day Placement

50 Challenge: Scalability Inputs Should handle tens of thousands of servers Placement Algorithm ILP (Integer Linear Programming) For 40 machines, D > 1 day Basic Greedy For 400 machines, D > 1 day Placement

51 Challenge: Scalability Inputs Should handle tens of thousands of servers Placement Algorithm Placement ILP (Integer Linear Programming) For 40 machines, D > 1 day Basic Greedy For 400 machines, D > 1 day Basic Greedy with our optimizations

52 Why is Basic Greedy not scalable? Generate Moves Compute Benefit (total reduction in infoleak) Pairwise Swap: 1-2 -> 2-1 N-way Swap: No Pick Best Move Make Move totalnummove > Budget Yes Exit

53 Why is Basic Greedy not scalable? Generate Moves Compute Benefit (total reduction in infoleak) Free Insert: 1 -> M1 Bottleneck #1: Pairwise Swap: 1-2 -> 2-1 Large Search Space N-way Swap: No Pick Best Move Make Move totalnummove > Budget Yes Exit

54 Why is Basic Greedy not scalable? No Generate Moves Compute Benefit (total reduction in infoleak) Pick Best Move Free Insert: 1 -> M1 Bottleneck #1: Pairwise Swap: 1-2 -> 2-1 Large Search Space N-way Swap: Bottleneck #2: Computing InfoLeak across all clients Make Move totalnummove > Budget Yes Exit

55 Why is Basic Greedy not scalable? Bottleneck #3: No Generate Moves Compute Benefit (total reduction in infoleak) Re-generating Pick Best Move move table after each move Make Move Free Insert: 1 -> M1 Bottleneck #1: Pairwise Swap: 1-2 -> 2-1 Large Search Space N-way Swap: Bottleneck #2: Computing InfoLeak across all clients totalnummove > Budget Yes Exit

56 Our Approach Bottlenecks Large Search Space Our Approach Prune Search Space Computing InfoLeak across all clients Incremental Benefit Computation Re-generating move table after each move Intra-Epoch Lazy Evaluation

57 Our Approach Bottlenecks Large Search Space Our Approach Prune Search Space Computing InfoLeak across all clients Incremental Benefit Computation Re-generating move table after each move Intra-Epoch Lazy Evaluation

58 Prune #1: Pruning Move Space Sets of all moves Insert 1 -> M1... Pairwise Swap 1-2 -> 2-1 N-way Swap.

59 Prune #1: Pruning Move Space Sets of all moves Insert 1 -> M1... Pairwise Swap 1-2 -> 2-1 N-way Swap. Nomad sets of all moves Free Insert 1 -> M1 Pairwise Swap 1-2 -> 2-1

60 Prune #2: Hierarchical Decomposition Sets of all free inserts Clients Machines M1 C1.. C1000 M2... M50000

61 Prune #2: Hierarchical Decomposition Sets of all free inserts Clients Machines M1 C1.. C1000 M2... M50000

62 Prune #2: Hierarchical Decomposition Sets of all free inserts Nomad sets of all free inserts Clients Machines M1 C1. C1000 Cluster1. Cluster25 C1.. C1000 M2... M50000

63 Prune #2: Hierarchical Decomposition Sets of all free inserts Nomad sets of all free inserts Clients Machines M1 C1. C1000 Cluster1. Cluster25 C1.. C1000 M2... M50000 C1 M1 M2.. M2000 Cluster1...

64 Our Work 1. Idea General side-channel defense via migration 2. Logic Characterize information leakage due to co-residency 3. Scalable Design Scalable migration strategy that can handle large cloud deployments 4. Practical Impact Practical OpenStack implementation with minimal modifications

65 System Implementation OpenStack v. Icehouse Clients in Cluster 1 Cloud Controller Clients in Cluster N Cluster 1 Placement Algorithm Cluster N Placement Algorithm Placements for Cluster 1 Placements for Cluster 1

66 System Implementation (One Cluster) Cluster 1 Placement Algorithm General Placement Computation OpenStack-specific Migration Engine Custom C++ ~2000 LOC OpenStack Icehouse: ~200 LOC in Controller Scheduler code Placement

67 System Implementation (One Cluster) Cluster 1 Placement Algorithm General Placement Computation OpenStack-specific Migration Engine Custom C++ ~2000 LOC OpenStack Icehouse: ~200 LOC in Controller Scheduler code Placement Requires minimal modifications to existing deployments

68 Key Evaluation Questions Information leakage resilience Scalability Impact on cloud applications Benefit/Cost of each design idea Resilience to strategic adversary

69 Information Leakage resilience <R,C>: Problem size of 20-machines Metric: InfoLeak * * -([t, t]) Nomad brings ~4.5x reduction in InfoLeak for 98 th percentile compared to static w.r.t. ILP.

70 Scalability Nomad placement algorithm is scalable to large deployments

71 Impact on cloud applications Replicated web-server (Wikibench) Each client : 3 replicated web servers, 1 worker In one epoch, at least 1 server migrates Norm. Throughput (Norm. T) = T B/D T B T B/D x 100 Overhead (Norm. T) ~0% for 95 th Norm T % for 50 th (median) Norm. T. 1.8% for 5 th Norm. T

72 Discussion Fast side-channel attacks Need out-of-band defense e.g., introduce cache noise, refresh secret Network Impact With techniques like incremental diffs, the transfer size is much less than base image Incentives for adoption Security-conscious clients opt-in Providers have new revenue streams More opportunities Fairness across clients

73 Conclusions Co-residency side-channel attacks: real/growing threats Current World : No Migration 1. Per-attack fixes 2. Require significant upgrades Nomad: Migration-as-a-Service 1. General solution 2. Needs minimal changes Nomad achieves: Information leakage resilience close to the ILP Scalable placement algorithm Practical system atop OpenStack with minimal modifications