Exploiting a Thermal Side Channel for Power Attacks in Multi-Tenant Data Centers

Transcription

1 Exploiting a Thermal Side Channel for Power Attacks in Multi-Tenant Data Centers Mohammad A. Islam, Shaolei Ren, and Adam Wierman Acknowledgement: NSF under grants CNS , CNS , and ECCS , AitF , CNS , and CNS

2 Multi-tenant data centers Mission-critical infrastructure Backbone of digital economy 50% growth by

3 Multi-tenant data centers Mission-critical infrastructure Backbone of digital economy 50% growth by 2020 A multi-tenant data center is a shared facility that houses multiple tenants, each managing its own servers 2

4 Multi-tenant data centers are everywhere 3

5 Multi-tenant data centers are everywhere Google, Amazon, MS, Fb :7.8% Multi-tenant: tenant: 37% Enterprise: 53% Percentage of electricity usage by data center type (source: NRDC 2015) 3

6 Using multi-tenant data centers for 4

7 Using multi-tenant data centers for Large IT companies Medium-scale companies IoT applications Lower latency Lower CapEx & OpEx Better privacy Higher scalability Apple houses 25% of its servers in multi-tenant data centers 4

8 Securing multi-tenant data centers is extremely important! Google, Amazon, MS, Fb :7.8% Multi-tenant: tenant: 37% Enterprise: 53% Percentage of electricity usage by data center type (source: NRDC 2015) 5

9 A cyber-physical view Utility 6

10 A cyber-physical view Utility ATS UPS Generator P D U P D U Managed by operator 6

11 A cyber-physical view Utility ATS UPS Generator P D U P D U Managed by operator Managed by tenants 6

12 A cyber-physical view Utility ATS UPS Generator P D U P D U Securing the cyberspace DDoS attack, network intrusion, privacy protection, etc. Managed by operator [Mirkovic, Sigcomm 04][Zhang CCS 12][Moon CCS 15][Dong CCS 17] Managed by tenants 6

13 A cyber-physical view How about physical security? Utility ATS UPS Generator P D U P D U Managed by operator Managed by tenants 6

14 A cyber-physical view How about physical security? 6

15 A cyber-physical view How about physical security? ATS UPS Generator P D U P D U % availability! 7

16 We revisit the conventional wisdom and find 8

17 We revisit the conventional wisdom and find Multi-tenant data centers are highly vulnerable to well-timed power attacks! 8

18 Why are multi-tenant data centers vulnerable to power attacks? What is the potential impact of power attacks? How could an attacker mount a power attack? How to defend a data center against power attacks? 9

19 Why are multi-tenant data centers vulnerable to power attacks? 9

20 When building a data center US$ per watt of data center capacity CapEx is 60+% of the total cost of ownership Utility ATS UPS Generator P D U P D U 10

21 When building a data center US$ per watt of data center capacity CapEx is 60+% of the total cost of ownership Utility ATS UPS Generator P D U P D U Oversubscribing the data center capacity is common! 10

22 When building a data center US$ per watt of data center capacity CapEx is 60+% of the total cost of ownership Utility ATS UPS Generator P D U P D U Oversubscribing the data center capacity is common! 60 kw 100 kw 60 kw Supply Sold capacity 10

23 When building a data center Oversubscribing the data center capacity is common! 60 kw 100 kw Supply is less than 60 kw Sold capacity 10

24 Rationale & safeguards Multiplex tenants power demand Limit on tenants power usage Infrastructure robustness and redundancy 11

25 Rationale & safeguards Multiplex tenants power demand Simultaneous peaks are very rare! Limit on tenants power usage Normal usage limited to 80% of tenant s subscribed capacity Only occasional peak usage is allowed Infrastructure robustness and redundancy Transient spikes are harmless 11

26 Rationale & safeguards Multiplex tenants power demand Simultaneous peaks are very rare! Limit on tenants power usage Normal usage limited to 80% of tenant s subscribed capacity Only occasional peak usage is allowed Infrastructure robustness and redundancy Transient spikes are harmless % availability! ATS UPS Generator P D U P D U 11

27 Rationale & safeguards Multiplex tenants power demand Simultaneous peaks are very rare! Limit on tenants power usage Normal usage limited to 80% of tenant s subscribed capacity Only occasional peak usage is allowed Infrastructure robustness and redundancy Transient spikes are harmless % availability?? availability! ATS UPS Generator P D U P D U 11

28 ATS UPS Generator P D U P D U 12

29 ATS UPS Generator P D U P D U 12

30 ATS UPS Generator P D U P D U Malicious Tenant Frequent capacity overloads 12

31 ATS UPS Generator P D U P D U New threat model: Power attack Well-timed power injection to overload the shared data center capacity, subject to all applicable usage constraints set by the operator Malicious Tenant Frequent capacity overloads 12

32 Why are multi-tenant data centers vulnerable to power attacks? Current safeguards are ineffective for well-timed power attacks 13

33 What is the potential impact of power attacks? 13

34 Compromising data center availability The outage risk is 280+ times higher during a capacity overload than otherwise Rather than rare events, data center outages could be much more frequent 14

35 Cost analysis Estimated impact of capacity overloads (5% of the time) on a 1MW-10,000 sqft data center 15

36 Cost analysis Estimated impact of capacity overloads (5% of the time) on a 1MW-10,000 sqft data center Million $ loss! 15

37 Cost analysis Estimated impact of capacity overloads (5% of the time) on a 1MW-10,000 sqft data center Million $ loss! Strong incentives: The attacker only spends US$ <500k (1-15% of the resulting loss)! Data center operator s competitor Against certain tenants to cause service disruptions Creating chaos 15

38 What is the potential impact of power attacks? Million dollar loss and service disruption 16

39 16

40 Attack opportunities are intermittent Random attacks are unlikely to be successful, while constant full power is prohibited 17

41 Attack opportunities are intermittent Random attacks are unlikely to be successful, while constant full power is prohibited Coarse timing (e.g., based on peak hours) is ineffective 17

42 Attack opportunities are intermittent Random attacks are unlikely to be successful, while constant full power is prohibited Coarse timing (e.g., based on peak hours) is ineffective How to achieve a precise timing for successful power attacks? 17

43 In a multi-tenant data center Tenants co-locate their servers in a shared data center space 18

44 In a multi-tenant data center Interconnected through physical processes that may leak power usage information 18

45 Power Heat 19

46 A thermal side channel Hot air can travel to nearby racks, affecting their inlet temperatures Demo of heat recirculation --- 5x speed viewing in Autodesk CFD 20

47 A thermal side channel Hot air can travel to nearby racks, affecting their inlet temperatures Sensor#2 Sensor#1 Temperature trace at select sensors Demo of heat recirculation --- 5x speed viewing in Autodesk CFD 20

48 A high temperature doesn t necessarily mean a high aggregate power usage 21

49 A high temperature doesn t necessarily mean a high aggregate power usage Heat recirculation is spatially non-uniform --- more significant among nearby racks! 21

50 A closer look at thermal network Ù ÜÜÛ Ù ÛÛ S1 Ù ÛÜÜ CRAH S2 S3 22

51 A closer look at thermal network h, Ù ÜÜÛ Ù ÛÛ S1 Ù ÛÜÜ CRAH h, S2 h, S3 22

52 A closer look at thermal network h, Ù ÜÜÛ Ù ÛÛ S1 Ù ÛÜÜ CRAH h, h, h, S2 h, S3 22

53 A closer look at thermal network h,» =» + º» Ê h, Ê +» (») Ù ÜÜÛ Ù ÛÛ S1 Ù ÛÜÜ CRAH h, h, S2 h, h, Sensor reading Heat recirculation impact MIMO model: benign servers, attacker servers, and sensors S3 22

54 A closer look at thermal network h,» =» + º» Ê h, Ê +» (») Ù ÜÜÛ Ù ÛÛ S1 Ù ÛÜÜ CRAH h, h, S2 h, h, Sensor reading Heat recirculation impact MIMO model: benign servers, attacker servers, and sensors S3 Power information is leaked and embedded into temperature readings 22

55 Rewriting the attacker s observation model»» ² ²» ²»» 23

56 Rewriting the attacker s observation model Temperature increase due to benign tenants Noise» ²»» ² º,» º,» º,» º,» º,» º,» º,» º,» Impact from previous slots» º» º» º» º» 23

57 Rewriting the attacker s observation model» ²»» ² º,» º,» º,» º,» º,» º,» º,» º,» Challenges: ² has a size of M by N K, very large for 500,1000 servers Difficult to obtain accurately, and high computational complexity 23

58 Rewriting the attacker s observation model A signal estimation problem with imperfect channel state information» ²»» ² º,» º,» º,» º,» º,» º,» º,» º,» Challenges: ² has a size of M by N K, very large for 500,1000 servers Difficult to obtain accurately, and high computational complexity 23

59 Approximate zone-level thermal network--- Divide data center into zones Perforated Server Racks Tiles C R A H C R A H C R A H C R A H Attacker ² = h, h, h, h, h, h, h, h, h, h, h, h, 24

60 Approximate zone-level thermal network--- Divide data center into zones Perforated Server Racks Tiles C R A H C R A H Zone 3 Zone 2 C R A H Attacker Zone 1 C R A H Zone-level heat recirculation matrix Obtained offline (say, through CFD) Replaced by one value ² = h, h, h, h, h, h, h, h, h, h, h, h, 24

61 Approximate zone-level thermal network--- Divide data center into zones Perforated Server Racks Tiles C R A H C R A H Zone 3 Zone 2 C R A H Attacker Zone 1 C R A H Zone-level heat recirculation matrix Obtained offline (say, through CFD) Error in ² 24

62 Estimating» from» = ²» +» 25

63 Solution: State-augmented robust Kalman filter Estimating» from» = ²» +»» is the augmented state,» is the observation Assumed state transition model:» = ²» + º Predict: Update:» = ²» ³ = ²³ + ³» =» ²» ³ = ² ³ ² + ³ ² = ³ ² ³» =» + ²» ³ = ² ² ² ³ 25

64 Solution: State-augmented robust Kalman filter Estimating» from» ²»»» is the augmented state,» is the observation Assumed state transition model:» ²» º Predict: Update:» ²» ³ ²³ ³»» ²» ³ ² ³ ² ³ ² ³ ² ³»» ²» Avg. error < 3% for estimating benign tenants aggregate power usage ³ ² ² ² ³ 25

65 An attack strategy Attack when the estimate of benign tenants power usage is sufficiently high Wait for some time before attacks Each attack lasts no more than, and no consecutive attacks 26

66 An attack strategy Attack when the estimate of benign tenants power usage is sufficiently high Wait for some time before attacks Each attack lasts no more than, and no consecutive attacks 26

67 An attack strategy Attack when the estimate of benign tenants power usage is sufficiently high Wait for some time before attacks Each attack lasts no more than, and no consecutive attacks e.g., running CPU-intensive computations 26

68 An attack strategy Attack when the estimate of benign tenants power usage is sufficiently high Wait for some time before attacks Each attack lasts no more than, and no consecutive attacks e.g., running CPU-intensive computations 26

69 An attack strategy Attack when the estimate of benign tenants power usage is sufficiently high Wait for some time before attacks Each attack lasts no more than, and no consecutive attacks 26

70 Illustration of well-timed power attacks Experimental settings Simulated real workload traces based on a HP data center layout Consider an attacker sharing a data center capacity of 200kW with benign tenants Attack for no more than 10% of the times 27

71 Illustration of well-timed power attacks Precise timing through a thermal side channel Experimental settings Simulated real workload traces based on a HP data center layout Consider an attacker sharing a data center capacity of 200kW with benign tenants Attack for no more than 10% of the times 27

72 Timing accuracy Attack more frequently with a lower triggering threshold True positive: % of attack opportunities detected Precision: % of attacks being successful 28

73 Timing accuracy Attack more frequently with a lower triggering threshold True positive: % of attack opportunities detected Precision: % of attacks being successful 54% TP (10% for random attacks), and 53% precision 28

74 How could an attacker mount a power attack? Exploiting physical side channels (e.g., thermal/acoustic networks ) 29

75 29

76 Randomizing physical side channels Thermal network model:»» ² ²» ²»» l e l e i s i s A Server Server Server A Server l d 60 F t 60 F o o H C 30

77 Randomizing physical side channels Thermal network model:»» ² ²» ²»» l e l e i s i s A Server Server Server A Server l d 60 F t 60 F o o H C Random supply air temperature Attacker can track the change! 30

78 Randomizing physical side channels Thermal network model:»» ² ²» ²»» l e l e i s i s A Server Server Server A Server l d 60 F t 60 F o o H C Random airflow with adaptive vent tiles Difficult to manage! 30

79 Randomizing physical side channels Thermal network model:»» ² ²» ²»» l e l e i s i s A Server Server Server A Server l d 60 F t 60 F o o H C Heat containment to reduce recirculation Require layout changes 30

80 Randomizing physical side channels Thermal network model:»» ² ²» ²»» l e l e i s i s A Server Server Server A Server l d 60 F t 60 F o o H C Finding and evicting suspicious tenants Intelligent power monitoring to find abnormal power usage patterns 30

81 How to defend a data center against power attacks? A comprehensive investigation required 31

82 A cyber-physical view How about physical security? Utility ATS UPS Generator P D U P D U Securing the cyberspace DDoS attack, network intrusion, privacy protection, etc. [Mirkovic, Sigcomm 04][Zhang CCS 12][Moon CCS 15][Dong CCS 17] 32

83 A cyber-physical view How about physical security? ATS UPS Generator P D U P D U A thermal side channel can help the attacker precisely time its power attacks 32

84 Thanks! 33