Making Searchable Encryption Scale to the Cloud. Ian Miers and Payman Mohassel

Transcription

1 Making Searchable Encryption Scale to the Cloud Ian Miers and Payman Mohassel

2 End to end Encryption No encryption Transport encryption End2End Encryption Service provider Service provider Service provider user user user user user user

3 E2E Encrypted Messaging Need Crypto Want Crypto

4 E2E Encrypted Messaging Ban Crypto Need Crypto Want Crypto

5 Deploying E2E encrypted messaging WhatsApp No feature loss Many users probably don t know they are using it imessage Same features as SMS WebRTC video chat

6 Search For some communication mechanisms, people expect search is the canonical example, but not the only one. Slack Any replacement

7 I m not particularly thrilled with building an apartment building which has the biggest bars on every window Jeff Bonforte (Yahoo VP mail and messagin)

8 E2E Encrypted Messaging Need Crypto Want Crypto

9 Searchable Encryption

10 An index for search The f 1 f 5 f 9 f 52 f 67 f 72 f 99 Man f 7 f 11 f 21 f 42 f 67 Bites f 2 Dog f 21 f 27 f 31

11 Index on client, store on server The f 1 f 5 f 9 f 52 f 67 f 72 f 99 Man f 7 f 1 1 f 2 1 f 4 2 f 6 7 Bites f 2 Dog f 2 1 f 2 7 f 3 1

12 Index on client, store on server f 21 f 27 f 31 Dog

13 Index on client, store on server The f 1 f 5 f 9 f 52 f 67 f 72 f 99 f 21 f 27 f 31 Man f 7 f 1 1 f 2 1 f 4 2 f 6 7 Bites f 2 Dog Dog f 2 1 f 2 7 f 3 1

14 An index for search The f 1 f 5 f 9 f 52 f 67 f 72 f 99 Man f 7 f 11 f 21 f 42 f 67 Bites f 2 Dog f 21 f 27 f 31

15 A Naïvely Encrypted Index H(k keyword) E(k,list of files) 8afa2 1c35f dc4cf 9f126

16 Index on client, store on server 8afa2 1c35f dc4cf H(k Dog ) 9f126

17 A Naïvely Encrypted Index H(k keyword) E(k,list of files) 8afa2 1c35f dc4cf 9f126 Leaks term frequency 8afa2 is the most frequent keyword The is the most frequent English word

18 An Inefficient Encrypted index H(k keyword KeyWord_ctr) 8afa2 E(k, f i ) 1c35f 41bb a5l9 5r6n d4c1 For a given keyword, each file containing it is stored in a separate random location This hides keyword frequency in a space efficient way Very inefficient to search: Requires one random read per result Results in a ~25-50x increase in I/O usage Yahoo! Mail search is already IO bound!!! Not viable for a server supporting multiple users who are not paying for it

19 Search at Cloud Scale Many small indexes < 1GB each > 1 Billion accounts Cannot store in memory Must use disk storage IO Bound Fragmented index causes massive increase in io for search A search for one keyword returning N documents takes N times as many reads.

20 Good news search queries are fairly simple Typically single keyword Conjunctive search nice, but not necessary Most searches are on meta data Searches on mail content are rare ~250 searches a second across all users ~300 million monthly active users But we must solve the IO issue.

21 An Inefficient Encrypted index H(k keyword KeyWord_ctr) 8afa2 E(k, f i ) 1c35f 41bb 9f126 dc4cf d4c1

22 IO Efficient search for static indexes

23 Chunked Encrypted Index H(k keyword chunk_ctr) E(k, chunk of files) 8afa2 1c35f 41bb 9f126 Assume we have all documents initially We break up the list into chunks Way more efficient to search Can scale to terabytes Cash et al (Crypto 13, NDSS 14) dc4cf d4c1

24 Problem: updates H(k keyword chunk_ctr) E(k, chunk of files) 8afa2 1c35f 41bb : lost DOG Dog is 9f126 Need to add to Dog entry. But that leaks what we updated 9f126 dc4cf d4c1

25 Problem: updates H(k keyword chunk_ctr) E(k, chunk of files) 8afa2 1c35f 41bb : lost DOG Dog is 9f126 Need to add to Dog entry. But that leaks what we updated 9f126 dc4cf d4c1

26 Problem: updates 8afa2 1c35f 41bb 9f126 dc4cf d4c1 fe52

27 Problem: updates 8afa2 1c35f 41bb 9f126 dc4cf d4c1 fe52

28 IO-DSSE: Scaling Dynamic Searchable Encryption to Millions of Indexes By Improving Locality

29 Obliviously Updateable Index Standard search index 8afa2 1c35f 41bb 9f126 dc4cf fe52 d4c1

30 Obliviously Updateable Index 8afa2 1c35f 41bb 9f126 dc4cf fe52 d4c1

31 Obliviously Updateable Index 8afa2 1c35f 41bb 9f126 dc4cf fe52 d4c1

32 Obliviously Updateable Index 8afa2 1c35f 41bb 9f126 dc4cf fe52 d4c1

33 Obliviously Updateable Index 8afa2 1c35f 41bb 9f126 dc4cf d4c1 fe52

34 Obliviously Updateable Index?

35 Chunked Encrypted Index 8afa2 1c35f 41bb 9f126 dc4cf d4c1 fe52

36 Buffer locally, put full chunks on server Zipfs f 1 law f 7 f 11 really f 2 Keywords have a power law distribution: common ones are really frequent, others are sparse We will end up with too many partial buckets on the client We can t upload partial buckets kills f 2 this f 1 f 5 f 9 f 52 f 67 f 72 idea f 2

37 We need an obliviously updatable index overflow Local Device Data center k 1 k 7 f 3 f 4 f 1 k 1 k 2 f 4 f 4 f 9 k 1 k 4 f 1, f 5, f 8, f 12 f 4, f 1, f 3, f 9 keyword qeuries k 3 f 1 f 8 k 3 f 1 f 8 f 9 k 1 f 2, f 3, f 8, f 11 Obliviously Updatable Index Full Block Index

38 Oblivious RAM ORAM hides locations of access to memory (both reads and writes) How to build ORAM 1. Encrypt memory 2. Shuffle memory locations on reads or writes to hide locations In Path ORAM, shuffling has logarithmic overhead.

39 OUI from Path ORAM RAM 1. Read(for search) 2. Shuffle 3. Read (for search) 4. Shuffle 5. Read/write for update 6. Shuffle

40 Client side stash Path ORAM

41 Client side stash Path ORAM

42 Client side stash Path ORAM

43 Client side stash Path ORAM

44 Client side stash Path ORAM

45 Client side stash Path ORAM

46 From ORAM to an OUI ORAM allows you to write to a location in memory without revealing the location Can add to a partial chunk without revealing we did so. Bandwidth costs get worse was ORAM gets larger Requires you to read and write Log(N)*B bytes for a read of B bytes from an ORAM of size N For 16GB of ORAM, server needs GB of space and reading 4KB takes 350KB read + 350KB write. Storing full index in ORAM requires too much bandwidth

47 From ORAM to an OUI ORAM hides both reads and writes Search explicitly leaks repeated reads Same files are returned each time. Same search token/hash used. No need to hide reads using ORAM Updates may happen in batches

48 OUI from Oblivious RAM 1. Read(for search) 2. Shuffle 3. Read (for search) 4. Shuffle 5. Read/write for update 6. Shuffle

49 Partial ORAM? 1. Read(for search) 2. Read (for search) 3. Read/write for update 4. Shuffle

50 OUI 1. Read(for search) 2. Shuffle 3. Read (for search) 4. Shuffle 5. Read/write for update 6. Shuffle

51 OUI 1. Read(for search) 2. Read (for search) 3. Shuffle + Shuffle 4. Read/write for update 5. Shuffle

52 OUI 1. Read(for search) 2. Read (for search) 3. Shuffle + Shuffle 4. Read/write for update 5. Shuffle

53 OUI 1. Read(for search) 2. Read (for search) 3. Shuffle + Shuffle 4. Read/write for update 5. Shuffle

54 Reads

55 Why not just read directly?

56 Leaks updates

57 OUI from ORAM Searching triggers a read and write of Log(n)*B data To avoid Log(N)*B read +write for each search Just read address for chunk for given keyword Defer read and write until later (i.e. when the phone is plugged in and on Wi-Fi) Search is constant bandwidth and has nice locality All updates must happen after deferred IO is done We get some savings from batching the IO together Multiple searches on the same keyword are free

58 OUI from ORAM Read directly from tree for search BUT Must complete full path read and write prior to any updates Call these deferred reads

59 Batched reads and writes Deferred (full reads ) reads and updates are not random events They will happen in groups either When an comes in we get many updates We might update the non local index only once a day (if system is not multi client) Batched reads and writes reduce the amount of data read and written For n full reads/ writes, The root is only updated once instead of n times Its children once instead n/2 times, etc

60 Deferred Reads

61 Deferred Reads

62 Deferred Reads

63 Deferred Reads + Batching

64 Batched update

65 Performance IO Savings (percentage) vs Simple encrypted index( including all previous works under purely dynamic insertion) Savings just for search (ignoring updates) Oblivious index from path ORAM

66 Conclusion Searchable encryption might be feasible for cloud based messaging with effort It pays to examine problems in context You can always get better performance by relaxing security assumptions Sometimes the relaxation is inherent to the setting and free

67 8afa2 1c35f 41bb 9f126 dc4cf d4c1 Updates Query local, ORAM, and index with efficient access Update : Buffer locally, overflow to ORAM, then commit full chunks to index Defer ORAM I/O from queries until update period Requirements 40 to 250mb of client storage to store a list of keywords Client has fast internet sometimes Ideally, client has large local buffer Local buffer keyword data Query on keyword Obliviously updateable Index (OUI)

68 Client side stash Path ORAM

69 Client side stash Path ORAM

70 Client side stash Path ORAM

71 Client side stash Path ORAM

72 Client side stash Path ORAM

73 Client side stash Path ORAM

74 From ORAM to an OUI ORAM allows you to write to a location in memory without revealing the location Can add to a partial chunk without revealing we did so. Bandwidth costs get worse was ORAM gets larger Requires you to read and write Log(N)*B bytes for a read of B bytes from an ORAM of size N For 16GB of ORAM, server needs GB of space and reading 4KB takes 350KB read + 350KB write. Storing full index in ORAM requires too much bandwidth

75 From ORAM to an OUI ORAM hides both reads and writes Search explicitly leaks repeated reads Same files are returned each time. Same search token/hash used. No need to hide reads using ORAM Updates may happen in batches

76 Reads

77 Why not just read directly?

78 Leaks updates

79 OUI from ORAM Read directly from tree for search BUT Must complete full path read and write prior to any updates Call these deferred reads

80 Batched reads and writes Deferred (full reads ) reads and updates are not random events They will happen in groups either When an comes in we get many updates We might update the non local index only once a day (if system is not multi client) Batched reads and writes reduce the amount of data read and written For n full reads/ writes, The root is only updated once instead of n times Its children once instead n/2 times, etc

81 Deferred Reads

82 Deferred Reads

83 Deferred Reads

84 Deferred Reads + Batching

85 Batched update