Pufferfish: A Semantic Approach to Customizable Privacy

Transcription

1 Pufferfish: A Semantic Approach to Customizable Privacy Ashwin Machanavajjhala ashwin AT cs.duke.edu Collaborators: Daniel Kifer (Penn State), Bolin Ding (UIUC, Microsoft Research) idash Privacy Workshop 9/29/2012 1

2 Outline Background How to define privacy? Correlations: A case for customizable privacy [Kifer-M SIGMOD 11] Pufferfish Privacy Framework [Kifer-M PODS 12] Case Study [Kifer-M PODS 12 & Ding-M 13] Privately handling correlations due to exact release of counts Open Challenges idash Privacy Workshop 9/29/2012 2

3 Data Privacy Problem Utility: Privacy: No breach about any individual Server D B Individual 1 r 1 Individual 2 r 2 Individual 3 r 3 Individual N r N idash Privacy Workshop 9/29/2012 6

4 Data Privacy in the real world Application Data Collector Third Party (adversary) Private Information Function (utility) Medical Hospital Epidemiologist Disease Correlation between disease and geography Genome analysis Hospital Statistician/ Researcher Genome Advertising Google/FB/Y! Advertiser Clicks/Brows ing Social Recommendations Facebook Another user Friend links / profile Correlation between genome and disease Number of clicks on an ad by age/region/gender Recommend other users or ads to users based on social network idash Privacy Workshop 9/29/2012 7

5 Many definitions & several attacks Sweeney et al. IJUFKS 02 K-Anonymity T-closeness Li et. al ICDE 07 L-diversity Machanavajjhala et. al TKDD 07 E-Privacy Linkage attack Background knowledge attack Minimality /Reconstruction attack de Finetti attack Composition attack Differential Privacy Machanavajjhala et. al VLDB 09 Dwork et. al ICALP 06 idash Privacy Workshop 9/29/

6 Differential Privacy Domain independent privacy definition that is independent of the attacker. Tolerates many attacks that other definitions are susceptible to. Avoids composition attacks Claimed to be tolerant against adversaries with arbitrary background knowledge. Allows simple, efficient and useful privacy mechanisms Used in a live US Census Product [M et al ICDE 08] idash Privacy Workshop 9/29/

7 No Free Lunch Theorem It is not possible to guarantee any utility in addition to privacy, without making assumptions about the data generating distribution [Kifer-Machanavajjhala SIGMOD 11] the background knowledge available to an adversary [Dwork-Naor JPC 10] idash Privacy Workshop 9/29/

8 Outline Background How to define privacy? Correlations: A case for customizable privacy [Kifer-M SIGMOD 11] Pufferfish Privacy Framework [Kifer-M PODS 12] Case Study [Kifer-M PODS 12 & Ding-M 12] Privately handling correlations due to exact release of counts Open Challenges idash Privacy Workshop 9/29/

9 Contingency tables Each tuple takes k=4 different values D Count(, ) idash Privacy Workshop 9/29/

10 Contingency tables Want to release counts privately???? D Count(, ) idash Privacy Workshop 9/29/

11 Laplace Mechanism 2 + Lap(1/ε) 2 + Lap(1/ε) 2 + Lap(1/ε) 8 + Lap(1/ε) D Mean : 8 Variance : 2/ε 2 Guarantees differential privacy. idash Privacy Workshop 9/29/

12 Marginal counts 2 + Lap(1/ε) 2 + Lap(1/ε) Lap(1/ε) 8 + Lap(1/ε) Auxiliary marginals published for following reasons: D 1. Legal: 2002 Supreme Court case Utah v. Evans 2. Contractual: Advertisers must know exact demographics at coarse granularities Does Laplace mechanism still guarantee privacy? idash Privacy Workshop 9/29/

13 Marginal counts 2 + Lap(1/ε) 2 + Lap(1/ε) Lap(1/ε) 8 + Lap(1/ε) D Count (, ) = 8 + Lap(1/ε) Count (, ) = 8 - Lap(1/ε) Count (, ) = 8 - Lap(1/ε) Count (, ) = 8 + Lap(1/ε) idash Privacy Workshop 9/29/

14 Marginal counts 2 + Lap(1/ε) 2 + Lap(1/ε) Lap(1/ε) 8 + Lap(1/ε) D Mean : 8 Variance : 2/ke 2 can reconstruct the table with high precision for large k idash Privacy Workshop 9/29/

15 For handling correlations Customizing Privacy Social networks [Kifer-M SIGMOD 11] Utility driven applications Realistic vs Worst-case adversaries [M et al PVLDB 09] Dealing with aggregate secrets [Kifer-M PODS 12] Qn: How to design principled privacy definitions customized to such scenarios? idash Privacy Workshop 9/29/

16 Outline Background How to define privacy? Correlations: A case for customizable privacy [Kifer-M SIGMOD 11] Pufferfish Privacy Framework [Kifer-M PODS 12] Case Study [Kifer-M PODS 12 & Ding-M 12] Privately handling correlations due to exact release of counts Open Challenges idash Privacy Workshop 9/29/

17 What is being kept secret? Pufferfish Semantics Who are the adversaries? How is information disclosure bounded? idash Privacy Workshop 9/29/

18 Sensitive Information Secrets: S be a set of potentially sensitive statements individual j s record is in the data, and j has Cancer individual j s record is not in the data Discriminative Pairs: Spairs is a subset of SxS. Mutually exclusive pairs of secrets. ( Bob is in the table, Bob is not in the table ) ( Bob has cancer, Bob has diabetes ) idash Privacy Workshop 9/29/

19 Adversaries An adversary can be completely characterized by his/her prior information about the data We do not assume computational limits Data Evolution Scenarios: set of all probability distributions that could have generated the data. No assumptions: All probability distributions over data instances are possible. I.I.D.: Set of all f such that: P(data = {r 1, r 2,, r k }) = f(r 1 ) x f(r 2 ) x x f(r k ) idash Privacy Workshop 9/29/

20 Information Disclosure Mechanism M satisfies ε-pufferfish(s, Spairs, D), if for every w ε Range(M), (s i, s j ) ε Spairs Θ ε D, such that P(s i θ) 0, P(s j θ) 0 P(M(data) = w s i, θ) e ε P(M(data) = w s j, θ) idash Privacy Workshop 9/29/

21 Pufferfish Semantic Guarantee Posterior odds of s i vs s j Prior odds of s i vs s j idash Privacy Workshop 9/29/

22 Applying Pufferfish to Differential Privacy Spairs: record j is in the table vs record j is not in the table record j is in the table with value x vs record j is not in the table Data evolution: Probability record j is in the table: π j Probability distribution over values of record j: f j For all θ = [f 1, f 2, f 3,, f k, π 1, π 2,, π k ] P[Data = D θ] = Π rj not in D (1-π j ) x Π rj in D π j x f j (r j ) idash Privacy Workshop 9/29/

23 Applying Pufferfish to Differential Privacy Spairs: record j is in the table vs record j is not in the table record j is in the table with value x vs record j is not in the table Data evolution: For all θ = [f 1, f 2, f 3,, f k, π 1, π 2,, π k ] P[Data = D θ] = Π rj not in D (1-π j ) x Π rj in D π j x f j (r j ) A mechanism M satisfies differential privacy if and only if it satisfies Pufferfish instantiated using Spairs and {θ} (as defined above) idash Privacy Workshop 9/29/

24 Differential Privacy Sensitive information: All pairs of secrets individual j is in the table with value x vs individual j is not in the table Adversary: Adversaries who believe the data is generated using any probability distribution that is independent across individuals Disclosure: ratio of the prior and posterior odds of the adversary is bounded by e ε idash Privacy Workshop 9/29/

25 Characterizing good privacy definition Any privacy definition that can be phrased as follows composes with itself. where I is the set of all tables. idash Privacy Workshop 9/29/

26 Outline Background How to define privacy? Correlations: A case for customizable privacy [Kifer-M SIGMOD 11] Pufferfish Privacy Framework [Kifer-M PODS 12] Case Study [Kifer-M PODS 12 & Ding-M 12] Privately handling correlations due to exact release of counts Open Challenges idash Privacy Workshop 9/29/

27 Induced Neighbor Privacy Differential Privacy: Neighboring tables differ in one value Induced neighbors: tables whose shortest path does not contain another table that is consistent with the marginals (e.g., D and D ) D D D idash Privacy Workshop 9/29/

28 Induced Neighbor Privacy For every pair of induced neighbors For every output D 1 D 2 Adversary should not be able to distinguish between any D 1 and D 2 based on any O O log Pr[A(D 1 ) = O] Pr[A(D 2 ) = O]. < ε (ε>0) idash Privacy Workshop 9/29/

29 Induced Neighbors Privacy and Pufferfish Given a set of count constraints Q, Spairs: record j is in the table vs record j is not in the table record j is in the table with value x vs record j is not in the table Data evolution: For all θ = [f 1, f 2, f 3,, f k, π 1, π 2,, π k ] P[Data = D θ] α Π rj not in D (1-π j ) x Π rj in D π j x f j (r j ), if D satisfies Q P[Data = D θ] = 0, if D does not satisfy Q A mechanism M satisfies induced neighbors privacy if and only if it satisfies Pufferfish instantiated using Spairs and {θ} idash Privacy Workshop 9/29/

30 Computing induced sensitivity 2D case: q all : outputs all the counts in a 2-D contingency table. Marginals: row and column sums. The induced-sensitivity of q all = min(2r, 2c). General Case: Deciding whether S in (q) > 0 is NP-hard. Conjecture: Computing S in (q) is hard (and complete) for the second level of the polynomial hierarchy. idash Privacy Workshop 9/29/

31 Outline Background How to define privacy? Correlations: A case for customizable privacy [Kifer-M SIGMOD 11] Pufferfish Privacy Framework [Kifer-M PODS 12] Case Study [Kifer-M PODS 12 & Ding-M 12] Privately handling correlations due to exact release of counts Open Challenges idash Privacy Workshop 9/29/

32 Privacy of social networks Open Questions Adversaries may use social network evolution models to infer sensitive information about edges in a network [Kifer-M SIGMOD 11] Can correlations in a social network be generatively described? Characterize necessary and sufficient conditions for resilience against classes of attacks. Sufficient conditions for composition. [Kifer-M PODS 12] Attack-resilient relaxations of differential privacy Existing privacy definitions do not provide sufficient utility for certain applications (e.g., social recommendations [M et al, VLDB 11]) idash Privacy Workshop 9/29/

33 Thank you [M et al PVLDB 11] A. Machanavajjhala, A. Korolova, A. Das Sarma, Personalized Social Recommendations Accurate or Private?, PVLDB 4(7) 2011 [Kifer-M SIGMOD 11] D. Kifer, A. Machanavajjhala, No Free Lunch in Data Privacy, SIGMOD 2011 [Kifer-M PODS 12] D. Kifer, A. Machanavajjhala, A Rigorous and Customizable Framework for Privacy, PODS 2012 [Ding-M 12] B. Ding, A. Machanavajjhala, Induced Neighbors Privacy(Work in progress), 2012 idash Privacy Workshop 9/29/