Survey of Techniques for Node Differential Privacy

Size: px

Start display at page:

Download "Survey of Techniques for Node Differential Privacy"

Baldwin Miles
5 years ago
Views:

1 Survey of Techniques for Noe Differential Privacy Sofya Raskhonikova Penn State University, on sabbatical at BU for privacy year, also visiting Harvar 1

Publishing information about graphs Many types of ata can be represente as graphs where noes correspon to iniviuals eges capture relationships Frienships in online social network Financial

2 Publishing information about graphs Many types of ata can be represente as graphs where noes correspon to iniviuals eges capture relationships Frienships in online social network Financial transactions communication Health networks (of octors an patients) Romantic relationships image source Image source: American J. Sociology, Bearman, Mooy, Stovel Privacy is a big issue! 2

Differential privacy (for graph ata) Graph G Truste curator A ( queries answers ) Users Government, researchers, businesses (or) malicious aversary Differential privacy [Dwork McSherry Nissim Smith

3 Differential privacy (for graph ata) Graph G Truste curator A ( queries answers ) Users Government, researchers, businesses (or) malicious aversary Differential privacy [Dwork McSherry Nissim Smith 06] An algorithm A is ε-ifferentially private if for all pairs of neighbors G, G an all sets of answers S: Pr A G S e ε Pr A G S image source 3

4 Two variants of ifferential privacy for graphs Ege ifferential privacy G: G : Two graphs are neighbors if they iffer in one ege. Noe ifferential privacy G: G : Two graphs are neighbors if one can be obtaine from the other by eleting a noe an its ajacent eges. 4

5 Differentially private analysis of graphs Graph G Truste curator A Two conflicting goals: utility an privacy Impossible to get both in the worst case Want: ifferentially private algorithms that are accurate on realistic graphs ifferentially private (for all graphs) accurate for a subclass of graphs ( queries answers ) Users Government, researchers, businesses (or) malicious aversary image source 5

6 Graph statistics Number of eges Counts of small subgraphs (e.g., triangles, k-triangles, k-stars) (If noes have colors, noe colors can be specifie in template graphs.) Degree istribution fraction of noes of egree Degree 6

7 Ege ifferentially private algorithms pre-2013: graph statistics an techniques number of triangles, MST cost [Nissim Raskhonikova Smith 07] Smooth sensitivity egree istribution [Hay Rastogi Miklau Suciu 09, Hay Li Miklau Jensen 09, Karwa Slavkovic 12, Kifer Lin 13] Global sensitivity an postprocessing small subgraph counts [Karwa Raskhonikova Smith Yaroslavtsev 11] Smooth sensitivity; Propose-Test-Release [Dwork Lei 09] cuts Ranom projections, global sensitivity [Blocki Blum Datta Sheffet 12] Iterative upates [Hart Rothblum 10, Gupta Roth Ullman 12] Kronecker graph moel parameters [Mir Wright 12] Postprocessing of [KRSY 11] 7

8 Other efinitions Ege private against Bayesian aversary (weaker privacy) small subgraph counts [Rastogi Hay Miklau Suciu 09] Noe zero-knowlege private (stronger privacy than DP) average egree, istances to nearest connecte, Eulerian, cycle-free graphs (privacy only for boune-egree graphs) [Gehrke Lui Pass 12] Sublinear-time algorithms + global sensitivity 8

9 Toay New techniques [Blocki Blum Datta Sheffet 13, Kasiviswanathan Nissim Raskhonikova Smith 13, Chen Zhou 13, Raskhonikova Smith] achieve noe ifferential privacy give better ege ifferentially private algorithms Guarantees for resulting algorithms noe ifferentially private for all graphs accurate for a subclass of graphs, which inclues graphs with sublinear (not necessarily constant) egree boun graphs where the tail of the egree istribution is not too heavy ense graphs goo performance in experiments on real graphs for simple statistics 9

10 New Techniques 1. Truncation + smooth sensitivity [BBDS 13, KNRS 13] Generic reuction to privacy over boune-egree graphs Fast 2. Lipschitz extensions [BBDS 13, KNRS 13] Releasing number of eges via max flow [KNRS 13] Releasing subgraph counts via linear programming [KNRS 13] Releasing egree istribution: via convex programming [RS] Slower, but more accurate 3. Recursive mechanism [CZ 13] (can be viewe as an efficient construction of Lipschitz extensions) Releasing (generalization of) subgraph counts The same accuracy as (2), but slower Unifying iea: ``projections on ``graphs with low sensitivity 10

11 Basic question Graph G Truste curator A ( statistic f approximation to f(g) ) Users Government, researchers, businesses (or) malicious aversary How accurately can an ε-ifferentially private algorithm release f(g)? 11

12 Challenge for noe privacy: high sensitivity Global sensitivity of a function f is f = max f G f G noe neighbors G,G Examples: f (G) is the number of eges in G. f (G) is the number of triangles in G. f = n. f = n 2. 12

13 Challenge for noe privacy: high sensitivity Global sensitivity of a function f is f = max f G f G noe neighbors G,G Local sensitivity LS f (G), max G : neighbor of G New measure of sensitivity [Chen Zhou 13] Down sensitivity DS f (G) is f G f G, is also high. max f G f G :subgraph neighbor of G G. Iea: project onto graphs with low own sensitivity. 13

14 Projections on graphs of small egree [BBDS 13,KNRS 13] Let G = family of all graphs, G = family of graphs of egree. G Notation. f = global sensitivity of f over G. f = global sensitivity of f over G. Observation. f is low for many useful f. Examples: f = (compare to f = n) G f = 2 (compare to f = n 2 ) Goal: privacy for all graphs Iea: ``Project on graphs in G for a carefully chosen << n. 14

15 Metho 1 Truncation + local-sensitivitybase frameworks 15

f How it works: Truncation T(G) outputs G with noes of egree > remove.

16 Metho 1: reuction to privacy over G [KNRS 13] Input: Algorithm B that is noe-dp over G Output: Algorithm A that is noe-dp over G, has accuracy similar to B on nice graphs Time(A) = Time(B) + O(m+n) Reuction works for all functions f How it works: Truncation T(G) outputs G with noes of egree > remove. Answer queries on T(G) instea of G G G high f low f via local-sensitivity-base frameworks [NRS 07,Dwork Lei 09, KRSY 11] T G A ε T S T(G) boun l on LS T (G) B ε/l 16

17 Metho 2 Lipschitz extensions 17

18 Metho 2: Lipschitz extensions [BBDS 13,KNRS 13] A function f is a Lipschitz extension of f from G to G if f agrees with f on G an f = f Release f via GS framework [DMNS 06] G G high f f = f low f f = f There exist Lipschitz extensions for all real-value functions Lipschitz extensions can be compute efficiently for subgraph counts [KNRS 13] egree istribution [RS] Vector of real values 18

19 Lipschitz extension of f : flow graph For a graph G=(V, E), efine flow graph of G: ' 2' s 3 3' t 4 4' 5 5' A ege (u, v ) iff u, v E. v flow (G) is the value of the maximum flow in this graph. Lemma. v flow (G)/2 is a Lipschitz extension of f. 19

20 Lipschitz extension of f : flow graph For a graph G=(V, E), efine flow graph of G: 1 2 1/ 1 eg v / eg v / 1' 2' s 3 3' t 4 4' 5 5' A ege (u, v ) iff u, v E. v flow (G) is the value of the maximum flow in this graph. Lemma. v flow (G)/2 is a Lipschitz extension of f. Proof: (1) v flow (G) = 2f (G) for all G G (2) v flow = 2 f 20

Lipschitz extension of f : flow graph For a graph G=(V, E), efine flow graph of G: 1 2 1 1' 2' s 3 3' t 4 5 4' 5' 6 6' v flow (G) is the value of

21 Lipschitz extension of f : flow graph For a graph G=(V, E), efine flow graph of G: ' 2' s 3 3' t 4 5 4' 5' 6 6' v flow (G) is the value of the maximum flow in this graph. Lemma. v flow (G)/2 is a Lipschitz extension of f. Proof: (1) v flow (G) = 2f (G) for all G G (2) v flow = 2 f = 2 21

22 Lipschitz extensions via linear programs For a graph G=([n], E), efine LP with variables x T for all triangles T: Maximize T= of G x T 2 T:v V(T) = f v LP (G) is the value of LP. Lemma. v LP (G) is a Lipschitz extension of f. Computable in time O(n + f G ) using [Plotkin Shmoys Taros] If we use δ instea of 2 0 x T 1 for all triangles T for all noes v as a boun, get a function with GS δ. It is a Lipschitz extension from a large set that inclues G. x T Can be generalize to other counting queries. 22

23 Lipschitz extension for a function that outputs a vector 23

24 Lipschitz extension of egree istribution via convex programming [RS] 1 1 1' 2 2' s 3 3' t 4 4' f v = flow into vertex v 5 flow graph of G 5' Can we use f v as a proxy for egree of v? Issue: max flow is not unique. Want: unique flow that has low global sensitivity. 24

25 Lipschitz extension of egree istribution via convex programming [RS] 1 1 1' 2 2' s f v = flow into vertex v Let h(x) = x(2 x) flow graph of G Iea: maximize v h(f v ) instea of v f v. Let φ be the flow maximizing v h(f v ), an f be the vector of s-out-flows in φ. f is unique, since h is strictly concave. Poly-time computable (e.g., [Lee Rao Srivastava 13]). 3' 4' 5' 2 t h(x) x 25

26 Lipschitz extension of egree istribution via convex programming [RS] 1 1 1' 2 2' s 3 3' 4 4' f 5 5' v = flow into vertex v φ = argmax v h(f v ). If G G, then f v = eg(v) for all v, since h is strictly increasing on [0,]. Lemma. l 1 global sensitivity f 3. 2 t h x = x(2 x) x 26

27 Lipschitz extension of egree istribution via convex programming [RS] s e s Lemma. l 1 global sensitivity f 3. Proof sketch: Consier g = φ new φ ol g is a union of simple s-t-paths an cycles of several types: 1. s-t-paths an cycles using e s. Contribute 2 to f new 2. s-t-paths using e t. 3. Cycles using e t. 4. Remaining paths an cycles. 1 1' 2' 3' 4' 5' 6' et t f ol 0 Do not exist. Use strict concavity of h 27 1

28 Releasing egree istribution: summary ' 2' s f v = flow into vertex v 1. Construct flow graph of G. 2. Compute s-out-flows f φ = argmax v h(f v ). 3. Release vector f, with Lap 3 per coorinate. ε 4. Use post-processing techniques by [Hay Rastogi Miklau Suciu 09, Hay Li Miklau Jensen 09, Karwa Slavkovic 12, Kifer Lin 13] to remove some noise. 3' 4' 5' 2 t h x = x(2 x) x 28

29 Summary 1. Truncation + smooth sensitivity [BBDS 13, KNRS 13] Generic reuction to privacy over boune-egree graphs Fast 2. Lipschitz extensions [BBDS 13, KNRS 13] Releasing number of eges via max flow [KNRS 13] Releasing subgraph counts via linear programming [KNRS 13] Releasing egree istribution: via convex programming [RS] Slower, but more accurate 3. Recursive mechanism [CZ 13] (can be viewe as an efficient construction of Lipschitz extensions) Releasing (generalization of) subgraph counts The same accuracy as (2), but slower 29

30 Experimental evaluation 30

5 Time, secs # Δs CA-AstroPh 18,772 396,220 504 0.

31 Experiments for the flow an LP metho [Lu] Graph # noes # eges Max egree Time, secs # eges CA-GrQc 5,242 28, CA-HepTh 9,877 51, Time, secs # Δs CA-AstroPh 18, , ,222 com-blp-ungraph 317,080 2,099, com-youtube-ungraph 1,134,890 5,975,248 28,

32 Other experimental results [Lu] showe that truncation is less accurate than flow an LPbase methos. [Chen Zhou 13] provie experimental evaluation on ranom an real-worl graphs. (Mostly) better accuracy than in [KRSY 11] for ege-dp algs. Longer running times than in flow- an LP-base methos implemente by [Lu] for noe-dp algorithms. 32

33 Conclusions We are close to having ege-private an noe-private algorithms that work well in practice for basic graph statistics. Interesting projection techniques that might be useful for esign of DP algorithms in other contexts. 33

34 Open questions New techniques: To which other queries o they apply? Specific queries: Releasing cuts with noe-dp Releasing pairwise istances between noes with DP 34

35 Open questions (continue) DP synthetic graphs Simultaneous release of answers to many queries What are the right notions of privacy for graph ata? What are the right ways to state utility guarantees? Some proposals in [KRSY 13, KNRS 13, Chen Zhou 13] Social networks have noe an ege attributes. What queries are useful? Hypergraphs (that capture relationships such as people appearing on the same photo ) 35

Graph Analysis with. Penn State University. Kobbi Nissim (Ben-Gurion U. and Harvard U.),

Graph Analysis with. Penn State University. Kobbi Nissim (Ben-Gurion U. and Harvard U.), Graph Analysis with Node Differential Privacy Sofya Raskhodnikova Penn State University Joint work with Shiva Kasiviswanathan (GE Research), Kobbi Nissim (Ben-Gurion U. and Harvard U.), Adam Smith (Penn