A Machine Learning Approach to Privacy-Preserving Data Mining Using Homomorphic Encryption

Transcription

1 A Machine Learning Approach to Privacy-Preserving Data Mining Using Homomorphic Encryption Seiichi Ozawa Center for Mathematical Data Science Graduate School of Engineering Kobe University

2 2 What is PPDM? Big data consists of lots of sensitive private information such as names, addresses, phone number, etc. Obviously, we should conduct proper masking to such sensitive data to analyze, but such masking could erase valuable information from a database. How can we analyze big data to extract useful rules in a legitimate way? Privacy-Preserving Data Mining (PPDM) Privacy-Preserving Machine Learning (PPML)

3 3 Approaches to PPDM (). Homomorphic Encryption A form of encryption that allows computation on ciphertexts. - Additive HE: Pailier - Multiplicative HE: Unpadded RSA, ElGamal - Fully HE: addition + mulitiplication 2. Garbled Circuits A cryptographic protocol that enables two-party secure computation in which two mistrusting parties can jointly evaluate a function over their private inputs without the presence of a trusted third party. (Wikipedia)

4 4 Approaches to PPDM (2) 3. Secret Sharing A form of approaches to distributing a secret amongst a group of participants, each of whom is allocated a share of the secret. The secret can be reconstructed only when a sufficient number, of possibly different types, of shares are combined together; individual shares are of no use on their own. (Wikipedia) 4. Perturbation Approaches Adding random noise to avoid from leaking information with a mechanism satisfying Differential Privacy. - Input Perturbation - Algorithm Perturbation - Output Perturbation

5 5 A New Direction Using PPDM - Fintech Current Approach New Approach Transaction Data ATM Data Internet Banking Bank A Bank B Bank C Other Data etc... Bank A Analyst Privacy-Preserving Data Mining Engine Transact ion Data Transact ion Data Integrate Automate ATM Data Internet Banking Bank B 5 ATM Data Internet Banking Bank C Individual Analysis - Detection of illegal money transfer - Calculate proper interest rate Machine Learning over Encrypted Data

6 6 Privacy-Preserving Platform on Cloud Computing Additively Homomorphic Encryption Privacy Preserving Extreme Learning Machine

7 7 Sharing Roles in Computation Data Contributor Nonlinear calculation with an activate function Multiplication and inner products Outsourced Server Summation of N data with Additive HE Data Analyst Calculating an inverse matrix and weights.

8 8 Privacy Preserving Extreme Learning Machine

9 9 Performance Evaluation Data Sets: 4 Bench Mark Datasets in Machine Leaning Repository Encryption: LWE base Homomorphic Encryption (L:#Hidden Units)

10 0 Privacy-Preserving Naïve Bayes Classification Classification Using Posterior Probability Naïve Bayes Classification x: input, λ: #classes Assume independency for x Probability Estimation m: #training samples, m i : #class i samples m it : #occurrences of x t in class i samples

11 Privacy-Preserving Naïve Bayes Classification Calculation of Posterior Probability d: dimensionality Obtain class i* with largest Posterior Probability for the other labels j Multiplying both sides by, Securely computed using homomorphic encryption

12 2 System Configuration - Multi-Party Computation CS: No access to Alice s secret key. Alice s public key CS2 knows Alice s secret key. - CS and CS2 do not collude. - All participants are assumed honest-but-curious. (Follow protocols but may want to know data information.)

13 3 Computation of E pk (m i d- ) (NOTE) provide Alice Alice s class labels Encrypted labels (homomorphic encryption) CS CS E pk (m i d- ) Addition on encrypted values Element-wise multiplication

14 4 Computation of E pk (m i d- Πm jt ) (NOTE) Key point of calculation of m jt Encrypted one-hot encoded feature t Encrypted labels And accumulate for every samples

15 5 Computation of E pk (m i d- Πm jt ) (NOTE) Key point of calculation of m jt Encrypted one-hot encoded feature t Encrypted labels Still CS cannot observe actual values

16 6 Maximization of Posterior Probability where Alice s secret key CS CS2 Construct Y CS2 Random rotation prevents CS2 from observing actual index (classification result) For enhanced security, we are considering adoption of garbled circuit method here

17 7 Sending Classification Result Rotation parameter k (ex. 2) Classification result is computed using information from both sides CS One-hot vector z Bob (ex) Classification result: 2 CS2 which has at the place of maximum column from rotated matrix (ex)

18 8 Performance Evaluation Data Sets: Iris dataset from UCI ML Repository (training:test=80:20) Preprocessing: Each feature (real-valued) are encoded into one-hot encoding(dimension: 5) Encryption: HELib (implementation of Brakerski-Gentry-Vaikuntanathan scheme) Prediction Accuracy (Smoothing: allocation of small value to 0 frequencies) Execution Time Protocol : computation of E pk (m i d- ) Protocol 2: computation of E pk (H) Protocol 3: obtaining classification result

19 9 Concluding Remarks. Privacy-Preserving Data Mining (PPDM) suggests a new direction of AI applications for Big Data. 2. Aggregation of big data provided by multiple organizations could bring a new impact in Big Data analysis. 3. Two machine learning approaches (i.e., PP-ELM and PP-NBC) are introduced. 4. The number of papers on PPDM/PPML is rapidly increasing at top conferences (ICML, USENIX, ACM CCS, etc.).