Dimensionality reduction as a defense against evasion attacks on machine learning classifiers

Transcription

1 Dimensionality reduction as a defense against evasion attacks on machine learning classifiers Arjun Nitin Bhagoji and Prateek Mittal Princeton University DC-Area Anonymity, Privacy, and Security Seminar, Fall

2 The Sixfold Path 1. Motivation 2. Machine learning, briefly 3. Adversaries and attacks 4. Defenses 5. Results 6. Ongoing Work and Extensions 2

3 Motivation 3

4 The The Ubiquity Ubiquity of of Machine Machine Learning learning Images Malware Machine Learning Systems Sound SVMs, Neural Networks, Random Forests, Cat Dog Malicious Benign Who are you? Open mail 4

5 Critical Applications of ML 5

6 Vulnerability of ML Modified by adversary Classified as 5 Classified as 0 Figure taken from Explaining and harnessing adversarial examples by Goodfellow et. al. 6

7 Machine Learning, Briefly 7

8 Typical ML Pipeline Training phase Training Data Labels Training data Training algorithm 1. Starts with f 0,untrained classifier 2. Optimizes to find f, which labels most data correctly Test phase Test Data Labels Trained ML Classifier y = f(x) Verify Test data Predicted labels 8 To find misclassification percentage

9 Support Vector Machines (SVMs) Support vectors Linear SVM on UCI Human Activity Recognition dataset Sitting vs. Walking Margin: Distance between parallel hyperplanes separating data Maximum margin separating hyperplane Image courtesy: Wikimedia Foundation Max. margin hyperplane: Halfway in between parallel hyperplanes 9

10 Adversaries and Attacks 10

11 Adversarial setup During the test phase (or once deployed) Trained ML Classifier y = f(x) x adv y adv = f(x adv ) y adv? = y Minimally modifies legitimate inputs to induce misclassification at test time Assume powerful adversary: has knowledge of trained classifier and input datasets Previous work has shown black-box ML systems can be reverse engineered enough to carry out evasion attacks using queries 11

12 Evasion Attack on Linear SVM x Classified as 7 Adversarial Class Original Class x adv Classified as 3! x adv = x w k. kw k k 2 Adversarial image with =2.0. Leads to 100% misclassification on test set. 2 [0, 1) Attack on Linear SVMs controls the amount of perturbation added (typically small) 12

13 Not just Images Images Szegedy et. al. (2014), Papernot et. al. (2015) Xu et. al. (2016) Malware Machine Learning Systems Sound SVMs, Neural Networks, Random Forests, Cat Malicious Dog Benign Who are you? Open mail Carlini et. al. (2016) 13

14 Defenses 14

15 Defense Desiderata Add defense here and/or modify the classifier x adv Trained ML Classifier y = f(x) y def = f def (x def ) y def = y Maintain classification accuracy (utility) Low efficiency overhead Improve security, i.e. resistance to adversarial samples Tunable, i.e. tradeoff utility, efficiency and security Effective in a range of settings 15

16 Limitations of Existing Defenses Focused on specific classifier families Resistance to adversary with knowledge of defense is unclear Only valid for specific attacks Case in point Proposed defense for neural networks of Papernot et. al. (2015) broken by modified attack in Carlini et. al. (2016) 16

17 Dimensionality reduction Preprocessing step for high-dimensional data Novel use as a defense against evasion attacks Various Algorithms tried Principal Component Analysis (PCA) Random Projections Kernel PCA 17

18 Principal Component Analysis d k u i : 1 n XT X u i = i u i k n d n Data Principal Components Reduced Dimension Data Principal component Use Principal Component Analysis (PCA) to reduce dimension Identifies top k directions of highest variance Directions: eigenvectors of covariance matrix 18

19 Reconstruction-based defense Step 1: Compute ˆx =, reconstructed input kx hx, u i iu i i=1 Initial adversarial example (Input may be benign or adversarial) Step 2: Find f(ˆx), where f( ) is the original classifier After reconstruction Intuition Perturbation added in existing attacks has low variance Reconstruction step removes perturbation 19

20 Re-training based defense f k Step 1: Train new classifier on (red. dim. training data) X train k Step 2: Project all inputs to k dimensions Step 3: Use f k to classify subsequent inputs Intuition For SVMs, margin increases for lower-dimensional classifiers 20

21 Results 21

22 Validation of defenses Do the defenses work for 1. different datasets? 2. various ML classifiers? 3. different attacks on the same classifier? 4. dimensionality reduction algorithms other than PCA? 22

23 Datasets used MNIST: Handwritten digits from 0 to 9. Extensively studied from the attack perspective. Enables visual evaluation of defenses. UCI HAR: Measurements obtained from a smartphone's accelerometer and gyroscope. Six activities: Walking, Walking Upstairs, Walking Downstairs, Sitting, Standing and Laying. 23

24 Linear SVM: Re-training Defense for MNIST Reconstruction 92.33% No defense 9.43% =0 24

25 Linear SVM: Reconstruction Defense for HAR 77.88% 28.74% 25

26 Classification accuracy HAR dataset 96.7% MNIST dataset 91.5% 91.2% 91.6% Takeaway: Defenses work for two different datasets with minimal utility loss 26

27 Neural Network: Reconstruction Defense for MNIST 96.86%, Utility: 97.71% 33.77%, Utility: 96.43% Re-training gives 7.17% misclassification at utility of 97.19%! 27

28 Ongoing Work and Extensions 28

29 Strategic attacks What if the adversary is aware of the defenses? For PCA defense, heuristically, adversary adds perturbation in directions with large projection along principal components Ongoing evaluations suggest defenses are effective even for strategic adversary 29

30 Extensions Formal definitions of classifier security Proofs for the effectiveness of dimensionality reduction Optimal attacks against various defenses and classifiers 30

31 That s all folks! Questions? 31

32 Backup slides 32

33 Evasion Attack on Neural Networks Classified as 5 Fast Sign Gradient attack x adv = x + sign(rj f (x,y, )) 2 [0, 1] Classified as 0! where J f ( ) is the loss function of the neural network Adversarial image with =0.15 Leads to 99% misclassification on test set. 33

34 Neural Networks p 1 p p 3 p C Input layer Output softmax layer Hidden layers Function that takes an input x and outputs a vector of probabilities y, giving the probability of each class 34

35 Motivation Machine Learning systems are ubiquitous BUT Vulnerable to adversarially modified inputs SO Good defenses are needed 35

36 Dimensionality reduction as a defense against evasion attacks on machine learning classifiers min r krk 2 subject to f(x + r) =l, Classified as 8 x + r 2 [0, 1] d. x re is the input, perturbation, r and Classified as 3 neural network. f 36

37 Linear SVM: Re-training Defense for MNIST 37

38 Linear SVM: Reconstruction Defense for Reconstruction MNIST 92.33% No defense 10.39% =0 38

39 Linear SVM: Re-training defense for HAR 39