CS230: Lecture 4 Attacking Networks with Adversarial Examples - Generative Adversarial Networks

Size: px

Start display at page:

Download "CS230: Lecture 4 Attacking Networks with Adversarial Examples - Generative Adversarial Networks"

Russell Ross
5 years ago
Views:

1 Go to and use the code CS230: Lecture 4 Attacking Networks with Adversarial Examples - Generative Adversarial Networks Kian Katanforoosh

2 Today s outline I. Attacking NNs with Adversarial Examples II. Generative Adversarial Networks

3 I. Adversarial examples Discovery (2013): several machine learning models, including state-of-the-art neural networks, are vulnerable to adversarial examples A. Attacking a network with adversarial examples B. Defenses against adversarial examples C. Why are neural networks vulnerable to adversarial examples? [Szegedy et al. (2013): Intriguing properties of neural networks] [Ian J. Goodfellow, Jonathon Shlens & Christian Szegedy (2015): Explaining and harnessing adversarial examples]

4 I. A. Attacking a network with adversarial examples Goal: Given a network pretrained on ImageNet, find an input image that is not a iguana but will be classified as an iguana. x Neural network (pretrained on ImageNet) 0.04! 0.85! ! car iguana tomato bike cat crab 1. Rephrasing what we want: Find x such that: ŷ(x) = y iguana = 0 1! 0 0! 0 2. Defining the loss function L( ŷ, y) = 1 2 ŷ(w,b,x) y iguana 2 2 Network (pretrained on ImageNet) After many iterations x L( ŷ, y)? 3. Optimize the image L x x = x α L x [Ian J. Goodfellow, Jonathon Shlens & Christian Szegedy (2015): Explaining and harnessing adversarial examples]

5 I. A. Attacking a network with adversarial examples Question: Will the forged image x look like an iguana?? Space of possible input images Space of images classified as iguanas Space of real images

6 I. A. Attacking a network with adversarial examples Goal: Given a network pretrained on ImageNet, find an input image that is a cat but will be classify as an iguana. Neural network (pretrained on ImageNet) 0.04! 0.85! ! car iguana tomato bike cat crab 1. Rephrasing what we want: Find x such that: And: x = x cat 0 1! ŷ(x) = y iguana = 0 0! 0 2. Defining the loss function L( ŷ, y) = 1 2 ŷ(w,b,x) y iguana 2 2 Network (pretrained on ImageNet) L( ŷ, y) 2 +λ x x cat 2 After many iterations x? 3. Optimize the image L x x = x α L x [Ian J. Goodfellow, Jonathon Shlens & Christian Szegedy (2015): Explaining and harnessing adversarial examples]

7 I. A. Attacking a network with adversarial examples 92% Cat 94% Iguana

8 I. A. Attacking a network with adversarial examples Space of possible input images Space of images classified as iguanas Space of real images Space of images that look real to humans

9 Menti [Alexey Kurakin, Ian J. Goodfellow, Samy Bengio (2017): Adversarial examples in the physical world]

I. B. Defenses against adversarial examples Menti Knowledge of the attacker: White-box Black-box Solution 1 Create a SafetyNet Solution 2 x = y = cat Train on correctly labelled adversarial examples

10 I. B. Defenses against adversarial examples Menti Knowledge of the attacker: White-box Black-box Solution 1 Create a SafetyNet Solution 2 x = y = cat Train on correctly labelled adversarial examples Solution 3 Adversarial training L new = L(W,b,x, y) + λl(w,b,x adv, y) Adversarial logit pairing L new = L(W,b,x, y) + λ f (x;w,b) f (x adv ;W,b) 2 2 [Lu et al. (2017): SafetyNet: Detecting and Rejecting Adversarial Examples Robustly] [Harini Kannan et al. (2018): Adversarial Logit Pairing]

11 I. C. Why are neural networks vulnerable to adversarial examples? See board. Do neural networks actually understand the data? [Yuan et al. (2017): Adversarial Examples: Attacks and Defenses for Deep Learning]

12 II. Generative Adversarial Networks (GANs) A. Motivation B. G/D Game C. Training GANs D. In terms of code E. Nice results [Ian J. Goodfellow, Jonathon Shlens & Christian Szegedy (2015): Explaining and harnessing adversarial examples]

13 II.A - Motivation Motivation: endowing computers with an understanding of our world. Goal: collect a lot of data, use it to train a model to generate similar data from scratch. Intuition: number of parameters of the model << amount of data

14 II.A - Motivation Probability distributions: Samples from the real data distribution real data distribution Matching distributions Goal Image space Samples from the generated distribution generated distribution Image space Image space [Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaogang Wang, Xiaolei Huang, Dimitris Metaxas (2017): StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks]

15 II. Generative Adversarial Networks (GANs) A. Motivation B. G/D Game C. Training GANs D. In terms of code E. Nice results [Ian J. Goodfellow, Jonathon Shlens & Christian Szegedy (2015): Explaining and harnessing adversarial examples]

II.B - G/D Game 100-d random code 0.47 0.

[Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaogang

16 II.B - G/D Game 100-d random code (64,64,3) generated image Generator G (Neural Network) z How can we train G to generate images from the true data distributions? [Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaogang Wang, Xiaolei Huang, Dimitris Metaxas (2017): StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks]

17 II.B - G/D Game 100-d random code Generator G (Neural Network) (64,64,3) generated image Run Adam simultaneously on two minibatches (true data / generated data) z Gradients Binary classification x Discriminator D (Neural Network) y = 0 if x = G(z) y = 1 otherwise Probability distributions Real images (database) Image space

II.B - G/D Game 100-d random code (64,64,3) generated image 0.47 0.

18 II.B - G/D Game 100-d random code (64,64,3) generated image Generator G (Neural Network) z Gradients Binary classification End goal: G is outputting images that are indistinguishable from real images for D x Discriminator D (Neural Network) y = 0 if x = G(z) y = 1 otherwise Probability distribution Real images (database) Image space

19 II.B - G/D Game Training procedure, we want to minimize: Labels: y real y gen is always 1 is always 0 The cost of the discriminator m real J ( D) = 1 y (i).log(d(x (i) )) m real!#### real i=1 "#### $ 1 i=1 (1 y (i) ).log(1 D(G(z (i) ))) gen!##### #" ####### $ cross-entropy 1: D should correctly label real data as 1 cross-entropy 2: D should correctly label generated data as 0 The cost of the generator J (G) = J ( D) = 1 log(1 D(G(z (i) ))) i=1 G should try to fool D: by minimizing the opposite of what D is trying to minimize

20 II. Generative Adversarial Networks (GANs) A. Motivation B. G/D Game C. Training GANs D. In terms of code E. Nice results [Ian J. Goodfellow, Jonathon Shlens & Christian Szegedy (2015): Explaining and harnessing adversarial examples]

21 II.C - Training GANs Saturating cost for the generator: min 1 log(1 D(G(z (i) ))) i=1 max 1 log(d(g(z (i) ))) i=1 min 1 log(d(g(z (i) ))) i=1 J (G) 5 0 Non-saturating cost J (G) = 1 log(d(g(z (i) ))) i=1-20 Saturating cost J (G) = 1 log(1 D(G(z (i) ))) i=1 0 D(G(z)) 1 [Ian Goodfellow (2014): NIPS Tutorial: GANs]

22 II.C - Training GANs Note that: min 1 log(1 D(G(z (i) ))) i=1 max 1 log(d(g(z (i) ))) i=1 min 1 log(d(g(z (i) ))) i=1 New training procedure, we want to minimize: m real J ( D) = 1 y (i).log(d(x (i) )) m real!#### real i=1 "#### $ 1 i=1 (1 y (i) ).log(1 D(G(z (i) ))) gen!##### #" ####### $ cross-entropy 1: D should correctly label real data as 1 cross-entropy 2: D should correctly label generated data as 0 J (G) = 1 log(d(g(z (i) ))) i=1 G should try to fool D: by minimizing this

23 [Lucic, Kurach et al. (2018): Are GANs Created Equal? A Large-Scale Study]

24 II.C - Training GANs Simultaneously training G/D? 5 Non-saturating cost 0 m g J (G) = 1 m g log(d(g(z (i) ))) for num_iterations: for k iterations: update D J (G) Saturating cost m g i=1 update G J (G) = 1 m g log(1 D(G(z (i) ))) i= D(G(z)) 1 [Ian Goodfellow (2014): NIPS Tutorial: GANs]

25 II.C - Training GANs 5 Recap: GANs training tips J (G) 0 Non-saturating cost Saturating cost m g J (G) = 1 m g log(d(g(z (i) ))) i=1 m g J (G) = 1 m g log(1 D(G(z (i) ))) i=1 Modification of the cost function - 0 D(G(z)) 1 Keep D up-to-date with respect to G (k update for D / 1 update for G) (not presented but important) Use Virtual Batchnorm And a lot more, GANs are hard to train! [Soumith et al. (2016): GanHacks] [Lucic, Kurach et al. (2018): Are GANs Created Equal? A Large-Scale Study]

26 II. Generative Adversarial Networks (GANs) A. Motivation B. G/D Game C. Training GANs D. In terms of code E. Nice results [Ian J. Goodfellow, Jonathon Shlens & Christian Szegedy (2015): Explaining and harnessing adversarial examples]

27 II. D. In terms of code [Erik Linder-Norén (Github): eriklindernoren/keras-gan: link]

28 II. D. In terms of code [Erik Linder-Norén (Github): eriklindernoren/keras-gan: link]

29 II. D. In terms of code [Erik Linder-Norén (Github): eriklindernoren/keras-gan: link]

30 II. Generative Adversarial Networks (GANs) A. Motivation B. G/D Game C. Training GANs D. In terms of code E. Nice results [Ian J. Goodfellow, Jonathon Shlens & Christian Szegedy (2015): Explaining and harnessing adversarial examples]

II.E - Nice results Operation on codes Code 1 (64,64,3)

92 Generator G (Neural Network) 1 Code 2 0.47 0.

2 (64,64,3) generated image Code 1 0.12 0.92 Code 2 0.47 0.

31 II.E - Nice results Operation on codes Code 1 (64,64,3) generated image Generator G (Neural Network) 1 Code Code 3 Generator G (Neural Network) (64,64,3) generated image 2 (64,64,3) generated image Code Code Code Generator G (Neural Network) Generator G (Neural Network) 2 Man with glasses - man + woman = woman with glasses [Radford et al. (2015): UNSUPERVISED REPRESENTATION LEARNING WITH DEEP CONVOLUTIONAL GENERATIVE ADVERSARIAL NETWORKS]

32 II.E - Nice results Image Generation: Samples from the generated distribution [Zhang et al. (2017): StackGAN++]

33 II.E - Nice results [Karras et al. (2018): A Style-Based Generator Architecture for Generative Adversarial Networks] v=ksljriaouma&feature=youtu.be

34 II.E - Nice results [Zhu, Park et al. (2017): Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks]

Unpaired images Horse images Zebra images [Zhu, Park et al.

35 II.E - Nice results Goal: Convert horses to zebras on images, and vice-versa. Data? Architecture? Cost? Unpaired images Horse images Zebra images [Zhu, Park et al. (2017): Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks]

36 II.E - Nice results y = 0 if x = G1(H ) y = 1 otherwise (x = z) Architecture? H2Z y = 0 if x = G2(Z ) y = 1 otherwise (x = h) H G1(H) Generator1 (H2Z) Discriminator1 G2(G1(H)) Generator2 (Z2H) Discriminator2 y = 0 if x = G1(H ) y = 1 otherwise (x = z) G1(G2(Z)) y = 0 if x = G2(Z ) y = 1 otherwise (x = h) Discriminator2 Generator1 (H2Z) G2(Z) Discriminator1 Z Generator2 (Z2H) [Zhu, Park et al. (2017): Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks] Z2H

37 II.E - Nice results Loss to minimize? J ( D1) 1 = mreal mreal H 1 (i) log(d1(z )) m log(1 D1(G1(H ))) i=1 gen i=1 J G1 y = 0 if x = G2(Z ) y = 1 otherwise J 1 (i) = log(d1(g1(h ))) mgen i=1 1 = mreal ( D 2) mreal J J D2 Z G2 1 (i) log(d2(h )) m log(1 D2(G2(Z ))) i=1 gen i=1 (i) J=J 1 (i) = log(d2(g2(z ))) mgen i=1 mgen cycle G G1( mgen mgen (G 2) D1 G2 (i) mgen (G1) G G1 G2( mgen y = 0 if x = G1(H ) y = 1 otherwise ( D1) +J (G1) +J ( D 2) +J (G 2) + λj cycle mgen 1 1 (i) (i) (i) (i) = G2(G1(H ) H 1 G1(G2(Z ) Z 1 mgen i=1 mgen i=1

38 II.E - Nice results CycleGANs: Face2ramen + Face detection [Shu Naritomi et al.: Face2Ramen] [Takuya Tako: Face2Ramen using CycleGAN] [Zhu, Park et al. (2017): Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks]

39 II.E - Nice results Pix2Pix: by Christopher Hesse. [Isola et al. (2017): Image-to-Image Translation with Conditional Adversarial Networks]

40 II.E - Nice results Super-resolution [Ledig et al. (2016): Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network]

41 II.E - Nice results Other applications: Beaulieu-Jones et al., Privacy-preserving generative deep neural networks support clinical data sharing. Hwang et al., Learning Beyond Human Expertise with Generative Models for Dental Restorations. Gomez et al., Unsupervised cipher cracking using discrete GANs. Many more

42 Announcements For Tuesday 02/04, 10am: C2M3 Quiz: Hyperparameter tuning, Batch Normalization, Programming Frameworks Programming assignment: Tensorflow C3M1 and C3M2 Quiz: Bird recognition in the city of Peacetopia (case study) Quiz: Autonomous driving (case study) This Friday 02/01: Hands-on section Meet with your mentor TA, to go over project proposal feedback (attendance required.)

43 II.E - Evaluating GANs How to evaluate GANs? Human annotators Inception Score (IS) web-application Indicate if image is fake or real ( ) IS(G) = exp Ε KL( p( y x) p( y)) x pg Image 1 Image 2 Image 3 Measure of image quality Measure of image diversity Image 4 Image 5 Image 6 n y j=1 IS(G) = exp 1 m b ŷ (i) m j b i=1 m b k=1 log ŷ (i) log( 1 ŷ (k ) ) j m j b [Lucic, Kurach et al. (2018): Are GANs Created Equal? A Large-Scale Study] [Salimans et al. (2016): Improved Techniques for Training GANs] Also: Fréchet Inception Distance (FID)

Introduction to Generative Adversarial Networks

Introduction to Generative Adversarial Networks Ian Goodfellow, OpenAI Research Scientist NIPS 2016 Workshop on Adversarial Training Barcelona, 2016-12-9 Adversarial Training A phrase whose usage is in