Introduction to GANs

Transcription

1 MedGAN ID-CGAN CoGAN LR-GAN CGAN IcGAN b-gan LS-GAN LAPGAN DiscoGANMPM-GAN AdaGAN LSGAN InfoGAN CatGAN AMGAN igan Introduction to GANs IAN SAGAN McGAN Ian Goodfellow, Staff Research Scientist, Google Brain MIX+GAN MGAN CVPR Tutorial on GANs BS-GAN FF-GAN C-VAE-GAN C-RNN-GAN MAGAN 3D-GAN CCGAN Salt Lake City, GoGAN DR-GAN DCGAN AC-GAN BiGAN GAWWN DualGAN CycleGAN Bayesian GAN AnoGAN GP-GAN EBGAN Context-RNN-GAN ALI f-gan MARTA-GAN ArtGAN MAD-GAN DTN BEGAN AL-CGAN MalGAN

2 Generative Modeling: Density Estimation Training Data Density Function

3 Generative Modeling: Sample Generation Training Data Sample Generator (CelebA) (Karras et al, 2017)

4 Adversarial Nets Framework D(x) tries to be near 1 D tries to make D(G(z)) near 0, G tries to make D(G(z)) near 1 Differentiable function D D x sampled from data x sampled from model Differentiable function G (Goodfellow et al., 2014) Input noise z

5 Self-Play 1959: Arthur Samuel s checkers agent (OpenAI, 2017) (Silver et al, 2017) (Bansal et al, 2017)

6 3.5 Years of Progress on Faces (Brundage et al, 2018)

7 <2 Years of Progress on ImageNet Odena et al 2016 monarch butterfly Miyato et al 2017 monarch butterfly goldfin Zhang et al 2018 monarch butterfly goldfinch

8 Self-Attention GAN State of the art FID on ImageNet: 1000 categories, 128x128 pixels Goldfish Redshank Tiger Cat Geyser Broccoli Stone Wall Indigo Bunting (Zhang et al., 2018) Saint Bernard

9 From GAN to SAGAN Depth and Convolution Class-conditional generation Spectral Normalization Hinge loss Two-timescale update rule Self-attention

10 No Convolution Needed to Solve Simple Tasks Original GAN, 2014

11 Depth and Convolution for Harder Tasks Original GAN (CIFAR-10) DCGAN (ImageNet) No convolution One convolutional layer Many convolutional layers (Radford et al, 2015)

12 From GAN to SAGAN Depth and Convolution Class-conditional generation Spectral Normalization Hinge loss Two-timescale update rule Self-attention

13 Class-Conditional GANs (Mirza and Osindero, 2014)

14 AC-GAN: Specialist Generators (Odena et al, 2016)

15 SN-GAN: Shared Generator (Miyato et al, 2017)

16 From GAN to SAGAN Depth and Convolution Class-conditional generation Spectral Normalization Hinge loss Two-timescale update rule Self-attention

17 Spectral Normalization (Miyato et al, 2017)

18 From GAN to SAGAN Depth and Convolution Class-conditional generation Spectral Normalization Hinge loss Two-timescale update rule Self-attention

19 Hinge Loss (Miyato et al 2017, Lim and Ye 2017, Tran et al 2017)

20 From GAN to SAGAN Depth and Convolution Class-conditional generation Spectral Normalization Hinge loss Two-timescale update rule Self-attention

21 Two-Timescale Update Rule

22 From GAN to SAGAN Depth and Convolution Class-conditional generation Spectral Normalization Hinge loss Two-timescale update rule Self-attention

23 Self-Attention Use layers from Wang et al 2018

24 Applying GANs Semi-supervised Learning Model-based optimization Extreme personalization Program synthesis

25 Supervised Discriminator for Semi-Supervised Learning Real Fake Real cat Real dog Fake Hidden units Hidden units Learn to read with 100 labels rather Input Input than 60,000 (Odena 2016, Salimans et al 2016)

26 Semi-Supervised Classification MNIST: 100 training labels -> 80 test mistakes SVHN: 1,000 training labels -> 4.3% test error CIFAR-10: 4,000 labels -> 14.4% test error (Dai et al 2017)

27 Designing DNA to optimize protein binding (Killoran et al, 2017)

28 Personalized GANufacturing (Hwang et al 2018)

29 SPIRAL Synthesizing Programs for Images Using Reinforced Adversarial Learning (Ganin et al, 2018)

30 Other applications Planning World Models for RL agents Fairness and Privacy Missing data Topics covered at workshop: Training data for other agents (Philip Isola, Taesung Park, Jun-Yan Zhu) Inference in other probabilistic models (Mihaela Rosca) Domain adaptation (Judy Hoffman) Imitation Learning (Stefano Ermon)

31 Track updates at the GAN Zoo

32 Questions