GENERATIVE ADVERSARIAL NETWORKS (GAN) Presented by Omer Stein and Moran Rubin
|
|
|
- Andrew Pearson
- 5 years ago
- Views:
Transcription
1 GENERATIVE ADVERSARIAL NETWORKS (GAN) Presented by Omer Stein and Moran Rubin
2 GENERATIVE MODEL Given a training dataset, x, try to estimate the distribution, Pdata(x) Explicitly or Implicitly (GAN) Explicitly estimation: Implicitly estimation:
3 GENERATIVE MODEL Given a training dataset, x, try to estimate the distribution, Pdata(x) Explicitly or Implicitly (GAN) Explicitly estimation: Variational Autoencoders (VAE) Implicitly estimation:
4 GENERATIVE MODEL Given a training dataset, x, try to estimate the distribution, Pdata(x) Explicitly or Implicitly (GAN) Explicitly estimation: Variational Autoencoders (VAE) Implicitly estimation: Generative Adversarial Networks (GAN)
5 GENERATIVE MODEL Given a training dataset, x, try to estimate the distribution, Pdata(x) Explicitly or Implicitly (GAN) Explicitly estimation: Variational Autoencoders (VAE) Implicitly estimation: Generative Adversarial Networks (GAN)
6 WHY GENERATIVE MODEL? Realistic samples for artwork, colorization, etc. Training GM can enable inference of latent representation that can be useful as general features GM can be trained with missing data and can provide predictions on inputs that are missing data GM enable machine learning to work with multimodel outputs (produce multiple different correct answers)
7 GENERATIVE ADVERSARIAL NETWORKS (GAN) Problem: Want to sample from complex, high dimensional training distribution. No direct way to do this! Solution: Sample from a simple distribution e.g. random noise. Learn complex transformation to training distribution.
8 GENERATIVE ADVERSARIAL NETWORKS (GAN) Problem: Want to sample from complex, high dimensional training distribution. No direct way to do this! Solution: Sample from a simple distribution e.g. random noise. Learn complex transformation to training distribution. Complex transformation = neural network!
9 GENERATIVE ADVERSARIAL NETWORKS (GAN) Problem: Want to sample from complex, high dimensional training distribution. No direct way to do this! Solution: Sample from a simple distribution e.g. random noise. Learn complex transformation to training distribution. Complex transformation = neural network!
10 Latent random variable TRAINING GAN : TWO PLAYER GAME x (1),,x (m) ~P data Real World Images Sample x~p data z (1),,z (m) ~P z Discriminator Real Fake Loss Generator Sample D x D(G(z)) G(z)~P g
11 Latent random variable TRAINING GAN : TWO PLAYER GAME x (1),,x (m) ~P data Real World Images Sample x~p data Discriminator Goal: Try to distinguish between real and fake images z (1),,z (m) ~P z Discriminator Real Fake Loss Generator Sample D x D(G(z)) G(z)~P g Generator Goal: Try to fool the discriminator by generating real looking images
12 Latent random variable TRAINING GAN : TWO PLAYER GAME Real World Images Sample police counterfeiter Real Fake Loss Sample
13 Latent random variable TRAINING GAN : TWO PLAYER GAME x (1),,x (m) ~P data Discriminator Goal: Discriminate between real and generated images: D x = 1, where x is a real image D G(z) = 0, where G(z) is a generative image Real Images Sample z (1),,z (m) ~P z x~p data Discriminator Real Fake Loss Generator G(z)~P g Sample D x D(G(z)) Generator Goal: generate an image G(z) such that: P g =P data D G Z = 1
14 TRAINING GAN : THE MINMAX GAME G(z) Generator network. Output in the dimensions of x. D(x) Discriminator network. Output in [0,1] where 1 means x is from p data. (log{d x } [-inf,0] ) min G max D V D, G = E x~pdata (x)[log{d x }]+ E z~pz (z)[log{1 D G z }] Discriminator output for real data Discriminator for output generated fake data G(z)
15 TRAINING GAN : THE MINMAX GAME min G max D V D, G = E x~pdata (x)[log{d x }]+ E z~pz (z)[log{1 D G z }] Alternate between: Gradient ascent on discriminator: max D [ E x~pdata (x)[log{d x }]+ E z~pz (z)[log{1 D G z }]] Gradient descent on generator: min G [E z~pz (z)[log{1 D G z }]]
16 TRAINING GAN : THE MINMAX GAME Equilibrium is a saddle point of the discriminator loss D G x = p data (x) p data x + p g (x) p g x =p data (x) 0. 5 Estimating this ratio using supervised learning is the key approximation mechanism used by GAN Data Discriminator Model distribution
17 TRAINING GAN : THE MINMAX GAME min G max D V D, G = E x~pdata (x)[log{d x }]+ E z~pz (z)[log{1 D G z }] Alternate between: Gradient ascent on discriminator: max D [ E x~pdata (x)[log{d x }]+ E z~pz (z)[log{1 D G z }]] Gradient descent on generator: Generator Loss min G [E z~pz (z)[log{1 D G z }]] In practice, optimizing this generator objective not work well!
18 TRAINING GAN : THE MINMAX GAME min G max D V D, G = E x~pdata (x)[log{d x }]+ E z~pz (z)[log{1 D G z }] Alternate between: Gradient ascent on discriminator: max D [ E x~pdata (x)[log{d x }]+ E z~pz (z)[log{1 D G z }]] Gradient ascent on generator: Generator Loss max G [E z~pz (z)[log{d G z }]] Maximizing likelihood of discriminator being wrong! Higher gradient for bad samples
19 Latent random variable TRAINING GAN : TWO PLAYER GAME x (1),,x (m) ~P data Real World Images Sample z (1),,z (m) ~P z Discriminator Real Fake Loss Generator x~p data Sample D x D(G(z)) G(z)~P g
20 Latent random variable TRAINING GAN : TWO PLAYER GAME z (1),,z (m) ~P z Generator
21 Latent random variable TRAINING GAN : TWO PLAYER GAME z (1),,z (m) ~P z Generator
22 Latent random variable TRAINING GAN : TWO PLAYER GAME z (1),,z (m) ~P z Generator
23 Latent random variable TRAINING GAN : TWO PLAYER GAME z (1),,z (m) ~P z Generator
24 DEEP CONVOLUTION GAN (DCGAN) Architecture key insights: Use batch normalization layers in both discriminator and generator. Use convolution with a stride greater than 1 instead of pooling layer. Use Adam optimizer instead of SGD. Use ReLU activation in generator for all layers except for the output, which uses Tanh. Use LeakyReLU activation in the discriminator for all layers.
25 DEEP CONVOLUTION GAN (DCGAN) Results: Images of bedrooms generated by a DCGAN Trained filters
26 DEEP CONVOLUTION GAN (DCGAN) Results: Vector space arithmetic Samples from the model
27 DEEP CONVOLUTION GAN (DCGAN) Results: Vector space arithmetic Samples from the model Average Z vectors, do arithmetic
28 DEEP CONVOLUTION GAN (DCGAN) Results: Vector space arithmetic
29 DEEP CONVOLUTION GAN (DCGAN) Our Results:
30 DEEP CONVOLUTION GAN (DCGAN) Our Results:
31 DEEP CONVOLUTION GAN (DCGAN) Our Results:
32 DEEP CONVOLUTION GAN (DCGAN) Our Results:
33 DEEP CONVOLUTION GAN (DCGAN) Our Results: The Generator never saw these faces! Only the Discriminative network!
34 GAN : APPLICATIONS Single image super-resolution Image to image translation Text to image Style transfer Video generation Semi-supervised learning And more
35 GAN : APPLICATIONS Single image super-resolution Image to image translation Text to image Style transfer Video generation Semi-supervised learning And more
36 GAN : APPLICATIONS Single image super-resolution From the paper Photo-Realistic Single Image Super- Resolution Using a Generative Adversarial Network
37 GAN : APPLICATIONS Single image super-resolution From the paper Photo-Realistic Single Image Super- Resolution Using a Generative Adversarial Network
38 GAN : APPLICATIONS Single image super-resolution Image to image translation Text to image Style transfer Video generation Semi-supervised learning And more
39 IMAGE TO IMAGE TRANSLATION
40 IMAGE TO IMAGE TRANSLATION A class of vision and graphics problems where the goal is to learn the mapping between an input image and an output image
41 IMAGE TO IMAGE TRANSLATION We will focus on 2 papers from 2017 dealing with unsupervised image to image translation Unpaired Image-to-Image Translation using Cycle- Consistent Adversarial Networks (UC Berkeley) - cyclegan Unsupervised Image-to-Image Translation Networks (NVIDIA) UNIT network
42 IMAGE TO IMAGE TRANSLATION Supervised approach Data {x i, y i } Goal: Learn mapping from x i y i across all pairs Unsupervised approach Data ( x, {y}) Goal: learn mapping from group x to group {y} by learning underlying structure
43 IMAGE TO IMAGE TRANSLATION Examples:
44 IMAGE TO IMAGE TRANSLATION Examples:
45 IMAGE TO IMAGE TRANSLATION Papers use similar approach to solving this problem: Use GAN generator discriminator model Use VAE latent space encoding as part generator architecture (though only UNIT trains VAE explicitly) Learn translation in both directions at once
46 IMAGE TO IMAGE TRANSLATION UNIT assumes a shared latent space: pair of corresponding images can be mapped to same latent representation
47 IMAGE TO IMAGE TRANSLATION UNIT
48 IMAGE TO IMAGE TRANSLATION UNIT
49 IMAGE TO IMAGE TRANSLATION UNIT Encoder-generator pair is a VAE E1(E2) encodes image x 1 (x 2 ) to a latent vector z G1 (G2) decodes a randomly perturbed version of the code z VAE1 and VAE2 share weights: high level layers of E1 and E2, and same for G1 and G2 VAE training ensure the reconstructed image and the original image are similar
50 IMAGE TO IMAGE TRANSLATION UNIT Generator-discriminator pair is a GAN Reconstruction is already trained by VAE, so D is only applied to translated images G1(G2) generates images for domain 1 (2) from a latent vector z D1 is trained to output True for x 1, and False for x (opposite for D2) GAN training ensures translated images resemble images in the target domain
51 IMAGE TO IMAGE TRANSLATION UNIT CYCLE CONSISTENCY Shared-latent space implies cycle consistency Added weight sharing to regularize training Forward translation: x*2 = G2(E1(x1)) Backward translation: x**1 = G1(E2(x*2)) Impose that original image x1 and cycle-translated image x*1 are equal: x1 = x**1 (and similarly: x2 = x**2)
52 IMAGE TO IMAGE TRANSLATION UNIT Explicit loss function:
53 IMAGE TO IMAGE TRANSLATION UNIT Explicit loss function:
54 IMAGE TO IMAGE TRANSLATION CycleGAN
55 IMAGE TO IMAGE TRANSLATION CycleGAN 1
56 IMAGE TO IMAGE TRANSLATION CycleGAN 1 1
57 IMAGE TO IMAGE TRANSLATION CycleGAN 1 1
58 IMAGE TO IMAGE TRANSLATION CycleGAN Explicit loss function:
59 IMAGE TO IMAGE TRANSLATION CycleGAN Generator architecture:
60 IMAGE TO IMAGE TRANSLATION CycleGAN Discriminator architecture
61 IMAGE TO IMAGE TRANSLATION CycleGAN Results:
62 IMAGE TO IMAGE TRANSLATION CycleGAN Results: video
63 IMAGE TO IMAGNE TRANSLATION UNIT Results: night day rain sunshine
64 IMAGE TO IMAGNE TRANSLATION UNIT Results:
65 IMAGE TO IMAGNE TRANSLATION UNIT Results:
66 SOME MORE RESULTS CycleGAN
67 REFERENCES Ian Goodfellow. NIPS 2016 Tutorial: Generative Adversarial Networks (2016) Jun-Yan Zhu, Tasung Park. Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks (2017) Ming-Yu Liu, Thomas Breuel. Unsupervised Image-to- Image Translation Networks (2017) Ian Goodfellow. Generative adversarial nets (2014) Christian Ledig. Photo-Realistic Single Image Super- Resolution Using a Generative Adversarial Network (2017)
68 Thanks for listening
69 TIPS AND TRICKS Labels improve subjective sample quality: (Denton et al 2015) Learning a conditional model p(y x) often gives much better samples from all classes than learning p(x) does. One-sided label smoothing: (Salimans et al 2016) D_cost = cross_entropy(1., D(data)) + cross_entropy(0., D(samples)) D*_cost = cross_entropy(.9, D(data)) + cross_entropy(0., D(samples)) Prevent extreme extrapolation behavior in the discriminator. Excellent regularizer that doesn t encourage misclassification.
Unsupervised Learning
Deep Learning for Graphics Unsupervised Learning Niloy Mitra Iasonas Kokkinos Paul Guerrero Vladimir Kim Kostas Rematas Tobias Ritschel UCL UCL/Facebook UCL Adobe Research U Washington UCL Timetable Niloy
Alternatives to Direct Supervision
CreativeAI: Deep Learning for Graphics Alternatives to Direct Supervision Niloy Mitra Iasonas Kokkinos Paul Guerrero Nils Thuerey Tobias Ritschel UCL UCL UCL TUM UCL Timetable Theory and Basics State of
Generative Adversarial Networks (GANs)
Generative Adversarial Networks (GANs) Hossein Azizpour Most of the slides are courtesy of Dr. Ian Goodfellow (Research Scientist at OpenAI) and from his presentation at NIPS 2016 tutorial Note. I am generally
Generative Modeling with Convolutional Neural Networks. Denis Dus Data Scientist at InData Labs
Generative Modeling with Convolutional Neural Networks Denis Dus Data Scientist at InData Labs What we will discuss 1. 2. 3. 4. Discriminative vs Generative modeling Convolutional Neural Networks How to
GAN Frontiers/Related Methods
GAN Frontiers/Related Methods Improving GAN Training Improved Techniques for Training GANs (Salimans, et. al 2016) CSC 2541 (07/10/2016) Robin Swanson (robin@cs.toronto.edu) Training GANs is Difficult
Lecture 19: Generative Adversarial Networks
Lecture 19: Generative Adversarial Networks Roger Grosse 1 Introduction Generative modeling is a type of machine learning where the aim is to model the distribution that a given set of data (e.g. images,
Introduction to Generative Adversarial Networks
Introduction to Generative Adversarial Networks Ian Goodfellow, OpenAI Research Scientist NIPS 2016 Workshop on Adversarial Training Barcelona, 2016-12-9 Adversarial Training A phrase whose usage is in
Autoencoders. Stephen Scott. Introduction. Basic Idea. Stacked AE. Denoising AE. Sparse AE. Contractive AE. Variational AE GAN.
Stacked Denoising Sparse Variational (Adapted from Paul Quint and Ian Goodfellow) Stacked Denoising Sparse Variational Autoencoding is training a network to replicate its input to its output Applications:
Generative Adversarial Network
Generative Adversarial Network Many slides from NIPS 2014 Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio Generative adversarial
Generative Adversarial Networks (GANs) Ian Goodfellow, Research Scientist MLSLP Keynote, San Francisco
Generative Adversarial Networks (GANs) Ian Goodfellow, Research Scientist MLSLP Keynote, San Francisco 2016-09-13 Generative Modeling Density estimation Sample generation Training examples Model samples
Variational Autoencoders. Sargur N. Srihari
Variational Autoencoders Sargur N. srihari@cedar.buffalo.edu Topics 1. Generative Model 2. Standard Autoencoder 3. Variational autoencoders (VAE) 2 Generative Model A variational autoencoder (VAE) is a
Generative Adversarial Networks (GANs) Based on slides from Ian Goodfellow s NIPS 2016 tutorial
Generative Adversarial Networks (GANs) Based on slides from Ian Goodfellow s NIPS 2016 tutorial Generative Modeling Density estimation Sample generation Training examples Model samples Next Video Frame
Introduction to GAN. Generative Adversarial Networks. Junheng(Jeff) Hao
Introduction to GAN Generative Adversarial Networks Junheng(Jeff) Hao Adversarial Training is the coolest thing since sliced bread. -- Yann LeCun Roadmap 1. Generative Modeling 2. GAN 101: What is GAN?
Deep Learning for Visual Manipulation and Synthesis
Deep Learning for Visual Manipulation and Synthesis Jun-Yan Zhu 朱俊彦 UC Berkeley 2017/01/11 @ VALSE What is visual manipulation? Image Editing Program input photo User Input result Desired output: stay
Introduction to Generative Adversarial Networks
Introduction to Generative Adversarial Networks Luke de Oliveira Vai Technologies Lawrence Berkeley National Laboratory @lukede0 @lukedeo lukedeo@vaitech.io https://ldo.io 1 Outline Why Generative Modeling?
Introduction to GAN. Generative Adversarial Networks. Junheng(Jeff) Hao
Introduction to GAN Generative Adversarial Networks Junheng(Jeff) Hao Adversarial Training is the coolest thing since sliced bread. -- Yann LeCun Roadmap 1. Generative Modeling 2. GAN 101: What is GAN?
Learning to generate with adversarial networks
Learning to generate with adversarial networks Gilles Louppe June 27, 2016 Problem statement Assume training samples D = {x x p data, x X } ; We want a generative model p model that can draw new samples
19: Inference and learning in Deep Learning
10-708: Probabilistic Graphical Models 10-708, Spring 2017 19: Inference and learning in Deep Learning Lecturer: Zhiting Hu Scribes: Akash Umakantha, Ryan Williamson 1 Classes of Deep Generative Models
Image Restoration with Deep Generative Models
Image Restoration with Deep Generative Models Raymond A. Yeh *, Teck-Yian Lim *, Chen Chen, Alexander G. Schwing, Mark Hasegawa-Johnson, Minh N. Do Department of Electrical and Computer Engineering, University
An Empirical Study of Generative Adversarial Networks for Computer Vision Tasks
An Empirical Study of Generative Adversarial Networks for Computer Vision Tasks Report for Undergraduate Project - CS396A Vinayak Tantia (Roll No: 14805) Guide: Prof Gaurav Sharma CSE, IIT Kanpur, India
Adversarially Learned Inference
Institut des algorithmes d apprentissage de Montréal Adversarially Learned Inference Aaron Courville CIFAR Fellow Université de Montréal Joint work with: Vincent Dumoulin, Ishmael Belghazi, Olivier Mastropietro,
(University Improving of Montreal) Generative Adversarial Networks with Denoising Feature Matching / 17
Improving Generative Adversarial Networks with Denoising Feature Matching David Warde-Farley 1 Yoshua Bengio 1 1 University of Montreal, ICLR,2017 Presenter: Bargav Jayaraman Outline 1 Introduction 2 Background
arxiv: v1 [cs.cv] 17 Nov 2016
![arxiv: v1 [cs.cv] 17 Nov 2016](/thumbs/86/93859848.jpg "arxiv: v1 [cs.cv] 17 Nov 2016")
Inverting The Generator Of A Generative Adversarial Network arxiv:1611.05644v1 [cs.cv] 17 Nov 2016 Antonia Creswell BICV Group Bioengineering Imperial College London ac2211@ic.ac.uk Abstract Anil Anthony
Reconstructing Pore Networks Using Generative Adversarial Networks
Reconstructing Pore Networks Using Generative Adversarial Networks Kelly Guan (kmguan@stanford.edu) I. INTRODUCTION Understanding fluid flow in porous media at the microscale is relevant to many fields,
Deep Fakes using Generative Adversarial Networks (GAN)
Deep Fakes using Generative Adversarial Networks (GAN) Tianxiang Shen UCSD La Jolla, USA tis038@eng.ucsd.edu Ruixian Liu UCSD La Jolla, USA rul188@eng.ucsd.edu Ju Bai UCSD La Jolla, USA jub010@eng.ucsd.edu
Lip Movement Synthesis from Text
Lip Movement Synthesis from Text 1 1 Department of Computer Science and Engineering Indian Institute of Technology, Kanpur July 20, 2017 (1Department of Computer Science Lipand Movement Engineering Synthesis
Generative Adversarial Nets. Priyanka Mehta Sudhanshu Srivastava
Generative Adversarial Nets Priyanka Mehta Sudhanshu Srivastava Outline What is a GAN? How does GAN work? Newer Architectures Applications of GAN Future possible applications Generative Adversarial Networks
Bidirectional GAN. Adversarially Learned Inference (ICLR 2017) Adversarial Feature Learning (ICLR 2017)
Bidirectional GAN Adversarially Learned Inference (ICLR 2017) V. Dumoulin 1, I. Belghazi 1, B. Poole 2, O. Mastropietro 1, A. Lamb 1, M. Arjovsky 3 and A. Courville 1 1 Universite de Montreal & 2 Stanford
CS230: Lecture 4 Attacking Networks with Adversarial Examples - Generative Adversarial Networks
Go to www.menti.com and use the code 91 41 37 CS230: Lecture 4 Attacking Networks with Adversarial Examples - Generative Adversarial Networks Kian Katanforoosh Today s outline I. Attacking NNs with Adversarial
Autoencoding Beyond Pixels Using a Learned Similarity Metric
Autoencoding Beyond Pixels Using a Learned Similarity Metric International Conference on Machine Learning, 2016 Anders Boesen Lindbo Larsen, Hugo Larochelle, Søren Kaae Sønderby, Ole Winther Technical
Deep Generative Models Variational Autoencoders
Deep Generative Models Variational Autoencoders Sudeshna Sarkar 5 April 2017 Generative Nets Generative models that represent probability distributions over multiple variables in some way. Directed Generative
arxiv: v1 [cs.cv] 5 Jul 2017
![arxiv: v1 [cs.cv] 5 Jul 2017](/thumbs/95/123923253.jpg "arxiv: v1 [cs.cv] 5 Jul 2017")
AlignGAN: Learning to Align Cross- Images with Conditional Generative Adversarial Networks Xudong Mao Department of Computer Science City University of Hong Kong xudonmao@gmail.com Qing Li Department of
Adversarial Machine Learning
MedGAN Progressive GAN CoGAN LR-GAN CGAN IcGAN BIM LS-GAN AffGAN LAPGAN DiscoGANMPM-GAN AdaGAN LSGAN InfoGAN ATN FGSM igan IAN Adversarial Machine Learning McGAN Ian Goodfellow, Staff Research Scientist,
Progress on Generative Adversarial Networks
Progress on Generative Adversarial Networks Wangmeng Zuo Vision Perception and Cognition Centre Harbin Institute of Technology Content Image generation: problem formulation Three issues about GAN Discriminate
Lab meeting (Paper review session) Stacked Generative Adversarial Networks
Lab meeting (Paper review session) Stacked Generative Adversarial Networks 2017. 02. 01. Saehoon Kim (Ph. D. candidate) Machine Learning Group Papers to be covered Stacked Generative Adversarial Networks
Generative Adversarial Networks (GANs) Ian Goodfellow, OpenAI Research Scientist NIPS 2016 tutorial Barcelona,
Generative Adversarial Networks (GANs) Ian Goodfellow, OpenAI Research Scientist NIPS 2016 tutorial Barcelona, 2016-12-4 Generative Modeling Density estimation Sample generation Training examples Model
Lecture 3 GANs and Their Applications in Image Generation
Lecture 3 GANs and Their Applications in Image Generation Lin ZHANG, PhD School of Software Engineering Tongji University Fall 2017 Outline Introduction Theoretical Part Application Part Existing Implementations
Paired 3D Model Generation with Conditional Generative Adversarial Networks
Accepted to 3D Reconstruction in the Wild Workshop European Conference on Computer Vision (ECCV) 2018 Paired 3D Model Generation with Conditional Generative Adversarial Networks Cihan Öngün Alptekin Temizel
Generative Models II. Phillip Isola, MIT, OpenAI DLSS 7/27/18
Generative Models II Phillip Isola, MIT, OpenAI DLSS 7/27/18 What s a generative model? For this talk: models that output high-dimensional data (Or, anything involving a GAN, VAE, PixelCNN, etc) Useful
Autoencoder. Representation learning (related to dictionary learning) Both the input and the output are x
Deep Learning 4 Autoencoder, Attention (spatial transformer), Multi-modal learning, Neural Turing Machine, Memory Networks, Generative Adversarial Net Jian Li IIIS, Tsinghua Autoencoder Autoencoder Unsupervised
Introduction to Generative Models (and GANs)
Introduction to Generative Models (and GANs) Haoqiang Fan fhq@megvii.com Nov. 2017 Figures adapted from NIPS 2016 Tutorial Generative Adversarial Networks Generative Models: Learning the Distributions
Implicit generative models: dual vs. primal approaches
Implicit generative models: dual vs. primal approaches Ilya Tolstikhin MPI for Intelligent Systems ilya@tue.mpg.de Machine Learning Summer School 2017 Tübingen, Germany Contents 1. Unsupervised generative
DOMAIN-ADAPTIVE GENERATIVE ADVERSARIAL NETWORKS FOR SKETCH-TO-PHOTO INVERSION
DOMAIN-ADAPTIVE GENERATIVE ADVERSARIAL NETWORKS FOR SKETCH-TO-PHOTO INVERSION Yen-Cheng Liu 1, Wei-Chen Chiu 2, Sheng-De Wang 1, and Yu-Chiang Frank Wang 1 1 Graduate Institute of Electrical Engineering,
Deep Generative Models and a Probabilistic Programming Library
Deep Generative Models and a Probabilistic Programming Library Discriminative (Deep) Learning Learn a (differentiable) function mapping from input to output x f(x; θ) y Gradient back-propagation Generative
DOMAIN-ADAPTIVE GENERATIVE ADVERSARIAL NETWORKS FOR SKETCH-TO-PHOTO INVERSION
2017 IEEE INTERNATIONAL WORKSHOP ON MACHINE LEARNING FOR SIGNAL PROCESSING, SEPT. 25 28, 2017, TOKYO, JAPAN DOMAIN-ADAPTIVE GENERATIVE ADVERSARIAL NETWORKS FOR SKETCH-TO-PHOTO INVERSION Yen-Cheng Liu 1,
Generative Adversarial Text to Image Synthesis
Generative Adversarial Text to Image Synthesis Scott Reed, Zeynep Akata, Xinchen Yan, Lajanugen Logeswaran, Bernt Schiele, Honglak Lee Presented by: Jingyao Zhan Contents Introduction Related Work Method
arxiv: v1 [cs.cv] 1 Aug 2017
![arxiv: v1 [cs.cv] 1 Aug 2017](/thumbs/87/97260731.jpg "arxiv: v1 [cs.cv] 1 Aug 2017")
Deep Generative Adversarial Neural Networks for Realistic Prostate Lesion MRI Synthesis Andy Kitchen a, Jarrel Seah b a,* Independent Researcher b STAT Innovations Pty. Ltd., PO Box 274, Ashburton VIC
GENERATIVE ADVERSARIAL NETWORK-BASED VIR-
GENERATIVE ADVERSARIAL NETWORK-BASED VIR- TUAL TRY-ON WITH CLOTHING REGION Shizuma Kubo, Yusuke Iwasawa, and Yutaka Matsuo The University of Tokyo Bunkyo-ku, Japan {kubo, iwasawa, matsuo}@weblab.t.u-tokyo.ac.jp
Bridging Theory and Practice of GANs
MedGAN ID-CGAN Progressive GAN LR-GAN CGAN IcGAN b-gan LS-GAN AffGAN LAPGAN LSGAN InfoGAN CatGAN SN-GAN DiscoGANMPM-GAN AdaGAN AMGAN igan IAN CoGAN Bridging Theory and Practice of GANs McGAN Ian Goodfellow,
UCLA UCLA Electronic Theses and Dissertations
UCLA UCLA Electronic Theses and Dissertations Title Application of Generative Adversarial Network on Image Style Transformation and Image Processing Permalink https://escholarship.org/uc/item/66w654x7
COMP 551 Applied Machine Learning Lecture 16: Deep Learning
COMP 551 Applied Machine Learning Lecture 16: Deep Learning Instructor: Ryan Lowe (ryan.lowe@cs.mcgill.ca) Slides mostly by: Class web page: www.cs.mcgill.ca/~hvanho2/comp551 Unless otherwise noted, all
Day 3 Lecture 1. Unsupervised Learning
Day 3 Lecture 1 Unsupervised Learning Semi-supervised and transfer learning Myth: you can t do deep learning unless you have a million labelled examples for your problem. Reality You can learn useful representations
Unsupervised Image-to-Image Translation Networks
Unsupervised Image-to-Image Translation Networks Ming-Yu Liu, Thomas Breuel, Jan Kautz NVIDIA {mingyul,tbreuel,jkautz}@nvidia.com Abstract Unsupervised image-to-image translation aims at learning a joint
arxiv: v1 [cs.cv] 7 Mar 2018
![arxiv: v1 [cs.cv] 7 Mar 2018](/thumbs/95/125565711.jpg "arxiv: v1 [cs.cv] 7 Mar 2018")
Accepted as a conference paper at the European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN) 2018 Inferencing Based on Unsupervised Learning of Disentangled
arxiv: v4 [cs.lg] 1 May 2018
![arxiv: v4 [cs.lg] 1 May 2018](/thumbs/92/110620456.jpg "arxiv: v4 [cs.lg] 1 May 2018")
Controllable Generative Adversarial Network arxiv:1708.00598v4 [cs.lg] 1 May 2018 Minhyeok Lee School of Electrical Engineering Korea University Seoul, Korea 02841 suam6409@korea.ac.kr Abstract Junhee
Tutorial on Keras CAP ADVANCED COMPUTER VISION SPRING 2018 KISHAN S ATHREY
Tutorial on Keras CAP 6412 - ADVANCED COMPUTER VISION SPRING 2018 KISHAN S ATHREY Deep learning packages TensorFlow Google PyTorch Facebook AI research Keras Francois Chollet (now at Google) Chainer Company
arxiv: v1 [cs.ne] 11 Jun 2018
![arxiv: v1 [cs.ne] 11 Jun 2018](/thumbs/80/80526645.jpg "arxiv: v1 [cs.ne] 11 Jun 2018")
Generative Adversarial Network Architectures For Image Synthesis Using Capsule Networks arxiv:1806.03796v1 [cs.ne] 11 Jun 2018 Yash Upadhyay University of Minnesota, Twin Cities Minneapolis, MN, 55414
arxiv: v4 [cs.lg] 3 Apr 2017
![arxiv: v4 [cs.lg] 3 Apr 2017](/thumbs/72/67150416.jpg "arxiv: v4 [cs.lg] 3 Apr 2017")
NIPS 2016 Tutorial: Generative Adversarial Networks arxiv:1701.00160v4 [cs.lg] 3 Apr 2017 Ian Goodfellow OpenAI, ian@openai.com Abstract This report summarizes the tutorial presented by the author at NIPS
Deep generative models of natural images
Spring 2016 1 Motivation 2 3 Variational autoencoders Generative adversarial networks Generative moment matching networks Evaluating generative models 4 Outline 1 Motivation 2 3 Variational autoencoders
Stochastic Simulation with Generative Adversarial Networks
Stochastic Simulation with Generative Adversarial Networks Lukas Mosser, Olivier Dubrule, Martin J. Blunt lukas.mosser15@imperial.ac.uk, o.dubrule@imperial.ac.uk, m.blunt@imperial.ac.uk (Deep) Generative
Generative Adversarial Network: a Brief Introduction. Lili Mou
Generative Adversarial Network: a Brief Introduction Lili Mou doublepower.mou@gmail.com Outline Generative adversarial net Conditional generative adversarial net Deep generative image models using Laplacian
Auto-Encoding Variational Bayes
Auto-Encoding Variational Bayes Diederik P (Durk) Kingma, Max Welling University of Amsterdam Ph.D. Candidate, advised by Max Durk Kingma D.P. Kingma Max Welling Problem class Directed graphical model:
A New CGAN Technique for Constrained Topology Design Optimization. Abstract
A New CGAN Technique for Constrained Topology Design Optimization M.-H. Herman Shen 1 and Liang Chen Department of Mechanical and Aerospace Engineering The Ohio State University Abstract This paper presents
Composable Unpaired Image to Image Translation
Composable Unpaired Image to Image Translation Laura Graesser New York University lhg256@nyu.edu Anant Gupta New York University ag4508@nyu.edu Abstract There has been remarkable recent work in unpaired
Supplementary Material: Unsupervised Domain Adaptation for Face Recognition in Unlabeled Videos
Supplementary Material: Unsupervised Domain Adaptation for Face Recognition in Unlabeled Videos Kihyuk Sohn 1 Sifei Liu 2 Guangyu Zhong 3 Xiang Yu 1 Ming-Hsuan Yang 2 Manmohan Chandraker 1,4 1 NEC Labs
ImageNet Classification with Deep Convolutional Neural Networks
ImageNet Classification with Deep Convolutional Neural Networks Alex Krizhevsky Ilya Sutskever Geoffrey Hinton University of Toronto Canada Paper with same name to appear in NIPS 2012 Main idea Architecture
Learning to Generate Images
Learning to Generate Images Jun-Yan Zhu Ph.D. at UC Berkeley Postdoc at MIT CSAIL Computer Vision before 2012 Cat Features Clustering Pooling Classification [LeCun et al, 1998], [Krizhevsky et al, 2012]
arxiv: v1 [cs.cv] 16 Jul 2017
![arxiv: v1 [cs.cv] 16 Jul 2017](/thumbs/85/91311140.jpg "arxiv: v1 [cs.cv] 16 Jul 2017")
enerative adversarial network based on resnet for conditional image restoration Paper: jc*-**-**-****: enerative Adversarial Network based on Resnet for Conditional Image Restoration Meng Wang, Huafeng
arxiv: v2 [cs.lg] 17 Dec 2018
![arxiv: v2 [cs.lg] 17 Dec 2018](/thumbs/95/125341191.jpg "arxiv: v2 [cs.lg] 17 Dec 2018")
Lu Mi 1 * Macheng Shen 2 * Jingzhao Zhang 2 * 1 MIT CSAIL, 2 MIT LIDS {lumi, macshen, jzhzhang}@mit.edu The authors equally contributed to this work. This report was a part of the class project for 6.867
Introduction to GANs
MedGAN ID-CGAN CoGAN LR-GAN CGAN IcGAN b-gan LS-GAN LAPGAN DiscoGANMPM-GAN AdaGAN LSGAN InfoGAN CatGAN AMGAN igan Introduction to GANs IAN SAGAN McGAN Ian Goodfellow, Staff Research Scientist, Google Brain
GANs for Exploiting Unlabeled Data. Presented by: Uriya Pesso Nimrod Gilboa Markevich
GANs for Exploiting Unlabeled Data Improved Techniques for Training GANs Learning from Simulated and Unsupervised Images through Adversarial Training Presented by: Uriya Pesso Nimrod Gilboa Markevich [
DEEP LEARNING PART THREE - DEEP GENERATIVE MODELS CS/CNS/EE MACHINE LEARNING & DATA MINING - LECTURE 17
DEEP LEARNING PART THREE - DEEP GENERATIVE MODELS CS/CNS/EE 155 - MACHINE LEARNING & DATA MINING - LECTURE 17 GENERATIVE MODELS DATA 3 DATA 4 example 1 DATA 5 example 2 DATA 6 example 3 DATA 7 number of
Dropout. Sargur N. Srihari This is part of lecture slides on Deep Learning:
Dropout Sargur N. srihari@buffalo.edu This is part of lecture slides on Deep Learning: http://www.cedar.buffalo.edu/~srihari/cse676 1 Regularization Strategies 1. Parameter Norm Penalties 2. Norm Penalties
One Network to Solve Them All Solving Linear Inverse Problems using Deep Projection Models
One Network to Solve Them All Solving Linear Inverse Problems using Deep Projection Models [Supplemental Materials] 1. Network Architecture b ref b ref +1 We now describe the architecture of the networks
Auto-encoder with Adversarially Regularized Latent Variables
Information Engineering Express International Institute of Applied Informatics 2017, Vol.3, No.3, P.11 20 Auto-encoder with Adversarially Regularized Latent Variables for Semi-Supervised Learning Ryosuke
MoonRiver: Deep Neural Network in C++
MoonRiver: Deep Neural Network in C++ Chung-Yi Weng Computer Science & Engineering University of Washington chungyi@cs.washington.edu Abstract Artificial intelligence resurges with its dramatic improvement
SiftingGAN: Generating and Sifting Labeled Samples to Improve the Remote Sensing Image Scene Classification Baseline in vitro
1 SiftingGAN: Generating and Sifting Labeled Samples to Improve the Remote Sensing Image Scene Classification Baseline in vitro Dongao Ma, Ping Tang, and Lijun Zhao arxiv:1809.04985v4 [cs.cv] 30 Nov 2018
GENERATIVE ADVERSARIAL NETWORKS FOR IMAGE STEGANOGRAPHY
GENERATIVE ADVERSARIAL NETWORKS FOR IMAGE STEGANOGRAPHY Denis Volkhonskiy 2,3, Boris Borisenko 3 and Evgeny Burnaev 1,2,3 1 Skolkovo Institute of Science and Technology 2 The Institute for Information
Controllable Generative Adversarial Network
Controllable Generative Adversarial Network arxiv:1708.00598v2 [cs.lg] 12 Sep 2017 Minhyeok Lee 1 and Junhee Seok 1 1 School of Electrical Engineering, Korea University, 145 Anam-ro, Seongbuk-gu, Seoul,
Learning Social Graph Topologies using Generative Adversarial Neural Networks
Learning Social Graph Topologies using Generative Adversarial Neural Networks Sahar Tavakoli 1, Alireza Hajibagheri 1, and Gita Sukthankar 1 1 University of Central Florida, Orlando, Florida sahar@knights.ucf.edu,alireza@eecs.ucf.edu,gitars@eecs.ucf.edu
Deep Manga Colorization with Color Style Extraction by Conditional Adversarially Learned Inference
Information Engineering Express International Institute of Applied Informatics 2017, Vol.3, No.4, P.55-66 Deep Manga Colorization with Color Style Extraction by Conditional Adversarially Learned Inference
Semi-Supervised Generative Adversarial Nets with Multiple Generators for SAR Image Recognition
sensors Article Semi-Supervised Generative Adversarial Nets with Multiple Generators for SAR Image Recognition Fei Gao 1, *, Fei Ma 1 ID, Jun Wang 1, Jinping Sun 1 ID, Erfu Yang 2 and Huiyu Zhou 3 1 School
Visual Recommender System with Adversarial Generator-Encoder Networks
Visual Recommender System with Adversarial Generator-Encoder Networks Bowen Yao Stanford University 450 Serra Mall, Stanford, CA 94305 boweny@stanford.edu Yilin Chen Stanford University 450 Serra Mall
Deep-Q: Traffic-driven QoS Inference using Deep Generative Network
Deep-Q: Traffic-driven QoS Inference using Deep Generative Network Shihan Xiao, Dongdong He, Zhibo Gong Network Technology Lab, Huawei Technologies Co., Ltd., Beijing, China 1 Background What is a QoS
GAN and Feature Representation. Hung-yi Lee
GAN and Feature Representation Hung-yi Lee Outline Generator (Decoder) Discrimi nator + Encoder GAN+Autoencoder x InfoGAN Encoder z Generator Discrimi (Decoder) x nator scalar Discrimi z Generator x scalar
Quantitative Evaluation of Generative Adversarial Networks and Improved Training Techniques
Quantitative Evaluation of Generative Adversarial Networks and Improved Training Techniques by Yadong Li to obtain the degree of Master of Science at the Delft University of Technology, to be defended
When Variational Auto-encoders meet Generative Adversarial Networks
When Variational Auto-encoders meet Generative Adversarial Networks Jianbo Chen Billy Fang Cheng Ju 14 December 2016 Abstract Variational auto-encoders are a promising class of generative models. In this
Data Set Extension with Generative Adversarial Nets
Department of Artificial Intelligence University of Groningen, The Netherlands Data Set Extension with Generative Adversarial Nets Master s Thesis Luuk Boulogne S2366681 Primary supervisor: Secondary supervisor:
Challenges motivating deep learning. Sargur N. Srihari
Challenges motivating deep learning Sargur N. srihari@cedar.buffalo.edu 1 Topics In Machine Learning Basics 1. Learning Algorithms 2. Capacity, Overfitting and Underfitting 3. Hyperparameters and Validation
Multi-Modal Generative Adversarial Networks
Multi-Modal Generative Adversarial Networks By MATAN BEN-YOSEF Under the supervision of PROF. DAPHNA WEINSHALL Faculty of Computer Science and Engineering THE HEBREW UNIVERSITY OF JERUSALEM A thesis submitted
Inverting The Generator Of A Generative Adversarial Network
1 Inverting The Generator Of A Generative Adversarial Network Antonia Creswell and Anil A Bharath, Imperial College London arxiv:1802.05701v1 [cs.cv] 15 Feb 2018 Abstract Generative adversarial networks
Adversarial Examples and Adversarial Training. Ian Goodfellow, Staff Research Scientist, Google Brain CS 231n, Stanford University,
Adversarial Examples and Adversarial Training Ian Goodfellow, Staff Research Scientist, Google Brain CS 231n, Stanford University, 2017-05-30 Overview What are adversarial examples? Why do they happen?
RenderBEGAN: Adversarial Generative Domain Adaptation
RenderBEGAN: Adversarial Generative Domain Adaptation Fabian Reimeier f.reimeier@fu-berlin.de Institute of Computer Science Freie Universität Berlin A thesis submitted for the degree of Master of Science
arxiv: v1 [stat.ml] 10 Dec 2018
![arxiv: v1 [stat.ml] 10 Dec 2018](/thumbs/88/116729438.jpg "arxiv: v1 [stat.ml] 10 Dec 2018")
1st Symposium on Advances in Approximate Bayesian Inference, 2018 1 7 Disentangled Dynamic Representations from Unordered Data arxiv:1812.03962v1 [stat.ml] 10 Dec 2018 Leonhard Helminger Abdelaziz Djelouah
Amortised MAP Inference for Image Super-resolution. Casper Kaae Sønderby, Jose Caballero, Lucas Theis, Wenzhe Shi & Ferenc Huszár ICLR 2017
Amortised MAP Inference for Image Super-resolution Casper Kaae Sønderby, Jose Caballero, Lucas Theis, Wenzhe Shi & Ferenc Huszár ICLR 2017 Super Resolution Inverse problem: Given low resolution representation
Building an Automatic Sprite Generator with Deep Convolutional Generative Adversarial Networks
Building an Automatic Sprite Generator with Deep Convolutional Generative Adversarial Networks Lewis Horsley School of Computer Science and Electronic Engineering, University of Essex, Colchester, UK lhorsl@essex.ac.uk
SYNTHESIS OF IMAGES BY TWO-STAGE GENERATIVE ADVERSARIAL NETWORKS. Qiang Huang, Philip J.B. Jackson, Mark D. Plumbley, Wenwu Wang
SYNTHESIS OF IMAGES BY TWO-STAGE GENERATIVE ADVERSARIAL NETWORKS Qiang Huang, Philip J.B. Jackson, Mark D. Plumbley, Wenwu Wang Centre for Vision, Speech and Signal Processing University of Surrey, Guildford,
INF 5860 Machine learning for image classification. Lecture 11: Visualization Anne Solberg April 4, 2018
INF 5860 Machine learning for image classification Lecture 11: Visualization Anne Solberg April 4, 2018 Reading material The lecture is based on papers: Deep Dream: https://research.googleblog.com/2015/06/inceptionism-goingdeeper-into-neural.html
arxiv: v1 [cs.cv] 1 May 2018
![arxiv: v1 [cs.cv] 1 May 2018](/thumbs/83/88735687.jpg "arxiv: v1 [cs.cv] 1 May 2018")
Conditional Image-to-Image Translation Jianxin Lin 1 Yingce Xia 1 Tao Qin 2 Zhibo Chen 1 Tie-Yan Liu 2 1 University of Science and Technology of China 2 Microsoft Research Asia linjx@mail.ustc.edu.cn {taoqin,
Machine Learning. Deep Learning. Eric Xing (and Pengtao Xie) , Fall Lecture 8, October 6, Eric CMU,
Machine Learning 10-701, Fall 2015 Deep Learning Eric Xing (and Pengtao Xie) Lecture 8, October 6, 2015 Eric Xing @ CMU, 2015 1 A perennial challenge in computer vision: feature engineering SIFT Spin image