Learning to Generate Images
|
|
|
- Charity Lynch
- 5 years ago
- Views:
Transcription
1 Learning to Generate Images Jun-Yan Zhu Ph.D. at UC Berkeley Postdoc at MIT CSAIL
2 Computer Vision before 2012 Cat Features Clustering Pooling Classification
3 [LeCun et al, 1998], [Krizhevsky et al, 2012] Computer Vision Now Features Clustering Pooling Classification Cat Deep Net Cat
![[Redmon et al., 2018] [Güler et al.](/docs-images/89/97791926/images/4-0.jpg ", 2018] [Zhao et al.")
![, 2017] Deep Learning for Computer](/docs-images/89/97791926/images/4-1.jpg "Vision 100 95 90 85 80 75 [Deng et")
![al. 2009] 70 Deep Net 2010 2011](/docs-images/89/97791926/images/4-2.jpg "2012 2013 2014 2015 2016 2017 Top")
4 [Redmon et al., 2018] [Güler et al., 2018] [Zhao et al., 2017] Deep Learning for Computer Vision [Deng et al. 2009] 70 Deep Net Top 5 accuracy on ImageNet benchmark Object detection Human understanding Autonomous driving
5 Can Deep Learning Help Graphics? Cat Modeling Texturing Lighting Rendering
6 Can Deep Learning Help Graphics? Cat Modeling Texturing Lighting Rendering Deep Net Good/Bad
7 Selecting the most attractive expressions Photos 101 Photos [Zhu et al. SIGGRAPH Asia 2014]
8 Selecting the most realistic composites Most realistic composites Least realistic composites [Zhu et al. ICCV 2015]
9 Can Deep Learning Help Graphics? Cat Modeling Texturing Lighting Rendering Deep Net Cat
10 Generating images is hard! Cat Modeling Texturing Lighting Rendering 8 Deep Net 28x28 pixels
11 Generative Adversarial Networks (GANs) [Goodfellow et al. 2014]
12 z G(z) Random code G Generator fake image aleju/cat-generator [Goodfellow et al. 2014]
13 z G(z) G D Real (1) or fake (0)? Random code Generator fake image Discriminator A two-player game: G tries to generate fake images that can fool D. D tries to detect fake images. [Goodfellow et al. 2014]
14 z G(z) G D fake (0.1) Random code Generator fake image Discriminator Learning objective (GANs) [Goodfellow et al. 2014]
15 z G(z) G D fake (0.1) Random code Generator fake image Discriminator x D real (0.9) Learning objective (GANs) real image [Goodfellow et al. 2014]
16 z G(z) G D fake (0.3) Random code Generator fake image Discriminator x D real (0.9) Learning objective (GANs) real image [Goodfellow et al. 2014]
17 Limitations of GANs No user control. vs Random code Output User input Output Low resolution and quality.
18 Contributions Co-authors: Phillip Isola, Taesung Park, Ting-Chun Wang Richard Zhang, Tinghui Zhou, Ming-Yu Liu, Andrew Tao Jan Kautz, Bryan Catanzaro, Alexei A. Efros
19 Goals: Improve Control, Quality, and Resolution pix2pix CycleGAN pix2pixhd GANs Conditional on user inputs. Learning without pairs. High quality and resolution.
20 Goals: Improve Control, Quality, and Resolution pix2pix CycleGAN pix2pixhd GANs Conditional on user inputs. Learning without pairs. High quality and resolution.
21 z G(z) G D Real or fake? Random code Generator Output image Discriminator Learning objective (GANs) [Goodfellow et al. 2014]
22 x G(x) G D Real or fake? Input image Generator Output image Discriminator Learning objective (pix2pix) [Isola, Zhu, Zhou, Efros, 2016]
23 x G(x) G D Real Input image Generator Output image Discriminator Learning objective (pix2pix) [Isola, Zhu, Zhou, Efros, 2016]
24 x G(x) G D Real too Input image Generator Output image Discriminator Learning objective (pix2pix) [Isola, Zhu, Zhou, Efros, 2016]
25 x G(x) G Generator D Real or fake pair? Discriminator Learning objective (pix2pix) [Isola, Zhu, Zhou, Efros, 2016]
26 #edges2cats Ivy Vitaly
27 x G(x) G Generator Real or fake pair? D Discriminator Input: Sketch Grayscale Output: Photo Color [Isola, Zhu, Zhou, Efros, 2016]
28 Automatic Colorization with pix2pix Input Output Input Output Input Output Data from [Russakovsky et al. 2015]
29 Interactive Colorization [Zhang*, Zhu*, Isola, Geng, Lin, Yu, Efros, 2017]
![2015]](/docs-images/89/97791926/images/30-9.jpg)
30 Edges Images Input Output Input Output Input Output Edges from [Xie & Tu, 2015]
![2012]](/docs-images/89/97791926/images/31-10.jpg)
31 Sketches Images Input Output Input Output Input Output Trained on Edges Images Data from [Eitz, Hays, Alexa, 2012]
32 Data from [maps.google.com] Input Output Groundtruth
33 Input Output Groundtruth Data from [maps.google.com]
34 Paired
35 Paired Unpaired
36 Goals: Improve Control, Quality, and Resolution pix2pix CycleGAN pix2pixhd GANs Conditional on user inputs. Learning without pairs. High quality and resolution.
37 Cycle-Consistent Adversarial Networks [Zhu*, Park*, Isola, and Efros, 2017]
![[Mark Twain, 1903]](/docs-images/89/97791926/images/38-1.jpg "[Zhu*, Park*, Isola,")
38 Cycle-Consistent Adversarial Networks [Mark Twain, 1903] [Zhu*, Park*, Isola, and Efros, 2017]
39 Cycle Consistency Loss x G(x) F(G x ) D Y (G x ) Reconstruction error F G x x 1 See also [Yi et al., 2017], [Kim et al, 2017] [Zhu*, Park*, Isola, and Efros, 2017]
40 Cycle Consistency Loss x G(x) F(G x ) y F(y) G(F x ) D Y (G x ) D G (F x ) Reconstruction error Reconstruction error F G x x 1 G F y y 1 See also [Yi et al., 2017], [Kim et al, 2017] [Zhu*, Park*, Isola, and Efros, 2017]
41 Horse Zebra
42 Orange Apple
43 Cezanne Ukiyo-e Collection Style Transfer Photograph Alexei Efros Monet Van Gogh
44 Monet s paintings photographic style
45 Why CycleGAN works
46 Style and Content Separation Paired Separation Content Unpaired Separation Adversarial Loss: change the style Style Cycle Consistency Loss: preserve the content Separating Style and Content with Bilinear Models [Tenenbaum and Freeman 2000 ] Two empirical assumptions: - content is easy to keep. - style is easy to change.
47 Neural Style Transfer [Gatys et al. 2015] Style and Content: - Content: feature difference - Style: Gram Matrix difference - Both losses are hard-coded.
48 horse zebra Input Style Image I Style image II Entire collection CycleGAN Photo Van Gogh Input Style image I Style image II Entire collection CycleGAN
49 Cycle Loss upper bounds Conditional Entropy Conditional Entropy High Conditional Entropy Low Conditional Entropy ALICE: Towards Understanding Adversarial Learning for Joint Distribution Matching [Li et al. NIPS 2017]. Also see [Tiao et al. 2018] CycleGAN as Approximate Bayesian Inference
50 Cycle Loss upper bounds Conditional Entropy Conditional Entropy ALICE: Towards Understanding Adversarial Learning for Joint Distribution Matching [Li et al. NIPS 2017]. Also see [Tiao et al. 2018] CycleGAN as Approximate Bayesian Inference
51 Customizing Gaming Experience Grand Theft Auto v (GTA5) Street view images in German cities Data from [Richter et al., 2016], [Cordts et al, 2016]
52 Customizing Gaming Experience Output image with Input German GTA5 CG street view style
53 Domain Adaptation with CycleGAN Train on GTA5 data Test on real images meaniou Per-pixel accuracy Oracle (Train and test on Real) Train on CG, test on Real See Judy Hoffman s talk at 14:30 Adversarial Domain Adaptation
54 Domain Adaptation with CycleGAN GTA5 data + Domain adaptation Test on real images meaniou Per-pixel accuracy Oracle (Train and test on Real) Train on CG, test on Real FCN in the wild [Previous STOA] See Judy Hoffman s talk at 14:30 Adversarial Domain Adaptation
![1 Train on CG, test on Real 17.9 54.0 FCN in the wild [Previous STOA] 27.](/docs-images/89/97791926/images/55-1.jpg "1 - Train on CycleGAN, test on Real 34.8 82.")
55 Domain Adaptation with CycleGAN Train on CycleGAN data Test on real images meaniou Per-pixel accuracy Oracle (Train and test on Real) Train on CG, test on Real FCN in the wild [Previous STOA] Train on CycleGAN, test on Real See Judy Hoffman s talk at 14:30 Adversarial Domain Adaptation
56 Failure case
57 Failure case
58 Open Source CycleGAN and pix2pix Among the most popular GitHub research projects since Among the most cited papers in Graphics/CV/ML since 2017.
59 CycleGAN in Classes CycleGAN results by students MS emoji Apple emoji MS emoji Input photo Stained glass art Roger Grosse, UoT Alena Harley, FastAI
60 Applications and Extentions Attribute Editing [Lu et al.] Object Editing [Liang et al.] Low-res Bald Bangs arxiv: Mask Input Output arxiv: Front/Character Transfer [Ignatov et al.] Data generation [Wang et al.] Input output arxiv: samples by CycleWGAN arxiv:
61 Photo Enhancement WESPE: Weakly Supervised Photo Enhancer for Digital Cameras. arxiv Andrey Ignatov, Nikolay Kobyshev, Kenneth Vanhoey, Radu Timofte, Luc Van Gool
62 Image Dehazing Cycle-Dehaze: Enhanced CycleGAN for Single Image Dehazing. CVPRW 2018 Deniz Engin Anıl Genc, Hazım Kemal Ekenel
63 Unsupervised Motion Retargeting Neural Kinematic Networks for Unsupervised Motion Retargetting. CVPR 2018 (oral) Ruben Villegas, Jimei Yang, Duygu Ceylan, Honglak Lee
64 Neural Kinematic Networks for Unsupervised Motion Retargetting. CVPR 2018 (oral) Ruben Villegas, Jimei Yang, Duygu Ceylan, Honglak Lee
65 Applications Beyond Computer Vision Medical Imaging and Biology [Wolterink et al., 2017] Voice conversion [Fang et al., 2018, Kaneko et al., 2017] Cryptography [CipherGAN: Gomez et al., ICLR 2018] Robotics NLP: Unsupervised machine translation. NLP: Text style transfer.
66 Input MR Generated CT Ground truth CT
67 Latest from #CycleGAN Input dog Output cat Input cat Output dog itok_msi
68 CycleGAN for Customized Gaming Cahintan Trivedi Battle royale games Low-res 256p/512p Fortnite Input PUBG Style Final result
69 Goals: Improve Control, Quality, and Resolution pix2pix CycleGAN pix2pixhd GANs Conditional on user inputs. Learning without pairs. High quality and resolution.
70 The Curse of Dimensionality Tree Building Car Road Sidewalk Pix2pix output
![, 2017] [Karras et al.](/docs-images/89/97791926/images/71-3.jpg ", 2018] D 2 High-res Discriminator")
71 pix2pixhd Low-res Discriminator D 1 Real/fake? G 1 Low-res Generator G 2 Coarse-to-fine High-res Generator Image Pyramid [Burt and Adelson, 1987] Also see [Zhang et al., 2017] [Karras et al., 2018] D 2 High-res Discriminator Real/fake? [Wang, Liu, Zhu, Tao, Kautz. Catanzaro, 2018]
72 pix2pixhd: Tree Building Car Road Sidewalk
73
74
75 pix2pixhd for sketch face
76 Improve Control, Still a long Quality, wayand to go Resolution pix2pix CycleGAN pix2pixhd GANs Learning to generate images from trillions of photos. Help more people tell their own visual stories.
77
78 Thank You! LynnHo
Generative Models II. Phillip Isola, MIT, OpenAI DLSS 7/27/18
Generative Models II Phillip Isola, MIT, OpenAI DLSS 7/27/18 What s a generative model? For this talk: models that output high-dimensional data (Or, anything involving a GAN, VAE, PixelCNN, etc) Useful
Deep Learning for Visual Manipulation and Synthesis
Deep Learning for Visual Manipulation and Synthesis Jun-Yan Zhu 朱俊彦 UC Berkeley 2017/01/11 @ VALSE What is visual manipulation? Image Editing Program input photo User Input result Desired output: stay
Generative Networks. James Hays Computer Vision
Generative Networks James Hays Computer Vision Interesting Illusion: Ames Window https://www.youtube.com/watch?v=ahjqe8eukhc https://en.wikipedia.org/wiki/ames_trapezoid Recap Unsupervised Learning Style
CS230: Lecture 4 Attacking Networks with Adversarial Examples - Generative Adversarial Networks
Go to www.menti.com and use the code 91 41 37 CS230: Lecture 4 Attacking Networks with Adversarial Examples - Generative Adversarial Networks Kian Katanforoosh Today s outline I. Attacking NNs with Adversarial
Unsupervised Cross-Domain Deep Image Generation
Unsupervised Cross-Domain Deep Image Generation Yaniv Taigman, Adam Polyak, Lior Wolf Facebook AI Research (FAIR) Tel Aviv Supervised Learning; {Xi, yi} àf Face Recognition (DeepFace / FAIR) Kaiming et
arxiv: v1 [cs.cv] 5 Jul 2017
![arxiv: v1 [cs.cv] 5 Jul 2017](/thumbs/95/123923253.jpg "arxiv: v1 [cs.cv] 5 Jul 2017")
AlignGAN: Learning to Align Cross- Images with Conditional Generative Adversarial Networks Xudong Mao Department of Computer Science City University of Hong Kong xudonmao@gmail.com Qing Li Department of
Deep Fakes using Generative Adversarial Networks (GAN)
Deep Fakes using Generative Adversarial Networks (GAN) Tianxiang Shen UCSD La Jolla, USA tis038@eng.ucsd.edu Ruixian Liu UCSD La Jolla, USA rul188@eng.ucsd.edu Ju Bai UCSD La Jolla, USA jub010@eng.ucsd.edu
GENERATIVE ADVERSARIAL NETWORKS (GAN) Presented by Omer Stein and Moran Rubin
GENERATIVE ADVERSARIAL NETWORKS (GAN) Presented by Omer Stein and Moran Rubin GENERATIVE MODEL Given a training dataset, x, try to estimate the distribution, Pdata(x) Explicitly or Implicitly (GAN) Explicitly
Two Routes for Image to Image Translation: Rule based vs. Learning based. Minglun Gong, Memorial Univ. Collaboration with Mr.
Two Routes for Image to Image Translation: Rule based vs. Learning based Minglun Gong, Memorial Univ. Collaboration with Mr. Zili Yi Introduction A brief history of image processing Image to Image translation
Progress on Generative Adversarial Networks
Progress on Generative Adversarial Networks Wangmeng Zuo Vision Perception and Cognition Centre Harbin Institute of Technology Content Image generation: problem formulation Three issues about GAN Discriminate
GAN Related Works. CVPR 2018 & Selective Works in ICML and NIPS. Zhifei Zhang
GAN Related Works CVPR 2018 & Selective Works in ICML and NIPS Zhifei Zhang Generative Adversarial Networks (GANs) 9/12/2018 2 Generative Adversarial Networks (GANs) Feedforward Backpropagation Real? z
Alternatives to Direct Supervision
CreativeAI: Deep Learning for Graphics Alternatives to Direct Supervision Niloy Mitra Iasonas Kokkinos Paul Guerrero Nils Thuerey Tobias Ritschel UCL UCL UCL TUM UCL Timetable Theory and Basics State of
Lecture 19: Generative Adversarial Networks
Lecture 19: Generative Adversarial Networks Roger Grosse 1 Introduction Generative modeling is a type of machine learning where the aim is to model the distribution that a given set of data (e.g. images,
Lecture 3 GANs and Their Applications in Image Generation
Lecture 3 GANs and Their Applications in Image Generation Lin ZHANG, PhD School of Software Engineering Tongji University Fall 2017 Outline Introduction Theoretical Part Application Part Existing Implementations
Unsupervised Learning
Deep Learning for Graphics Unsupervised Learning Niloy Mitra Iasonas Kokkinos Paul Guerrero Vladimir Kim Kostas Rematas Tobias Ritschel UCL UCL/Facebook UCL Adobe Research U Washington UCL Timetable Niloy
arxiv: v1 [cs.cv] 1 Nov 2018
![arxiv: v1 [cs.cv] 1 Nov 2018](/thumbs/89/100791471.jpg "arxiv: v1 [cs.cv] 1 Nov 2018")
Examining Performance of Sketch-to-Image Translation Models with Multiclass Automatically Generated Paired Training Data Dichao Hu College of Computing, Georgia Institute of Technology, 801 Atlantic Dr
What was Monet seeing while painting? Translating artworks to photo-realistic images M. Tomei, L. Baraldi, M. Cornia, R. Cucchiara
What was Monet seeing while painting? Translating artworks to photo-realistic images M. Tomei, L. Baraldi, M. Cornia, R. Cucchiara COMPUTER VISION IN THE ARTISTIC DOMAIN The effectiveness of Computer Vision
Unsupervised Deep Learning. James Hays slides from Carl Doersch and Richard Zhang
Unsupervised Deep Learning James Hays slides from Carl Doersch and Richard Zhang Recap from Previous Lecture We saw two strategies to get structured output while using deep learning With object detection,
GENERATIVE ADVERSARIAL NETWORK-BASED VIR-
GENERATIVE ADVERSARIAL NETWORK-BASED VIR- TUAL TRY-ON WITH CLOTHING REGION Shizuma Kubo, Yusuke Iwasawa, and Yutaka Matsuo The University of Tokyo Bunkyo-ku, Japan {kubo, iwasawa, matsuo}@weblab.t.u-tokyo.ac.jp
An Empirical Study of Generative Adversarial Networks for Computer Vision Tasks
An Empirical Study of Generative Adversarial Networks for Computer Vision Tasks Report for Undergraduate Project - CS396A Vinayak Tantia (Roll No: 14805) Guide: Prof Gaurav Sharma CSE, IIT Kanpur, India
Lecture 7: Semantic Segmentation
Semantic Segmentation CSED703R: Deep Learning for Visual Recognition (207F) Segmenting images based on its semantic notion Lecture 7: Semantic Segmentation Bohyung Han Computer Vision Lab. bhhanpostech.ac.kr
REGION AVERAGE POOLING FOR CONTEXT-AWARE OBJECT DETECTION
REGION AVERAGE POOLING FOR CONTEXT-AWARE OBJECT DETECTION Kingsley Kuan 1, Gaurav Manek 1, Jie Lin 1, Yuan Fang 1, Vijay Chandrasekhar 1,2 Institute for Infocomm Research, A*STAR, Singapore 1 Nanyang Technological
arxiv: v3 [cs.cv] 24 Nov 2017
![arxiv: v3 [cs.cv] 24 Nov 2017](/thumbs/72/67417859.jpg "arxiv: v3 [cs.cv] 24 Nov 2017")
Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks Jun-Yan Zhu Taesung Park Phillip Isola Alexei A. Efros Berkeley AI Research (BAIR) laboratory, UC Berkeley arxiv:1703.10593v3
arxiv: v1 [cs.cv] 6 Sep 2018
![arxiv: v1 [cs.cv] 6 Sep 2018](/thumbs/85/92251412.jpg "arxiv: v1 [cs.cv] 6 Sep 2018")
arxiv:1809.01890v1 [cs.cv] 6 Sep 2018 Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto
DOMAIN-ADAPTIVE GENERATIVE ADVERSARIAL NETWORKS FOR SKETCH-TO-PHOTO INVERSION
DOMAIN-ADAPTIVE GENERATIVE ADVERSARIAL NETWORKS FOR SKETCH-TO-PHOTO INVERSION Yen-Cheng Liu 1, Wei-Chen Chiu 2, Sheng-De Wang 1, and Yu-Chiang Frank Wang 1 1 Graduate Institute of Electrical Engineering,
arxiv: v1 [cs.cv] 7 Mar 2018
![arxiv: v1 [cs.cv] 7 Mar 2018](/thumbs/95/125565711.jpg "arxiv: v1 [cs.cv] 7 Mar 2018")
Accepted as a conference paper at the European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN) 2018 Inferencing Based on Unsupervised Learning of Disentangled
Unsupervised Image-to-Image Translation with Stacked Cycle-Consistent Adversarial Networks
Unsupervised Image-to-Image Translation with Stacked Cycle-Consistent Adversarial Networks Minjun Li 1,2, Haozhi Huang 2, Lin Ma 2, Wei Liu 2, Tong Zhang 2, Yu-Gang Jiang 1 1 Shanghai Key Lab of Intelligent
DOMAIN-ADAPTIVE GENERATIVE ADVERSARIAL NETWORKS FOR SKETCH-TO-PHOTO INVERSION
2017 IEEE INTERNATIONAL WORKSHOP ON MACHINE LEARNING FOR SIGNAL PROCESSING, SEPT. 25 28, 2017, TOKYO, JAPAN DOMAIN-ADAPTIVE GENERATIVE ADVERSARIAL NETWORKS FOR SKETCH-TO-PHOTO INVERSION Yen-Cheng Liu 1,
Synscapes A photorealistic syntehtic dataset for street scene parsing Jonas Unger Department of Science and Technology Linköpings Universitet.
Synscapes A photorealistic syntehtic dataset for street scene parsing Jonas Unger Department of Science and Technology Linköpings Universitet 7D Labs VINNOVA https://7dlabs.com Photo-realistic image synthesis
Introduction to Generative Adversarial Networks
Introduction to Generative Adversarial Networks Ian Goodfellow, OpenAI Research Scientist NIPS 2016 Workshop on Adversarial Training Barcelona, 2016-12-9 Adversarial Training A phrase whose usage is in
Fully Convolutional Networks for Semantic Segmentation
Fully Convolutional Networks for Semantic Segmentation Jonathan Long* Evan Shelhamer* Trevor Darrell UC Berkeley Chaim Ginzburg for Deep Learning seminar 1 Semantic Segmentation Define a pixel-wise labeling
Introduction to GANs
MedGAN ID-CGAN CoGAN LR-GAN CGAN IcGAN b-gan LS-GAN LAPGAN DiscoGANMPM-GAN AdaGAN LSGAN InfoGAN CatGAN AMGAN igan Introduction to GANs IAN SAGAN McGAN Ian Goodfellow, Staff Research Scientist, Google Brain
GraphGAN: Graph Representation Learning with Generative Adversarial Nets
The 32 nd AAAI Conference on Artificial Intelligence (AAAI 2018) New Orleans, Louisiana, USA GraphGAN: Graph Representation Learning with Generative Adversarial Nets Hongwei Wang 1,2, Jia Wang 3, Jialin
Joint Inference in Image Databases via Dense Correspondence. Michael Rubinstein MIT CSAIL (while interning at Microsoft Research)
Joint Inference in Image Databases via Dense Correspondence Michael Rubinstein MIT CSAIL (while interning at Microsoft Research) My work Throughout the year (and my PhD thesis): Temporal Video Analysis
arxiv: v1 [cs.cv] 8 May 2018
![arxiv: v1 [cs.cv] 8 May 2018](/thumbs/83/87299988.jpg "arxiv: v1 [cs.cv] 8 May 2018")
arxiv:1805.03189v1 [cs.cv] 8 May 2018 Learning image-to-image translation using paired and unpaired training samples Soumya Tripathy 1, Juho Kannala 2, and Esa Rahtu 1 1 Tampere University of Technology
arxiv: v1 [stat.ml] 14 Sep 2017
![arxiv: v1 [stat.ml] 14 Sep 2017](/thumbs/78/77167912.jpg "arxiv: v1 [stat.ml] 14 Sep 2017")
The Conditional Analogy GAN: Swapping Fashion Articles on People Images Nikolay Jetchev Zalando Research nikolay.jetchev@zalando.de Urs Bergmann Zalando Research urs.bergmann@zalando.de arxiv:1709.04695v1
arxiv: v2 [cs.cv] 13 Jun 2017
![arxiv: v2 [cs.cv] 13 Jun 2017](/thumbs/80/81053395.jpg "arxiv: v2 [cs.cv] 13 Jun 2017")
Style Transfer for Anime Sketches with Enhanced Residual U-net and Auxiliary Classifier GAN arxiv:1706.03319v2 [cs.cv] 13 Jun 2017 Lvmin Zhang, Yi Ji and Xin Lin School of Computer Science and Technology,
Controllable Generative Adversarial Network
Controllable Generative Adversarial Network arxiv:1708.00598v2 [cs.lg] 12 Sep 2017 Minhyeok Lee 1 and Junhee Seok 1 1 School of Electrical Engineering, Korea University, 145 Anam-ro, Seongbuk-gu, Seoul,
A FRAMEWORK OF EXTRACTING MULTI-SCALE FEATURES USING MULTIPLE CONVOLUTIONAL NEURAL NETWORKS. Kuan-Chuan Peng and Tsuhan Chen
A FRAMEWORK OF EXTRACTING MULTI-SCALE FEATURES USING MULTIPLE CONVOLUTIONAL NEURAL NETWORKS Kuan-Chuan Peng and Tsuhan Chen School of Electrical and Computer Engineering, Cornell University, Ithaca, NY
Proceedings of the International MultiConference of Engineers and Computer Scientists 2018 Vol I IMECS 2018, March 14-16, 2018, Hong Kong
, March 14-16, 2018, Hong Kong , March 14-16, 2018, Hong Kong , March 14-16, 2018, Hong Kong , March 14-16, 2018, Hong Kong TABLE I CLASSIFICATION ACCURACY OF DIFFERENT PRE-TRAINED MODELS ON THE TEST DATA
Seeing 3D chairs: Exemplar part-based 2D-3D alignment using a large dataset of CAD models
Seeing 3D chairs: Exemplar part-based 2D-3D alignment using a large dataset of CAD models Mathieu Aubry (INRIA) Daniel Maturana (CMU) Alexei Efros (UC Berkeley) Bryan Russell (Intel) Josef Sivic (INRIA)
arxiv: v2 [cs.cv] 23 Dec 2017
![arxiv: v2 [cs.cv] 23 Dec 2017](/thumbs/75/72736245.jpg "arxiv: v2 [cs.cv] 23 Dec 2017")
TextureGAN: Controlling Deep Image Synthesis with Texture Patches Wenqi Xian 1 Patsorn Sangkloy 1 Varun Agrawal 1 Amit Raj 1 Jingwan Lu 2 Chen Fang 2 Fisher Yu 3 James Hays 1 1 Georgia Institute of Technology
High-Resolution Image Dehazing with respect to Training Losses and Receptive Field Sizes
High-Resolution Image Dehazing with respect to Training osses and Receptive Field Sizes Hyeonjun Sim, Sehwan Ki, Jae-Seok Choi, Soo Ye Kim, Soomin Seo, Saehun Kim, and Munchurl Kim School of EE, Korea
arxiv: v1 [cs.cv] 19 Dec 2017
![arxiv: v1 [cs.cv] 19 Dec 2017](/thumbs/79/79999778.jpg "arxiv: v1 [cs.cv] 19 Dec 2017")
ComboGA: Unrestrained Scalability for Image Domain Translation ariv:171.06909v1 [cs.cv] 19 Dec 017 Asha Anoosheh Computer Vision Lab ashaa@ethz.ch Abstract Eirikur Agustsson Computer Vision Lab aeirikur@ethz.ch
Content-Based Image Recovery
Content-Based Image Recovery Hong-Yu Zhou and Jianxin Wu National Key Laboratory for Novel Software Technology Nanjing University, China zhouhy@lamda.nju.edu.cn wujx2001@nju.edu.cn Abstract. We propose
arxiv: v1 [cs.cv] 17 Nov 2016
![arxiv: v1 [cs.cv] 17 Nov 2016](/thumbs/86/93859848.jpg "arxiv: v1 [cs.cv] 17 Nov 2016")
Inverting The Generator Of A Generative Adversarial Network arxiv:1611.05644v1 [cs.cv] 17 Nov 2016 Antonia Creswell BICV Group Bioengineering Imperial College London ac2211@ic.ac.uk Abstract Anil Anthony
A GAN framework for Instance Segmentation using the Mutex Watershed Algorithm
A GAN framework for Instance Segmentation using the Mutex Watershed Algorithm Mandikal Vikram National Institute of Technology Karnataka, India 15it217.vikram@nitk.edu.in Steffen Wolf HCI/IWR, University
Sky is Not the Limit: Semantic-Aware Sky Replacement
Sky is Not the Limit: Semantic-Aware Sky Replacement ACM Transactions on Graphics (SIGGRAPH), 2016 Yi-Hsuan Tsai UC Merced Xiaohui Shen Adobe Research Zhe Lin Adobe Research Kalyan Sunkavalli Adobe Research
Generative Adversarial Network
Generative Adversarial Network Many slides from NIPS 2014 Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio Generative adversarial
Amodal and Panoptic Segmentation. Stephanie Liu, Andrew Zhou
Amodal and Panoptic Segmentation Stephanie Liu, Andrew Zhou This lecture: 1. 2. 3. 4. Semantic Amodal Segmentation Cityscapes Dataset ADE20K Dataset Panoptic Segmentation Semantic Amodal Segmentation Yan
Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform. Xintao Wang Ke Yu Chao Dong Chen Change Loy
Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform Xintao Wang Ke Yu Chao Dong Chen Change Loy Problem enlarge 4 times Low-resolution image High-resolution image Previous
Deep Manga Colorization with Color Style Extraction by Conditional Adversarially Learned Inference
Information Engineering Express International Institute of Applied Informatics 2017, Vol.3, No.4, P.55-66 Deep Manga Colorization with Color Style Extraction by Conditional Adversarially Learned Inference
Learning to generate 3D shapes
Learning to generate 3D shapes Subhransu Maji College of Information and Computer Sciences University of Massachusetts, Amherst http://people.cs.umass.edu/smaji August 10, 2018 @ Caltech Creating 3D shapes
Composable Unpaired Image to Image Translation
Composable Unpaired Image to Image Translation Laura Graesser New York University lhg256@nyu.edu Anant Gupta New York University ag4508@nyu.edu Abstract There has been remarkable recent work in unpaired
Unified, real-time object detection
Unified, real-time object detection Final Project Report, Group 02, 8 Nov 2016 Akshat Agarwal (13068), Siddharth Tanwar (13699) CS698N: Recent Advances in Computer Vision, Jul Nov 2016 Instructor: Gaurav
Lab meeting (Paper review session) Stacked Generative Adversarial Networks
Lab meeting (Paper review session) Stacked Generative Adversarial Networks 2017. 02. 01. Saehoon Kim (Ph. D. candidate) Machine Learning Group Papers to be covered Stacked Generative Adversarial Networks
Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks Jun-Yan Zhu Taesung Park Phillip Isola Alexei A. Efros Berkeley AI Research (BAIR) laboratory, UC Berkeley Monet Photos Zebras
Transfer Learning. Style Transfer in Deep Learning
Transfer Learning & Style Transfer in Deep Learning 4-DEC-2016 Gal Barzilai, Ram Machlev Deep Learning Seminar School of Electrical Engineering Tel Aviv University Part 1: Transfer Learning in Deep Learning
Regionlet Object Detector with Hand-crafted and CNN Feature
Regionlet Object Detector with Hand-crafted and CNN Feature Xiaoyu Wang Research Xiaoyu Wang Research Ming Yang Horizon Robotics Shenghuo Zhu Alibaba Group Yuanqing Lin Baidu Overview of this section Regionlet
arxiv: v1 [cs.cv] 3 Apr 2019
![arxiv: v1 [cs.cv] 3 Apr 2019](/thumbs/95/125074959.jpg "arxiv: v1 [cs.cv] 3 Apr 2019")
Unpaired Thermal to Visible Spectrum Transfer using Adversarial Training Adam Nyberg 1[0000 0001 8764 8499], Abdelrahman Eldesokey 1[0000 0003 3292 7153], David Bergström 2[0000 0003 2414 4482], and David
TextureGAN: Controlling Deep Image Synthesis with Texture Patches
TextureGAN: Controlling Deep Image Synthesis with Texture Patches Wenqi Xian 1 Patsorn Sangkloy 1 Varun Agrawal 1 Amit Raj 1 Jingwan Lu 2 Chen Fang 2 Fisher Yu 3 James Hays 1,4 1 Georgia Institute of Technology
Learning Deep Structured Models for Semantic Segmentation. Guosheng Lin
Learning Deep Structured Models for Semantic Segmentation Guosheng Lin Semantic Segmentation Outline Exploring Context with Deep Structured Models Guosheng Lin, Chunhua Shen, Ian Reid, Anton van dan Hengel;
JOINT DETECTION AND SEGMENTATION WITH DEEP HIERARCHICAL NETWORKS. Zhao Chen Machine Learning Intern, NVIDIA
JOINT DETECTION AND SEGMENTATION WITH DEEP HIERARCHICAL NETWORKS Zhao Chen Machine Learning Intern, NVIDIA ABOUT ME 5th year PhD student in physics @ Stanford by day, deep learning computer vision scientist
Detecting and Parsing of Visual Objects: Humans and Animals. Alan Yuille (UCLA)
Detecting and Parsing of Visual Objects: Humans and Animals Alan Yuille (UCLA) Summary This talk describes recent work on detection and parsing visual objects. The methods represent objects in terms of
Human Pose Estimation with Deep Learning. Wei Yang
Human Pose Estimation with Deep Learning Wei Yang Applications Understand Activities Family Robots American Heist (2014) - The Bank Robbery Scene 2 What do we need to know to recognize a crime scene? 3
THERMAL TEXTURE GENERATION AND 3D MODEL RECONSTRUCTION USING SFM AND GAN
THERMAL TEXTURE GENERATION AND 3D MODEL RECONSTRUCTION USING SFM AND GAN V. V. Kniaz a,b, V. A. Mizginov a a State Res. Institute of Aviation Systems (GosNIIAS), 125319, 7, Victorenko str., Moscow, Russia
Adversarial Machine Learning
MedGAN Progressive GAN CoGAN LR-GAN CGAN IcGAN BIM LS-GAN AffGAN LAPGAN DiscoGANMPM-GAN AdaGAN LSGAN InfoGAN ATN FGSM igan IAN Adversarial Machine Learning McGAN Ian Goodfellow, Staff Research Scientist,
Unsupervised Visual Attribute Transfer with Reconfigurable Generative Adversarial Networks
Unsupervised Visual Attribute Transfer with Reconfigurable Generative Adversarial Networks Taeksoo Kim, Byoungjip Kim, Moonsu Cha, Jiwon Kim SK T-Brain {jazzsaxmafia,bjkim,ckanstnzja,jk}@sktbrain.com arxiv:1707.09798v1
A Unified Feature Disentangler for Multi-Domain Image Translation and Manipulation
A Unified Feature Disentangler for Multi-Domain Image Translation and Manipulation Alexander H. Liu 1 Yen-Cheng Liu 2 Yu-Ying Yeh 3 Yu-Chiang Frank Wang 1,4 1 National Taiwan University, Taiwan 2 Georgia
arxiv: v1 [cs.cv] 11 Oct 2018
![arxiv: v1 [cs.cv] 11 Oct 2018](/thumbs/94/121703201.jpg "arxiv: v1 [cs.cv] 11 Oct 2018")
SingleGAN: Image-to-Image Translation by a Single-Generator Network using Multiple Generative Adversarial Learning Xiaoming Yu, Xing Cai, Zhenqiang Ying, Thomas Li, and Ge Li arxiv:1810.04991v1 [cs.cv]
SYNTHESIS OF IMAGES BY TWO-STAGE GENERATIVE ADVERSARIAL NETWORKS. Qiang Huang, Philip J.B. Jackson, Mark D. Plumbley, Wenwu Wang
SYNTHESIS OF IMAGES BY TWO-STAGE GENERATIVE ADVERSARIAL NETWORKS Qiang Huang, Philip J.B. Jackson, Mark D. Plumbley, Wenwu Wang Centre for Vision, Speech and Signal Processing University of Surrey, Guildford,
Stanford University Packing and Padding, Coupled Multi-index for Accurate Image Retrieval.
Liang Zheng Nationality: China Information Systems Technology and Design pillar Singapore University of Technology and Design 8 Somapah Road Singapore 487372 Date of Birth: 11 Jun 1987 Phone: +65 84036274
arxiv: v1 [cs.cv] 3 Dec 2018
![arxiv: v1 [cs.cv] 3 Dec 2018](/thumbs/91/106616262.jpg "arxiv: v1 [cs.cv] 3 Dec 2018")
SPLAT: Semantic Pixel-Level Adaptation Transforms for Detection arxiv:1812.00929v1 [cs.cv] 3 Dec 2018 Eric Tzeng UC Berkeley etzeng@eecs.berkeley.edu Abstract Domain adaptation of visual detectors is a
Yiqi Yan. May 10, 2017
Yiqi Yan May 10, 2017 P a r t I F u n d a m e n t a l B a c k g r o u n d s Convolution Single Filter Multiple Filters 3 Convolution: case study, 2 filters 4 Convolution: receptive field receptive field
Generative Semantic Manipulation with Mask-Contrasting GAN
Generative Semantic Manipulation with Mask-Contrasting GAN Xiaodan Liang 1, Hao Zhang 1, Liang Lin 2, and Eric Xing 1 1 Carnegie Mellon University, {xiaodan1, hao, epxing}@cs.cmu.edu 2 Sun Yat-sen University,
XNet: GAN Latent Space Constraints
: GAN Latent Space Constraints Omry Sendik Tel Aviv University Tel Aviv, Israel omrysendik@gmail.com Dani Lischinski Hebrew University Jerusalem, Israel danix3d@gmail.com Danny CohenOr Tel Aviv University
Paired 3D Model Generation with Conditional Generative Adversarial Networks
Accepted to 3D Reconstruction in the Wild Workshop European Conference on Computer Vision (ECCV) 2018 Paired 3D Model Generation with Conditional Generative Adversarial Networks Cihan Öngün Alptekin Temizel
Photo-realistic Renderings for Machines Seong-heum Kim
Photo-realistic Renderings for Machines 20105034 Seong-heum Kim CS580 Student Presentations 2016.04.28 Photo-realistic Renderings for Machines Scene radiances Model descriptions (Light, Shape, Material,
arxiv: v3 [cs.cv] 30 Mar 2018
![arxiv: v3 [cs.cv] 30 Mar 2018](/thumbs/78/78600089.jpg "arxiv: v3 [cs.cv] 30 Mar 2018")
Learning to Generate Time-Lapse Videos Using Multi-Stage Dynamic Generative Adversarial Networks Wei Xiong Wenhan Luo Lin Ma Wei Liu Jiebo Luo Tencent AI Lab University of Rochester {wxiong5,jluo}@cs.rochester.edu
arxiv: v2 [cs.cv] 2 Dec 2017
![arxiv: v2 [cs.cv] 2 Dec 2017](/thumbs/86/94865601.jpg "arxiv: v2 [cs.cv] 2 Dec 2017")
Learning to Generate Time-Lapse Videos Using Multi-Stage Dynamic Generative Adversarial Networks Wei Xiong, Wenhan Luo, Lin Ma, Wei Liu, and Jiebo Luo Department of Computer Science, University of Rochester,
Martian lava field, NASA, Wikipedia
Martian lava field, NASA, Wikipedia Old Man of the Mountain, Franconia, New Hampshire Pareidolia http://smrt.ccel.ca/203/2/6/pareidolia/ Reddit for more : ) https://www.reddit.com/r/pareidolia/top/ Pareidolia
GAN and Feature Representation. Hung-yi Lee
GAN and Feature Representation Hung-yi Lee Outline Generator (Decoder) Discrimi nator + Encoder GAN+Autoencoder x InfoGAN Encoder z Generator Discrimi (Decoder) x nator scalar Discrimi z Generator x scalar
Geometry-aware Traffic Flow Analysis by Detection and Tracking
Geometry-aware Traffic Flow Analysis by Detection and Tracking 1,2 Honghui Shi, 1 Zhonghao Wang, 1,2 Yang Zhang, 1,3 Xinchao Wang, 1 Thomas Huang 1 IFP Group, Beckman Institute at UIUC, 2 IBM Research,
arxiv: v1 [cs.cv] 1 May 2018
![arxiv: v1 [cs.cv] 1 May 2018](/thumbs/83/88735687.jpg "arxiv: v1 [cs.cv] 1 May 2018")
Conditional Image-to-Image Translation Jianxin Lin 1 Yingce Xia 1 Tao Qin 2 Zhibo Chen 1 Tie-Yan Liu 2 1 University of Science and Technology of China 2 Microsoft Research Asia linjx@mail.ustc.edu.cn {taoqin,
arxiv: v3 [cs.cv] 22 Feb 2018
![arxiv: v3 [cs.cv] 22 Feb 2018](/thumbs/76/73136133.jpg "arxiv: v3 [cs.cv] 22 Feb 2018")
A Closed-form Solution to Photorealistic Image Stylization Yijun Li 1, Ming-Yu Liu 2, Xueting Li 1, Ming-Hsuan Yang 1,2, and Jan Kautz 2 1 University of California, Merced 2 NVIDIA {yli62,xli75,mhyang}@ucmerced.edu
Supplementary Material: Unsupervised Domain Adaptation for Face Recognition in Unlabeled Videos
Supplementary Material: Unsupervised Domain Adaptation for Face Recognition in Unlabeled Videos Kihyuk Sohn 1 Sifei Liu 2 Guangyu Zhong 3 Xiang Yu 1 Ming-Hsuan Yang 2 Manmohan Chandraker 1,4 1 NEC Labs
Generative Semantic Manipulation with Contrasting GAN
Generative Semantic Manipulation with Contrasting GAN Xiaodan Liang, Hao Zhang, Eric P. Xing Carnegie Mellon University and Petuum Inc. {xiaodan1, hao, epxing}@cs.cmu.edu arxiv:1708.00315v1 [cs.cv] 1 Aug
Bidirectional GAN. Adversarially Learned Inference (ICLR 2017) Adversarial Feature Learning (ICLR 2017)
Bidirectional GAN Adversarially Learned Inference (ICLR 2017) V. Dumoulin 1, I. Belghazi 1, B. Poole 2, O. Mastropietro 1, A. Lamb 1, M. Arjovsky 3 and A. Courville 1 1 Universite de Montreal & 2 Stanford
Web-Scale Image Search and Their Applications
Web-Scale Image Search and Their Applications Sung-Eui Yoon KAIST http://sglab.kaist.ac.kr Project Guidelines: Project Topics Any topics related to the course theme are okay You can find topics by browsing
A Closed-form Solution to Photorealistic Image Stylization
A Closed-form Solution to Photorealistic Image Stylization Yijun Li 1, Ming-Yu Liu 2, Xueting Li 1, Ming-Hsuan Yang 1,2, Jan Kautz 2 1 University of California, Merced 2 NVIDIA {yli62,xli75,mhyang}@ucmerced.edu
Learning from 3D Data
Learning from 3D Data Thomas Funkhouser Princeton University* * On sabbatical at Stanford and Google Disclaimer: I am talking about the work of these people Shuran Song Andy Zeng Fisher Yu Yinda Zhang
Towards Automatic Icon Design using Machine Learning
Moses Soh (msoh) moses.soh@gmail.com Abstract We propose a deep learning approach for automatic colorization of icons. The system maps grayscale outlines to fully colored and stylized icons using a Convolutional
arxiv: v1 [cs.cv] 9 Aug 2018
![arxiv: v1 [cs.cv] 9 Aug 2018](/thumbs/83/87638223.jpg "arxiv: v1 [cs.cv] 9 Aug 2018")
Controllable Image-to-Video Translation: A Case Study on Facial Expression Generation Lijie Fan MIT CSAIL Wenbing Huang Tencent AI Lab Chuang Gan MIT-Waston Lab Junzhou Huang Tencent AI Lab Boqing Gong
Fast Guided Global Interpolation for Depth and. Yu Li, Dongbo Min, Minh N. Do, Jiangbo Lu
Fast Guided Global Interpolation for Depth and Yu Li, Dongbo Min, Minh N. Do, Jiangbo Lu Introduction Depth upsampling and motion interpolation are often required to generate a dense, high-quality, and
345 Park Ave,, San Jose, CA Homepage:
Jimei Yang Contact Information Research Interests Working Experience 345 Park Ave,, San Jose, CA 95110 E-mail: jimyang@adobe.com Homepage: https://eng.ucmerced.edu/people/jyang44 Deep Learning, Computer
Computer Vision Lecture 16
Computer Vision Lecture 16 Deep Learning for Object Categorization 14.01.2016 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Announcements Seminar registration period
arxiv: v2 [cs.cv] 25 Jul 2018
![arxiv: v2 [cs.cv] 25 Jul 2018](/thumbs/93/114177872.jpg "arxiv: v2 [cs.cv] 25 Jul 2018")
Unpaired Photo-to-Caricature Translation on Faces in the Wild Ziqiang Zheng a, Chao Wang a, Zhibin Yu a, Nan Wang a, Haiyong Zheng a,, Bing Zheng a arxiv:1711.10735v2 [cs.cv] 25 Jul 2018 a No. 238 Songling
IDENTIFYING ANALOGIES ACROSS DOMAINS
IDENTIFYING ANALOGIES ACROSS DOMAINS Yedid Hoshen 1 and Lior Wolf 1,2 1 Facebook AI Research 2 Tel Aviv University ABSTRACT Identifying analogies across domains without supervision is an important task
High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs
High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs Ting-Chun Wang1 Ming-Yu Liu1 Jun-Yan Zhu2 Andrew Tao1 Jan Kautz1 1 2 NVIDIA Corporation UC Berkeley Bryan Catanzaro1 Cascaded
CyCADA: Cycle-Consistent Adversarial Domain Adaptation
Judy Hoffman 1 Eric Tzeng 1 Taesung Park 1 Jun-Yan Zhu 1 Phillip Isola 12 Kate Saenko 3 Alexei A. Efros 1 Trevor Darrell 1 Abstract Domain adaptation is critical for success in new, unseen environments.