Pixel-level Generative Model
|
|
- Shannon Shields
- 5 years ago
- Views:
Transcription
1 Pixel-level Generative Model Generative Image Modeling Using Spatial LSTMs (2015NIPS) L. Theis and M. Bethge University of Tübingen, Germany Pixel Recurrent Neural Networks (2016ICML) A. van den Oord, N. Kalchbrenner and K. Kavukcuoglu Google DeepMind, UK Conditional Image Generation with PixelCNN Decoders (2016NIPS) A. van den Oord, N. Kalchbrenner, O. Vinyals, L. Espeholt, A. Graves and K. Kavukcuoglu Google DeepMind, UK Presented by Qi WEI June 23rd,
2 Open problem How to generate a good image? Target: model the distribution of natural images Expectation: expressive, tractable and scalable Difficulty: strong statistical dependencies over hundreds of pixels Strategy of pixel-level generation: Explicit density model Use chain rule to decompose likelihood of an image x into product of 1-d distributions p(x; θ) }{{} likelihood of image x = i,j p(x i,j x <ij ; θ) }{{} conditional on all previous pixels (1) maximize likelihood of training data 2
3 PixelRNN Generate image pixels starting from corner Dependency on previous pixels modeled using an RNN (LSTM) 3
4 PixelRNN Generate image pixels starting from corner Dependency on previous pixels modeled using an RNN (LSTM) 4
5 PixelRNN Generate image pixels starting from corner Dependency on previous pixels modeled using an RNN (LSTM) 5
6 PixelRNN Generate image pixels starting from corner Dependency on previous pixels modeled using an RNN (LSTM) 6
7 PixelCNN Also generate image pixels starting from corner Dependency on previous pixels modeled using a CNN Comparison with PixelRNN: Training PixelCNN is faster than PixelRNN Generation still proceed sequentially: still slow 7
8 1 Spatial LSTMs 2 PixelRNN 3 PixelCNN 8
9 Spatial LSTMs Spatial LSTMs A recurrent image model based on multi-dimensional LSTM Target: handle long-range dependencies that are central to object and scene understanding Advantage: scale to images of arbitrary size likelihood is computationally tractable Model Structure: MCGSM: mixtures of conditional Gaussian scale mixtures Spatial LSTM: a special case of the multidimensional LSTM (Graves & Schmidhuber, 2009) RIDE: Recurrent image density estimator, i.e., MCGSM + SLSTM 9
10 Spatial LSTMs 10 Recurrent model of natural images MCGSM+RIDE Figure: A: pixel dependency. B: causal neighborhood. C: a visualization of the proposed recurrent image model with two layers of spatial LSTMs.
11 Spatial LSTMs 11 MCGSMs: mixtures of conditional Gaussian scale mixtures Distribution of any parametric model with parameters θ: p(x; θ) = i,j p(x i,j x <ij ; θ) (2) Improve the representational power: p(x; {θ ij }) = i,j p(x i,j x <ij ; θ ij ) (3) The conditional distribution in an MCGSM: p(x ij x <ij, θ ij ) = c,s p(c, s x <ij, θ ij ) p(x i,j x <ij, c, s, θ ij ) (4) }{{}}{{} gate expert
12 Spatial LSTMs 12 MCGSMs: mixtures of conditional Gaussian scale mixtures The conditional distribution in an MCGSM: p(x ij x <ij, θ ij ) = c,s p(c, s x <ij, θ ij ) p(x i,j x <ij, c, s, θ ij ) (5) }{{}}{{} gate expert p(c, s x <ij, θ ij ) exp(η cs 1 2 eαcs x T <ijk c x <ij ) p(x i,j x <ij, c, s) = N (x ij ; a T c x <ij, e αcs ) where K c is positive definite. To reduce the number of parameters: (6) K c n β 2 cnb n b T n (7)
13 Spatial LSTMs Spatial LSTM Operations: where c ij = g ij i ij + c i,j 1 f c ij + c i 1,j f r ij h ij = o ij tanh(c ij ) g ij o ij i ij f r ij f c ij = tanh σ σ σ σ T A,b x <ij h i,j 1 h i 1,j (8) (9) where σ is the logistic sigmoid function indicates a pointwise product T A,b is an affine transformation which depends on the only parameters of the network A and b i ij, o ij, f c ij, f r ij are gating units 13
14 Spatial LSTMs 14 RIDE: recurrent image density estimator Factorized MCGSM: p(x ij x <ij ) = p(x ij h ij ) (10) the state of the hidden vector only depends on pixels in x <ij and does not violate the factorization law allows this RIDE to use pixels of a much larger region for prediction nonlinearly transform the pixels before applying the MCGSM increase the representational power of the model by stacking spatial LSTMs
15 Spatial LSTMs Experimental Results Natural Images Figure: Average log-likelihoods and log-likelihood rates for image patches (w.o./w. DC) and large images extracted from BSDS300 [25]. Figure: Average log-likelihood rates for image patches and large images extracted from van Hateren s dataset [48]. 15
16 Spatial LSTMs Experimental Results (cont.) Dead leaves Figure: From top to bottom: A 256 by 256 pixel crop of a texture [2], a sample generated by an MCGSM trained on the full texture [7], and a sample generated by RIDE (highlight D104, D34). 16
17 Spatial LSTMs 17 Experimental Results (cont.) Texture synthesis and inpainting Figure: The center portion of a texture (left and center) was reconstructed by sampling from the posterior distribution of RIDE (right).
18 Spatial LSTMs 18 Summary Pros Cons RIDE: a deep but tractable recurrent image model based on spatial LSTMs Superior performance in quantitative comparisons Able to capture many different statistical patterns Only works on grayscale images
19 PixelRNN 19 PixelRNN PixelRNN: a DNN sequentially predicting the pixels in an image along the two spatial dimensions, similarly to RIDE with Novelty: fast two dimensional recurrent layers residual connections in RNN handle RGB images Advantages: scale to images of arbitrary size get better log-likelihood scores on natural images generate more crisp, varied and globally coherent samples
20 PixelRNN 20 Model structure For one band image: p(x) = p(x i x 1,, x i 1 ) (11) n 2 i=1 For RGB images: p(x i x <i ) = p(x i,r x <i )p(x i,g x <i, x i,r )p(x i,b x <i, x i,r, x i,g ) (12) Pixels as Discrete Variables: Each channel variable x i, simply takes one of 256 distinct values.
21 PixelRNN 21 Model structure (cont.) Row LSTM & Diagonal BiLSTM
22 PixelRNN 22 Model structure (cont.) Residual Connections Figure: Left: PixelCNN Right: PixelRNN
23 PixelRNN 23 Experiment Results Residual Connections MNIST
24 PixelRNN 24 Experiment Results MNIST
25 PixelRNN 25 Experiment Results Generate Images (32 32) (a) CIFAR-10 (b) ImageNet
26 PixelRNN Experiment Results ImageNet (64 64) (c) normal model (d) multi-scale model 26
27 PixelRNN 27 Experiment Results Image Completions (32 32)
28 PixelCNN 28 PixelCNN Target: generating images conditional on any vector, e.g., labels/tags/embeddings Advantages: generate diverse, realistic scenes representing distinct animals, objects, landscapes and structures serve as a powerful decoder improve the log-likelihood of PixelRNN on ImageNet much faster to train than PixelRNN
29 PixelCNN 29 Visualization of PixelCNN Figure: Left: visualization of PixelCNN. Middle: masked convolution filter. Right: blind spot in the receptive field.
30 PixelCNN 30 Gated Convolutional Layers Joint distribution of pixels over an image: p(x) = p(x i x 1,, x i 1 ) (13) n 2 i=1 Replace RELUs between masked convolutions with the gated activation unit: y = tanh(w k,f x) σ(w k,g x) (14)
31 PixelCNN 31 Conditional PixelCNN Distribution of conditional PixelCNN: h is location-independent p(x h) = p(x i x 1,, x i 1, h) (15) n 2 i=1 y = tanh(w k,f x + Vk,f T h) σ(w k,g x + Vk,g T h) (16) h is location-dependent y = tanh(w k,f x + V k,f s) σ(w k,g x + V k,g s) (17) where s = m(h) is a spatial representation with a deconvolutional neural network m().
32 PixelCNN 32 Gated PixelCNN Figure: A single Layer in the Gated PixelCNN architecture.
33 PixelCNN 33 Experiment Results Unconditional Modeling with Gated PixelCNN (a) CIFAR-10 (b) ImageNet
34 PixelCNN Experiment Results (Cont ) Conditioning on ImageNet Classes 34
35 PixelCNN Experiment Results (Cont ) Conditioning on Portrait Embeddings 35
36 PixelCNN 36 Experiment Results (Cont ) PixelCNN Auto Encoder
37 PixelCNN 37 Summary Pros Cons provides a way to calculate likelihood training more stable than GANs works for both discrete and continuous data faster than PixelRNN assumes the order of generation: top to down, left to right slow in sampling as in PixelRNN slow in training (though faster than PixelRNN): PixelCNN++ (from OpenAI) converges in 5 days on 8 Titan for CIFAR dataset.
38 PixelCNN 38 Take-home Message Both PixelRNN and PixelCNN can Quantitative: explicitly compute likelihood p(x) which is high Qualitative: generate visually good image samples and PixelRNN: slow to train and slow to generate PixelCNN: fast to train and slow to generate Other interesting works: PixelVAE, PixelCNN++, PixelGAN,...
Amortised MAP Inference for Image Super-resolution. Casper Kaae Sønderby, Jose Caballero, Lucas Theis, Wenzhe Shi & Ferenc Huszár ICLR 2017
Amortised MAP Inference for Image Super-resolution Casper Kaae Sønderby, Jose Caballero, Lucas Theis, Wenzhe Shi & Ferenc Huszár ICLR 2017 Super Resolution Inverse problem: Given low resolution representation
More informationOutline GF-RNN ReNet. Outline
Outline Gated Feedback Recurrent Neural Networks. arxiv1502. Introduction: RNN & Gated RNN Gated Feedback Recurrent Neural Networks (GF-RNN) Experiments: Character-level Language Modeling & Python Program
More informationBc. David Honzátko. Generative Neural Networks in Image Reconstruction
MASTER THESIS Bc. David Honzátko Generative Neural Networks in Image Reconstruction Department of Software and Computer Science Education Supervisor of the master thesis: Study programme: Study branch:
More informationPIXELCNN++: IMPROVING THE PIXELCNN WITH DISCRETIZED LOGISTIC MIXTURE LIKELIHOOD AND OTHER MODIFICATIONS
PIXELCNN++: IMPROVING THE PIXELCNN WITH DISCRETIZED LOGISTIC MIXTURE LIKELIHOOD AND OTHER MODIFICATIONS Tim Salimans, Andrej Karpathy, Xi Chen, Diederik P. Kingma {tim,karpathy,peter,dpkingma}@openai.com
More informationLSTM and its variants for visual recognition. Xiaodan Liang Sun Yat-sen University
LSTM and its variants for visual recognition Xiaodan Liang xdliang328@gmail.com Sun Yat-sen University Outline Context Modelling with CNN LSTM and its Variants LSTM Architecture Variants Application in
More informationVideo Pixel Networks. Abstract. 1. Introduction
Nal Kalchbrenner 1 Aäron van den Oord 1 Karen Simonyan 1 Ivo Danihelka 1 Oriol Vinyals 1 Alex Graves 1 Koray Kavukcuoglu 1 Abstract We propose a probabilistic video model, the Video Pixel Network (VPN),
More informationRNNs as Directed Graphical Models
RNNs as Directed Graphical Models Sargur Srihari srihari@buffalo.edu This is part of lecture slides on Deep Learning: http://www.cedar.buffalo.edu/~srihari/cse676 1 10. Topics in Sequence Modeling Overview
More informationAutoencoder. Representation learning (related to dictionary learning) Both the input and the output are x
Deep Learning 4 Autoencoder, Attention (spatial transformer), Multi-modal learning, Neural Turing Machine, Memory Networks, Generative Adversarial Net Jian Li IIIS, Tsinghua Autoencoder Autoencoder Unsupervised
More information27: Hybrid Graphical Models and Neural Networks
10-708: Probabilistic Graphical Models 10-708 Spring 2016 27: Hybrid Graphical Models and Neural Networks Lecturer: Matt Gormley Scribes: Jakob Bauer Otilia Stretcu Rohan Varma 1 Motivation We first look
More informationMachine Learning. Deep Learning. Eric Xing (and Pengtao Xie) , Fall Lecture 8, October 6, Eric CMU,
Machine Learning 10-701, Fall 2015 Deep Learning Eric Xing (and Pengtao Xie) Lecture 8, October 6, 2015 Eric Xing @ CMU, 2015 1 A perennial challenge in computer vision: feature engineering SIFT Spin image
More informationMachine Learning 13. week
Machine Learning 13. week Deep Learning Convolutional Neural Network Recurrent Neural Network 1 Why Deep Learning is so Popular? 1. Increase in the amount of data Thanks to the Internet, huge amount of
More informationNeural Machine Translation In Linear Time
Neural Machine Translation In Linear Time Authors: Nal Kalchbrenner, Lasse Espeholt, Karen Simonyan, Aaron van den Oord, Alex Graves, Koray Kavukcuoglu Presenter: SunMao sm4206 YuZheng yz2978 OVERVIEW
More informationDeep Generative Models Variational Autoencoders
Deep Generative Models Variational Autoencoders Sudeshna Sarkar 5 April 2017 Generative Nets Generative models that represent probability distributions over multiple variables in some way. Directed Generative
More informationBidirectional Recurrent Convolutional Networks for Video Super-Resolution
Bidirectional Recurrent Convolutional Networks for Video Super-Resolution Qi Zhang & Yan Huang Center for Research on Intelligent Perception and Computing (CRIPAC) National Laboratory of Pattern Recognition
More informationarxiv: v2 [cs.cv] 14 May 2018
ContextVP: Fully Context-Aware Video Prediction Wonmin Byeon 1234, Qin Wang 1, Rupesh Kumar Srivastava 3, and Petros Koumoutsakos 1 arxiv:1710.08518v2 [cs.cv] 14 May 2018 Abstract Video prediction models
More informationDeep generative models of natural images
Spring 2016 1 Motivation 2 3 Variational autoencoders Generative adversarial networks Generative moment matching networks Evaluating generative models 4 Outline 1 Motivation 2 3 Variational autoencoders
More informationarxiv: v3 [cs.lg] 30 Dec 2016
Video Ladder Networks Francesco Cricri Nokia Technologies francesco.cricri@nokia.com Xingyang Ni Tampere University of Technology xingyang.ni@tut.fi arxiv:1612.01756v3 [cs.lg] 30 Dec 2016 Mikko Honkala
More informationarxiv: v1 [cs.lg] 8 Dec 2016 Abstract
An Architecture for Deep, Hierarchical Generative Models Philip Bachman phil.bachman@maluuba.com Maluuba Research arxiv:1612.04739v1 [cs.lg] 8 Dec 2016 Abstract We present an architecture which lets us
More informationCapsule Networks. Eric Mintun
Capsule Networks Eric Mintun Motivation An improvement* to regular Convolutional Neural Networks. Two goals: Replace max-pooling operation with something more intuitive. Keep more info about an activated
More informationMachine Learning: Chenhao Tan University of Colorado Boulder LECTURE 15
Machine Learning: Chenhao Tan University of Colorado Boulder LECTURE 15 Slides adapted from Jordan Boyd-Graber Machine Learning: Chenhao Tan Boulder 1 of 21 Logistics HW3 available on Github, due on October
More informationDeep Image Inpainting
Deep Image Inpainting Charles Burlin cburlin@stanford.edu Yoann Le Calonnec yoann@stanford.edu Louis Duperier duperier@stanford.edu Abstract We present a new take on several image inpaiting techniques
More informationOne Network to Solve Them All Solving Linear Inverse Problems using Deep Projection Models
One Network to Solve Them All Solving Linear Inverse Problems using Deep Projection Models [Supplemental Materials] 1. Network Architecture b ref b ref +1 We now describe the architecture of the networks
More informationImage Captioning with Object Detection and Localization
Image Captioning with Object Detection and Localization Zhongliang Yang, Yu-Jin Zhang, Sadaqat ur Rehman, Yongfeng Huang, Department of Electronic Engineering, Tsinghua University, Beijing 100084, China
More informationQumran Letter Restoration by Rotation and Reflection Modified PixelCNN
Qumran Letter Restoration by Rotation and Reflection Modified PixelCNN Lior Uzan Blavatnik School of Computer Science Tel Aviv University Tel Aviv, Israel lioruzan@mail.tau.ac.il Nachum Dershowitz Blavatnik
More informationRecurrent Convolutional Neural Networks for Scene Labeling
Recurrent Convolutional Neural Networks for Scene Labeling Pedro O. Pinheiro, Ronan Collobert Reviewed by Yizhe Zhang August 14, 2015 Scene labeling task Scene labeling: assign a class label to each pixel
More informationEmpirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling Authors: Junyoung Chung, Caglar Gulcehre, KyungHyun Cho and Yoshua Bengio Presenter: Yu-Wei Lin Background: Recurrent Neural
More informationMoonRiver: Deep Neural Network in C++
MoonRiver: Deep Neural Network in C++ Chung-Yi Weng Computer Science & Engineering University of Washington chungyi@cs.washington.edu Abstract Artificial intelligence resurges with its dramatic improvement
More informationGenerative Adversarial Text to Image Synthesis
Generative Adversarial Text to Image Synthesis Scott Reed, Zeynep Akata, Xinchen Yan, Lajanugen Logeswaran, Bernt Schiele, Honglak Lee Presented by: Jingyao Zhan Contents Introduction Related Work Method
More informationDeep Learning with Tensorflow AlexNet
Machine Learning and Computer Vision Group Deep Learning with Tensorflow http://cvml.ist.ac.at/courses/dlwt_w17/ AlexNet Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton, "Imagenet classification
More informationSEMANTIC COMPUTING. Lecture 8: Introduction to Deep Learning. TU Dresden, 7 December Dagmar Gromann International Center For Computational Logic
SEMANTIC COMPUTING Lecture 8: Introduction to Deep Learning Dagmar Gromann International Center For Computational Logic TU Dresden, 7 December 2018 Overview Introduction Deep Learning General Neural Networks
More informationControllable Generative Adversarial Network
Controllable Generative Adversarial Network arxiv:1708.00598v2 [cs.lg] 12 Sep 2017 Minhyeok Lee 1 and Junhee Seok 1 1 School of Electrical Engineering, Korea University, 145 Anam-ro, Seongbuk-gu, Seoul,
More informationDeep Learning in Visual Recognition. Thanks Da Zhang for the slides
Deep Learning in Visual Recognition Thanks Da Zhang for the slides Deep Learning is Everywhere 2 Roadmap Introduction Convolutional Neural Network Application Image Classification Object Detection Object
More informationCOMP 551 Applied Machine Learning Lecture 16: Deep Learning
COMP 551 Applied Machine Learning Lecture 16: Deep Learning Instructor: Ryan Lowe (ryan.lowe@cs.mcgill.ca) Slides mostly by: Class web page: www.cs.mcgill.ca/~hvanho2/comp551 Unless otherwise noted, all
More informationImageNet Classification with Deep Convolutional Neural Networks
ImageNet Classification with Deep Convolutional Neural Networks Alex Krizhevsky Ilya Sutskever Geoffrey Hinton University of Toronto Canada Paper with same name to appear in NIPS 2012 Main idea Architecture
More informationAsynchronous Parallel Learning for Neural Networks and Structured Models with Dense Features
Asynchronous Parallel Learning for Neural Networks and Structured Models with Dense Features Xu SUN ( 孙栩 ) Peking University xusun@pku.edu.cn Motivation Neural networks -> Good Performance CNN, RNN, LSTM
More informationDeep Learning for Computer Vision II
IIIT Hyderabad Deep Learning for Computer Vision II C. V. Jawahar Paradigm Shift Feature Extraction (SIFT, HoG, ) Part Models / Encoding Classifier Sparrow Feature Learning Classifier Sparrow L 1 L 2 L
More information(University Improving of Montreal) Generative Adversarial Networks with Denoising Feature Matching / 17
Improving Generative Adversarial Networks with Denoising Feature Matching David Warde-Farley 1 Yoshua Bengio 1 1 University of Montreal, ICLR,2017 Presenter: Bargav Jayaraman Outline 1 Introduction 2 Background
More informationRecovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform. Xintao Wang Ke Yu Chao Dong Chen Change Loy
Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform Xintao Wang Ke Yu Chao Dong Chen Change Loy Problem enlarge 4 times Low-resolution image High-resolution image Previous
More informationMachine Learning. MGS Lecture 3: Deep Learning
Dr Michel F. Valstar http://cs.nott.ac.uk/~mfv/ Machine Learning MGS Lecture 3: Deep Learning Dr Michel F. Valstar http://cs.nott.ac.uk/~mfv/ WHAT IS DEEP LEARNING? Shallow network: Only one hidden layer
More informationVideo Compression Using Recurrent Convolutional Neural Networks
Video Compression Using Recurrent Convolutional Neural Networks Cedric Yue Sik Kin Electrical Engineering cedyue@stanford.edu Berk Coker Computer Science bcoker@stanford.edu Abstract The demand for video
More informationA Wavenet for Speech Denoising
A Wavenet for Speech Denoising Jordi Pons work done in collaboration with Dario Rethage and Xavier Serra Music Technology Group (Universitat Pompeu Fabra, Barcelona) Summer 2017 Presented at Pandora and
More informationDeep Learning for Visual Manipulation and Synthesis
Deep Learning for Visual Manipulation and Synthesis Jun-Yan Zhu 朱俊彦 UC Berkeley 2017/01/11 @ VALSE What is visual manipulation? Image Editing Program input photo User Input result Desired output: stay
More informationEND-TO-END CHINESE TEXT RECOGNITION
END-TO-END CHINESE TEXT RECOGNITION Jie Hu 1, Tszhang Guo 1, Ji Cao 2, Changshui Zhang 1 1 Department of Automation, Tsinghua University 2 Beijing SinoVoice Technology November 15, 2017 Presentation at
More informationDEEP LEARNING PART THREE - DEEP GENERATIVE MODELS CS/CNS/EE MACHINE LEARNING & DATA MINING - LECTURE 17
DEEP LEARNING PART THREE - DEEP GENERATIVE MODELS CS/CNS/EE 155 - MACHINE LEARNING & DATA MINING - LECTURE 17 GENERATIVE MODELS DATA 3 DATA 4 example 1 DATA 5 example 2 DATA 6 example 3 DATA 7 number of
More informationRestricted Boltzmann Machines. Shallow vs. deep networks. Stacked RBMs. Boltzmann Machine learning: Unsupervised version
Shallow vs. deep networks Restricted Boltzmann Machines Shallow: one hidden layer Features can be learned more-or-less independently Arbitrary function approximator (with enough hidden units) Deep: two
More informationBayesian model ensembling using meta-trained recurrent neural networks
Bayesian model ensembling using meta-trained recurrent neural networks Luca Ambrogioni l.ambrogioni@donders.ru.nl Umut Güçlü u.guclu@donders.ru.nl Yağmur Güçlütürk y.gucluturk@donders.ru.nl Julia Berezutskaya
More informationAn Abstract Domain for Certifying Neural Networks. Department of Computer Science
An Abstract Domain for Certifying Neural Networks Gagandeep Singh Timon Gehr Markus Püschel Martin Vechev Department of Computer Science Adversarial input perturbations Neural network f 8 I " Neural network
More informationAuxiliary Guided Autoregressive Variational Autoencoders
Auxiliary Guided Autoregressive Variational Autoencoders Thomas Lucas, Jakob Verbeek To cite this version: Thomas Lucas, Jakob Verbeek. Auxiliary Guided Autoregressive Variational Autoencoders. 2017.
More informationSingle Image Depth Estimation via Deep Learning
Single Image Depth Estimation via Deep Learning Wei Song Stanford University Stanford, CA Abstract The goal of the project is to apply direct supervised deep learning to the problem of monocular depth
More informationDEEP LEARNING OF COMPRESSED SENSING OPERATORS WITH STRUCTURAL SIMILARITY (SSIM) LOSS
DEEP LEARNING OF COMPRESSED SENSING OPERATORS WITH STRUCTURAL SIMILARITY (SSIM) LOSS ABSTRACT Compressed sensing (CS) is a signal processing framework for efficiently reconstructing a signal from a small
More informationKnow your data - many types of networks
Architectures Know your data - many types of networks Fixed length representation Variable length representation Online video sequences, or samples of different sizes Images Specific architectures for
More informationTutorial on Keras CAP ADVANCED COMPUTER VISION SPRING 2018 KISHAN S ATHREY
Tutorial on Keras CAP 6412 - ADVANCED COMPUTER VISION SPRING 2018 KISHAN S ATHREY Deep learning packages TensorFlow Google PyTorch Facebook AI research Keras Francois Chollet (now at Google) Chainer Company
More informationPSU Student Research Symposium 2017 Bayesian Optimization for Refining Object Proposals, with an Application to Pedestrian Detection Anthony D.
PSU Student Research Symposium 2017 Bayesian Optimization for Refining Object Proposals, with an Application to Pedestrian Detection Anthony D. Rhodes 5/10/17 What is Machine Learning? Machine learning
More informationDeep Learning and Its Applications
Convolutional Neural Network and Its Application in Image Recognition Oct 28, 2016 Outline 1 A Motivating Example 2 The Convolutional Neural Network (CNN) Model 3 Training the CNN Model 4 Issues and Recent
More informationCOMP9444 Neural Networks and Deep Learning 7. Image Processing. COMP9444 c Alan Blair, 2017
COMP9444 Neural Networks and Deep Learning 7. Image Processing COMP9444 17s2 Image Processing 1 Outline Image Datasets and Tasks Convolution in Detail AlexNet Weight Initialization Batch Normalization
More informationDeep Learning. Deep Learning. Practical Application Automatically Adding Sounds To Silent Movies
http://blog.csdn.net/zouxy09/article/details/8775360 Automatic Colorization of Black and White Images Automatically Adding Sounds To Silent Movies Traditionally this was done by hand with human effort
More informationModeling Sequences Conditioned on Context with RNNs
Modeling Sequences Conditioned on Context with RNNs Sargur Srihari srihari@buffalo.edu This is part of lecture slides on Deep Learning: http://www.cedar.buffalo.edu/~srihari/cse676 1 10. Topics in Sequence
More informationDeep Learning. Vladimir Golkov Technical University of Munich Computer Vision Group
Deep Learning Vladimir Golkov Technical University of Munich Computer Vision Group 1D Input, 1D Output target input 2 2D Input, 1D Output: Data Distribution Complexity Imagine many dimensions (data occupies
More informationSemantic Segmentation. Zhongang Qi
Semantic Segmentation Zhongang Qi qiz@oregonstate.edu Semantic Segmentation "Two men riding on a bike in front of a building on the road. And there is a car." Idea: recognizing, understanding what's in
More informationImplicit Mixtures of Restricted Boltzmann Machines
Implicit Mixtures of Restricted Boltzmann Machines Vinod Nair and Geoffrey Hinton Department of Computer Science, University of Toronto 10 King s College Road, Toronto, M5S 3G5 Canada {vnair,hinton}@cs.toronto.edu
More informationGenerative Adversarial Networks (GANs) Ian Goodfellow, Research Scientist MLSLP Keynote, San Francisco
Generative Adversarial Networks (GANs) Ian Goodfellow, Research Scientist MLSLP Keynote, San Francisco 2016-09-13 Generative Modeling Density estimation Sample generation Training examples Model samples
More informationNeural Networks and Deep Learning
Neural Networks and Deep Learning Example Learning Problem Example Learning Problem Celebrity Faces in the Wild Machine Learning Pipeline Raw data Feature extract. Feature computation Inference: prediction,
More informationarxiv: v2 [cs.cv] 24 Jul 2017
One-Step Time-Dependent Future Video Frame Prediction with a Convolutional Encoder-Decoder Neural Network Vedran Vukotić 1,2,3, Silvia-Laura Pintea 2, Christian Raymond 1,3, Guillaume Gravier 1,4, and
More informationDEEP LEARNING REVIEW. Yann LeCun, Yoshua Bengio & Geoffrey Hinton Nature Presented by Divya Chitimalla
DEEP LEARNING REVIEW Yann LeCun, Yoshua Bengio & Geoffrey Hinton Nature 2015 -Presented by Divya Chitimalla What is deep learning Deep learning allows computational models that are composed of multiple
More informationVariational Autoencoders. Sargur N. Srihari
Variational Autoencoders Sargur N. srihari@cedar.buffalo.edu Topics 1. Generative Model 2. Standard Autoencoder 3. Variational autoencoders (VAE) 2 Generative Model A variational autoencoder (VAE) is a
More informationConditional Random Fields as Recurrent Neural Networks
BIL722 - Deep Learning for Computer Vision Conditional Random Fields as Recurrent Neural Networks S. Zheng, S. Jayasumana, B. Romera-Paredes V. Vineet, Z. Su, D. Du, C. Huang, P.H.S. Torr Introduction
More information19: Inference and learning in Deep Learning
10-708: Probabilistic Graphical Models 10-708, Spring 2017 19: Inference and learning in Deep Learning Lecturer: Zhiting Hu Scribes: Akash Umakantha, Ryan Williamson 1 Classes of Deep Generative Models
More informationRecurrent Neural Networks and Transfer Learning for Action Recognition
Recurrent Neural Networks and Transfer Learning for Action Recognition Andrew Giel Stanford University agiel@stanford.edu Ryan Diaz Stanford University ryandiaz@stanford.edu Abstract We have taken on the
More informationCS489/698: Intro to ML
CS489/698: Intro to ML Lecture 14: Training of Deep NNs Instructor: Sun Sun 1 Outline Activation functions Regularization Gradient-based optimization 2 Examples of activation functions 3 5/28/18 Sun Sun
More informationDisguised Face Identification (DFI) with Facial KeyPoints using Spatial Fusion Convolutional Network. Nathan Sun CIS601
Disguised Face Identification (DFI) with Facial KeyPoints using Spatial Fusion Convolutional Network Nathan Sun CIS601 Introduction Face ID is complicated by alterations to an individual s appearance Beard,
More informationCS 543: Final Project Report Texture Classification using 2-D Noncausal HMMs
CS 543: Final Project Report Texture Classification using 2-D Noncausal HMMs Felix Wang fywang2 John Wieting wieting2 Introduction We implement a texture classification algorithm using 2-D Noncausal Hidden
More informationAkarsh Pokkunuru EECS Department Contractive Auto-Encoders: Explicit Invariance During Feature Extraction
Akarsh Pokkunuru EECS Department 03-16-2017 Contractive Auto-Encoders: Explicit Invariance During Feature Extraction 1 AGENDA Introduction to Auto-encoders Types of Auto-encoders Analysis of different
More informationDynamic Routing Between Capsules
Report Explainable Machine Learning Dynamic Routing Between Capsules Author: Michael Dorkenwald Supervisor: Dr. Ullrich Köthe 28. Juni 2018 Inhaltsverzeichnis 1 Introduction 2 2 Motivation 2 3 CapusleNet
More informationGraphical Models, Bayesian Method, Sampling, and Variational Inference
Graphical Models, Bayesian Method, Sampling, and Variational Inference With Application in Function MRI Analysis and Other Imaging Problems Wei Liu Scientific Computing and Imaging Institute University
More informationPhoto-realistic Renderings for Machines Seong-heum Kim
Photo-realistic Renderings for Machines 20105034 Seong-heum Kim CS580 Student Presentations 2016.04.28 Photo-realistic Renderings for Machines Scene radiances Model descriptions (Light, Shape, Material,
More informationRecurrent Neural Network (RNN) Industrial AI Lab.
Recurrent Neural Network (RNN) Industrial AI Lab. For example (Deterministic) Time Series Data Closed- form Linear difference equation (LDE) and initial condition High order LDEs 2 (Stochastic) Time Series
More informationarxiv: v2 [cs.ne] 10 Nov 2018
Number Sequence Prediction Problems for Evaluating Computational Powers of Neural Networks Hyoungwook Nam College of Liberal Studies Seoul National University Seoul, Korea hwnam8@snu.ac.kr Segwang Kim
More informationPredCNN: Predictive Learning with Cascade Convolutions
PredCNN: Predictive Learning with Cascade Convolutions Ziru Xu, Yunbo Wang, Mingsheng Long, and Jianmin Wang KLiss, MOE; School of Software, Tsinghua University, China National Engineering Laboratory for
More informationConvolutional Neural Networks
Lecturer: Barnabas Poczos Introduction to Machine Learning (Lecture Notes) Convolutional Neural Networks Disclaimer: These notes have not been subjected to the usual scrutiny reserved for formal publications.
More informationGenerative Modeling with Convolutional Neural Networks. Denis Dus Data Scientist at InData Labs
Generative Modeling with Convolutional Neural Networks Denis Dus Data Scientist at InData Labs What we will discuss 1. 2. 3. 4. Discriminative vs Generative modeling Convolutional Neural Networks How to
More information16-785: Integrated Intelligence in Robotics: Vision, Language, and Planning. Spring 2018 Lecture 14. Image to Text
16-785: Integrated Intelligence in Robotics: Vision, Language, and Planning Spring 2018 Lecture 14. Image to Text Input Output Classification tasks 4/1/18 CMU 16-785: Integrated Intelligence in Robotics
More informationDiffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting
Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting Yaguang Li Joint work with Rose Yu, Cyrus Shahabi, Yan Liu Page 1 Introduction Traffic congesting is wasteful of time,
More informationTowards Conceptual Compression
Towards Conceptual Compression Karol Gregor karolg@google.com Frederic Besse fbesse@google.com Danilo Jimenez Rezende danilor@google.com Ivo Danihelka danihelka@google.com Daan Wierstra wierstra@google.com
More informationAdaDepth: Unsupervised Content Congruent Adaptation for Depth Estimation
AdaDepth: Unsupervised Content Congruent Adaptation for Depth Estimation Introduction Supplementary material In the supplementary material, we present additional qualitative results of the proposed AdaDepth
More informationarxiv: v1 [cs.cv] 14 Dec 2016
Detect, Replace, Refine: Deep Structured Prediction For Pixel Wise Labeling arxiv:1612.04770v1 [cs.cv] 14 Dec 2016 Spyros Gidaris University Paris-Est, LIGM Ecole des Ponts ParisTech spyros.gidaris@imagine.enpc.fr
More informationDeep learning for dense per-pixel prediction. Chunhua Shen The University of Adelaide, Australia
Deep learning for dense per-pixel prediction Chunhua Shen The University of Adelaide, Australia Image understanding Classification error Convolution Neural Networks 0.3 0.2 0.1 Image Classification [Krizhevsky
More informationCS231N Section. Video Understanding 6/1/2018
CS231N Section Video Understanding 6/1/2018 Outline Background / Motivation / History Video Datasets Models Pre-deep learning CNN + RNN 3D convolution Two-stream What we ve seen in class so far... Image
More informationShow, Discriminate, and Tell: A Discriminatory Image Captioning Model with Deep Neural Networks
Show, Discriminate, and Tell: A Discriminatory Image Captioning Model with Deep Neural Networks Zelun Luo Department of Computer Science Stanford University zelunluo@stanford.edu Te-Lin Wu Department of
More informationContextVP: Fully Context-Aware Video Prediction
ContextVP: Fully Context-Aware Video Prediction Wonmin Byeon 1,2,3,4, Qin Wang 2, Rupesh Kumar Srivastava 4, and Petros Koumoutsakos 2 1 NVIDIA, Santa Clara, CA, USA wbyeon@nvidia com 2 ETH Zurich, Zurich,
More informationWhat Do We Understand About Convolutional Networks?
arxiv:1803.08834v1 [cs.cv] 23 Mar 2018 What Do We Understand About Convolutional Networks? Isma Hadji and Richard P. Wildes Department of Electrical Engineering and Computer Science York University Toronto,
More informationComputer Vision I - Filtering and Feature detection
Computer Vision I - Filtering and Feature detection Carsten Rother 30/10/2015 Computer Vision I: Basics of Image Processing Roadmap: Basics of Digital Image Processing Computer Vision I: Basics of Image
More informationarxiv: v1 [cs.cv] 17 Nov 2016
Inverting The Generator Of A Generative Adversarial Network arxiv:1611.05644v1 [cs.cv] 17 Nov 2016 Antonia Creswell BICV Group Bioengineering Imperial College London ac2211@ic.ac.uk Abstract Anil Anthony
More informationarxiv: v1 [stat.ml] 10 Dec 2018
1st Symposium on Advances in Approximate Bayesian Inference, 2018 1 7 Disentangled Dynamic Representations from Unordered Data arxiv:1812.03962v1 [stat.ml] 10 Dec 2018 Leonhard Helminger Abdelaziz Djelouah
More informationObject Recognition Using Pictorial Structures. Daniel Huttenlocher Computer Science Department. In This Talk. Object recognition in computer vision
Object Recognition Using Pictorial Structures Daniel Huttenlocher Computer Science Department Joint work with Pedro Felzenszwalb, MIT AI Lab In This Talk Object recognition in computer vision Brief definition
More informationLayerwise Interweaving Convolutional LSTM
Layerwise Interweaving Convolutional LSTM Tiehang Duan and Sargur N. Srihari Department of Computer Science and Engineering The State University of New York at Buffalo Buffalo, NY 14260, United States
More informationConvolutional Neural Networks
NPFL114, Lecture 4 Convolutional Neural Networks Milan Straka March 25, 2019 Charles University in Prague Faculty of Mathematics and Physics Institute of Formal and Applied Linguistics unless otherwise
More informationLSTM: An Image Classification Model Based on Fashion-MNIST Dataset
LSTM: An Image Classification Model Based on Fashion-MNIST Dataset Kexin Zhang, Research School of Computer Science, Australian National University Kexin Zhang, U6342657@anu.edu.au Abstract. The application
More informationNeural style transfer
1/32 Neural style transfer Victor Kitov v.v.kitov@yandex.ru 2/32 Neural style transfer Input: content image, style image. Style transfer - application of artistic style from style image to content image.
More informationCSC 578 Neural Networks and Deep Learning
CSC 578 Neural Networks and Deep Learning Fall 2018/19 7. Recurrent Neural Networks (Some figures adapted from NNDL book) 1 Recurrent Neural Networks 1. Recurrent Neural Networks (RNNs) 2. RNN Training
More informationPixelCNN Models with Auxiliary Variables for Natural Image Modeling
Alexander Kolesnikov 1 Christoph H. Lampert 1 Abstract We study probabilistic models of natural images and extend the autoregressive family of PixelCNN architectures by incorporating auxiliary variables.
More information