Bidirectional Recurrent Convolutional Networks for Video Super-Resolution
|
|
- Rosamund Neal
- 5 years ago
- Views:
Transcription
1 Bidirectional Recurrent Convolutional Networks for Video Super-Resolution Qi Zhang & Yan Huang Center for Research on Intelligent Perception and Computing (CRIPAC) National Laboratory of Pattern Recognition (NLPR) Institute of Automation, Chinese Academy of Sciences (CASIA) May 10, 2017
2 2 CRIPAC CRIPAC mainly focuses on the following research topics related to national public security. Biometrics Image and Video Analysis Big Data and Multi-modal Computing Content Security and Authentication Sensing and Information Acquisition CRAPIC receives regular fundings from various Government departments or agencies. It is also supported by funds of R&D projects from many other national and international sources. CRIPAC members publish widely in leading national and international journals and conferences such as IEEE Transactions on PAMI, IEEE Transactions on Image Processing, International Journal of Computer Vision, Pattern Recognition, Pattern Recognition Letters, ICCV, ECCV, CVPR, ACCV, ICPR, ICIP, etc.
3 3 NVAIL Artificial Intelligence Laboratory Researches on artificial intelligence and deep learning
4 4 Outline 1 Deep Learning 2 Recurrent Convolutional Networks 3 Application to Video Super-Resolution 4 Future Work
5 5 Outline 1 Deep Learning 2 Recurrent Convolutional Networks 3 Application to Video Super-Resolution 4 Future Work
6 6 Deep Neural Networks (DNN) Originate from: simple/complex cell, Hubel and Wiesel efficient error backpropagation, Linnainmaa deep neocognitron, convolution, Fukushima autoencoder, Ballard backpropagation for CNN, Lecun fundamental deep learning problem, Hochreiter deep recurrent neural network, Schmidhuber supervised LSTM RNN, Schmidhuber Two drawbacks: Large numbers of parameters High computational cost Small training set Over-fitting problem
7 Two Recent Developments Big Data Cheap Computation Video surveillance data size (PB) DNN can thus be fitted efficiently 7
8 Deep Learning The Resurgence of DNN Breakthrough in 2006 ImageNet: 74% vs. 85% RNN for sequence analysis Activity recognition, CVPR2015 Video caption, CVPR2015 Deep Learning promotes the fast development areas2014 of various visual computing Representation learning CNN for visual tasks DeepFace, CVPR2014 RCNN for detection, CVPR2014 8
9 9 Outline 1 Deep Learning 2 Recurrent Convolutional Networks 3 Application to Video Super-Resolution 4 Future Work
10 10 Deep Neural Networks (DNN) y x R d, h R n, W R d n h = σ xw, σ t = 1 1+e t h W x Sigmoid function σ t
11 11 Recurrent Neural Networks (RNN) y Temporal dependency modeling y h h 1 U h 2 U h 3 W W W W x x 1 x 2 x 3 DNN RNN x t R d, h t R n, W R d n, U R n n h t = σ x t W + h t 1 U
12 12 Recurrent Convolutional Networks (RCN) DNN: Deep Neural Networks RNN: Recurrent Neural Networks CNN: Convolutional Neural Networks DNN CNN Convolutional Sequential Sequential RNN RCN Convolutional
13 13 Applications of RCN Video SR, NIPS15 & TPAMI17 Scene Labeling, NIPS15 Weather Nowcasting, NIPS15 Action Recognition, ICLR15 Object Recognition, CVPR15 Person ReID, CVPR16
14 14 Outline 1 Deep Learning 2 Recurrent Convolutional Networks 3 Application to Video Super-Resolution 4 Future Work
15 Video Super-Resolution Display High-resolution devices High-resolution videos Display Super-resolution: denoising, deblurring, upscaling Low-resolution videos A great need for super resolving low-resolution videos 15
16 Two Main Approaches (1/2) 1. Single-Image super-resolution [1-6] One-to-One scheme, super resolve each video frame independently Ignore the intrinsic temporal dependency relation of video frames Low computational complexity, fast [1] Dong et al., Learning a deep convolutional network for image super resolution. ECCV, [2] Timofte et al., Anchored neighborhood regression for fast example-based super resolution. ICCV, [3] Zeyde et al., On single image scale-up using sparse-representations. Curves and Surfaces, [4] Yang et al., Image super-resolution via sparse representation. IEEE TIP, [5] Bevilacqua et al., Low-complexity single-image super resolution. BMVC, [6] Chang et al., Super-resolution through neighbor embedding. CVPR,
17 Two Main Approaches (2/2) 2. Multi-Frame super-resolution [7-11] Many-to-One scheme, use multiple adjacent frames to super resolve a frame Model the temporal dependency relation by motion estimation High computational complexity, slow [7] Liu and Sun, On bayesian adaptive video super resolution. IEEE PAMI, [8] Takeda et al., Super-resolution without explicit subpixel motion estimation. IEEE TIP, [9] Mitzel et al., Video super resolution using duality based tv-l 1 optical flow. PR, [10] Protter et al. Generalizing the nonlocal-means to super-resolution reconstruction. IEEE TIP, [11] Fransens et al., Optical flow based super-resolution: A probabilistic approach. CVIU,
18 Motivation RNN: Recurrent Neural Networks SR: Super-Resolution RNN can model long-term contextual information of temporal sequences well Convolutional operation can scale to full videos of any spatial size and temporal step Propose bidirectional recurrent convolutional networks, different from vanilla RNN: 1. Commonly-used full connections are replaced with weight -sharing convolutions 2. Conditional convolutions are added for learning visual-temporal dependency relation 18
19 19 Bidirectional Recurrent Convolutional Networks learn spatial dependency between a low-resolution frame and its highresolution result model long-term temporal dependency relation across video frames enhance visual-temporal dependency relation modeling
20 Learning Define an end-to-end mapping O from low-resolution frames X to high-resolution frames Y Learning proceeds by optimizing the Mean Square Error (MSE) between predicted frames O(X) and Y stochastic gradient descent L = O X Y 2 small learning rate in the output layer: 1e-4 20
21 Experiments Train the model on 25 YUV format video sequences volume-based training number of volumes: roughly 41,000 volume size: Test on a variety of real world videos severe motion blur motion aliasing complex motions Training videos Testing videos 21
22 PSNR Comparison PSNR: peak signal-to-noise ratio Table1: The results of PSNR (db) and test time (sec) on the test video sequences. Surpass state-of-the-art methods in PSNR, due to the effective [1] Video enhancer. version [4] Bevilacqua et al., Low-complexity single-image super resolution. BMVC, [5] Chang et al., Super-resolution through neighbor embedding. CVPR, [6] Dong temporal et al., Learning a dependency deep convolutional network modelling for image super resolution. ECCV, [20] Takeda et al., Super-resolution without explicit subpixel motion estimation. IEEE TIP, [22] Timofte et al., Anchored neighborhood regression for fast example-based super resolution. ICCV, [24] Yang et al., Image super-resolution via sparse representation. IEEE TIP, [25] Zeyde et al., On single image scale-up using sparse-representations. Curves and Surfaces,
23 23 Model Architecture Investigate the impact of our model architecture on the performance Take a simplified network containing only feedfoward (v) convolution as a benchmark Study its variants by successively adding the bidirectional (b), recurrent (r)and conditional (t) schemes Table1: The results of PSNR (db) by variants of BRCN on the testing video sequences.
24 24 Running Time Figure: Speed vs. PSNR for all the comparison methods. Outperform both single-image and multi-frame SR methods Achieve comparable speed with the fastest single-image SR methods
25 25 Closeup Comparison Our method is able to recover more image details than others, Figure: Comparison among original frames (2th, 3th and 4th frames, from the top row to the bottom) of the Dancing video and super resolved results by Bicubic, 3DSKR, ANR and BRCN, under respectively. severe motion conditions
26 26 Example Upscaling factor: Comparison: Bicubic (top) Ours (bottom)
27 Conclusion Bidirectional Recurrent Convolutional Networks bidirectional recurrent and conditional convolutions an end-to-end framework, without pre/post-processing well performance and fast speed For more details, please refer to the following papers: 1. Yan Huang, Wei Wang, and Liang Wang, Bidirectional Recurrent Convolutional Networks for Multi-Frame Super-Resolution. Advances in Neural Information Processing Systems (NIPS), pp , Yan Huang, Wei Wang, and Liang Wang, Video Super-Resolution via Bidirectional Recurrent Convolutional Networks, IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2017, Accepted 27
28 28 Outline 1 Deep Learning 2 Recurrent Convolutional Networks 3 Application to Video Super-Resolution 4 Future Work
29 Future Work For performance improvement extend our model to have a deeper architecture, e.g., based on 19 layers VGG net incorporate some effective strategies, e.g., motion ensemble and residual connection For speed acceleration replace the used pre-upsampling by learning diverse upsampling filters with deconvolution layers Others collect a large-scale high-resolution video dataset, and try to learn our model directly from raw videos 29
30 30 Acknowledgement NVAIL Artificial Intelligence Laboratory Sponsor excellent hardware resources
31 THANK YOU
Bidirectional Recurrent Convolutional Networks for Multi-Frame Super-Resolution
Bidirectional Recurrent Convolutional Networks for Multi-Frame Super-Resolution Yan Huang 1 Wei Wang 1 Liang Wang 1,2 1 Center for Research on Intelligent Perception and Computing National Laboratory of
More informationLSTM and its variants for visual recognition. Xiaodan Liang Sun Yat-sen University
LSTM and its variants for visual recognition Xiaodan Liang xdliang328@gmail.com Sun Yat-sen University Outline Context Modelling with CNN LSTM and its Variants LSTM Architecture Variants Application in
More informationarxiv: v2 [cs.cv] 11 Nov 2016
Accurate Image Super-Resolution Using Very Deep Convolutional Networks Jiwon Kim, Jung Kwon Lee and Kyoung Mu Lee Department of ECE, ASRI, Seoul National University, Korea {j.kim, deruci, kyoungmu}@snu.ac.kr
More informationA Novel Multi-Frame Color Images Super-Resolution Framework based on Deep Convolutional Neural Network. Zhe Li, Shu Li, Jianmin Wang and Hongyang Wang
5th International Conference on Measurement, Instrumentation and Automation (ICMIA 2016) A Novel Multi-Frame Color Images Super-Resolution Framewor based on Deep Convolutional Neural Networ Zhe Li, Shu
More informationMachine Learning 13. week
Machine Learning 13. week Deep Learning Convolutional Neural Network Recurrent Neural Network 1 Why Deep Learning is so Popular? 1. Increase in the amount of data Thanks to the Internet, huge amount of
More informationIMAGE SUPER-RESOLUTION BASED ON DICTIONARY LEARNING AND ANCHORED NEIGHBORHOOD REGRESSION WITH MUTUAL INCOHERENCE
IMAGE SUPER-RESOLUTION BASED ON DICTIONARY LEARNING AND ANCHORED NEIGHBORHOOD REGRESSION WITH MUTUAL INCOHERENCE Yulun Zhang 1, Kaiyu Gu 2, Yongbing Zhang 1, Jian Zhang 3, and Qionghai Dai 1,4 1 Shenzhen
More informationIntroduction. Prior work BYNET: IMAGE SUPER RESOLUTION WITH A BYPASS CONNECTION NETWORK. Bjo rn Stenger. Rakuten Institute of Technology
BYNET: IMAGE SUPER RESOLUTION WITH A BYPASS CONNECTION NETWORK Jiu Xu Yeongnam Chae Bjo rn Stenger Rakuten Institute of Technology ABSTRACT This paper proposes a deep residual network, ByNet, for the single
More informationOne Network to Solve Them All Solving Linear Inverse Problems using Deep Projection Models
One Network to Solve Them All Solving Linear Inverse Problems using Deep Projection Models [Supplemental Materials] 1. Network Architecture b ref b ref +1 We now describe the architecture of the networks
More informationRecovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform. Xintao Wang Ke Yu Chao Dong Chen Change Loy
Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform Xintao Wang Ke Yu Chao Dong Chen Change Loy Problem enlarge 4 times Low-resolution image High-resolution image Previous
More informationConvolutional Neural Networks. Computer Vision Jia-Bin Huang, Virginia Tech
Convolutional Neural Networks Computer Vision Jia-Bin Huang, Virginia Tech Today s class Overview Convolutional Neural Network (CNN) Training CNN Understanding and Visualizing CNN Image Categorization:
More informationarxiv: v1 [cs.cv] 3 Jan 2017
Learning a Mixture of Deep Networks for Single Image Super-Resolution Ding Liu, Zhaowen Wang, Nasser Nasrabadi, and Thomas Huang arxiv:1701.00823v1 [cs.cv] 3 Jan 2017 Beckman Institute, University of Illinois
More informationarxiv: v1 [cs.cv] 8 Feb 2018
DEEP IMAGE SUPER RESOLUTION VIA NATURAL IMAGE PRIORS Hojjat S. Mousavi, Tiantong Guo, Vishal Monga Dept. of Electrical Engineering, The Pennsylvania State University arxiv:802.0272v [cs.cv] 8 Feb 208 ABSTRACT
More informationProceedings of the International MultiConference of Engineers and Computer Scientists 2018 Vol I IMECS 2018, March 14-16, 2018, Hong Kong
, March 14-16, 2018, Hong Kong , March 14-16, 2018, Hong Kong , March 14-16, 2018, Hong Kong , March 14-16, 2018, Hong Kong TABLE I CLASSIFICATION ACCURACY OF DIFFERENT PRE-TRAINED MODELS ON THE TEST DATA
More informationEfficient Module Based Single Image Super Resolution for Multiple Problems
Efficient Module Based Single Image Super Resolution for Multiple Problems Dongwon Park Kwanyoung Kim Se Young Chun School of ECE, Ulsan National Institute of Science and Technology, 44919, Ulsan, South
More informationOPTICAL Character Recognition systems aim at converting
ICDAR 2015 COMPETITION ON TEXT IMAGE SUPER-RESOLUTION 1 Boosting Optical Character Recognition: A Super-Resolution Approach Chao Dong, Ximei Zhu, Yubin Deng, Chen Change Loy, Member, IEEE, and Yu Qiao
More informationSingle Image Super-Resolution via Iterative Collaborative Representation
Single Image Super-Resolution via Iterative Collaborative Representation Yulun Zhang 1(B), Yongbing Zhang 1, Jian Zhang 2, aoqian Wang 1, and Qionghai Dai 1,3 1 Graduate School at Shenzhen, Tsinghua University,
More informationDCGANs for image super-resolution, denoising and debluring
DCGANs for image super-resolution, denoising and debluring Qiaojing Yan Stanford University Electrical Engineering qiaojing@stanford.edu Wei Wang Stanford University Electrical Engineering wwang23@stanford.edu
More informationImage Super-Resolution Using Dense Skip Connections
Image Super-Resolution Using Dense Skip Connections Tong Tong, Gen Li, Xiejie Liu, Qinquan Gao Imperial Vision Technology Fuzhou, China {ttraveltong,ligen,liu.xiejie,gqinquan}@imperial-vision.com Abstract
More informationarxiv: v1 [cs.cv] 14 Jul 2017
Temporal Modeling Approaches for Large-scale Youtube-8M Video Understanding Fu Li, Chuang Gan, Xiao Liu, Yunlong Bian, Xiang Long, Yandong Li, Zhichao Li, Jie Zhou, Shilei Wen Baidu IDL & Tsinghua University
More informationMultilayer and Multimodal Fusion of Deep Neural Networks for Video Classification
Multilayer and Multimodal Fusion of Deep Neural Networks for Video Classification Xiaodong Yang, Pavlo Molchanov, Jan Kautz INTELLIGENT VIDEO ANALYTICS Surveillance event detection Human-computer interaction
More informationarxiv: v4 [cs.cv] 25 Mar 2018
Frame-Recurrent Video Super-Resolution Mehdi S. M. Sajjadi 1,2 msajjadi@tue.mpg.de Raviteja Vemulapalli 2 ravitejavemu@google.com 1 Max Planck Institute for Intelligent Systems 2 Google Matthew Brown 2
More informationarxiv: v1 [cs.cv] 6 Nov 2015
Seven ways to improve example-based single image super resolution Radu Timofte Computer Vision Lab D-ITET, ETH Zurich timofter@vision.ee.ethz.ch Rasmus Rothe Computer Vision Lab D-ITET, ETH Zurich rrothe@vision.ee.ethz.ch
More informationDENSE BYNET: RESIDUAL DENSE NETWORK FOR IMAGE SUPER RESOLUTION. Bjo rn Stenger2
DENSE BYNET: RESIDUAL DENSE NETWORK FOR IMAGE SUPER RESOLUTION Jiu Xu1 Yeongnam Chae2 1 Bjo rn Stenger2 Ankur Datta1 Rakuten Institute of Technology, Boston Rakuten Institute of Technology, Tokyo 2 ABSTRACT
More informationCOMP 551 Applied Machine Learning Lecture 16: Deep Learning
COMP 551 Applied Machine Learning Lecture 16: Deep Learning Instructor: Ryan Lowe (ryan.lowe@cs.mcgill.ca) Slides mostly by: Class web page: www.cs.mcgill.ca/~hvanho2/comp551 Unless otherwise noted, all
More informationCNN for Low Level Image Processing. Huanjing Yue
CNN for Low Level Image Processing Huanjing Yue 2017.11 1 Deep Learning for Image Restoration General formulation: min Θ L( x, x) s. t. x = F(y; Θ) Loss function Parameters to be learned Key issues The
More informationAn Attention-Based Approach for Single Image Super Resolution
An Attention-Based Approach for Single Image Super Resolution Yuan Liu 1,2,3, Yuancheng Wang 1, Nan Li 1,Xu Cheng 4, Yifeng Zhang 1,2,3,, Yongming Huang 1, Guojun Lu 5 1 School of Information Science and
More informationUDNet: Up-Down Network for Compact and Efficient Feature Representation in Image Super-Resolution
UDNet: Up-Down Network for Compact and Efficient Feature Representation in Image Super-Resolution Chang Chen Xinmei Tian Zhiwei Xiong Feng Wu University of Science and Technology of China Abstract Recently,
More informationSeven ways to improve example-based single image super resolution
Seven ways to improve example-based single image super resolution Radu Timofte CVL, D-ITET, ETH Zurich radu.timofte@vision.ee.ethz.ch Rasmus Rothe CVL, D-ITET, ETH Zurich rrothe@vision.ee.ethz.ch Luc Van
More informationMultimodal Gesture Recognition using Multi-stream Recurrent Neural Network
Multimodal Gesture Recognition using Multi-stream Recurrent Neural Network Noriki Nishida and Hideki Nakayama Machine Perception Group Graduate School of Information Science and Technology The University
More informationDeep Networks for Image Super-Resolution with Sparse Prior
Deep Networks for Image Super-Resolution with Sparse Prior Zhaowen Wang Ding Liu Jianchao Yang Wei Han Thomas Huang Beckman Institute, University of Illinois at Urbana-Champaign, Urbana, IL Adobe Research,
More informationYiqi Yan. May 10, 2017
Yiqi Yan May 10, 2017 P a r t I F u n d a m e n t a l B a c k g r o u n d s Convolution Single Filter Multiple Filters 3 Convolution: case study, 2 filters 4 Convolution: receptive field receptive field
More informationLearning a Deep Convolutional Network for Image Super-Resolution
Learning a Deep Convolutional Network for Image Super-Resolution Chao Dong 1, Chen Change Loy 1, Kaiming He 2, and Xiaoou Tang 1 1 Department of Information Engineering, The Chinese University of Hong
More informationPixel-level Generative Model
Pixel-level Generative Model Generative Image Modeling Using Spatial LSTMs (2015NIPS) L. Theis and M. Bethge University of Tübingen, Germany Pixel Recurrent Neural Networks (2016ICML) A. van den Oord,
More informationFast and Accurate Image Super-Resolution Using A Combined Loss
Fast and Accurate Image Super-Resolution Using A Combined Loss Jinchang Xu 1, Yu Zhao 1, Yuan Dong 1, Hongliang Bai 2 1 Beijing University of Posts and Telecommunications, 2 Beijing Faceall Technology
More informationarxiv: v1 [cs.cv] 7 May 2018
arxiv:1805.02335v1 [cs.cv] 7 May 2018 Skeleton-Based Action Recognition with Spatial Reasoning and Temporal Stack Learning Chenyang Si 1,3, Ya Jing 1,3, Wei Wang 1,3,, Liang Wang 1,2,3, and Tieniu Tan
More informationRecurrent Convolutional Neural Networks for Scene Labeling
Recurrent Convolutional Neural Networks for Scene Labeling Pedro O. Pinheiro, Ronan Collobert Reviewed by Yizhe Zhang August 14, 2015 Scene labeling task Scene labeling: assign a class label to each pixel
More informationarxiv: v2 [cs.cv] 14 May 2018
ContextVP: Fully Context-Aware Video Prediction Wonmin Byeon 1234, Qin Wang 1, Rupesh Kumar Srivastava 3, and Petros Koumoutsakos 1 arxiv:1710.08518v2 [cs.cv] 14 May 2018 Abstract Video prediction models
More informationImage Upscaling and Fuzzy ARTMAP Neural Network
IOSR Journal of Computer Engineering (IOSR-JCE) e-issn: 2278-0661,p-ISSN: 2278-8727, Volume 17, Issue 4, Ver. II (July Aug. 2015), PP 79-85 www.iosrjournals.org Image Upscaling and Fuzzy ARTMAP Neural
More informationStructured Prediction using Convolutional Neural Networks
Overview Structured Prediction using Convolutional Neural Networks Bohyung Han bhhan@postech.ac.kr Computer Vision Lab. Convolutional Neural Networks (CNNs) Structured predictions for low level computer
More informationMulti-Input Cardiac Image Super-Resolution using Convolutional Neural Networks
Multi-Input Cardiac Image Super-Resolution using Convolutional Neural Networks Ozan Oktay, Wenjia Bai, Matthew Lee, Ricardo Guerrero, Konstantinos Kamnitsas, Jose Caballero, Antonio de Marvao, Stuart Cook,
More informationLearning Visual Semantics: Models, Massive Computation, and Innovative Applications
Learning Visual Semantics: Models, Massive Computation, and Innovative Applications Part II: Visual Features and Representations Liangliang Cao, IBM Watson Research Center Evolvement of Visual Features
More informationSpatial Localization and Detection. Lecture 8-1
Lecture 8: Spatial Localization and Detection Lecture 8-1 Administrative - Project Proposals were due on Saturday Homework 2 due Friday 2/5 Homework 1 grades out this week Midterm will be in-class on Wednesday
More informationA Novel Image Super-resolution Reconstruction Algorithm based on Modified Sparse Representation
, pp.162-167 http://dx.doi.org/10.14257/astl.2016.138.33 A Novel Image Super-resolution Reconstruction Algorithm based on Modified Sparse Representation Liqiang Hu, Chaofeng He Shijiazhuang Tiedao University,
More informationStudy of Residual Networks for Image Recognition
Study of Residual Networks for Image Recognition Mohammad Sadegh Ebrahimi Stanford University sadegh@stanford.edu Hossein Karkeh Abadi Stanford University hosseink@stanford.edu Abstract Deep neural networks
More informationDeep Learning. Deep Learning. Practical Application Automatically Adding Sounds To Silent Movies
http://blog.csdn.net/zouxy09/article/details/8775360 Automatic Colorization of Black and White Images Automatically Adding Sounds To Silent Movies Traditionally this was done by hand with human effort
More informationDeep learning for object detection. Slides from Svetlana Lazebnik and many others
Deep learning for object detection Slides from Svetlana Lazebnik and many others Recent developments in object detection 80% PASCAL VOC mean0average0precision0(map) 70% 60% 50% 40% 30% 20% 10% Before deep
More informationA DEEP DICTIONARY MODEL FOR IMAGE SUPER-RESOLUTION. Jun-Jie Huang and Pier Luigi Dragotti
A DEEP DICTIONARY MODEL FOR IMAGE SUPER-RESOLUTION Jun-Jie Huang and Pier Luigi Dragotti Communications and Signal Processing Group CSP), Imperial College London, UK ABSTRACT Inspired by the recent success
More informationAsynchronous Parallel Learning for Neural Networks and Structured Models with Dense Features
Asynchronous Parallel Learning for Neural Networks and Structured Models with Dense Features Xu SUN ( 孙栩 ) Peking University xusun@pku.edu.cn Motivation Neural networks -> Good Performance CNN, RNN, LSTM
More informationRTSR: Enhancing Real-time H.264 Video Streaming using Deep Learning based Video Super Resolution Spring 2017 CS570 Project Presentation June 8, 2017
RTSR: Enhancing Real-time H.264 Video Streaming using Deep Learning based Video Super Resolution Spring 2017 CS570 Project Presentation June 8, 2017 Team 16 Soomin Kim Leslie Tiong Youngki Kwon Insu Jang
More informationObject detection with CNNs
Object detection with CNNs 80% PASCAL VOC mean0average0precision0(map) 70% 60% 50% 40% 30% 20% 10% Before CNNs After CNNs 0% 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 year Region proposals
More informationBoosting face recognition via neural Super-Resolution
Boosting face recognition via neural Super-Resolution Guillaume Berger, Cle ment Peyrard and Moez Baccouche Orange Labs - 4 rue du Clos Courtel, 35510 Cesson-Se vigne - France Abstract. We propose a two-step
More informationSingle Image Super Resolution of Textures via CNNs. Andrew Palmer
Single Image Super Resolution of Textures via CNNs Andrew Palmer What is Super Resolution (SR)? Simple: Obtain one or more high-resolution images from one or more low-resolution ones Many, many applications
More informationProgressive Neural Architecture Search
Progressive Neural Architecture Search Chenxi Liu, Barret Zoph, Maxim Neumann, Jonathon Shlens, Wei Hua, Li-Jia Li, Li Fei-Fei, Alan Yuille, Jonathan Huang, Kevin Murphy 09/10/2018 @ECCV 1 Outline Introduction
More informationDeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution and Fully Connected CRFs
DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution and Fully Connected CRFs Zhipeng Yan, Moyuan Huang, Hao Jiang 5/1/2017 1 Outline Background semantic segmentation Objective,
More informationFlow-Based Video Recognition
Flow-Based Video Recognition Jifeng Dai Visual Computing Group, Microsoft Research Asia Joint work with Xizhou Zhu*, Yuwen Xiong*, Yujie Wang*, Lu Yuan and Yichen Wei (* interns) Talk pipeline Introduction
More informationVideo Compression Using Recurrent Convolutional Neural Networks
Video Compression Using Recurrent Convolutional Neural Networks Cedric Yue Sik Kin Electrical Engineering cedyue@stanford.edu Berk Coker Computer Science bcoker@stanford.edu Abstract The demand for video
More informationArtificial Neural Networks. Introduction to Computational Neuroscience Ardi Tampuu
Artificial Neural Networks Introduction to Computational Neuroscience Ardi Tampuu 7.0.206 Artificial neural network NB! Inspired by biology, not based on biology! Applications Automatic speech recognition
More informationEncoder-Decoder Networks for Semantic Segmentation. Sachin Mehta
Encoder-Decoder Networks for Semantic Segmentation Sachin Mehta Outline > Overview of Semantic Segmentation > Encoder-Decoder Networks > Results What is Semantic Segmentation? Input: RGB Image Output:
More informationHallucinating Very Low-Resolution Unaligned and Noisy Face Images by Transformative Discriminative Autoencoders
Hallucinating Very Low-Resolution Unaligned and Noisy Face Images by Transformative Discriminative Autoencoders Xin Yu, Fatih Porikli Australian National University {xin.yu, fatih.porikli}@anu.edu.au Abstract
More informationComputer Vision Lecture 16
Computer Vision Lecture 16 Deep Learning for Object Categorization 14.01.2016 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Announcements Seminar registration period
More informationRecovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform Supplementary Material
Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform Supplementary Material Xintao Wang 1 Ke Yu 1 Chao Dong 2 Chen Change Loy 1 1 CUHK - SenseTime Joint Lab, The Chinese
More informationText Recognition in Videos using a Recurrent Connectionist Approach
Author manuscript, published in "ICANN - 22th International Conference on Artificial Neural Networks, Lausanne : Switzerland (2012)" DOI : 10.1007/978-3-642-33266-1_22 Text Recognition in Videos using
More informationDeep Back-Projection Networks For Super-Resolution Supplementary Material
Deep Back-Projection Networks For Super-Resolution Supplementary Material Muhammad Haris 1, Greg Shakhnarovich 2, and Norimichi Ukita 1, 1 Toyota Technological Institute, Japan 2 Toyota Technological Institute
More informationDeep Learning in Visual Recognition. Thanks Da Zhang for the slides
Deep Learning in Visual Recognition Thanks Da Zhang for the slides Deep Learning is Everywhere 2 Roadmap Introduction Convolutional Neural Network Application Image Classification Object Detection Object
More informationFAST: A Framework to Accelerate Super-Resolution Processing on Compressed Videos
FAST: A Framework to Accelerate Super-Resolution Processing on Compressed Videos Zhengdong Zhang, Vivienne Sze Massachusetts Institute of Technology {zhangzd, sze}@mit.edu Abstract State-of-the-art super-resolution
More informationDeep learning for dense per-pixel prediction. Chunhua Shen The University of Adelaide, Australia
Deep learning for dense per-pixel prediction Chunhua Shen The University of Adelaide, Australia Image understanding Classification error Convolution Neural Networks 0.3 0.2 0.1 Image Classification [Krizhevsky
More informationDigital Image Restoration
Digital Image Restoration Blur as a chance and not a nuisance Filip Šroubek sroubekf@utia.cas.cz www.utia.cas.cz Institute of Information Theory and Automation Academy of Sciences of the Czech Republic
More informationMachine Learning. Deep Learning. Eric Xing (and Pengtao Xie) , Fall Lecture 8, October 6, Eric CMU,
Machine Learning 10-701, Fall 2015 Deep Learning Eric Xing (and Pengtao Xie) Lecture 8, October 6, 2015 Eric Xing @ CMU, 2015 1 A perennial challenge in computer vision: feature engineering SIFT Spin image
More informationCode Mania Artificial Intelligence: a. Module - 1: Introduction to Artificial intelligence and Python:
Code Mania 2019 Artificial Intelligence: a. Module - 1: Introduction to Artificial intelligence and Python: 1. Introduction to Artificial Intelligence 2. Introduction to python programming and Environment
More informationBilevel Sparse Coding
Adobe Research 345 Park Ave, San Jose, CA Mar 15, 2013 Outline 1 2 The learning model The learning algorithm 3 4 Sparse Modeling Many types of sensory data, e.g., images and audio, are in high-dimensional
More informationTutorial on Keras CAP ADVANCED COMPUTER VISION SPRING 2018 KISHAN S ATHREY
Tutorial on Keras CAP 6412 - ADVANCED COMPUTER VISION SPRING 2018 KISHAN S ATHREY Deep learning packages TensorFlow Google PyTorch Facebook AI research Keras Francois Chollet (now at Google) Chainer Company
More informationarxiv: v1 [cs.cv] 18 Dec 2018 Abstract
SREdgeNet: Edge Enhanced Single Image Super Resolution using Dense Edge Detection Network and Feature Merge Network Kwanyoung Kim, Se Young Chun Ulsan National Institute of Science and Technology (UNIST),
More informationDeep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution
Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution Wei-Sheng Lai 1 Jia-Bin Huang 2 Narendra Ahuja 3 Ming-Hsuan Yang 1 1 University of California, Merced 2 Virginia Tech 3 University
More informationDeep Neural Networks:
Deep Neural Networks: Part II Convolutional Neural Network (CNN) Yuan-Kai Wang, 2016 Web site of this course: http://pattern-recognition.weebly.com source: CNN for ImageClassification, by S. Lazebnik,
More informationDiffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting
Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting Yaguang Li Joint work with Rose Yu, Cyrus Shahabi, Yan Liu Page 1 Introduction Traffic congesting is wasteful of time,
More informationSingle Image Super Resolution - When Model Adaptation Matters
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 1 Single Image Super Resolution - When Model Adaptation Matters arxiv:1703.10889v1 [cs.cv] 31 Mar 2017 Yudong Liang, Radu Timofte, Member, IEEE,
More informationAnchored Neighborhood Regression for Fast Example-Based Super-Resolution
Anchored Neighborhood Regression for Fast Example-Based Super-Resolution Radu Timofte 1,2, Vincent De Smet 1, and Luc Van Gool 1,2 1 KU Leuven, ESAT-PSI / iminds, VISICS 2 ETH Zurich, D-ITET, Computer
More informationArbitrary Style Transfer in Real-Time with Adaptive Instance Normalization. Presented by: Karen Lucknavalai and Alexandr Kuznetsov
Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization Presented by: Karen Lucknavalai and Alexandr Kuznetsov Example Style Content Result Motivation Transforming content of an image
More informationPrediction of Pedestrian Trajectories Final Report
Prediction of Pedestrian Trajectories Final Report Mingchen Li (limc), Yiyang Li (yiyang7), Gendong Zhang (zgdsh29) December 15, 2017 1 Introduction As the industry of automotive vehicles growing rapidly,
More informationRECURRENT NEURAL NETWORKS
RECURRENT NEURAL NETWORKS Methods Traditional Deep-Learning based Non-machine Learning Machine-Learning based method Supervised SVM MLP CNN RNN (LSTM) Localizati on GPS, SLAM Self Driving Perception Pedestrian
More informationRestricted Boltzmann Machines. Shallow vs. deep networks. Stacked RBMs. Boltzmann Machine learning: Unsupervised version
Shallow vs. deep networks Restricted Boltzmann Machines Shallow: one hidden layer Features can be learned more-or-less independently Arbitrary function approximator (with enough hidden units) Deep: two
More informationComputer Vision Lecture 16
Announcements Computer Vision Lecture 16 Deep Learning Applications 11.01.2017 Seminar registration period starts on Friday We will offer a lab course in the summer semester Deep Robot Learning Topic:
More informationFast and Accurate Single Image Super-Resolution via Information Distillation Network
Fast and Accurate Single Image Super-Resolution via Information Distillation Network Recently, due to the strength of deep convolutional neural network (CNN), many CNN-based SR methods try to train a deep
More informationDeep Video Super-Resolution Network Using Dynamic Upsampling Filters Without Explicit Motion Compensation
Deep Video Super-Resolution Network Using Dynamic Upsampling Filters Without Explicit Motion Compensation Younghyun Jo Seoung Wug Oh Jaeyeon Kang Seon Joo Kim Yonsei University Abstract Video super-resolution
More informationLearning to Match. Jun Xu, Zhengdong Lu, Tianqi Chen, Hang Li
Learning to Match Jun Xu, Zhengdong Lu, Tianqi Chen, Hang Li 1. Introduction The main tasks in many applications can be formalized as matching between heterogeneous objects, including search, recommendation,
More informationSingle Image Super-resolution using Deformable Patches
Single Image Super-resolution using Deformable Patches Yu Zhu 1, Yanning Zhang 1, Alan L. Yuille 2 1 School of Computer Science, Northwestern Polytechnical University, China 2 Department of Statistics,
More informationA FRAMEWORK OF EXTRACTING MULTI-SCALE FEATURES USING MULTIPLE CONVOLUTIONAL NEURAL NETWORKS. Kuan-Chuan Peng and Tsuhan Chen
A FRAMEWORK OF EXTRACTING MULTI-SCALE FEATURES USING MULTIPLE CONVOLUTIONAL NEURAL NETWORKS Kuan-Chuan Peng and Tsuhan Chen School of Electrical and Computer Engineering, Cornell University, Ithaca, NY
More informationComputer Vision Lecture 16
Computer Vision Lecture 16 Deep Learning Applications 11.01.2017 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Announcements Seminar registration period starts
More informationFeature-Fused SSD: Fast Detection for Small Objects
Feature-Fused SSD: Fast Detection for Small Objects Guimei Cao, Xuemei Xie, Wenzhe Yang, Quan Liao, Guangming Shi, Jinjian Wu School of Electronic Engineering, Xidian University, China xmxie@mail.xidian.edu.cn
More informationarxiv: v1 [cs.cv] 30 Nov 2018
Super-Resolution based on Image-Adapted CNN Denoisers: Incorporating Generalization of Training Data and Internal Learning in Test Time arxiv:1811.12866v1 [cs.cv] 30 Nov 2018 Tom Tirer Tel Aviv University,
More informationHuman Detection and Tracking for Video Surveillance: A Cognitive Science Approach
Human Detection and Tracking for Video Surveillance: A Cognitive Science Approach Vandit Gajjar gajjar.vandit.381@ldce.ac.in Ayesha Gurnani gurnani.ayesha.52@ldce.ac.in Yash Khandhediya khandhediya.yash.364@ldce.ac.in
More informationarxiv: v1 [cs.cv] 31 Dec 2018 Abstract
Image Super-Resolution via RL-CSC: When Residual Learning Meets olutional Sparse Coding Menglei Zhang, Zhou Liu, Lei Yu School of Electronic and Information, Wuhan University, China {zmlhome, liuzhou,
More informationROBUST INTERNAL EXEMPLAR-BASED IMAGE ENHANCEMENT. Yang Xian 1 and Yingli Tian 1,2
ROBUST INTERNAL EXEMPLAR-BASED IMAGE ENHANCEMENT Yang Xian 1 and Yingli Tian 1,2 1 The Graduate Center, 2 The City College, The City University of New York, New York, Email: yxian@gc.cuny.edu; ytian@ccny.cuny.edu
More informationEnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis Supplementary
EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis Supplementary Mehdi S. M. Sajjadi Bernhard Schölkopf Michael Hirsch Max Planck Institute for Intelligent Systems Spemanstr.
More informationAn Effective Single-Image Super-Resolution Model Using Squeeze-and-Excitation Networks
An Effective Single-Image Super-Resolution Model Using Squeeze-and-Excitation Networks Kangfu Mei 1, Juncheng Li 2, Luyao 1, Mingwen Wang 1, Aiwen Jiang 1 Jiangxi Normal University 1 East China Normal
More informationFAST: A Framework to Accelerate Super- Resolution Processing on Compressed Videos
FAST: A Framework to Accelerate Super- Resolution Processing on Compressed Videos Zhengdong Zhang, Vivienne Sze Massachusetts Institute of Technology http://www.mit.edu/~sze/fast.html 1 Super-Resolution
More informationSUPPLEMENTARY MATERIAL
SUPPLEMENTARY MATERIAL Zhiyuan Zha 1,3, Xin Liu 2, Ziheng Zhou 2, Xiaohua Huang 2, Jingang Shi 2, Zhenhong Shang 3, Lan Tang 1, Yechao Bai 1, Qiong Wang 1, Xinggan Zhang 1 1 School of Electronic Science
More informationDeepFace: Closing the Gap to Human-Level Performance in Face Verification
DeepFace: Closing the Gap to Human-Level Performance in Face Verification Report on the paper Artem Komarichev February 7, 2016 Outline New alignment technique New DNN architecture New large dataset with
More informationObject Detection Lecture Introduction to deep learning (CNN) Idar Dyrdal
Object Detection Lecture 10.3 - Introduction to deep learning (CNN) Idar Dyrdal Deep Learning Labels Computational models composed of multiple processing layers (non-linear transformations) Used to learn
More informationHuman Pose Estimation with Deep Learning. Wei Yang
Human Pose Estimation with Deep Learning Wei Yang Applications Understand Activities Family Robots American Heist (2014) - The Bank Robbery Scene 2 What do we need to know to recognize a crime scene? 3
More information