Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform. Xintao Wang Ke Yu Chao Dong Chen Change Loy
|
|
- Christina Perry
- 5 years ago
- Views:
Transcription
1 Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform Xintao Wang Ke Yu Chao Dong Chen Change Loy
2 Problem enlarge 4 times Low-resolution image High-resolution image
3 Previous work Contemporary SR algorithms are mostly CNN-based methods [1]. Most of CNN-based methods use pixel-wise loss function. (MSE-based model) good at recovering edges and smooth areas not good at texture recovery Adversarial loss is introduced in SRGAN [2] and EnhanceNet [3]. (GAN-based model) encourage the network to favor solutions that look more like natural images visual quality of reconstruction is significantly improved SRCNN SRGAN Ground-truth [1] C. Dong, C. C. Loy, K. He, and X. Tang. Learning a deep convolutional network for image super-resolution. In ECCV, [2] C. Ledig, L. Theis, F. Husz ar, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, et al. Photo-realistic single image super-resolution using a generative adversarial network. In CVPR, [3] M. S. Sajjadi, B. Sch olkopf, and M. Hirsch. EnhanceNet: Single image super-resolution through automated texture synthesis. In ICCV, 2017.
4 Motivation building x4 plant x4 building plant swap s
5 Semantic categorical building water animal sky grass plant mountain
6 Issues 1. How to represent the semantic categorical? Our approach: explore semantic segmentation probability maps as the categorical up to pixel level. 2. How categorical can be incorporated into the reconstruction process effectively? Our approach: propose a novel Spatial Feature Transform that is capable of altering the network behavior conditioned on other information.
7 Represent categorical Contemporary CNN segmentation network [1] fine-tuned on LR images K categories ResNet 101 argmax probability maps semantic categorical [1] Z. Liu, X. Li, P. Luo, C.-C. Loy, and X. Tang. Semantic image segmentation via deep parsing network. In ICCV, 2015.
8 Examples on segmentation Input LR images Ground-truth Segments on HR images Segments on LR images sky grass building mountain plant water animal background
9 Incorporate conditions Categorical Ψ = (P 1, P 2,, P K ) Ψ probability maps P 1, P 2,, P K? y = G θ (x Ψ) CNN for SR y = G θ (x) input LR image x net G parametrized by θ restored image y
10 Spatial Feature Transform By learning a mapping function M, the Ψ is modeled by a pair of affine transformation parameters (γ, β). M: Ψ (γ, β) The modulation is then carried out by an affine transformation on feature maps F. SFT F γ, β = γ F+ β y = G θ (x Ψ) M: Ψ (γ, β) SFT F γ, β = γ F+ β y = G θ (x γ, β)
11 Conv Conv Conv Conv Conv Conv Conv Conv Conv SFT layer Conv SFT layer Conv Residual block Residual block SFT layer Conv Upsampling Conv Conv Conv Spatial Feature Transform Residual block SFT layer Segmentation probability maps Condition Network conditions Shared SFT conditions features + γ i β i
12 loss function Generator Adversarial loss [1] encourage the network to generate images that reside on the manifold of natural images min θ max η Ε y~phr logd η y + Ε x~plr log(1 D η G θ (x) ) Compete Discriminator Perceptual loss [2] use a pre-trained 19-layer VGG network (features before conv54) optimize a super-resolution model in a feature space φ VGG y φ VGG y 2 2 [1] Goodfellow, Ian, et al. Generative adversarial nets. In NIPS [2] J. Johnson, A. Alahi, and L. Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. In ECCV, 2016.
13 Spatial condition The modulation parameters (γ, β) have a close relationship with probability maps P and contain spatial information. Input P building map P grass map γ map of C 6 β map of C 7 LR patch Restored
14 Delicate modulation LR patch P plant map γ map of C 51 β map of C 1 Restored P grass map γ map of C 14 β map of C 5
15 Results SRCNN SRGAN EnhanceNet SFT-Net (ours) GT PSNR: 24.83dB PSNR: 23.36dB PSNR: 22.71dB PSNR: 22.90dB
16 Results Bicubic SRCNN VDSR LapSRN DRRN MemNet EnhanceNet SRGAN SFT-Net (ours) GT MSE-based method GAN-based method
17 User study part I Ours EnhanceNet Ours SRGAN sky building grass animal plant water mountain
18 User study part II GT Ours MemNet SRCNN % 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Rank-1 Rank-2 Rank-3 Rank-4
19 animal plant water mountain grass sky building Impact of different s bicubic building sky grass mountain water plant animal building sky grass mountain water plant animal building
20 animal plant water mountain grass sky building Impact of different s bicubic building sky grass mountain water plant animal mountain bicubic building sky grass mountain water plant animal
21 Other conditioning methods Input concatenation Compositional mapping [1] FiLM [2] [1] S. Zhu, S. Fidler, R. Urtasun, D. Lin, and C. C. Loy. Be your own prada: Fashion synthesis with structural coherence. In ICCV, [2] E. Perez, F. Strub, H. de Vries, V. Dumoulin, and A. Courville. FiLM: Visual reasoning with a general conditioning layer. arxiv preprint arxiv: , 2017.
22 Comparison with other conditioning methods SFT-Net (ours) Input concatenation Compositional mapping FiLM
23 Robustness to out-of-category SRGAN Ours SRGAN Ours
24 Conclusion Explore semantic segmentation maps as categorical for realistic texture recovery. Propose a novel Spatial Feature Transform layer to efficiently incorporate the categorical conditions into a CNN-based SR network. Extensive comparisons and a user study demonstrate the capability of SFT-Net in generating realistic and visually pleasing textures.
25 Crafting a Toolchain for Image Restoration by Deep Reinforcement Learning Ke Yu Chao Dong Liang Lin Chen Change Loy
26 Image Restoration There are many individual tasks Denoising Deblurring JPEG Deblocking Super-Resolution Towards more complicated distortions Address multiple levels of degradation in one task Address multiple individual tasks [3] [1, 2]
27 Image Restoration A New Setting Consider multiple distortions simultaneously Real-world: Image capture and storage Synthetic: Gaussian blur, Gaussian noise and JPEG compression Real-world Scenario Gaussian Blur Gaussian Noise JPEG Compression Synthetic Setting Our New Task
28 Motivation Can we use a single CNN to address multiple distortions? Inefficient: Require a huge network to handle all the possibilities Inflexible: All kinds of distorted images are processed with the same structure Find a more efficient and flexible approach! Process different distortion in a different way
29 Method Decision Making Progressively restore the image quality Treat image restoration as a decision making process Artifacts! Blurry! Noisy! Try a Good enough :) deblocking deblurring denoising tool
30 Method Overview Our framework requires a toolbox and an agent Agent Toolbox Agent Toolbox
31 Method Toolbox We design 12 tools, each of which addresses a simple task 3-layer CNN [4] 8-layer CNN
32 Method Agent Use reinforcement learning to address tool selection current distorted image action at last step 12 tools stopping State Action Reward: PSNR gain at each step Structure : I 1 Input Image v 1 S 1 Feature Extractor One-hot Encoder Agent LSTM v 1
33 Method Joint Training Challenge of Middle State Intermediate results after several steps of processing None of the tools has seen these intermediate results Joint Training forward backward toolchain 1 forward toolchain 2 backward MSE loss MSE loss
34 Experimental Results Dataset: DIV2K [5] Comparison with generic models for image restoration VDSR [1] DnCNN [3]
35 Experimental Results Quantitative results on DIV2K Competitive performance Better generality Runtime Analyses More efficient
36 Experimental Results Qualitative results on DIV2K Mild (unseen) Moderate Severe (unseen) Input 1 st step 2 nd step 3 rd step VDSR-s VDSR [1]
37 Experimental Results Qualitative results on real-world images Input 1 st step 2 nd step 3 rd step VDSR [1]
38 Experimental Results Ablation Study Joint training Stopping action
39 Conclusion Contributions Address image restoration in a reinforcement learning framework Propose joint learning to cope with middle processing state Dynamically formed toolchain performs competitively against human-designed networks with less computational complexity Future work Incorporate more tools (trained with GAN loss) Handle spatial-variant distortions
40 Thanks! Q & A
41 Reference [1] J. Kim, J. Kwon Lee, and K. Mu Lee. Accurate image super-resolution using very deep convolutional networks. In CVPR, [2] Y. Tai, J. Yang, X. Liu, and C. Xu. Memnet: A persistent memory network for image restoration. In ICCV, [3] K. Zhang,W. Zuo, Y. Chen, D. Meng, and L. Zhang. Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. TIP, [4] C. Dong, C. C. Loy, K. He, and X. Tang. Image super-resolution using deep convolutional networks. TPAMI, 38(2): , [5] E. Agustsson and R. Timofte. Ntire 2017 challenge on single image super-resolution: Dataset and study. In CVPR Workshop, 2017.
Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform Supplementary Material
Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform Supplementary Material Xintao Wang 1 Ke Yu 1 Chao Dong 2 Chen Change Loy 1 1 CUHK - SenseTime Joint Lab, The Chinese
More informationRecovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform
Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform Xintao Wang 1 Ke Yu 1 Chao Dong 2 Chen Change Loy 1 1 CUHK - SenseTime Joint Lab, The Chinese University of Hong
More informationCNN for Low Level Image Processing. Huanjing Yue
CNN for Low Level Image Processing Huanjing Yue 2017.11 1 Deep Learning for Image Restoration General formulation: min Θ L( x, x) s. t. x = F(y; Θ) Loss function Parameters to be learned Key issues The
More informationEfficient Module Based Single Image Super Resolution for Multiple Problems
Efficient Module Based Single Image Super Resolution for Multiple Problems Dongwon Park Kwanyoung Kim Se Young Chun School of ECE, Ulsan National Institute of Science and Technology, 44919, Ulsan, South
More informationDCGANs for image super-resolution, denoising and debluring
DCGANs for image super-resolution, denoising and debluring Qiaojing Yan Stanford University Electrical Engineering qiaojing@stanford.edu Wei Wang Stanford University Electrical Engineering wwang23@stanford.edu
More informationImage Super-Resolution Using Dense Skip Connections
Image Super-Resolution Using Dense Skip Connections Tong Tong, Gen Li, Xiejie Liu, Qinquan Gao Imperial Vision Technology Fuzhou, China {ttraveltong,ligen,liu.xiejie,gqinquan}@imperial-vision.com Abstract
More informationDeep Back-Projection Networks For Super-Resolution Supplementary Material
Deep Back-Projection Networks For Super-Resolution Supplementary Material Muhammad Haris 1, Greg Shakhnarovich 2, and Norimichi Ukita 1, 1 Toyota Technological Institute, Japan 2 Toyota Technological Institute
More informationFast and Accurate Image Super-Resolution Using A Combined Loss
Fast and Accurate Image Super-Resolution Using A Combined Loss Jinchang Xu 1, Yu Zhao 1, Yuan Dong 1, Hongliang Bai 2 1 Beijing University of Posts and Telecommunications, 2 Beijing Faceall Technology
More informationarxiv: v1 [cs.cv] 10 Apr 2018
arxiv:1804.03360v1 [cs.cv] 10 Apr 2018 Reference-Conditioned Super-Resolution by Neural Texture Transfer Zhifei Zhang 1, Zhaowen Wang 2, Zhe Lin 2, and Hairong Qi 1 1 Department of Electrical Engineering
More informationarxiv: v1 [cs.cv] 18 Dec 2018 Abstract
SREdgeNet: Edge Enhanced Single Image Super Resolution using Dense Edge Detection Network and Feature Merge Network Kwanyoung Kim, Se Young Chun Ulsan National Institute of Science and Technology (UNIST),
More informationEnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis Supplementary
EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis Supplementary Mehdi S. M. Sajjadi Bernhard Schölkopf Michael Hirsch Max Planck Institute for Intelligent Systems Spemanstr.
More informationSupplementary Material: Unsupervised Domain Adaptation for Face Recognition in Unlabeled Videos
Supplementary Material: Unsupervised Domain Adaptation for Face Recognition in Unlabeled Videos Kihyuk Sohn 1 Sifei Liu 2 Guangyu Zhong 3 Xiang Yu 1 Ming-Hsuan Yang 2 Manmohan Chandraker 1,4 1 NEC Labs
More informationDensely Connected High Order Residual Network for Single Frame Image Super Resolution
Densely Connected High Order Residual Network for Single Frame Image Super Resolution Yiwen Huang Department of Computer Science Wenhua College, Wuhan, China nickgray0@gmail.com Ming Qin Department of
More informationarxiv: v1 [cs.cv] 5 Jul 2017
AlignGAN: Learning to Align Cross- Images with Conditional Generative Adversarial Networks Xudong Mao Department of Computer Science City University of Hong Kong xudonmao@gmail.com Qing Li Department of
More informationBalanced Two-Stage Residual Networks for Image Super-Resolution
Balanced Two-Stage Residual Networks for Image Super-Resolution Yuchen Fan *, Honghui Shi, Jiahui Yu, Ding Liu, Wei Han, Haichao Yu, Zhangyang Wang, Xinchao Wang, and Thomas S. Huang Beckman Institute,
More informationarxiv: v2 [cs.cv] 4 Nov 2018
arxiv:1811.00344v2 [cs.cv] 4 Nov 2018 Analyzing Perception-Distortion Tradeoff using Enhanced Perceptual Super-resolution Network Subeesh Vasu 1, Nimisha Thekke Madam 2, and Rajagopalan A.N. 3 Indian Institute
More informationLearning a Single Convolutional Super-Resolution Network for Multiple Degradations
Learning a Single Convolutional Super-Resolution Network for Multiple Degradations Kai Zhang,2, Wangmeng Zuo, Lei Zhang 2 School of Computer Science and Technology, Harbin Institute of Technology, Harbin,
More informationarxiv: v2 [cs.cv] 17 Sep 2018
arxiv:1809.00219v2 [cs.cv] 17 Sep 2018 ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks Xintao Wang 1, Ke Yu 1, Shixiang Wu 2, Jinjin Gu 3, Yihao Liu 4, Chao Dong 2, Chen Change Loy 5,
More informationarxiv: v2 [cs.cv] 16 Apr 2018
arxiv:1804.04829v2 [cs.cv] 16 Apr 2018 Learning Warped Guidance for Blind Face Restoration Xiaoming Li 1, Ming Liu 1, Yuting Ye 1, Wangmeng Zuo 1( ), Liang Lin 2, and Ruigang Yang 3 1 School of Computer
More informationarxiv: v1 [cs.cv] 13 Sep 2018
arxiv:1809.04789v1 [cs.cv] 13 Sep 2018 Deep Learning-based Image Super-Resolution Considering Quantitative and Perceptual Quality Jun-Ho Choi, Jun-Hyuk Kim, Manri Cheon, and Jong-Seok Lee School of Integrated
More informationSingle Image Super Resolution of Textures via CNNs. Andrew Palmer
Single Image Super Resolution of Textures via CNNs Andrew Palmer What is Super Resolution (SR)? Simple: Obtain one or more high-resolution images from one or more low-resolution ones Many, many applications
More informationProgress on Generative Adversarial Networks
Progress on Generative Adversarial Networks Wangmeng Zuo Vision Perception and Cognition Centre Harbin Institute of Technology Content Image generation: problem formulation Three issues about GAN Discriminate
More informationarxiv: v2 [cs.cv] 19 Apr 2019
arxiv:1809.04789v2 [cs.cv] 19 Apr 2019 Deep Learning-based Image Super-Resolution Considering Quantitative and Perceptual Quality Jun-Ho Choi, Jun-Hyuk Kim, Manri Cheon, and Jong-Seok Lee School of Integrated
More informationarxiv: v1 [cs.cv] 25 Dec 2017
Deep Blind Image Inpainting Yang Liu 1, Jinshan Pan 2, Zhixun Su 1 1 School of Mathematical Sciences, Dalian University of Technology 2 School of Computer Science and Engineering, Nanjing University of
More informationDENSE BYNET: RESIDUAL DENSE NETWORK FOR IMAGE SUPER RESOLUTION. Bjo rn Stenger2
DENSE BYNET: RESIDUAL DENSE NETWORK FOR IMAGE SUPER RESOLUTION Jiu Xu1 Yeongnam Chae2 1 Bjo rn Stenger2 Ankur Datta1 Rakuten Institute of Technology, Boston Rakuten Institute of Technology, Tokyo 2 ABSTRACT
More informationarxiv: v1 [cs.cv] 20 Jul 2018
arxiv:1807.07930v1 [cs.cv] 20 Jul 2018 Photorealistic Video Super Resolution Eduardo Pérez-Pellitero 1, Mehdi S. M. Sajjadi 1, Michael Hirsch 2, and Bernhard Schölkopf 12 1 Max Planck Institute for Intelligent
More informationIntroduction. Prior work BYNET: IMAGE SUPER RESOLUTION WITH A BYPASS CONNECTION NETWORK. Bjo rn Stenger. Rakuten Institute of Technology
BYNET: IMAGE SUPER RESOLUTION WITH A BYPASS CONNECTION NETWORK Jiu Xu Yeongnam Chae Bjo rn Stenger Rakuten Institute of Technology ABSTRACT This paper proposes a deep residual network, ByNet, for the single
More informationarxiv: v2 [cs.cv] 24 May 2018
Learning a Single Convolutional Super-Resolution Network for Multiple Degradations Kai Zhang 1,2,3, Wangmeng Zuo 1, Lei Zhang 2 1 School of Computer Science and Technology, Harbin Institute of Technology,
More informationImage Restoration: From Sparse and Low-rank Priors to Deep Priors
Image Restoration: From Sparse and Low-rank Priors to Deep Priors Lei Zhang 1, Wangmeng Zuo 2 1 Dept. of computing, The Hong Kong Polytechnic University, 2 School of Computer Science and Technology, Harbin
More informationFAST: A Framework to Accelerate Super- Resolution Processing on Compressed Videos
FAST: A Framework to Accelerate Super- Resolution Processing on Compressed Videos Zhengdong Zhang, Vivienne Sze Massachusetts Institute of Technology http://www.mit.edu/~sze/fast.html 1 Super-Resolution
More informationSINGLE image super-resolution (SR) aims to reconstruct
Fast and Accurate Image Super-Resolution with Deep Laplacian Pyramid Networks Wei-Sheng Lai, Jia-Bin Huang, Narendra Ahuja, and Ming-Hsuan Yang 1 arxiv:1710.01992v3 [cs.cv] 9 Aug 2018 Abstract Convolutional
More informationFSRNet: End-to-End Learning Face Super-Resolution with Facial Priors
FSRNet: End-to-End Learning Face Super-Resolution with Facial Priors Yu Chen 1,5 Ying Tai 2 Xiaoming Liu 3 Chunhua Shen 4 Jian Yang 1 1 Nanjing University of Science and Technology 2 Youtu Lab, Tencent
More informationPhysics-Based Generative Adversarial Models for Image Restoration and Beyond
1 Physics-Based Generative Adversarial Models for Image Restoration and Beyond Jinshan Pan, Yang Liu, Jiangxin Dong, Jiawei Zhang, Jimmy Ren, Jinhui Tang, Yu-Wing Tai and Ming-Hsuan Yang arxiv:1808.00605v1
More informationIEGAN: Multi-purpose Perceptual Quality Image Enhancement Using Generative Adversarial Network
IEGAN: Multi-purpose Perceptual Quality Image Enhancement Using Generative Adversarial Network Soumya Shubhra Ghosh 1, Yang Hua 1, Sankha Subhra Mukherjee 2, Neil Robertson 1,2 1 EEECS/ECIT, Queen s University
More informationSuper-Resolution on Image and Video
Super-Resolution on Image and Video Jason Liu Stanford University liujas00@stanford.edu Max Spero Stanford University maxspero@stanford.edu Allan Raventos Stanford University aravento@stanford.edu Abstract
More informationSINGLE image super-resolution (SR) aims to reconstruct
Fast and Accurate Image Super-Resolution with Deep Laplacian Pyramid Networks Wei-Sheng Lai, Jia-Bin Huang, Narendra Ahuja, and Ming-Hsuan Yang 1 arxiv:1710.01992v2 [cs.cv] 11 Oct 2017 Abstract Convolutional
More informationFast and Accurate Single Image Super-Resolution via Information Distillation Network
Fast and Accurate Single Image Super-Resolution via Information Distillation Network Recently, due to the strength of deep convolutional neural network (CNN), many CNN-based SR methods try to train a deep
More informationAn Attention-Based Approach for Single Image Super Resolution
An Attention-Based Approach for Single Image Super Resolution Yuan Liu 1,2,3, Yuancheng Wang 1, Nan Li 1,Xu Cheng 4, Yifeng Zhang 1,2,3,, Yongming Huang 1, Guojun Lu 5 1 School of Information Science and
More informationarxiv: v1 [cs.lg] 21 Dec 2018
Multimodal Sensor Fusion In Single Thermal image Super-Resolution Feras Almasri 1 and Olivier Debeir 2 arxiv:1812.09276v1 [cs.lg] 21 Dec 2018 Dept.LISA - Laboratory of Image Synthesis and Analysis, Universit
More informationarxiv:submit/ [cs.cv] 27 Mar 2018
Ours SRGAN Input Super-FAN: Integrated facial landmark localization and super-resolution of real-world low resolution faces in arbitrary poses with GANs arxiv:submit/2208568 [cs.cv] 27 Mar 2018 Adrian
More informationDeep Video Super-Resolution Network Using Dynamic Upsampling Filters Without Explicit Motion Compensation
Deep Video Super-Resolution Network Using Dynamic Upsampling Filters Without Explicit Motion Compensation Younghyun Jo Seoung Wug Oh Jaeyeon Kang Seon Joo Kim Yonsei University Abstract Video super-resolution
More informationFSRNet: End-to-End Learning Face Super-Resolution with Facial Priors
: End-to-End Learning Face Super-Resolution with Facial Priors Yu Chen 1 Ying Tai 2 Xiaoming Liu 3 Chunhua Shen 4 Jian Yang 1 1 Nanjing University of Science and Technology 2 Youtu Lab, Tencent 3 Michigan
More informationAccelerated very deep denoising convolutional neural network for image super-resolution NTIRE2017 factsheet
Accelerated very deep denoising convolutional neural network for image super-resolution NTIRE2017 factsheet Yunjin Chen, Kai Zhang and Wangmeng Zuo April 17, 2017 1 Team details Team name HIT-ULSee Team
More informationDeep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution
Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution Wei-Sheng Lai 1 Jia-Bin Huang 2 Narendra Ahuja 3 Ming-Hsuan Yang 1 1 University of California, Merced 2 Virginia Tech 3 University
More informationDOMAIN-ADAPTIVE GENERATIVE ADVERSARIAL NETWORKS FOR SKETCH-TO-PHOTO INVERSION
DOMAIN-ADAPTIVE GENERATIVE ADVERSARIAL NETWORKS FOR SKETCH-TO-PHOTO INVERSION Yen-Cheng Liu 1, Wei-Chen Chiu 2, Sheng-De Wang 1, and Yu-Chiang Frank Wang 1 1 Graduate Institute of Electrical Engineering,
More informationExample-Based Image Super-Resolution Techniques
Example-Based Image Super-Resolution Techniques Mark Sabini msabini & Gili Rusak gili December 17, 2016 1 Introduction With the current surge in popularity of imagebased applications, improving content
More informationarxiv: v1 [cs.cv] 31 Dec 2018 Abstract
Image Super-Resolution via RL-CSC: When Residual Learning Meets olutional Sparse Coding Menglei Zhang, Zhou Liu, Lei Yu School of Electronic and Information, Wuhan University, China {zmlhome, liuzhou,
More informationBidirectional Recurrent Convolutional Networks for Video Super-Resolution
Bidirectional Recurrent Convolutional Networks for Video Super-Resolution Qi Zhang & Yan Huang Center for Research on Intelligent Perception and Computing (CRIPAC) National Laboratory of Pattern Recognition
More informationIRGUN : Improved Residue based Gradual Up-Scaling Network for Single Image Super Resolution
IRGUN : Improved Residue based Gradual Up-Scaling Network for Single Image Super Resolution Manoj Sharma, Rudrabha Mukhopadhyay, Avinash Upadhyay, Sriharsha Koundinya, Ankit Shukla, Santanu Chaudhury.
More informationA Novel Multi-Frame Color Images Super-Resolution Framework based on Deep Convolutional Neural Network. Zhe Li, Shu Li, Jianmin Wang and Hongyang Wang
5th International Conference on Measurement, Instrumentation and Automation (ICMIA 2016) A Novel Multi-Frame Color Images Super-Resolution Framewor based on Deep Convolutional Neural Networ Zhe Li, Shu
More informationImage Super Resolution Based on Fusing Multiple Convolution Neural Networks
Image Super Resolution Based on Fusing Multiple Convolution Neural Networks Haoyu Ren, Mostafa El-Khamy, Jungwon Lee SAMSUNG SEMICONDUCTOR INC. 4921 Directors Place, San Diego, CA, US {haoyu.ren, mostafa.e,
More informationarxiv: v1 [cs.cv] 15 Sep 2016
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network arxiv:1609.04802v1 [cs.cv] 15 Sep 2016 Christian Ledig 1, Lucas Theis 1, Ferenc Huszár 1, Jose Caballero 1, Andrew Aitken
More informationTask-Aware Image Downscaling
Task-Aware Image Downscaling Heewon Kim, Myungsub Choi, Bee Lim, and Kyoung Mu Lee Department of ECE, ASRI, Seoul National University, Seoul, Korea {ghimhw,cms6539,biya999,kyoungmu}@snu.ac.kr https://cv.snu.ac.kr
More informationOPTICAL Character Recognition systems aim at converting
ICDAR 2015 COMPETITION ON TEXT IMAGE SUPER-RESOLUTION 1 Boosting Optical Character Recognition: A Super-Resolution Approach Chao Dong, Ximei Zhu, Yubin Deng, Chen Change Loy, Member, IEEE, and Yu Qiao
More informationarxiv: v2 [cs.cv] 19 Sep 2016
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network arxiv:1609.04802v2 [cs.cv] 19 Sep 2016 Christian Ledig, Lucas Theis, Ferenc Huszár, Jose Caballero, Andrew Aitken, Alykhan
More informationDOMAIN-ADAPTIVE GENERATIVE ADVERSARIAL NETWORKS FOR SKETCH-TO-PHOTO INVERSION
2017 IEEE INTERNATIONAL WORKSHOP ON MACHINE LEARNING FOR SIGNAL PROCESSING, SEPT. 25 28, 2017, TOKYO, JAPAN DOMAIN-ADAPTIVE GENERATIVE ADVERSARIAL NETWORKS FOR SKETCH-TO-PHOTO INVERSION Yen-Cheng Liu 1,
More informationRTSR: Enhancing Real-time H.264 Video Streaming using Deep Learning based Video Super Resolution Spring 2017 CS570 Project Presentation June 8, 2017
RTSR: Enhancing Real-time H.264 Video Streaming using Deep Learning based Video Super Resolution Spring 2017 CS570 Project Presentation June 8, 2017 Team 16 Soomin Kim Leslie Tiong Youngki Kwon Insu Jang
More informationFeature Super-Resolution: Make Machine See More Clearly
Feature Super-Resolution: Make Machine See More Clearly Weimin Tan, Bo Yan, Bahetiyaer Bare School of Computer Science, Shanghai Key Laboratory of Intelligent Information Processing, Fudan University {wmtan14,
More informationControllable Generative Adversarial Network
Controllable Generative Adversarial Network arxiv:1708.00598v2 [cs.lg] 12 Sep 2017 Minhyeok Lee 1 and Junhee Seok 1 1 School of Electrical Engineering, Korea University, 145 Anam-ro, Seongbuk-gu, Seoul,
More informationResidual Dense Network for Image Super-Resolution
Residual Dense Network for Image Super-Resolution Yulun Zhang 1, Yapeng Tian 2,YuKong 1, Bineng Zhong 1, Yun Fu 1,3 1 Department of Electrical and Computer Engineering, Northeastern University, Boston,
More informationarxiv: v1 [cs.cv] 30 Nov 2018
Super-Resolution based on Image-Adapted CNN Denoisers: Incorporating Generalization of Training Data and Internal Learning in Test Time arxiv:1811.12866v1 [cs.cv] 30 Nov 2018 Tom Tirer Tel Aviv University,
More informationAn Effective Single-Image Super-Resolution Model Using Squeeze-and-Excitation Networks
An Effective Single-Image Super-Resolution Model Using Squeeze-and-Excitation Networks Kangfu Mei 1, Juncheng Li 2, Luyao 1, Mingwen Wang 1, Aiwen Jiang 1 Jiangxi Normal University 1 East China Normal
More informationarxiv: v1 [cs.cv] 7 Mar 2018
Deep Back-Projection Networks For Super-Resolution Muhammad Haris 1, Greg Shakhnarovich, and Norimichi Ukita 1, 1 Toyota Technological Institute, Japan Toyota Technological Institute at Chicago, United
More informationLearning to Super-Resolve Blurry Face and Text Images
Learning to Super-Resolve Blurry Face and Text Images Xiangyu Xu,2,3 Deqing Sun 3,4 Jinshan Pan 5 Yujin Zhang Hanspeter Pfister 3 Ming-Hsuan Yang 2 Tsinghua University 2 University of California, Merced
More informationAttribute Augmented Convolutional Neural Network for Face Hallucination
Attribute Augmented Convolutional Neural Network for Face Hallucination Cheng-Han Lee 1 Kaipeng Zhang 1 Hu-Cheng Lee 1 Chia-Wen Cheng 2 Winston Hsu 1 1 National Taiwan University 2 The University of Texas
More informationCT Image Denoising with Perceptive Deep Neural Networks
June 2017, Xi'an CT Image Denoising with Perceptive Deep Neural Networks Qingsong Yang, and Ge Wang Department of Biomedical Engineering Rensselaer Polytechnic Institute Troy, NY, USA Email: wangg6@rpi.edu
More informationSINGLE image restoration (IR) aims to generate a visually
Concat 1x1 JOURNAL OF L A T E X CLASS FILES, VOL. 13, NO. 9, SEPTEMBER 2014 1 Residual Dense Network for Image Restoration Yulun Zhang, Yapeng Tian, Yu Kong, Bineng Zhong, and Yun Fu, Fellow, IEEE arxiv:1812.10477v1
More informationarxiv: v1 [cs.cv] 16 Dec 2018
Efficient Super Resolution Using Binarized Neural Network arxiv:1812.06378v1 [cs.cv] 16 Dec 2018 Yinglan Ma Adobe Inc. Research yingma@adobe.com Abstract Deep convolutional neural networks (DCNNs) have
More informationFast and Accurate Single Image Super-Resolution via Information Distillation Network
Fast and Accurate Single Image Super-Resolution via Information Distillation Network Zheng Hui, Xiumei Wang, Xinbo Gao School of Electronic Engineering, Xidian University Xi an, China zheng hui@aliyun.com,
More informationCARN: Convolutional Anchored Regression Network for Fast and Accurate Single Image Super-Resolution
CARN: Convolutional Anchored Regression Network for Fast and Accurate Single Image Super-Resolution Yawei Li 1, Eirikur Agustsson 1, Shuhang Gu 1, Radu Timofte 1, and Luc Van Gool 1,2 1 ETH Zürich, Sternwartstrasse
More informationImage Restoration with Deep Generative Models
Image Restoration with Deep Generative Models Raymond A. Yeh *, Teck-Yian Lim *, Chen Chen, Alexander G. Schwing, Mark Hasegawa-Johnson, Minh N. Do Department of Electrical and Computer Engineering, University
More informationVideo Frame Interpolation Using Recurrent Convolutional Layers
2018 IEEE Fourth International Conference on Multimedia Big Data (BigMM) Video Frame Interpolation Using Recurrent Convolutional Layers Zhifeng Zhang 1, Li Song 1,2, Rong Xie 2, Li Chen 1 1 Institute of
More informationarxiv: v3 [cs.cv] 6 Dec 2017
Deep Generative Adversarial Compression Artifact Removal Leonardo Galteri, Lorenzo Seidenari, Marco Bertini, Alberto Del Bimbo MICC, University of Florence arxiv:1704.02518v3 [cs.cv] 6 Dec 2017 https://www.micc.unifi.it/
More informationOne Network to Solve Them All Solving Linear Inverse Problems using Deep Projection Models
One Network to Solve Them All Solving Linear Inverse Problems using Deep Projection Models [Supplemental Materials] 1. Network Architecture b ref b ref +1 We now describe the architecture of the networks
More informationSingle Image Super-resolution. Slides from Libin Geoffrey Sun and James Hays
Single Image Super-resolution Slides from Libin Geoffrey Sun and James Hays Cs129 Computational Photography James Hays, Brown, fall 2012 Types of Super-resolution Multi-image (sub-pixel registration) Single-image
More informationLOW-RESOLUTION and noisy images are always. Simultaneously Color-Depth Super-Resolution with Conditional Generative Adversarial Network
1 Simultaneously Color-Depth Super-Resolution with Conditional Generative Adversarial Network Lijun Zhao, Huihui Bai, Member, IEEE, Jie Liang, Senior Member, IEEE, Bing Zeng, Fellow, IEEE, Anhong Wang,
More informationSuper-Resolving Very Low-Resolution Face Images with Supplementary Attributes
Super-Resolving Very Low-Resolution Face Images with Supplementary Attributes Xin Yu Basura Fernando Richard Hartley Fatih Porikli Australian National University {xin.yu,basura.fernando,richard.hartley,fatih.porikli}@anu.edu.au
More informationBlind Image Deblurring Using Dark Channel Prior
Blind Image Deblurring Using Dark Channel Prior Jinshan Pan 1,2,3, Deqing Sun 2,4, Hanspeter Pfister 2, and Ming-Hsuan Yang 3 1 Dalian University of Technology 2 Harvard University 3 UC Merced 4 NVIDIA
More informationMulti-Input Cardiac Image Super-Resolution using Convolutional Neural Networks
Multi-Input Cardiac Image Super-Resolution using Convolutional Neural Networks Ozan Oktay, Wenjia Bai, Matthew Lee, Ricardo Guerrero, Konstantinos Kamnitsas, Jose Caballero, Antonio de Marvao, Stuart Cook,
More informationHallucinating Very Low-Resolution Unaligned and Noisy Face Images by Transformative Discriminative Autoencoders
Hallucinating Very Low-Resolution Unaligned and Noisy Face Images by Transformative Discriminative Autoencoders Xin Yu, Fatih Porikli Australian National University {xin.yu, fatih.porikli}@anu.edu.au Abstract
More informationGENERATIVE ADVERSARIAL NETWORK-BASED VIR-
GENERATIVE ADVERSARIAL NETWORK-BASED VIR- TUAL TRY-ON WITH CLOTHING REGION Shizuma Kubo, Yusuke Iwasawa, and Yutaka Matsuo The University of Tokyo Bunkyo-ku, Japan {kubo, iwasawa, matsuo}@weblab.t.u-tokyo.ac.jp
More informationConvolutional Neural Networks + Neural Style Transfer. Justin Johnson 2/1/2017
Convolutional Neural Networks + Neural Style Transfer Justin Johnson 2/1/2017 Outline Convolutional Neural Networks Convolution Pooling Feature Visualization Neural Style Transfer Feature Inversion Texture
More informationarxiv: v4 [cs.cv] 25 Mar 2018
Frame-Recurrent Video Super-Resolution Mehdi S. M. Sajjadi 1,2 msajjadi@tue.mpg.de Raviteja Vemulapalli 2 ravitejavemu@google.com 1 Max Planck Institute for Intelligent Systems 2 Google Matthew Brown 2
More informationarxiv: v1 [cs.cv] 8 Feb 2018
DEEP IMAGE SUPER RESOLUTION VIA NATURAL IMAGE PRIORS Hojjat S. Mousavi, Tiantong Guo, Vishal Monga Dept. of Electrical Engineering, The Pennsylvania State University arxiv:802.0272v [cs.cv] 8 Feb 208 ABSTRACT
More informationarxiv: v1 [cs.cv] 10 Jul 2017
Enhanced Deep Residual Networks for Single Image SuperResolution Bee Lim Sanghyun Son Heewon Kim Seungjun Nah Kyoung Mu Lee Department of ECE, ASRI, Seoul National University, 08826, Seoul, Korea forestrainee@gmail.com,
More informationarxiv: v5 [cs.cv] 4 Oct 2018
Fast, Accurate, and Lightweight Super-Resolution with Cascading Residual Network Namhyuk Ahn [0000 0003 1990 9516], Byungkon Kang [0000 0001 8541 2861], and Kyung-Ah Sohn [0000 0001 8941 1188] arxiv:1803.08664v5
More informationarxiv: v1 [cs.cv] 7 Aug 2017
MemNet: A Persistent Memory Network for Image Restoration Ying Tai 1, Jian Yang 1, Xiaoming Liu 2, and Chunyan Xu 1 1 Department of Computer Science and Engineering, Nanjing University of Science and Technology
More informationFace Super-resolution Guided by Facial Component Heatmaps
Face Super-resolution Guided by Facial Component Heatmaps Xin Yu 1, Basura Fernando 1, Bernard Ghanem 2, Fatih Porikli 1, Richard Hartley 1 1 Australian National University, 2 King Abdullah University
More informationMulti-scale Residual Network for Image Super-Resolution
Multi-scale Residual Network for Image Super-Resolution Juncheng Li 1[0000 0001 7314 6754], Faming Fang 1[0000 0003 4511 4813], Kangfu Mei 2[0000 0001 8949 9597], and Guixu Zhang 1[0000 0003 4720 6607]
More informationUDNet: Up-Down Network for Compact and Efficient Feature Representation in Image Super-Resolution
UDNet: Up-Down Network for Compact and Efficient Feature Representation in Image Super-Resolution Chang Chen Xinmei Tian Zhiwei Xiong Feng Wu University of Science and Technology of China Abstract Recently,
More informationarxiv: v1 [cs.cv] 23 Mar 2018
arxiv:1803.08664v1 [cs.cv] 23 Mar 2018 Fast, Accurate, and, Lightweight Super-Resolution with Cascading Residual Network Namhyuk Ahn, Byungkon Kang, and Kyung-Ah Sohn Department of Computer Engineering,
More informationConnecting Image Denoising and High-Level Vision Tasks via Deep Learning
1 Connecting Image Denoising and High-Level Vision Tasks via Deep Learning Ding Liu, Student Member, IEEE, Bihan Wen, Student Member, IEEE, Jianbo Jiao, Student Member, IEEE, Xianming Liu, Zhangyang Wang,
More informationFast Guided Global Interpolation for Depth and. Yu Li, Dongbo Min, Minh N. Do, Jiangbo Lu
Fast Guided Global Interpolation for Depth and Yu Li, Dongbo Min, Minh N. Do, Jiangbo Lu Introduction Depth upsampling and motion interpolation are often required to generate a dense, high-quality, and
More informationDeep Learning for Visual Manipulation and Synthesis
Deep Learning for Visual Manipulation and Synthesis Jun-Yan Zhu 朱俊彦 UC Berkeley 2017/01/11 @ VALSE What is visual manipulation? Image Editing Program input photo User Input result Desired output: stay
More informationarxiv: v2 [cs.cv] 2 Dec 2017
Learning to Generate Time-Lapse Videos Using Multi-Stage Dynamic Generative Adversarial Networks Wei Xiong, Wenhan Luo, Lin Ma, Wei Liu, and Jiebo Luo Department of Computer Science, University of Rochester,
More informationDeep Generative Adversarial Compression Artifact Removal
Deep Generative Adversarial Compression Artifact Removal Leonardo Galteri, Lorenzo Seidenari, Marco Bertini, Alberto Del Bimbo MICC, University of Florence https://www.micc.unifi.it/ Abstract Compression
More informationLearning Warped Guidance for Blind Face Restoration
Learning Warped Guidance for Blind Face Restoration Xiaoming Li 1[0000 0003 3844 9308], Ming Liu 1[0000 0001 9136 8481], Yuting Ye 1[0000 0002 3922 148X], Wangmeng Zuo 1( )[0000 0002 3330 783X], Liang
More informationarxiv: v1 [cs.cv] 19 Oct 2017
Be Your Own Prada: Fashion Synthesis with Structural Coherence Shizhan Zhu 1 Sanja Fidler 2,3 Raquel Urtasun 2,3,4 Dahua Lin 1 Chen Change Loy 1 1 Department of Information Engineering, The Chinese University
More informationGAN-D: Generative Adversarial Networks for Image Deconvolution
GAN-D: Generative Adversarial Networks for Image Deconvolution Ha Yeon Lee, Jin Myung Kwak, Byunghyun Ban, Seon Jin Na, Se Ra Lee, Heung-Kyu Lee School of Computing and Department of Bio and Brain Engineering
More informationLearning Data Terms for Non-blind Deblurring Supplemental Material
Learning Data Terms for Non-blind Deblurring Supplemental Material Jiangxin Dong 1, Jinshan Pan 2, Deqing Sun 3, Zhixun Su 1,4, and Ming-Hsuan Yang 5 1 Dalian University of Technology dongjxjx@gmail.com,
More information