Image restoration with Convolutional Neural Networks
In this project, we explore the ability of Convolutional Neural Networks to restore degraded images. This page lists related publications and various suplementary material including datasets, evaluation scripts, and trained networks.
Convolutional Neural Networks for Direct Text Deblurring
Authors: Hradiš Michal, Kotera Jan, Zemčík Pavel and Šroubek Filip
Abstract: In this work we address the problem of blind deconvolution and denoising. We focus on restoration of text documents and we show that this type of highly structured data can be successfully restored by a convolutional neural network. The networks are trained to reconstruct high-quality images directly from blurry inputs without assuming any specific blur and noise models. We demonstrate the performance of the convolutional networks on a large set of text documents and on a combination of realistic de-focus and camera shake blur kernels. On this artificial data, the convolutional networks significantly outperform existing blind deconvolution methods, including those optimized for text, in terms of image quality and OCR accuracy. In fact, the networks outperform even state-of-the-art non-blind methods for anything but the lowest noise levels. The approach is validated on real photos taken by various devices.
[Paper] [Extended abstract] [BMVC presentation]
Contact person: Michal Hradiš
Suplementatry downloads:
- L15 convolutional networks in Caffe format trained on artificial data. The networks expect text at DPI 120-150, reasonable orientation, and reasonable black and white levels. The networks expect mean [103.9, 116.8, 123.7] to be subtracted from inputs. The inputs shuld be further multiplied by 0.004.
- Python script for CNN deblurring. 2016-06-22 updated for easier use.
- Image quality test dataset and results of L15-CNN
- OCR test dataset, results of L15-CNN, results of baseline methods , OCR evaluation data and scripts .
- CNN deblurring results on real photograps.
wget http://www.fit.vutbr.cz/~ihradis/CNN-Deblur/BMVC_large_patches.tar.aa wget http://www.fit.vutbr.cz/~ihradis/CNN-Deblur/BMVC_large_patches.tar.ab wget http://www.fit.vutbr.cz/~ihradis/CNN-Deblur/BMVC_large_patches.tar.ac wget http://www.fit.vutbr.cz/~ihradis/CNN-Deblur/BMVC_large_patches.tar.ad wget http://www.fit.vutbr.cz/~ihradis/CNN-Deblur/BMVC_large_patches.tar.ae wget http://www.fit.vutbr.cz/~ihradis/CNN-Deblur/BMVC_large_patches.tar.af wget http://www.fit.vutbr.cz/~ihradis/CNN-Deblur/BMVC_large_patches.tar.ag wget http://www.fit.vutbr.cz/~ihradis/CNN-Deblur/BMVC_large_patches.tar.ah wget http://www.fit.vutbr.cz/~ihradis/CNN-Deblur/BMVC_large_patches.tar.ai wget http://www.fit.vutbr.cz/~ihradis/CNN-Deblur/BMVC_large_patches.tar.aj wget http://www.fit.vutbr.cz/~ihradis/CNN-Deblur/BMVC_large_patches.tar.ak wget http://www.fit.vutbr.cz/~ihradis/CNN-Deblur/BMVC_large_patches.tar.al wget http://www.fit.vutbr.cz/~ihradis/CNN-Deblur/BMVC_large_patches.tar.am wget http://www.fit.vutbr.cz/~ihradis/CNN-Deblur/BMVC_large_patches.tar.an wget http://www.fit.vutbr.cz/~ihradis/CNN-Deblur/BMVC_large_patches.tar.ao wget http://www.fit.vutbr.cz/~ihradis/CNN-Deblur/BMVC_large_patches.tar.ap wget http://www.fit.vutbr.cz/~ihradis/CNN-Deblur/BMVC_large_patches.tar.aq cat BMVC_large_patches.tar.* | tar x
BibTex entry:
@INPROCEEDINGS{ author = {Michal Hradi{\v{s}} and Jan Kotera and Pavel Zem{\v{c}}{\'{i}}k and Filip {\v{S}}roubek}, title = {Convolutional Neural Networks for Direct Text Deblurring}, booktitle = {Proceedings of BMVC 2015}, year = {2015}, location = {Swansea, GB}, publisher = {The British Machine Vision Association and Society for Pattern Recognition}, ISBN = {1-901725-53-7}, url = {http://www.fit.vutbr.cz/research/view_pub.php?id=10922} }
CNN for License Plate Motion Deblurring
Authors: Pavel Svoboda, Michal Hradiš, Lukas Maršík, Pavel Zemčík
Abstract: In this work we explore the previously proposed approach of direct blind deconvolution and denoising with convolutional neural networks in a situation where the blur kernels are partially constrained. We focus on blurred images from a real-life traffic surveillance system, on which we, for the first time, demonstrate that neural networks trained on artificial data provide superior reconstruction quality on real images compared to traditional blind deconvolution methods. The training data is easy to obtain by blurring sharp photos from a target system with a very rough approximation of the expected blur kernels, thereby allowing custom CNNs to be trained for a specific application (image content and blur range). Additionally, we evaluate the behavior and limits of the CNNs with respect to blur direction range and length.
[arXiv paper] [poster] [Results1] [Results2] [Results3] [Results4] [Results5] [Results6]
Contact person: Michal Hradiš
Suplementatry downloads:
- L15 convolutional networks in Caffe format trained on artificial data. It is The network is the one used to get final results in the paper.
- Test data from the paper. 721 cropped license plates with motion blur from real trafic camera system together with ground truth annotations and deblurred results from the paper.
Compression Artifacts Removal Using Convolutional Neural Networks
Authors: Pavel Svoboda, Michal Hradiš, David Bařina, Pavel Zemčík
Abstract: This paper shows that it is possible to train large and deep convolutional neural networks (CNN) for JPEG compression artifacts reduction, and that such networks can provide significantly better reconstruction quality compared to previously used smaller networks as well as to any other state-of-the-art methods. We were able to train networks with 8 layers in a single step and in relatively short time by combining residual learning, skip architecture, and symmetric weight initialization. We provide further insights into convolution networks for JPEG artifact reduction by evaluating three different objectives, generalization with respect to training dataset size, and generalization with respect to JPEG quality level.
[Paper]
Suplementatry downloads:
- L08 residual network from the paper trained for grayscale JPEG quality 20.
- L04 residual network from the paper trained for grayscale JPEG quality 20.
- Reconstructions from the paper. Contains all images from experiments reported in the paper. [1.7GB]
Contact person: Michal Hradiš
@ARTICLE{ author = {Pavel Svoboda and Michal Hradi{\v{s}} and David Ba{\v{r}}ina and Pavel Zem{\v{c}}{\'{i}}k}, title = {Compression Artifacts Removal Using Convolutional Neural Networks}, pages = {63--72}, journal = {Journal of WSCG}, volume = {24}, number = {2}, year = {2016}, ISSN = {1213-6972}, language = {english}, url = {http://www.fit.vutbr.cz/research/view_pub.php.en?id=11176} }
Author of this webpage: Michal Hradiš