Project

Image restoration with Convolutional Neural Networks

Examples of real photo deblurring from the paper.

In this project, we explore the ability of Convolutional Neural Networks to restore degraded images. This page lists related publications and various suplementary material including datasets, evaluation scripts, and trained networks.

Convolutional Neural Networks for Direct Text Deblurring

Authors: Hradiš Michal, Kotera Jan, Zemčík Pavel and Šroubek Filip

Abstract: In this work we address the problem of blind deconvolution and denoising. We focus on restoration of text documents and we show that this type of highly structured data can be successfully restored by a convolutional neural network. The networks are trained to reconstruct high-quality images directly from blurry inputs without assuming any specific blur and noise models. We demonstrate the performance of the convolutional networks on a large set of text documents and on a combination of realistic de-focus and camera shake blur kernels. On this artificial data, the convolutional networks significantly outperform existing blind deconvolution methods, including those optimized for text, in terms of image quality and OCR accuracy. In fact, the networks outperform even state-of-the-art non-blind methods for anything but the lowest noise levels. The approach is validated on real photos taken by various devices.

[Paper] [Extended abstract] [BMVC presentation]

Contact person: Michal Hradiš

Suplementatry downloads:

[NEW] Training data - 66k 300x300px image patches. Each patch is extracted from a different document page. Each blur kernel is unique. These images are generated exactly the same way as for the BMVC paper (the artificial image experiments).
wget http://www.fit.vutbr.cz/~ihradis/CNN-Deblur/BMVC_large_patches.tar.aa
wget http://www.fit.vutbr.cz/~ihradis/CNN-Deblur/BMVC_large_patches.tar.ab
wget http://www.fit.vutbr.cz/~ihradis/CNN-Deblur/BMVC_large_patches.tar.ac
wget http://www.fit.vutbr.cz/~ihradis/CNN-Deblur/BMVC_large_patches.tar.ad
wget http://www.fit.vutbr.cz/~ihradis/CNN-Deblur/BMVC_large_patches.tar.ae
wget http://www.fit.vutbr.cz/~ihradis/CNN-Deblur/BMVC_large_patches.tar.af
wget http://www.fit.vutbr.cz/~ihradis/CNN-Deblur/BMVC_large_patches.tar.ag
wget http://www.fit.vutbr.cz/~ihradis/CNN-Deblur/BMVC_large_patches.tar.ah
wget http://www.fit.vutbr.cz/~ihradis/CNN-Deblur/BMVC_large_patches.tar.ai
wget http://www.fit.vutbr.cz/~ihradis/CNN-Deblur/BMVC_large_patches.tar.aj
wget http://www.fit.vutbr.cz/~ihradis/CNN-Deblur/BMVC_large_patches.tar.ak
wget http://www.fit.vutbr.cz/~ihradis/CNN-Deblur/BMVC_large_patches.tar.al
wget http://www.fit.vutbr.cz/~ihradis/CNN-Deblur/BMVC_large_patches.tar.am
wget http://www.fit.vutbr.cz/~ihradis/CNN-Deblur/BMVC_large_patches.tar.an
wget http://www.fit.vutbr.cz/~ihradis/CNN-Deblur/BMVC_large_patches.tar.ao
wget http://www.fit.vutbr.cz/~ihradis/CNN-Deblur/BMVC_large_patches.tar.ap
wget http://www.fit.vutbr.cz/~ihradis/CNN-Deblur/BMVC_large_patches.tar.aq
cat BMVC_large_patches.tar.* | tar x

Cite as: HRADIŠ Michal, KOTERA Jan, ZEMČÍK Pavel a ŠROUBEK Filip. Convolutional Neural Networks for Direct Text Deblurring. In: Proceedings of BMVC 2015, Swansea, The British Machine Vision Association and Society for Pattern Recognition, 2015, ISBN 1-901725-53-7.

BibTex entry:

@INPROCEEDINGS{
   author = {Michal Hradi{\v{s}} and Jan Kotera and Pavel
	    Zem{\v{c}}{\'{i}}k and Filip {\v{S}}roubek},
   title = {Convolutional Neural Networks for Direct Text Deblurring},
   booktitle = {Proceedings of BMVC 2015},
   year = {2015},
   location = {Swansea, GB},
   publisher = {The British Machine Vision Association and Society for
	   Pattern Recognition},
   ISBN = {1-901725-53-7},
   url = {http://www.fit.vutbr.cz/research/view_pub.php?id=10922}
}

CNN for License Plate Motion Deblurring

Authors: Pavel Svoboda, Michal Hradiš, Lukas Maršík, Pavel Zemčík

Abstract: In this work we explore the previously proposed approach of direct blind deconvolution and denoising with convolutional neural networks in a situation where the blur kernels are partially constrained. We focus on blurred images from a real-life traffic surveillance system, on which we, for the first time, demonstrate that neural networks trained on artificial data provide superior reconstruction quality on real images compared to traditional blind deconvolution methods. The training data is easy to obtain by blurring sharp photos from a target system with a very rough approximation of the expected blur kernels, thereby allowing custom CNNs to be trained for a specific application (image content and blur range). Additionally, we evaluate the behavior and limits of the CNNs with respect to blur direction range and length.

[arXiv paper] [poster] [Results1] [Results2] [Results3] [Results4] [Results5] [Results6]

Contact person: Michal Hradiš

Suplementatry downloads:

  • L15 convolutional networks in Caffe format trained on artificial data. It is The network is the one used to get final results in the paper.
  • Test data from the paper. 721 cropped license plates with motion blur from real trafic camera system together with ground truth annotations and deblurred results from the paper.

Compression Artifacts Removal Using Convolutional Neural Networks

Authors: Pavel Svoboda, Michal Hradiš, David Bařina, Pavel Zemčík

Abstract: This paper shows that it is possible to train large and deep convolutional neural networks (CNN) for JPEG compression artifacts reduction, and that such networks can provide significantly better reconstruction quality compared to previously used smaller networks as well as to any other state-of-the-art methods. We were able to train networks with 8 layers in a single step and in relatively short time by combining residual learning, skip architecture, and symmetric weight initialization. We provide further insights into convolution networks for JPEG artifact reduction by evaluating three different objectives, generalization with respect to training dataset size, and generalization with respect to JPEG quality level.

[Paper]

Suplementatry downloads:

Contact person: Michal Hradiš

Cite as: SVOBODA Pavel, HRADIŠ Michal, BAŘINA David and ZEMČÍK Pavel. Compression Artifacts Removal Using Convolutional Neural Networks. Journal of WSCG. Plzeň: 2016, vol. 24, no. 2, pp. 63-72. ISSN 1213-6972.

@ARTICLE{
	   author = {Pavel Svoboda and Michal Hradi{\v{s}} and David Ba{\v{r}}ina
		and Pavel Zem{\v{c}}{\'{i}}k},
	   title = {Compression Artifacts Removal Using Convolutional Neural
		Networks},
	   pages = {63--72},
	   journal = {Journal of WSCG},
	   volume = {24},
	   number = {2},
	   year = {2016},
	   ISSN = {1213-6972},
	   language = {english},
	   url = {http://www.fit.vutbr.cz/research/view_pub.php.en?id=11176}
	}

Author of this webpage: Michal Hradiš