Video restoration based on deep learning: a comprehensive survey (original) (raw)

Editorial: Introduction to the Issue on Deep Learning for Image/Video Restoration and Compression

2021

T HE huge success of deep-learning–based approaches in computer vision has inspired research in learned solutions to classic image/video processing problems, such as denoising, deblurring, dehazing, deraining, super-resolution (SR), and compression. Hence, learning-based methods have emerged as a promising nonlinear signal-processing framework for image/video restoration and compression. Recent works have shown that learned models can achieve significant performance gains, especially in terms of perceptual quality measures, over traditional methods. Hence, the state of the art in image restoration and compression is getting redefined. This special issue covers the state of the art in learned image/video restoration and compression to promote further progress in innovative architectures and training methods for effective and efficient networks for image/video restoration and compression. In the following, we provide a short overview of the state of the art in learned image and video ...

Image restoration using deep learning

2016

We propose a new image restoration method that reduces noise and blur in degraded images. In contrast to many state of the art methods, our method does not rely on intensive iterative approaches, instead it uses a pre-trained convolutional neural network.

State of the Art on: Deep Image Denoising

2020

Recent technological and methodological advances have allowed the employment of deep learning techniques, in particular deep artificial neural networks, in a large variety of fields. One of the fields that most is benefiting from the introduction of deep learning is image processing and computer vision, which mainly exploits convolutional neural networks (CNNs) for addressing visual understanding problems. For instance, the use of CNNs for image classification and object detection has led to outstanding results. In the last years, deep learning models have been successfully employed also for the tasks of image restoration. Starting from a corrupted image (e.g. noisy, blurred), the goal of image restoration is to recover the original image (i.e. the clean image). Depending on the type of corruption, image restoration tasks can be divided into deblurring, super-resolution, denoising, inpainting, text removal and many others. In particular, the main focus of our research work is image ...

NTIRE 2019 Challenge on Video Deblurring and Super-Resolution: Dataset and Study

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2019

This paper introduces a novel large dataset for video deblurring, video super-resolution and studies the state-ofthe-art as emerged from the NTIRE 2019 video restoration challenges. The video deblurring and video superresolution challenges are each the first challenge of its kind, with 4 competitions, hundreds of participants and tens of proposed solutions. Our newly collected REalistic and Diverse Scenes dataset (REDS) was employed by the challenges. In our study, we compare the solutions from the challenges to a set of representative methods from the literature and evaluate them on our proposed REDS dataset. We find that the NTIRE 2019 challenges push the state-of-theart in video deblurring and super-resolution, reaching compelling performance on our newly proposed REDS dataset.

Image Restoration and Enhancement Using Deep Learning

International Journal of Engineering Applied Sciences and Technology

During the process of image acquisition, sometimes images are degraded because of various reasons like low resolution of camera, motion blur, noise etc. This paper presents the work associated with the Image Restoration & Enhancement techniques. The process of recovering degraded image is known as Image Restoration. Image restoration includes denoising image, image inpainting, etc. Here we proposed Convolution Neural Network (CNN) with Median Filter for removing noise, Region filling Exemplar Based Inpainting Algorithm for image inpainting. Image enhancement is one amongst the problem in image processing. Haze, low lighting etc. are the various problems in images. The aim of Image enhancement is to process an image such that result is more suitable than original image for specific application. Here for haze removal we implement dark channel prior algorithm and for lightning low-light image we proposed functions. Image enhancement improves the appearance of the image.

Can fully convolutional networks perform well for general image restoration problems?

2017 Fifteenth IAPR International Conference on Machine Vision Applications (MVA), 2017

We present a fully convolutional network(FCN) based approach for color image restoration. FCNs have recently shown remarkable performance for high-level vision problem like semantic segmentation. In this paper, we investigate if FCN models can show promising performance for low-level problems like image restoration as well. We propose a fully convolutional model, that learns a direct end-to-end mapping between the corrupted images as input and the desired clean images as output. Our proposed method takes inspiration from domain transformation techniques but presents a data-driven task specific approach where filters for novel basis projection, task dependent coefficient alterations, and image reconstruction are represented as convolutional networks. Experimental results show that our FCN model outperforms traditional sparse coding based methods and demonstrates competitive performance compared to the state-of-the-art methods for image denoising. We further show that our proposed model can solve the difficult problem of blind image inpainting and can produce reconstructed images of impressive visual quality.

Blind Restoration Using Convolution Neural Network

Iraqi Journal of Information and Communications Technology, 2021

Image restoration is a branch of image processing that involves a mathematical deterioration and restoration model to restore an original image from a degraded image. This research aims to restore blurred images that have been corrupted by a known or unknown degradation function. Image restoration approaches can be classified into 2 groups based on degradation feature knowledge: blind and non-blind techniques. In our research, we adopt the type of blind algorithm. A deep learning method (SR) has been proposed for single image super-resolution. This approach can directly learn an end-to-end mapping between low-resolution images and high-resolution images. The mapping is expressed by a deep convolutional neural network (CNN). The proposed restoration system must overcome and deal with the challenges that the degraded images have unknown kernel blur, to deblur degraded images as an estimation from original images with a minimum rate of error.

Research Project Proposal: Deep Image Denoising

2020

The goal of image restoration is to recover the original, clean image, starting from a corrupted image. Depending on the type of corruption, image restoration tasks can be divided into deblurring, super-resolution, denoising, inpainting, text removal and many others. In particular, image denoising has the goal of estimating the clean image from its observed version corrupted by noise. Modern image denoising lies at the intersection of signal processing, computer science and machine learning. Indeed, recent technological and methodological advances in deep learning have allowed the employment of convolutional neural networks (CNNs) for image restoration purposes. The obvious application for image denoising is to provide to the user a pleasant and clear image by removing as much noise as possible, without losing details. From a technical point of view, a denoised image is an essential prerequisite for more high-level computer vision tasks and complex deep learning pipelines (e.g. auto...

Image Restoration Using Deep Regulated Convolutional Networks

2019

While the depth of convolutional neural networks has attracted substantial attention in the deep learning research, the width of these networks has recently received greater interest. The width of networks, defined as the size of the receptive fields and the density of the channels, has demonstrated crucial importance in low-level vision tasks such as image denoising and restoration. However, the limited generalization ability, due to the increased width of networks, creates a bottleneck in designing wider networks. In this paper, we propose the Deep Regulated Convolutional Network (RC-Net), a deep network composed of regulated sub-network blocks cascaded by skip-connections, to overcome this bottleneck. Specifically, the Regulated Convolution block (RC-block), featured by a combination of large and small convolution filters, balances the effectiveness of prominent feature extraction and the generalization ability of the network. RC-Nets have several compelling advantages: they embr...

Deep Residual Autoencoder for quality independent JPEG restoration

2019

In this paper we propose a deep residual autoencoder exploiting Residual-in-Residual Dense Blocks (RRDB) to remove artifacts in JPEG compressed images that is independent from the Quality Factor (QF) used. The proposed approach leverages both the learning capacity of deep residual networks and prior knowledge of the JPEG compression pipeline. The proposed model operates in the YCbCr color space and performs JPEG artifact restoration in two phases using two different autoencoders: the first one restores the luma channel exploiting 2D convolutions; the second one, using the restored luma channel as a guide, restores the chroma channels explotining 3D convolutions. Extensive experimental results on three widely used benchmark datasets (i.e. LIVE1, BDS500, and CLASSIC-5) show that our model is able to outperform the state of the art with respect to all the evaluation metrics considered (i.e. PSNR, PSNR-B, and SSIM). This results is remarkable since the approaches in the state of the art...