Exploring and Evaluating Image Restoration Potential in Dynamic Scenes
- URL: http://arxiv.org/abs/2203.11754v2
- Date: Wed, 23 Mar 2022 13:15:17 GMT
- Title: Exploring and Evaluating Image Restoration Potential in Dynamic Scenes
- Authors: Cheng Zhang, Shaolin Su, Yu Zhu, Qingsen Yan, Jinqiu Sun, Yanning
Zhang
- Abstract summary: In dynamic scenes, images often suffer from dynamic blur due to superposition of motions or low signal-noise ratio.
We propose a novel concept, referring to image restoration potential (IRP)
- Score: 44.37018725642948
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: In dynamic scenes, images often suffer from dynamic blur due to superposition
of motions or low signal-noise ratio resulted from quick shutter speed when
avoiding motions. Recovering sharp and clean results from the captured images
heavily depends on the ability of restoration methods and the quality of the
input. Although existing research on image restoration focuses on developing
models for obtaining better restored results, fewer have studied to evaluate
how and which input image leads to superior restored quality. In this paper, to
better study an image's potential value that can be explored for restoration,
we propose a novel concept, referring to image restoration potential (IRP).
Specifically, We first establish a dynamic scene imaging dataset containing
composite distortions and applied image restoration processes to validate the
rationality of the existence to IRP. Based on this dataset, we investigate
several properties of IRP and propose a novel deep model to accurately predict
IRP values. By gradually distilling and selective fusing the degradation
features, the proposed model shows its superiority in IRP prediction. Thanks to
the proposed model, we are then able to validate how various image restoration
related applications are benefited from IRP prediction. We show the potential
usages of IRP as a filtering principle to select valuable frames, an auxiliary
guidance to improve restoration models, and even an indicator to optimize
camera settings for capturing better images under dynamic scenarios.
Related papers
- OneRestore: A Universal Restoration Framework for Composite Degradation [33.556183375565034]
In real-world scenarios, image impairments often manifest as composite degradations, presenting a complex interplay of elements such as low light, haze, rain, and snow.
Our study proposes a versatile imaging model that consolidates four physical corruption paradigms to accurately represent complex, composite degradation scenarios.
OneRestore is a novel transformer-based framework designed for adaptive, controllable scene restoration.
arXiv Detail & Related papers (2024-07-05T16:27:00Z) - Photo-Realistic Image Restoration in the Wild with Controlled Vision-Language Models [14.25759541950917]
This work leverages a capable vision-language model and a synthetic degradation pipeline to learn image restoration in the wild (wild IR)
Our base diffusion model is the image restoration SDE (IR-SDE)
arXiv Detail & Related papers (2024-04-15T12:34:21Z) - Boosting Image Restoration via Priors from Pre-trained Models [54.83907596825985]
We learn an additional lightweight module called Pre-Train-Guided Refinement Module (PTG-RM) to refine restoration results of a target restoration network with OSF.
PTG-RM effectively enhances restoration performance of various models across different tasks, including low-light enhancement, deraining, deblurring, and denoising.
arXiv Detail & Related papers (2024-03-11T15:11:57Z) - Analysis of Deep Image Prior and Exploiting Self-Guidance for Image
Reconstruction [13.277067849874756]
We study how DIP recovers information from undersampled imaging measurements.
We introduce a self-driven reconstruction process that concurrently optimize both the network weights and the input.
Our method incorporates a novel denoiser regularization term which enables robust and stable joint estimation of both the network input and reconstructed image.
arXiv Detail & Related papers (2024-02-06T15:52:23Z) - Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild [57.06779516541574]
SUPIR (Scaling-UP Image Restoration) is a groundbreaking image restoration method that harnesses generative prior and the power of model scaling up.
We collect a dataset comprising 20 million high-resolution, high-quality images for model training, each enriched with descriptive text annotations.
arXiv Detail & Related papers (2024-01-24T17:58:07Z) - Efficient Test-Time Adaptation for Super-Resolution with Second-Order
Degradation and Reconstruction [62.955327005837475]
Image super-resolution (SR) aims to learn a mapping from low-resolution (LR) to high-resolution (HR) using paired HR-LR training images.
We present an efficient test-time adaptation framework for SR, named SRTTA, which is able to quickly adapt SR models to test domains with different/unknown degradation types.
arXiv Detail & Related papers (2023-10-29T13:58:57Z) - Reconstruction Distortion of Learned Image Compression with
Imperceptible Perturbations [69.25683256447044]
We introduce an attack approach designed to effectively degrade the reconstruction quality of Learned Image Compression (LIC)
We generate adversarial examples by introducing a Frobenius norm-based loss function to maximize the discrepancy between original images and reconstructed adversarial examples.
Experiments conducted on the Kodak dataset using various LIC models demonstrate effectiveness.
arXiv Detail & Related papers (2023-06-01T20:21:05Z) - Invertible Rescaling Network and Its Extensions [118.72015270085535]
In this work, we propose a novel invertible framework to model the bidirectional degradation and restoration from a new perspective.
We develop invertible models to generate valid degraded images and transform the distribution of lost contents.
Then restoration is made tractable by applying the inverse transformation on the generated degraded image together with a randomly-drawn latent variable.
arXiv Detail & Related papers (2022-10-09T06:58:58Z) - REPNP: Plug-and-Play with Deep Reinforcement Learning Prior for Robust
Image Restoration [30.966005373669027]
We propose a novel deep reinforcement learning (DRL) based framework dubbed RePNP.
Results demonstrate that the proposed RePNP is robust to the observation model used in the.
scheme dubbed RePNP.
RePNP achieves better results subjective to model deviation with fewer model parameters.
arXiv Detail & Related papers (2022-07-25T10:56:10Z) - Inverting Adversarially Robust Networks for Image Synthesis [37.927552662984034]
We propose the use of robust representations as a perceptual primitive for feature inversion models.
We empirically show that adopting robust representations as an image prior significantly improves the reconstruction accuracy of CNN-based feature inversion models.
Following these findings, we propose an encoding-decoding network based on robust representations and show its advantages for applications such as anomaly detection, style transfer and image denoising.
arXiv Detail & Related papers (2021-06-13T05:51:00Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.