Gated Multi-Resolution Transfer Network for Burst Restoration and
Enhancement
- URL: http://arxiv.org/abs/2304.06703v1
- Date: Thu, 13 Apr 2023 17:54:00 GMT
- Title: Gated Multi-Resolution Transfer Network for Burst Restoration and
Enhancement
- Authors: Nancy Mehta, Akshay Dudhane, Subrahmanyam Murala, Syed Waqas Zamir,
Salman Khan, Fahad Shahbaz Khan
- Abstract summary: We propose a novel Gated Multi-Resolution Transfer Network (GMTNet) to reconstruct a spatially precise high-quality image from a burst of low-quality raw images.
Detailed experimental analysis on five datasets validates our approach and sets a state-of-the-art for burst super-resolution, burst denoising, and low-light burst enhancement.
- Score: 75.25451566988565
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Burst image processing is becoming increasingly popular in recent years.
However, it is a challenging task since individual burst images undergo
multiple degradations and often have mutual misalignments resulting in ghosting
and zipper artifacts. Existing burst restoration methods usually do not
consider the mutual correlation and non-local contextual information among
burst frames, which tends to limit these approaches in challenging cases.
Another key challenge lies in the robust up-sampling of burst frames. The
existing up-sampling methods cannot effectively utilize the advantages of
single-stage and progressive up-sampling strategies with conventional and/or
recent up-samplers at the same time. To address these challenges, we propose a
novel Gated Multi-Resolution Transfer Network (GMTNet) to reconstruct a
spatially precise high-quality image from a burst of low-quality raw images.
GMTNet consists of three modules optimized for burst processing tasks:
Multi-scale Burst Feature Alignment (MBFA) for feature denoising and alignment,
Transposed-Attention Feature Merging (TAFM) for multi-frame feature
aggregation, and Resolution Transfer Feature Up-sampler (RTFU) to up-scale
merged features and construct a high-quality output image. Detailed
experimental analysis on five datasets validates our approach and sets a
state-of-the-art for burst super-resolution, burst denoising, and low-light
burst enhancement.
Related papers
- QMambaBSR: Burst Image Super-Resolution with Query State Space Model [55.56075874424194]
Burst super-resolution aims to reconstruct high-resolution images with higher quality and richer details by fusing the sub-pixel information from multiple burst low-resolution frames.
In BusrtSR, the key challenge lies in extracting the base frame's content complementary sub-pixel details while simultaneously suppressing high-frequency noise disturbance.
We introduce a novel Query Mamba Burst Super-Resolution (QMambaBSR) network, which incorporates a Query State Space Model (QSSM) and Adaptive Up-sampling module (AdaUp)
arXiv Detail & Related papers (2024-08-16T11:15:29Z) - CRNet: A Detail-Preserving Network for Unified Image Restoration and Enhancement Task [44.14681936953848]
Composite Refinement Network (CRNet) can perform unified image restoration and enhancement.
CRNet explicitly separates and strengthens high and low-frequency information through pooling layers.
Our model secured third place in the first track of the Bracketing Image Restoration and Enhancement Challenge.
arXiv Detail & Related papers (2024-04-22T12:33:18Z) - Exposure Bracketing is All You Need for Unifying Image Restoration and Enhancement Tasks [50.822601495422916]
We propose to utilize exposure bracketing photography to unify image restoration and enhancement tasks.
Due to the difficulty in collecting real-world pairs, we suggest a solution that first pre-trains the model with synthetic paired data.
In particular, a temporally modulated recurrent network (TMRNet) and self-supervised adaptation method are proposed.
arXiv Detail & Related papers (2024-01-01T14:14:35Z) - Holistic Dynamic Frequency Transformer for Image Fusion and Exposure Correction [18.014481087171657]
The correction of exposure-related issues is a pivotal component in enhancing the quality of images.
This paper proposes a novel methodology that leverages the frequency domain to improve and unify the handling of exposure correction tasks.
Our proposed method achieves state-of-the-art results, paving the way for more sophisticated and unified solutions in exposure correction.
arXiv Detail & Related papers (2023-09-03T14:09:14Z) - Hybrid-Supervised Dual-Search: Leveraging Automatic Learning for
Loss-free Multi-Exposure Image Fusion [60.221404321514086]
Multi-exposure image fusion (MEF) has emerged as a prominent solution to address the limitations of digital imaging in representing varied exposure levels.
This paper presents a Hybrid-Supervised Dual-Search approach for MEF, dubbed HSDS-MEF, which introduces a bi-level optimization search scheme for automatic design of both network structures and loss functions.
arXiv Detail & Related papers (2023-09-03T08:07:26Z) - Super-resolution Reconstruction of Single Image for Latent features [8.857209365343646]
Single-image super-resolution (SISR) typically focuses on restoring various degraded low-resolution (LR) images to a single high-resolution (HR) image.
It is often challenging for models to simultaneously maintain high quality and rapid sampling while preserving diversity in details and texture features.
This challenge can lead to issues such as model collapse, lack of rich details and texture features in the reconstructed HR images, and excessive time consumption for model sampling.
arXiv Detail & Related papers (2022-11-16T09:37:07Z) - Burst Image Restoration and Enhancement [86.08546447144377]
The goal of Burst Image Restoration is to effectively combine complimentary cues across multiple burst frames to generate high-quality outputs.
We create a set of emphpseudo-burst features that combine complimentary information from all the input burst frames to seamlessly exchange information.
Our approach delivers state of the art performance on burst super-resolution and low-light image enhancement tasks.
arXiv Detail & Related papers (2021-10-07T17:58:56Z) - Deep Burst Super-Resolution [165.90445859851448]
We propose a novel architecture for the burst super-resolution task.
Our network takes multiple noisy RAW images as input, and generates a denoised, super-resolved RGB image as output.
In order to enable training and evaluation on real-world data, we additionally introduce the BurstSR dataset.
arXiv Detail & Related papers (2021-01-26T18:57:21Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.