Self-Supervised Super-Resolution for Multi-Exposure Push-Frame
Satellites
- URL: http://arxiv.org/abs/2205.02031v1
- Date: Wed, 4 May 2022 12:42:57 GMT
- Title: Self-Supervised Super-Resolution for Multi-Exposure Push-Frame
Satellites
- Authors: Ngoc Long Nguyen, J\'er\'emy Anger, Axel Davy, Pablo Arias, and
Gabriele Facciolo
- Abstract summary: The proposed method can handle the signal-dependent noise in the inputs, process sequences of any length, and be robust to inaccuracies in the exposure times.
It can be trained end-to-end with self-supervision, without requiring ground truth high resolution frames.
We evaluate the proposed method on synthetic and real data and show that it outperforms existing single-exposure approaches.
- Score: 13.267489927661797
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Modern Earth observation satellites capture multi-exposure bursts of
push-frame images that can be super-resolved via computational means. In this
work, we propose a super-resolution method for such multi-exposure sequences, a
problem that has received very little attention in the literature. The proposed
method can handle the signal-dependent noise in the inputs, process sequences
of any length, and be robust to inaccuracies in the exposure times.
Furthermore, it can be trained end-to-end with self-supervision, without
requiring ground truth high resolution frames, which makes it especially suited
to handle real data. Central to our method are three key contributions: i) a
base-detail decomposition for handling errors in the exposure times, ii) a
noise-level-aware feature encoding for improved fusion of frames with varying
signal-to-noise ratio and iii) a permutation invariant fusion strategy by
temporal pooling operators. We evaluate the proposed method on synthetic and
real data and show that it outperforms by a significant margin existing
single-exposure approaches that we adapted to the multi-exposure case.
Related papers
- Accelerating Diffusion for SAR-to-Optical Image Translation via Adversarial Consistency Distillation [5.234109158596138]
We propose a new training framework for SAR-to-optical image translation.
Our method employs consistency distillation to reduce iterative inference steps and integrates adversarial learning to ensure image clarity and minimize color shifts.
The results demonstrate that our approach significantly improves inference speed by 131 times while maintaining the visual quality of the generated images.
arXiv Detail & Related papers (2024-07-08T16:36:12Z) - Efficient Diffusion Model for Image Restoration by Residual Shifting [63.02725947015132]
This study proposes a novel and efficient diffusion model for image restoration.
Our method avoids the need for post-acceleration during inference, thereby avoiding the associated performance deterioration.
Our method achieves superior or comparable performance to current state-of-the-art methods on three classical IR tasks.
arXiv Detail & Related papers (2024-03-12T05:06:07Z) - PolMERLIN: Self-Supervised Polarimetric Complex SAR Image Despeckling
with Masked Networks [2.580765958706854]
Despeckling is a crucial noise reduction task in improving the quality of synthetic aperture radar (SAR) images.
Existing methods deal solely with single-polarization images and cannot handle the multi-polarization images captured by modern satellites.
We propose a novel self-supervised despeckling approach called channel masking, which exploits the relationship between polarizations.
arXiv Detail & Related papers (2024-01-15T07:06:36Z) - Exposure Bracketing is All You Need for Unifying Image Restoration and Enhancement Tasks [50.822601495422916]
We propose to utilize exposure bracketing photography to unify image restoration and enhancement tasks.
Due to the difficulty in collecting real-world pairs, we suggest a solution that first pre-trains the model with synthetic paired data.
In particular, a temporally modulated recurrent network (TMRNet) and self-supervised adaptation method are proposed.
arXiv Detail & Related papers (2024-01-01T14:14:35Z) - Unified Frequency-Assisted Transformer Framework for Detecting and
Grounding Multi-Modal Manipulation [109.1912721224697]
We present the Unified Frequency-Assisted transFormer framework, named UFAFormer, to address the DGM4 problem.
By leveraging the discrete wavelet transform, we decompose images into several frequency sub-bands, capturing rich face forgery artifacts.
Our proposed frequency encoder, incorporating intra-band and inter-band self-attentions, explicitly aggregates forgery features within and across diverse sub-bands.
arXiv Detail & Related papers (2023-09-18T11:06:42Z) - Holistic Dynamic Frequency Transformer for Image Fusion and Exposure
Correction [19.088429172840428]
The correction of exposure-related issues is a pivotal component in enhancing the quality of images.
This paper proposes a novel methodology that leverages the frequency domain to improve and unify the handling of exposure correction tasks.
Our proposed method achieves state-of-the-art results, paving the way for more sophisticated and unified solutions in exposure correction.
arXiv Detail & Related papers (2023-09-03T14:09:14Z) - Embracing Compact and Robust Architectures for Multi-Exposure Image
Fusion [50.598654017728045]
We propose a search-based paradigm, involving self-alignment and detail repletion modules for robust multi-exposure image fusion.
By utilizing scene relighting and deformable convolutions, the self-alignment module can accurately align images despite camera movement.
We realize the state-of-the-art performance in comparison to various competitive schemes, yielding a 4.02% and 29.34% improvement in PSNR for general and misaligned scenarios.
arXiv Detail & Related papers (2023-05-20T17:01:52Z) - Gated Multi-Resolution Transfer Network for Burst Restoration and
Enhancement [75.25451566988565]
We propose a novel Gated Multi-Resolution Transfer Network (GMTNet) to reconstruct a spatially precise high-quality image from a burst of low-quality raw images.
Detailed experimental analysis on five datasets validates our approach and sets a state-of-the-art for burst super-resolution, burst denoising, and low-light burst enhancement.
arXiv Detail & Related papers (2023-04-13T17:54:00Z) - Handheld Burst Super-Resolution Meets Multi-Exposure Satellite Imagery [7.9716992946722804]
We adapt a state-of-the-art kernel regression technique for smartphone camera burst super-resolution to satellites.
We leverage the local structure of the image to optimally steer the fusion kernels, limiting blur in the final high-resolution prediction.
We extend this approach to predict from a sequence of multi-exposure low-resolution frames a high-resolution and noise-free one.
arXiv Detail & Related papers (2023-03-10T12:13:31Z) - Aliasing is your Ally: End-to-End Super-Resolution from Raw Image Bursts [70.80220990106467]
This presentation addresses the problem of reconstructing a high-resolution image from multiple lower-resolution snapshots captured from slightly different viewpoints in space and time.
Key challenges for solving this problem include (i) aligning the input pictures with sub-pixel accuracy, (ii) handling raw (noisy) images for maximal faithfulness to native camera data, and (iii) designing/learning an image prior (regularizer) well suited to the task.
arXiv Detail & Related papers (2021-04-13T13:39:43Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.