Self-Supervised Super-Resolution for Multi-Exposure Push-Frame Satellites

• URL: http://arxiv.org/abs/2205.02031v1
• Date: Wed, 4 May 2022 12:42:57 GMT
• Title: Self-Supervised Super-Resolution for Multi-Exposure Push-Frame Satellites
• Authors: Ngoc Long Nguyen, J\'er\'emy Anger, Axel Davy, Pablo Arias, and Gabriele Facciolo
• Abstract summary: The proposed method can handle the signal-dependent noise in the inputs, process sequences of any length, and be robust to inaccuracies in the exposure times. It can be trained end-to-end with self-supervision, without requiring ground truth high resolution frames. We evaluate the proposed method on synthetic and real data and show that it outperforms existing single-exposure approaches.
• Score: 13.267489927661797
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern Earth observation satellites capture multi-exposure bursts of push-frame images that can be super-resolved via computational means. In this work, we propose a super-resolution method for such multi-exposure sequences, a problem that has received very little attention in the literature. The proposed method can handle the signal-dependent noise in the inputs, process sequences of any length, and be robust to inaccuracies in the exposure times. Furthermore, it can be trained end-to-end with self-supervision, without requiring ground truth high resolution frames, which makes it especially suited to handle real data. Central to our method are three key contributions: i) a base-detail decomposition for handling errors in the exposure times, ii) a noise-level-aware feature encoding for improved fusion of frames with varying signal-to-noise ratio and iii) a permutation invariant fusion strategy by temporal pooling operators. We evaluate the proposed method on synthetic and real data and show that it outperforms by a significant margin existing single-exposure approaches that we adapted to the multi-exposure case.

Related papers

• QMambaBSR: Burst Image Super-Resolution with Query State Space Model [55.56075874424194]
  Burst super-resolution aims to reconstruct high-resolution images with higher quality and richer details by fusing the sub-pixel information from multiple burst low-resolution frames. In BusrtSR, the key challenge lies in extracting the base frame's content complementary sub-pixel details while simultaneously suppressing high-frequency noise disturbance. We introduce a novel Query Mamba Burst Super-Resolution (QMambaBSR) network, which incorporates a Query State Space Model (QSSM) and Adaptive Up-sampling module (AdaUp)
  arXiv  Detail & Related papers  (2024-08-16T11:15:29Z)
• Accelerating Diffusion for SAR-to-Optical Image Translation via Adversarial Consistency Distillation [5.234109158596138]
  We propose a new training framework for SAR-to-optical image translation. Our method employs consistency distillation to reduce iterative inference steps and integrates adversarial learning to ensure image clarity and minimize color shifts. The results demonstrate that our approach significantly improves inference speed by 131 times while maintaining the visual quality of the generated images.
  arXiv  Detail & Related papers  (2024-07-08T16:36:12Z)
• Digging into contrastive learning for robust depth estimation with diffusion models [55.62276027922499]
  We propose a novel robust depth estimation method called D4RD. It features a custom contrastive learning mode tailored for diffusion models to mitigate performance degradation in complex environments. In experiments, D4RD surpasses existing state-of-the-art solutions on synthetic corruption datasets and real-world weather conditions.
  arXiv  Detail & Related papers  (2024-04-15T14:29:47Z)
• Efficient Diffusion Model for Image Restoration by Residual Shifting [63.02725947015132]
  This study proposes a novel and efficient diffusion model for image restoration. Our method avoids the need for post-acceleration during inference, thereby avoiding the associated performance deterioration. Our method achieves superior or comparable performance to current state-of-the-art methods on three classical IR tasks.
  arXiv  Detail & Related papers  (2024-03-12T05:06:07Z)
• PolMERLIN: Self-Supervised Polarimetric Complex SAR Image Despeckling with Masked Networks [2.580765958706854]
  Despeckling is a crucial noise reduction task in improving the quality of synthetic aperture radar (SAR) images. Existing methods deal solely with single-polarization images and cannot handle the multi-polarization images captured by modern satellites. We propose a novel self-supervised despeckling approach called channel masking, which exploits the relationship between polarizations.
  arXiv  Detail & Related papers  (2024-01-15T07:06:36Z)
• Holistic Dynamic Frequency Transformer for Image Fusion and Exposure Correction [18.014481087171657]
  The correction of exposure-related issues is a pivotal component in enhancing the quality of images. This paper proposes a novel methodology that leverages the frequency domain to improve and unify the handling of exposure correction tasks. Our proposed method achieves state-of-the-art results, paving the way for more sophisticated and unified solutions in exposure correction.
  arXiv  Detail & Related papers  (2023-09-03T14:09:14Z)
• Searching a Compact Architecture for Robust Multi-Exposure Image Fusion [55.37210629454589]
  Two major stumbling blocks hinder the development, including pixel misalignment and inefficient inference. This study introduces an architecture search-based paradigm incorporating self-alignment and detail repletion modules for robust multi-exposure image fusion. The proposed method outperforms various competitive schemes, achieving a noteworthy 3.19% improvement in PSNR for general scenarios and an impressive 23.5% enhancement in misaligned scenarios.
  arXiv  Detail & Related papers  (2023-05-20T17:01:52Z)
• Gated Multi-Resolution Transfer Network for Burst Restoration and Enhancement [75.25451566988565]
  We propose a novel Gated Multi-Resolution Transfer Network (GMTNet) to reconstruct a spatially precise high-quality image from a burst of low-quality raw images. Detailed experimental analysis on five datasets validates our approach and sets a state-of-the-art for burst super-resolution, burst denoising, and low-light burst enhancement.
  arXiv  Detail & Related papers  (2023-04-13T17:54:00Z)
• Handheld Burst Super-Resolution Meets Multi-Exposure Satellite Imagery [7.9716992946722804]
  We adapt a state-of-the-art kernel regression technique for smartphone camera burst super-resolution to satellites. We leverage the local structure of the image to optimally steer the fusion kernels, limiting blur in the final high-resolution prediction. We extend this approach to predict from a sequence of multi-exposure low-resolution frames a high-resolution and noise-free one.
  arXiv  Detail & Related papers  (2023-03-10T12:13:31Z)
• Aliasing is your Ally: End-to-End Super-Resolution from Raw Image Bursts [70.80220990106467]
  This presentation addresses the problem of reconstructing a high-resolution image from multiple lower-resolution snapshots captured from slightly different viewpoints in space and time. Key challenges for solving this problem include (i) aligning the input pictures with sub-pixel accuracy, (ii) handling raw (noisy) images for maximal faithfulness to native camera data, and (iii) designing/learning an image prior (regularizer) well suited to the task.
  arXiv  Detail & Related papers  (2021-04-13T13:39:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.