SAFNet: Selective Alignment Fusion Network for Efficient HDR Imaging

• URL: http://arxiv.org/abs/2407.16308v1
• Date: Tue, 23 Jul 2024 09:02:35 GMT
• Title: SAFNet: Selective Alignment Fusion Network for Efficient HDR Imaging
• Authors: Lingtong Kong, Bo Li, Yike Xiong, Hao Zhang, Hong Gu, Jinwei Chen,
• Abstract summary: Selective Alignment Fusion Network (SAFNet) for HDR imaging is proposed. It can focus the model on finding valuable regions while estimating their easily detectable and meaningful motion. Experiments on public and newly developed challenging datasets show that proposed SAFNet not only exceeds previous SOTA competitors quantitatively and qualitatively, but also runs order of magnitude faster.
• Score: 14.466061918935937
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-exposure High Dynamic Range (HDR) imaging is a challenging task when facing truncated texture and complex motion. Existing deep learning-based methods have achieved great success by either following the alignment and fusion pipeline or utilizing attention mechanism. However, the large computation cost and inference delay hinder them from deploying on resource limited devices. In this paper, to achieve better efficiency, a novel Selective Alignment Fusion Network (SAFNet) for HDR imaging is proposed. After extracting pyramid features, it jointly refines valuable area masks and cross-exposure motion in selected regions with shared decoders, and then fuses high quality HDR image in an explicit way. This approach can focus the model on finding valuable regions while estimating their easily detectable and meaningful motion. For further detail enhancement, a lightweight refine module is introduced which enjoys privileges from previous optical flow, selection masks and initial prediction. Moreover, to facilitate learning on samples with large motion, a new window partition cropping method is presented during training. Experiments on public and newly developed challenging datasets show that proposed SAFNet not only exceeds previous SOTA competitors quantitatively and qualitatively, but also runs order of magnitude faster. Code and dataset is available at https://github.com/ltkong218/SAFNet.

Related papers

• RBSR: Efficient and Flexible Recurrent Network for Burst Super-Resolution [57.98314517861539]
  Burst super-resolution (BurstSR) aims at reconstructing a high-resolution (HR) image from a sequence of low-resolution (LR) and noisy images. In this paper, we suggest fusing cues frame-by-frame with an efficient and flexible recurrent network.
  arXiv  Detail & Related papers  (2023-06-30T12:14:13Z)
• Searching a Compact Architecture for Robust Multi-Exposure Image Fusion [55.37210629454589]
  Two major stumbling blocks hinder the development, including pixel misalignment and inefficient inference. This study introduces an architecture search-based paradigm incorporating self-alignment and detail repletion modules for robust multi-exposure image fusion. The proposed method outperforms various competitive schemes, achieving a noteworthy 3.19% improvement in PSNR for general scenarios and an impressive 23.5% enhancement in misaligned scenarios.
  arXiv  Detail & Related papers  (2023-05-20T17:01:52Z)
• SMAE: Few-shot Learning for HDR Deghosting with Saturation-Aware Masked Autoencoders [97.64072440883392]
  We propose a novel semi-supervised approach to realize few-shot HDR imaging via two stages of training, called SSHDR. Unlikely previous methods, directly recovering content and removing ghosts simultaneously, which is hard to achieve optimum. Experiments demonstrate that SSHDR outperforms state-of-the-art methods quantitatively and qualitatively within and across different datasets.
  arXiv  Detail & Related papers  (2023-04-14T03:42:51Z)
• Ultra-High-Definition Low-Light Image Enhancement: A Benchmark and Transformer-Based Method [51.30748775681917]
  We consider the task of low-light image enhancement (LLIE) and introduce a large-scale database consisting of images at 4K and 8K resolution. We conduct systematic benchmarking studies and provide a comparison of current LLIE algorithms. As a second contribution, we introduce LLFormer, a transformer-based low-light enhancement method.
  arXiv  Detail & Related papers  (2022-12-22T09:05:07Z)
• LWGNet: Learned Wirtinger Gradients for Fourier Ptychographic Phase Retrieval [14.588976801396576]
  We propose a hybrid model-driven residual network that combines the knowledge of the forward imaging system with a deep data-driven network. Unlike other conventional unrolling techniques, LWGNet uses fewer stages while performing at par or even better than existing traditional and deep learning techniques. This improvement in performance for low-bit depth and low-cost sensors has the potential to bring down the cost of FPM imaging setup significantly.
  arXiv  Detail & Related papers  (2022-08-08T17:22:54Z)
• HerosNet: Hyperspectral Explicable Reconstruction and Optimal Sampling Deep Network for Snapshot Compressive Imaging [41.91463343106411]
  Hyperspectral imaging is an essential imaging modality for a wide range of applications, especially in remote sensing, agriculture, and medicine. Inspired by existing hyperspectral cameras that are either slow, expensive, or bulky, reconstructing hyperspectral images (HSIs) from a low-budget snapshot measurement has drawn wide attention. Recent deep unfolding networks (DUNs) for spectral snapshot sensing (SCI) have achieved remarkable success. In this paper, we propose a novel Hyperspectral Explicable Reconstruction and Optimal Sampling deep Network for SCI, dubbed HerosNet, which includes several phases under the ISTA-unfolding framework.
  arXiv  Detail & Related papers  (2021-12-12T13:42:49Z)
• Best-Buddy GANs for Highly Detailed Image Super-Resolution [71.13466303340192]
  We consider the single image super-resolution (SISR) problem, where a high-resolution (HR) image is generated based on a low-resolution (LR) input. Most methods along this line rely on a predefined single-LR-single-HR mapping, which is not flexible enough for the SISR task. We propose best-buddy GANs (Beby-GAN) for rich-detail SISR. Relaxing the immutable one-to-one constraint, we allow the estimated patches to dynamically seek the best supervision.
  arXiv  Detail & Related papers  (2021-03-29T02:58:27Z)
• Deep Burst Super-Resolution [165.90445859851448]
  We propose a novel architecture for the burst super-resolution task. Our network takes multiple noisy RAW images as input, and generates a denoised, super-resolved RGB image as output. In order to enable training and evaluation on real-world data, we additionally introduce the BurstSR dataset.
  arXiv  Detail & Related papers  (2021-01-26T18:57:21Z)
• The Mertens Unrolled Network (MU-Net): A High Dynamic Range Fusion Neural Network for Through the Windshield Driver Recognition [1.758759574398869]
  Face recognition in unconstrained environments poses a number of unique challenges ranging from glare, poor illumination, driver pose and motion. We further develop the hardware and software of a custom vehicle imaging system to better overcome these challenges. We name the Mertens Unrolled Network (MU-Net) for the purpose of fine-tuning the HDR output of through-windshield images.
  arXiv  Detail & Related papers  (2020-02-27T16:57:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.