HDRTransDC: High Dynamic Range Image Reconstruction with Transformer Deformation Convolution

• URL: http://arxiv.org/abs/2403.06831v2
• Date: Thu, 29 Aug 2024 11:57:39 GMT
• Title: HDRTransDC: High Dynamic Range Image Reconstruction with Transformer Deformation Convolution
• Authors: Shuaikang Shang, Xuejing Kang, Anlong Ming,
• Abstract summary: High Dynamic Range (CAM) imaging aims to generate an artifact-free HDR image with realistic details by fusing multi-exposure Low Dynamic Range (LDR) images. For the purpose of eliminating fusion distortions, we propose DWFB to spatially adaptively select useful information across frames.
• Score: 21.870772317331447
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: High Dynamic Range (HDR) imaging aims to generate an artifact-free HDR image with realistic details by fusing multi-exposure Low Dynamic Range (LDR) images. Caused by large motion and severe under-/over-exposure among input LDR images, HDR imaging suffers from ghosting artifacts and fusion distortions. To address these critical issues, we propose an HDR Transformer Deformation Convolution (HDRTransDC) network to generate high-quality HDR images, which consists of the Transformer Deformable Convolution Alignment Module (TDCAM) and the Dynamic Weight Fusion Block (DWFB). To solve the ghosting artifacts, the proposed TDCAM extracts long-distance content similar to the reference feature in the entire non-reference features, which can accurately remove misalignment and fill the content occluded by moving objects. For the purpose of eliminating fusion distortions, we propose DWFB to spatially adaptively select useful information across frames to effectively fuse multi-exposed features. Extensive experiments show that our method quantitatively and qualitatively achieves state-of-the-art performance.

Related papers

• Generating Content for HDR Deghosting from Frequency View [56.103761824603644]
  Recent Diffusion Models (DMs) have been introduced in HDR imaging field. DMs require extensive iterations with large models to estimate entire images. We propose the Low-Frequency aware Diffusion (LF-Diff) model for ghost-free HDR imaging.
  arXiv  Detail & Related papers  (2024-04-01T01:32:11Z)
• Towards High-quality HDR Deghosting with Conditional Diffusion Models [88.83729417524823]
  High Dynamic Range (LDR) images can be recovered from several Low Dynamic Range (LDR) images by existing Deep Neural Networks (DNNs) techniques. DNNs still generate ghosting artifacts when LDR images have saturation and large motion. We formulate the HDR deghosting problem as an image generation that leverages LDR features as the diffusion model's condition.
  arXiv  Detail & Related papers  (2023-11-02T01:53:55Z)
• IFT: Image Fusion Transformer for Ghost-free High Dynamic Range Imaging [26.127764855477782]
  Multi-frame high dynamic range ( HDR) imaging aims to reconstruct ghost-free images with photo-realistic details from content-complementary but spatially misaligned low dynamic range (LDR) images. Existing HDR algorithms are prone to producing ghosting artifacts as their methods fail to capture long-range dependencies between LDR frames with large motion in dynamic scenes. We propose a novel image fusion transformer, referred to as IFT, which presents a fast global patch searching (FGPS) module followed by a self-cross fusion module (SCF) for ghost-free HDR imaging.
  arXiv  Detail & Related papers  (2023-09-26T15:38:52Z)
• Alignment-free HDR Deghosting with Semantics Consistent Transformer [76.91669741684173]
  High dynamic range imaging aims to retrieve information from multiple low-dynamic range inputs to generate realistic output. Existing methods often focus on the spatial misalignment across input frames caused by the foreground and/or camera motion. We propose a novel alignment-free network with a Semantics Consistent Transformer (SCTNet) with both spatial and channel attention modules.
  arXiv  Detail & Related papers  (2023-05-29T15:03:23Z)
• Gated Multi-Resolution Transfer Network for Burst Restoration and Enhancement [75.25451566988565]
  We propose a novel Gated Multi-Resolution Transfer Network (GMTNet) to reconstruct a spatially precise high-quality image from a burst of low-quality raw images. Detailed experimental analysis on five datasets validates our approach and sets a state-of-the-art for burst super-resolution, burst denoising, and low-light burst enhancement.
  arXiv  Detail & Related papers  (2023-04-13T17:54:00Z)
• Multi-Exposure HDR Composition by Gated Swin Transformer [8.619880437958525]
  This paper provides a novel multi-exposure fusion model based on Swin Transformer. We exploit the long distance contextual dependency in the exposure-space pyramid by the self-attention mechanism. Experiments show that our model achieves the accuracy on par with current top performing multi-exposure HDR imaging models.
  arXiv  Detail & Related papers  (2023-03-15T15:38:43Z)
• Ghost-free High Dynamic Range Imaging via Hybrid CNN-Transformer and Structure Tensor [12.167049432063132]
  We present a hybrid model consisting of a convolutional encoder and a Transformer decoder to generate ghost-free HDR images. In the encoder, a context aggregation network and non-local attention block are adopted to optimize multi-scale features. The decoder based on Swin Transformer is utilized to improve the reconstruction capability of the proposed model.
  arXiv  Detail & Related papers  (2022-12-01T15:43:32Z)
• Deep Progressive Feature Aggregation Network for High Dynamic Range Imaging [24.94466716276423]
  We propose a deep progressive feature aggregation network for improving HDR imaging quality in dynamic scenes. Our method implicitly samples high-correspondence features and aggregates them in a coarse-to-fine manner for alignment. Experiments show that our proposed method can achieve state-of-the-art performance under different scenes.
  arXiv  Detail & Related papers  (2022-08-04T04:37:35Z)
• HDR-GAN: HDR Image Reconstruction from Multi-Exposed LDR Images with Large Motions [62.44802076971331]
  We propose a novel GAN-based model, HDR-GAN, for synthesizing HDR images from multi-exposed LDR images. By incorporating adversarial learning, our method is able to produce faithful information in the regions with missing content.
  arXiv  Detail & Related papers  (2020-07-03T11:42:35Z)
• Single-Image HDR Reconstruction by Learning to Reverse the Camera Pipeline [100.5353614588565]
  We propose to incorporate the domain knowledge of the LDR image formation pipeline into our model. We model the HDRto-LDR image formation pipeline as the (1) dynamic range clipping, (2) non-linear mapping from a camera response function, and (3) quantization. We demonstrate that the proposed method performs favorably against state-of-the-art single-image HDR reconstruction algorithms.
  arXiv  Detail & Related papers  (2020-04-02T17:59:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.