HDRVideo-GAN: Deep Generative HDR Video Reconstruction

• URL: http://arxiv.org/abs/2110.11795v1
• Date: Fri, 22 Oct 2021 14:02:03 GMT
• Title: HDRVideo-GAN: Deep Generative HDR Video Reconstruction
• Authors: Mrinal Anand, Nidhin Harilal, Chandan Kumar, Shanmuganathan Raman
• Abstract summary: We propose an end-to-end GAN-based framework for HDR video reconstruction from LDR sequences with alternating exposures. We first extract clean LDR frames from noisy LDR video with alternating exposures with a denoising network trained in a self-supervised setting. We then align the neighboring alternating-exposure frames to a reference frame and then reconstruct high-quality HDR frames in a complete adversarial setting.
• Score: 19.837271879354184
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: High dynamic range (HDR) videos provide a more visually realistic experience than the standard low dynamic range (LDR) videos. Despite having significant progress in HDR imaging, it is still a challenging task to capture high-quality HDR video with a conventional off-the-shelf camera. Existing approaches rely entirely on using dense optical flow between the neighboring LDR sequences to reconstruct an HDR frame. However, they lead to inconsistencies in color and exposure over time when applied to alternating exposures with noisy frames. In this paper, we propose an end-to-end GAN-based framework for HDR video reconstruction from LDR sequences with alternating exposures. We first extract clean LDR frames from noisy LDR video with alternating exposures with a denoising network trained in a self-supervised setting. Using optical flow, we then align the neighboring alternating-exposure frames to a reference frame and then reconstruct high-quality HDR frames in a complete adversarial setting. To further improve the robustness and quality of generated frames, we incorporate temporal stability-based regularization term along with content and style-based losses in the cost function during the training procedure. Experimental results demonstrate that our framework achieves state-of-the-art performance and generates superior quality HDR frames of a video over the existing methods.

Related papers

• Exposure Completing for Temporally Consistent Neural High Dynamic Range Video Rendering [17.430726543786943]
  We propose a novel paradigm to render HDR frames via completing the absent exposure information. Our approach involves interpolating neighbor LDR frames in the time dimension to reconstruct LDR frames for the absent exposures. This benefits the fusing process for HDR results, reducing noise and ghosting artifacts therefore improving temporal consistency.
  arXiv  Detail & Related papers  (2024-07-18T09:13:08Z)
• Diffusion-Promoted HDR Video Reconstruction [45.73396977607666]
  High dynamic range (LDR) video reconstruction aims to generate HDR videos from low dynamic range (LDR) frames captured with alternating exposures. Most existing works solely rely on the regression-based paradigm, leading to adverse effects such as ghosting artifacts and missing details in saturated regions. We propose a diffusion-promoted method for HDR video reconstruction, termed HDR-V-Diff, which incorporates a diffusion model to capture the HDR distribution.
  arXiv  Detail & Related papers  (2024-06-12T13:38:10Z)
• HDR-GS: Efficient High Dynamic Range Novel View Synthesis at 1000x Speed via Gaussian Splatting [76.5908492298286]
  Existing HDR NVS methods are mainly based on NeRF. They suffer from long training time and slow inference speed. We propose a new framework, High Dynamic Range Gaussian Splatting (-GS)
  arXiv  Detail & Related papers  (2024-05-24T00:46:58Z)
• Event-based Asynchronous HDR Imaging by Temporal Incident Light Modulation [54.64335350932855]
  We propose a Pixel-Asynchronous HDR imaging system, based on key insights into the challenges in HDR imaging. Our proposed Asyn system integrates the Dynamic Vision Sensors (DVS) with a set of LCD panels. The LCD panels modulate the irradiance incident upon the DVS by altering their transparency, thereby triggering the pixel-independent event streams.
  arXiv  Detail & Related papers  (2024-03-14T13:45:09Z)
• Self-Supervised High Dynamic Range Imaging with Multi-Exposure Images in Dynamic Scenes [58.66427721308464]
  Self is a self-supervised reconstruction method that only requires dynamic multi-exposure images during training. Self achieves superior results against the state-of-the-art self-supervised methods, and comparable performance to supervised ones.
  arXiv  Detail & Related papers  (2023-10-03T07:10:49Z)
• Online Overexposed Pixels Hallucination in Videos with Adaptive Reference Frame Selection [90.35085487641773]
  Low dynamic range (LDR) cameras cannot deal with wide dynamic range inputs, frequently leading to local overexposure issues. We present a learning-based system to reduce these artifacts without resorting to complex processing mechanisms.
  arXiv  Detail & Related papers  (2023-08-29T17:40:57Z)
• LAN-HDR: Luminance-based Alignment Network for High Dynamic Range Video Reconstruction [20.911738532410766]
  We propose an end-to-end HDR video composition framework, which aligns LDR frames in feature space and then merges aligned features into an HDR frame. In training, we adopt a temporal loss, in addition to frame reconstruction losses, to enhance temporal consistency and thus reduce flickering.
  arXiv  Detail & Related papers  (2023-08-22T01:43:00Z)
• DeepHS-HDRVideo: Deep High Speed High Dynamic Range Video Reconstruction [23.341594337637545]
  We propose to align the input LDR frames using a pre-trained video frame network. This results in better alignment of LDR frames, since we circumvent the error-prone exposure matching step. We also present the first method to generate high FPS HDR videos.
  arXiv  Detail & Related papers  (2022-10-10T04:27:45Z)
• Snapshot HDR Video Construction Using Coded Mask [25.12198906401246]
  This study utilize 3D-CNNs to perform a joint demosaicking, denoising, and HDR video reconstruction from coded LDR video. The obtained results are promising and could lead to affordable HDR video capture using conventional cameras.
  arXiv  Detail & Related papers  (2021-12-05T09:32:11Z)
• HDR-GAN: HDR Image Reconstruction from Multi-Exposed LDR Images with Large Motions [62.44802076971331]
  We propose a novel GAN-based model, HDR-GAN, for synthesizing HDR images from multi-exposed LDR images. By incorporating adversarial learning, our method is able to produce faithful information in the regions with missing content.
  arXiv  Detail & Related papers  (2020-07-03T11:42:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.