HDR Imaging for Dynamic Scenes with Events

• URL: http://arxiv.org/abs/2404.03210v1
• Date: Thu, 4 Apr 2024 05:33:06 GMT
• Title: HDR Imaging for Dynamic Scenes with Events
• Authors: Li Xiaopeng, Zeng Zhaoyuan, Fan Cien, Zhao Chen, Deng Lei, Yu Lei,
• Abstract summary: We propose an Event-based HDRI framework within a Self-supervised learning paradigm, which generalizes HDRI performance in real-world dynamic scenarios. A self-supervised learning strategy is carried out by learning cross-domain conversions from blurry LDR images to sharp LDR images. We construct large-scale synthetic and real-world datasets to evaluate the effectiveness of our method.
• Score: 2.750189317612375
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: High dynamic range imaging (HDRI) for real-world dynamic scenes is challenging because moving objects may lead to hybrid degradation of low dynamic range and motion blur. Existing event-based approaches only focus on a separate task, while cascading HDRI and motion deblurring would lead to sub-optimal solutions, and unavailable ground-truth sharp HDR images aggravate the predicament. To address these challenges, we propose an Event-based HDRI framework within a Self-supervised learning paradigm, i.e., Self-EHDRI, which generalizes HDRI performance in real-world dynamic scenarios. Specifically, a self-supervised learning strategy is carried out by learning cross-domain conversions from blurry LDR images to sharp LDR images, which enables sharp HDR images to be accessible in the intermediate process even though ground-truth sharp HDR images are missing. Then, we formulate the event-based HDRI and motion deblurring model and conduct a unified network to recover the intermediate sharp HDR results, where both the high dynamic range and high temporal resolution of events are leveraged simultaneously for compensation. We construct large-scale synthetic and real-world datasets to evaluate the effectiveness of our method. Comprehensive experiments demonstrate that the proposed Self-EHDRI outperforms state-of-the-art approaches by a large margin. The codes, datasets, and results are available at https://lxp-whu.github.io/Self-EHDRI.

Related papers

• HistoHDR-Net: Histogram Equalization for Single LDR to HDR Image Translation [12.45632443397018]
  High Dynamic Range ( HDR) imaging aims to replicate the high visual quality and clarity of real-world scenes. The literature offers various data-driven methods for HDR image reconstruction from Low Dynamic Range (LDR) counterparts. A common limitation of these approaches is missing details in regions of the reconstructed HDR images. We propose a simple and effective method, Histo-Net, to recover the fine details.
  arXiv  Detail & Related papers  (2024-02-08T20:14:46Z)
• Towards High-quality HDR Deghosting with Conditional Diffusion Models [88.83729417524823]
  High Dynamic Range (LDR) images can be recovered from several Low Dynamic Range (LDR) images by existing Deep Neural Networks (DNNs) techniques. DNNs still generate ghosting artifacts when LDR images have saturation and large motion. We formulate the HDR deghosting problem as an image generation that leverages LDR features as the diffusion model's condition.
  arXiv  Detail & Related papers  (2023-11-02T01:53:55Z)
• Self-Supervised High Dynamic Range Imaging with Multi-Exposure Images in Dynamic Scenes [58.66427721308464]
  Self is a self-supervised reconstruction method that only requires dynamic multi-exposure images during training. Self achieves superior results against the state-of-the-art self-supervised methods, and comparable performance to supervised ones.
  arXiv  Detail & Related papers  (2023-10-03T07:10:49Z)
• SMAE: Few-shot Learning for HDR Deghosting with Saturation-Aware Masked Autoencoders [97.64072440883392]
  We propose a novel semi-supervised approach to realize few-shot HDR imaging via two stages of training, called SSHDR. Unlikely previous methods, directly recovering content and removing ghosts simultaneously, which is hard to achieve optimum. Experiments demonstrate that SSHDR outperforms state-of-the-art methods quantitatively and qualitatively within and across different datasets.
  arXiv  Detail & Related papers  (2023-04-14T03:42:51Z)
• GlowGAN: Unsupervised Learning of HDR Images from LDR Images in the Wild [74.52723408793648]
  We present the first method for learning a generative model of HDR images from in-the-wild LDR image collections in a fully unsupervised manner. The key idea is to train a generative adversarial network (GAN) to generate HDR images which, when projected to LDR under various exposures, are indistinguishable from real LDR images. Experiments show that our method GlowGAN can synthesize photorealistic HDR images in many challenging cases such as landscapes, lightning, or windows.
  arXiv  Detail & Related papers  (2022-11-22T15:42:08Z)
• Self-supervised HDR Imaging from Motion and Exposure Cues [14.57046548797279]
  We propose a novel self-supervised approach for learnable HDR estimation that alleviates the need for HDR ground-truth labels. Experimental results show that the HDR models trained using our proposed self-supervision approach achieve performance competitive with those trained under full supervision.
  arXiv  Detail & Related papers  (2022-03-23T10:22:03Z)
• HDR-cGAN: Single LDR to HDR Image Translation using Conditional GAN [24.299931323012757]
  Low Dynamic Range (LDR) cameras are incapable of representing the wide dynamic range of the real-world scene. We propose a deep learning based approach to recover details in the saturated areas while reconstructing the HDR image. We present a novel conditional GAN (cGAN) based framework trained in an end-to-end fashion over the HDR-REAL and HDR-SYNTH datasets.
  arXiv  Detail & Related papers  (2021-10-04T18:50:35Z)
• NTIRE 2021 Challenge on High Dynamic Range Imaging: Dataset, Methods and Results [56.932867490888015]
  This paper reviews the first challenge on high-dynamic range imaging that was part of the New Trends in Image Restoration and Enhancement (NTIRE) workshop, held in conjunction with CVPR 2021. The challenge aims at estimating a HDR image from one or multiple respective low-dynamic range (LDR) observations, which might suffer from under- or over-exposed regions and different sources of noise.
  arXiv  Detail & Related papers  (2021-06-02T19:45:16Z)
• HDR-GAN: HDR Image Reconstruction from Multi-Exposed LDR Images with Large Motions [62.44802076971331]
  We propose a novel GAN-based model, HDR-GAN, for synthesizing HDR images from multi-exposed LDR images. By incorporating adversarial learning, our method is able to produce faithful information in the regions with missing content.
  arXiv  Detail & Related papers  (2020-07-03T11:42:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.