Self-supervised HDR Imaging from Motion and Exposure Cues

• URL: http://arxiv.org/abs/2203.12311v1
• Date: Wed, 23 Mar 2022 10:22:03 GMT
• Title: Self-supervised HDR Imaging from Motion and Exposure Cues
• Authors: Michal Nazarczuk and Sibi Catley-Chandar and Ales Leonardis and Eduardo P\'erez Pellitero
• Abstract summary: We propose a novel self-supervised approach for learnable HDR estimation that alleviates the need for HDR ground-truth labels. Experimental results show that the HDR models trained using our proposed self-supervision approach achieve performance competitive with those trained under full supervision.
• Score: 14.57046548797279
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent High Dynamic Range (HDR) techniques extend the capabilities of current cameras where scenes with a wide range of illumination can not be accurately captured with a single low-dynamic-range (LDR) image. This is generally accomplished by capturing several LDR images with varying exposure values whose information is then incorporated into a merged HDR image. While such approaches work well for static scenes, dynamic scenes pose several challenges, mostly related to the difficulty of finding reliable pixel correspondences. Data-driven approaches tackle the problem by learning an end-to-end mapping with paired LDR-HDR training data, but in practice generating such HDR ground-truth labels for dynamic scenes is time-consuming and requires complex procedures that assume control of certain dynamic elements of the scene (e.g. actor pose) and repeatable lighting conditions (stop-motion capturing). In this work, we propose a novel self-supervised approach for learnable HDR estimation that alleviates the need for HDR ground-truth labels. We propose to leverage the internal statistics of LDR images to create HDR pseudo-labels. We separately exploit static and well-exposed parts of the input images, which in conjunction with synthetic illumination clipping and motion augmentation provide high quality training examples. Experimental results show that the HDR models trained using our proposed self-supervision approach achieve performance competitive with those trained under full supervision, and are to a large extent superior to previous methods that equally do not require any supervision.

Related papers

• HDR Imaging for Dynamic Scenes with Events [2.750189317612375]
  We propose an Event-based HDRI framework within a Self-supervised learning paradigm, which generalizes HDRI performance in real-world dynamic scenarios. A self-supervised learning strategy is carried out by learning cross-domain conversions from blurry LDR images to sharp LDR images. We construct large-scale synthetic and real-world datasets to evaluate the effectiveness of our method.
  arXiv  Detail & Related papers  (2024-04-04T05:33:06Z)
• Event-based Asynchronous HDR Imaging by Temporal Incident Light Modulation [54.64335350932855]
  We propose a Pixel-Asynchronous HDR imaging system, based on key insights into the challenges in HDR imaging. Our proposed Asyn system integrates the Dynamic Vision Sensors (DVS) with a set of LCD panels. The LCD panels modulate the irradiance incident upon the DVS by altering their transparency, thereby triggering the pixel-independent event streams.
  arXiv  Detail & Related papers  (2024-03-14T13:45:09Z)
• Self-Supervised High Dynamic Range Imaging with Multi-Exposure Images in Dynamic Scenes [58.66427721308464]
  Self is a self-supervised reconstruction method that only requires dynamic multi-exposure images during training. Self achieves superior results against the state-of-the-art self-supervised methods, and comparable performance to supervised ones.
  arXiv  Detail & Related papers  (2023-10-03T07:10:49Z)
• RawHDR: High Dynamic Range Image Reconstruction from a Single Raw Image [36.17182977927645]
  High dynamic range (RGB) images capture much more intensity levels than standard ones. Current methods predominantly generate HDR images from 8-bit low dynamic range (LDR) s images that have been degraded by the camera processing pipeline. Unlike existing methods, the core idea of this work is to incorporate more informative Raw sensor data to generate HDR images.
  arXiv  Detail & Related papers  (2023-09-05T07:58:21Z)
• SMAE: Few-shot Learning for HDR Deghosting with Saturation-Aware Masked Autoencoders [97.64072440883392]
  We propose a novel semi-supervised approach to realize few-shot HDR imaging via two stages of training, called SSHDR. Unlikely previous methods, directly recovering content and removing ghosts simultaneously, which is hard to achieve optimum. Experiments demonstrate that SSHDR outperforms state-of-the-art methods quantitatively and qualitatively within and across different datasets.
  arXiv  Detail & Related papers  (2023-04-14T03:42:51Z)
• GlowGAN: Unsupervised Learning of HDR Images from LDR Images in the Wild [74.52723408793648]
  We present the first method for learning a generative model of HDR images from in-the-wild LDR image collections in a fully unsupervised manner. The key idea is to train a generative adversarial network (GAN) to generate HDR images which, when projected to LDR under various exposures, are indistinguishable from real LDR images. Experiments show that our method GlowGAN can synthesize photorealistic HDR images in many challenging cases such as landscapes, lightning, or windows.
  arXiv  Detail & Related papers  (2022-11-22T15:42:08Z)
• HDR-cGAN: Single LDR to HDR Image Translation using Conditional GAN [24.299931323012757]
  Low Dynamic Range (LDR) cameras are incapable of representing the wide dynamic range of the real-world scene. We propose a deep learning based approach to recover details in the saturated areas while reconstructing the HDR image. We present a novel conditional GAN (cGAN) based framework trained in an end-to-end fashion over the HDR-REAL and HDR-SYNTH datasets.
  arXiv  Detail & Related papers  (2021-10-04T18:50:35Z)
• Beyond Visual Attractiveness: Physically Plausible Single Image HDR Reconstruction for Spherical Panoramas [60.24132321381606]
  We introduce the physical illuminance constraints to our single-shot HDR reconstruction framework. Our method can generate HDRs which are not only visually appealing but also physically plausible.
  arXiv  Detail & Related papers  (2021-03-24T01:51:19Z)
• HDR-GAN: HDR Image Reconstruction from Multi-Exposed LDR Images with Large Motions [62.44802076971331]
  We propose a novel GAN-based model, HDR-GAN, for synthesizing HDR images from multi-exposed LDR images. By incorporating adversarial learning, our method is able to produce faithful information in the regions with missing content.
  arXiv  Detail & Related papers  (2020-07-03T11:42:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.