End-to-End Differentiable Learning to HDR Image Synthesis for Multi-exposure Images

• URL: http://arxiv.org/abs/2006.15833v2
• Date: Fri, 18 Dec 2020 07:19:13 GMT
• Title: End-to-End Differentiable Learning to HDR Image Synthesis for Multi-exposure Images
• Authors: Jung Hee Kim, Siyeong Lee, and Suk-Ju Kang
• Abstract summary: High dynamic range () image reconstruction based on the multiple exposure stack from a given single exposure utilizes a deep learning framework to generate high-quality HDR images. We tackle the problem in stack reconstruction-based methods by proposing a novel framework with a fully differentiable high dynamic range imaging (I) process. In other words, our differentiable HDR synthesis layer helps the deep neural network to train to create multi-exposure stacks while reflecting the precise correlations between multi-exposure images in the HDRI process.
• Score: 23.895981099137533
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recently, high dynamic range (HDR) image reconstruction based on the multiple exposure stack from a given single exposure utilizes a deep learning framework to generate high-quality HDR images. These conventional networks focus on the exposure transfer task to reconstruct the multi-exposure stack. Therefore, they often fail to fuse the multi-exposure stack into a perceptually pleasant HDR image as the inversion artifacts occur. We tackle the problem in stack reconstruction-based methods by proposing a novel framework with a fully differentiable high dynamic range imaging (HDRI) process. By explicitly using the loss, which compares the network's output with the ground truth HDR image, our framework enables a neural network that generates the multiple exposure stack for HDRI to train stably. In other words, our differentiable HDR synthesis layer helps the deep neural network to train to create multi-exposure stacks while reflecting the precise correlations between multi-exposure images in the HDRI process. In addition, our network uses the image decomposition and the recursive process to facilitate the exposure transfer task and to adaptively respond to recursion frequency. The experimental results show that the proposed network outperforms the state-of-the-art quantitative and qualitative results in terms of both the exposure transfer tasks and the whole HDRI process.

Related papers

• HDRT: Infrared Capture for HDR Imaging [8.208995723545502]
  We propose a new approach, High Dynamic Range Thermal (HDRT), for HDR acquisition using a separate, commonly available, thermal infrared (IR) sensor. We propose a novel deep neural method (HDRTNet) which combines IR and SDR content to generate HDR images. We show substantial quantitative and qualitative quality improvements on both over- and under-exposed images, showing that our approach is robust to capturing in multiple different lighting conditions.
  arXiv  Detail & Related papers  (2024-06-08T13:43:44Z)
• Generating Content for HDR Deghosting from Frequency View [56.103761824603644]
  Recent Diffusion Models (DMs) have been introduced in HDR imaging field. DMs require extensive iterations with large models to estimate entire images. We propose the Low-Frequency aware Diffusion (LF-Diff) model for ghost-free HDR imaging.
  arXiv  Detail & Related papers  (2024-04-01T01:32:11Z)
• Self-Supervised High Dynamic Range Imaging with Multi-Exposure Images in Dynamic Scenes [58.66427721308464]
  Self is a self-supervised reconstruction method that only requires dynamic multi-exposure images during training. Self achieves superior results against the state-of-the-art self-supervised methods, and comparable performance to supervised ones.
  arXiv  Detail & Related papers  (2023-10-03T07:10:49Z)
• SMAE: Few-shot Learning for HDR Deghosting with Saturation-Aware Masked Autoencoders [97.64072440883392]
  We propose a novel semi-supervised approach to realize few-shot HDR imaging via two stages of training, called SSHDR. Unlikely previous methods, directly recovering content and removing ghosts simultaneously, which is hard to achieve optimum. Experiments demonstrate that SSHDR outperforms state-of-the-art methods quantitatively and qualitatively within and across different datasets.
  arXiv  Detail & Related papers  (2023-04-14T03:42:51Z)
• GlowGAN: Unsupervised Learning of HDR Images from LDR Images in the Wild [74.52723408793648]
  We present the first method for learning a generative model of HDR images from in-the-wild LDR image collections in a fully unsupervised manner. The key idea is to train a generative adversarial network (GAN) to generate HDR images which, when projected to LDR under various exposures, are indistinguishable from real LDR images. Experiments show that our method GlowGAN can synthesize photorealistic HDR images in many challenging cases such as landscapes, lightning, or windows.
  arXiv  Detail & Related papers  (2022-11-22T15:42:08Z)
• Single-Image HDR Reconstruction by Multi-Exposure Generation [8.656080193351581]
  We propose a weakly supervised learning method that inverts the physical image formation process for HDR reconstruction. Our neural network can invert the camera response to reconstruct pixel irradiance before synthesizing multiple exposures. Our experiments show that our proposed model can effectively reconstruct HDR images.
  arXiv  Detail & Related papers  (2022-10-28T05:12:56Z)
• FlexHDR: Modelling Alignment and Exposure Uncertainties for Flexible HDR Imaging [0.9185931275245008]
  We present a new HDR imaging technique that models alignment and exposure uncertainties to produce high quality HDR results. We introduce a strategy that learns to jointly align and assess the alignment and exposure reliability using an HDR-aware, uncertainty-driven attention map. Experimental results show our method can produce better quality HDR images with up to 0.8dB PSNR improvement to the state-of-the-art.
  arXiv  Detail & Related papers  (2022-01-07T14:27:17Z)
• A Two-stage Deep Network for High Dynamic Range Image Reconstruction [0.883717274344425]
  This study tackles the challenges of single-shot LDR to HDR mapping by proposing a novel two-stage deep network. Notably, our proposed method aims to reconstruct an HDR image without knowing hardware information, including camera response function (CRF) and exposure settings.
  arXiv  Detail & Related papers  (2021-04-19T15:19:17Z)
• HDR-GAN: HDR Image Reconstruction from Multi-Exposed LDR Images with Large Motions [62.44802076971331]
  We propose a novel GAN-based model, HDR-GAN, for synthesizing HDR images from multi-exposed LDR images. By incorporating adversarial learning, our method is able to produce faithful information in the regions with missing content.
  arXiv  Detail & Related papers  (2020-07-03T11:42:35Z)
• Single-Image HDR Reconstruction by Learning to Reverse the Camera Pipeline [100.5353614588565]
  We propose to incorporate the domain knowledge of the LDR image formation pipeline into our model. We model the HDRto-LDR image formation pipeline as the (1) dynamic range clipping, (2) non-linear mapping from a camera response function, and (3) quantization. We demonstrate that the proposed method performs favorably against state-of-the-art single-image HDR reconstruction algorithms.
  arXiv  Detail & Related papers  (2020-04-02T17:59:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.