Exposure Fusion for Hand-held Camera Inputs with Optical Flow and PatchMatch

• URL: http://arxiv.org/abs/2304.04464v1
• Date: Mon, 10 Apr 2023 09:06:37 GMT
• Title: Exposure Fusion for Hand-held Camera Inputs with Optical Flow and PatchMatch
• Authors: Ru Li, Guanghui Liu, Bing Zeng, Shuaicheng Liu
• Abstract summary: We propose a hybrid synthesis method for multi-exposure image fusion taken by hand-held cameras. Our method can deal with such motions and maintain the exposure information of each input effectively. Experiment results demonstrate the effectiveness and robustness of our method.
• Score: 53.149395644547226
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper proposes a hybrid synthesis method for multi-exposure image fusion taken by hand-held cameras. Motions either due to the shaky camera or caused by dynamic scenes should be compensated before any content fusion. Any misalignment can easily cause blurring/ghosting artifacts in the fused result. Our hybrid method can deal with such motions and maintain the exposure information of each input effectively. In particular, the proposed method first applies optical flow for a coarse registration, which performs well with complex non-rigid motion but produces deformations at regions with missing correspondences. The absence of correspondences is due to the occlusions of scene parallax or the moving contents. To correct such error registration, we segment images into superpixels and identify problematic alignments based on each superpixel, which is further aligned by PatchMatch. The method combines the efficiency of optical flow and the accuracy of PatchMatch. After PatchMatch correction, we obtain a fully aligned image stack that facilitates a high-quality fusion that is free from blurring/ghosting artifacts. We compare our method with existing fusion algorithms on various challenging examples, including the static/dynamic, the indoor/outdoor and the daytime/nighttime scenes. Experiment results demonstrate the effectiveness and robustness of our method.

Related papers

• Decomposition-based and Interference Perception for Infrared and Visible Image Fusion in Complex Scenes [4.919706769234434]
  We propose a decomposition-based and interference perception image fusion method. We classify the pixels of visible image from the degree of scattering of light transmission, based on which we then separate the detail and energy information of the image. This refined decomposition facilitates the proposed model in identifying more interfering pixels that are in complex scenes.
  arXiv  Detail & Related papers  (2024-02-03T09:27:33Z)
• Neural Spline Fields for Burst Image Fusion and Layer Separation [40.9442467471977]
  We propose a versatile intermediate representation: a two-layer alpha-composited image plus flow model constructed with neural spline fields. Our method is able to jointly fuse a burst image capture into one high-resolution reconstruction and decompose it into transmission and obstruction layers. We find that, with no post-processing steps or learned priors, our generalizable model is able to outperform existing dedicated single-image and multi-view obstruction removal approaches.
  arXiv  Detail & Related papers  (2023-12-21T18:54:19Z)
• Searching a Compact Architecture for Robust Multi-Exposure Image Fusion [55.37210629454589]
  Two major stumbling blocks hinder the development, including pixel misalignment and inefficient inference. This study introduces an architecture search-based paradigm incorporating self-alignment and detail repletion modules for robust multi-exposure image fusion. The proposed method outperforms various competitive schemes, achieving a noteworthy 3.19% improvement in PSNR for general scenarios and an impressive 23.5% enhancement in misaligned scenarios.
  arXiv  Detail & Related papers  (2023-05-20T17:01:52Z)
• Joint Video Multi-Frame Interpolation and Deblurring under Unknown Exposure Time [101.91824315554682]
  In this work, we aim ambitiously for a more realistic and challenging task - joint video multi-frame and deblurring under unknown exposure time. We first adopt a variant of supervised contrastive learning to construct an exposure-aware representation from input blurred frames. We then build our video reconstruction network upon the exposure and motion representation by progressive exposure-adaptive convolution and motion refinement.
  arXiv  Detail & Related papers  (2023-03-27T09:43:42Z)
• Parallax-Tolerant Unsupervised Deep Image Stitching [57.76737888499145]
  We propose UDIS++, a parallax-tolerant unsupervised deep image stitching technique. First, we propose a robust and flexible warp to model the image registration from global homography to local thin-plate spline motion. To further eliminate the parallax artifacts, we propose to composite the stitched image seamlessly by unsupervised learning for seam-driven composition masks.
  arXiv  Detail & Related papers  (2023-02-16T10:40:55Z)
• Time Lens++: Event-based Frame Interpolation with Parametric Non-linear Flow and Multi-scale Fusion [47.57998625129672]
  We introduce multi-scale feature-level fusion and computing one-shot non-linear inter-frame motion from events and images. We show that our method improves the reconstruction quality by up to 0.2 dB in terms of PSNR and up to 15% in LPIPS score.
  arXiv  Detail & Related papers  (2022-03-31T17:14:58Z)
• Neural Re-rendering for Full-frame Video Stabilization [144.9918806873405]
  We present an algorithm for full-frame video stabilization by first estimating dense warp fields. Full-frame stabilized frames can then be synthesized by fusing warped contents from neighboring frames.
  arXiv  Detail & Related papers  (2021-02-11T18:59:45Z)
• Pixel-Wise Motion Deblurring of Thermal Videos [26.6875886332029]
  Uncooled microbolometers can enable robots to see in the absence of visible illumination by imaging the "heat" radiated from the scene. Despite this ability to see in the dark, these sensors suffer from significant motion blur. This paper formulates reversing the effect of thermal inertia at a single pixel as a Least Absolute Shrinkage and Selection Operator (LASSO) problem.
  arXiv  Detail & Related papers  (2020-06-08T22:35:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.