Learning to See Through Obstructions

• URL: http://arxiv.org/abs/2004.01180v1
• Date: Thu, 2 Apr 2020 17:59:12 GMT
• Title: Learning to See Through Obstructions
• Authors: Yu-Lun Liu, Wei-Sheng Lai, Ming-Hsuan Yang, Yung-Yu Chuang, Jia-Bin Huang
• Abstract summary: We present a learning-based approach for removing unwanted obstructions from a short sequence of images captured by a moving camera. Our method leverages the motion differences between the background and the obstructing elements to recover both layers. We show that training on synthetically generated data transfers well to real images.
• Score: 117.77024641706451
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a learning-based approach for removing unwanted obstructions, such as window reflections, fence occlusions or raindrops, from a short sequence of images captured by a moving camera. Our method leverages the motion differences between the background and the obstructing elements to recover both layers. Specifically, we alternate between estimating dense optical flow fields of the two layers and reconstructing each layer from the flow-warped images via a deep convolutional neural network. The learning-based layer reconstruction allows us to accommodate potential errors in the flow estimation and brittle assumptions such as brightness consistency. We show that training on synthetically generated data transfers well to real images. Our results on numerous challenging scenarios of reflection and fence removal demonstrate the effectiveness of the proposed method.

Related papers

• Neural Spline Fields for Burst Image Fusion and Layer Separation [40.9442467471977]
  We propose a versatile intermediate representation: a two-layer alpha-composited image plus flow model constructed with neural spline fields. Our method is able to jointly fuse a burst image capture into one high-resolution reconstruction and decompose it into transmission and obstruction layers. We find that, with no post-processing steps or learned priors, our generalizable model is able to outperform existing dedicated single-image and multi-view obstruction removal approaches.
  arXiv  Detail & Related papers  (2023-12-21T18:54:19Z)
• Unsupervised Restoration of Weather-affected Images using Deep Gaussian Process-based CycleGAN [92.15895515035795]
  We describe an approach for supervising deep networks that are based on CycleGAN. We introduce new losses for training CycleGAN that lead to more effective training, resulting in high-quality reconstructions. We demonstrate that the proposed method can be effectively applied to different restoration tasks like de-raining, de-hazing and de-snowing.
  arXiv  Detail & Related papers  (2022-04-23T01:30:47Z)
• Iterative Gradient Encoding Network with Feature Co-Occurrence Loss for Single Image Reflection Removal [6.370905925442655]
  We propose an iterative gradient encoding network for single image reflection removal. Our method can remove reflection favorably against the existing state-of-the-art method on all imaging settings.
  arXiv  Detail & Related papers  (2021-03-29T19:29:29Z)
• Unsupervised Single-Image Reflection Separation Using Perceptual Deep Image Priors [6.333390830515411]
  We propose a novel unsupervised framework for single-image reflection separation. We optimize the parameters of two cross-coupled deep convolutional networks on a target image to generate two exclusive background and reflection layers. Our results show that our method significantly outperforms the closest unsupervised method in the literature for removing reflections from single images.
  arXiv  Detail & Related papers  (2020-09-01T21:08:30Z)
• Learning to See Through Obstructions with Layered Decomposition [117.77024641706451]
  We present a learning-based approach for removing unwanted obstructions from moving images. Our method leverages motion differences between the background and obstructing elements to recover both layers. We show that the proposed approach learned from synthetically generated data performs well to real images.
  arXiv  Detail & Related papers  (2020-08-11T17:59:31Z)
• Polarized Reflection Removal with Perfect Alignment in the Wild [66.48211204364142]
  We present a novel formulation to removing reflection from polarized images in the wild. We first identify the misalignment issues of existing reflection removal datasets. We build a new dataset with more than 100 types of glass in which obtained transmission images are perfectly aligned with input mixed images.
  arXiv  Detail & Related papers  (2020-03-28T13:29:31Z)
• Self-Supervised Linear Motion Deblurring [112.75317069916579]
  Deep convolutional neural networks are state-of-the-art for image deblurring. We present a differentiable reblur model for self-supervised motion deblurring. Our experiments demonstrate that self-supervised single image deblurring is really feasible.
  arXiv  Detail & Related papers  (2020-02-10T20:15:21Z)
• Single image reflection removal via learning with multi-image constraints [50.54095311597466]
  We propose a novel learning-based solution that combines the advantages of the aforementioned approaches and overcomes their drawbacks. Our algorithm works by learning a deep neural network to optimize the target with joint constraints enhanced among multiple input images. Our algorithm runs in real-time and state-of-the-art reflection removal performance on real images.
  arXiv  Detail & Related papers  (2019-12-08T06:10:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.