Related papers: Learning to See Through Obstructions with Layered Decomposition

Learning to See Through Obstructions with Layered Decomposition

URL: http://arxiv.org/abs/2008.04902v3
Date: Sun, 25 Jul 2021 06:47:11 GMT
Title: Learning to See Through Obstructions with Layered Decomposition
Authors: Yu-Lun Liu, Wei-Sheng Lai, Ming-Hsuan Yang, Yung-Yu Chuang, Jia-Bin Huang
Abstract summary: We present a learning-based approach for removing unwanted obstructions from moving images. Our method leverages motion differences between the background and obstructing elements to recover both layers. We show that the proposed approach learned from synthetically generated data performs well to real images.
Score: 117.77024641706451
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: We present a learning-based approach for removing unwanted obstructions, such as window reflections, fence occlusions, or adherent raindrops, from a short sequence of images captured by a moving camera. Our method leverages motion differences between the background and obstructing elements to recover both layers. Specifically, we alternate between estimating dense optical flow fields of the two layers and reconstructing each layer from the flow-warped images via a deep convolutional neural network. This learning-based layer reconstruction module facilitates accommodating potential errors in the flow estimation and brittle assumptions, such as brightness consistency. We show that the proposed approach learned from synthetically generated data performs well to real images. Experimental results on numerous challenging scenarios of reflection and fence removal demonstrate the effectiveness of the proposed method.

Related papers

Oscillation Inversion: Understand the structure of Large Flow Model through the Lens of Inversion Method [60.88467353578118]
We show that a fixed-point-inspired iterative approach to invert real-world images does not achieve convergence, instead oscillating between distinct clusters. We introduce a simple and fast distribution transfer technique that facilitates image enhancement, stroke-based recoloring, as well as visual prompt-guided image editing.
arXiv Detail & Related papers (2024-11-17T17:45:37Z)
Neural Spline Fields for Burst Image Fusion and Layer Separation [40.9442467471977]
We propose a versatile intermediate representation: a two-layer alpha-composited image plus flow model constructed with neural spline fields. Our method is able to jointly fuse a burst image capture into one high-resolution reconstruction and decompose it into transmission and obstruction layers. We find that, with no post-processing steps or learned priors, our generalizable model is able to outperform existing dedicated single-image and multi-view obstruction removal approaches.
arXiv Detail & Related papers (2023-12-21T18:54:19Z)
Iterative Gradient Encoding Network with Feature Co-Occurrence Loss for Single Image Reflection Removal [6.370905925442655]
We propose an iterative gradient encoding network for single image reflection removal. Our method can remove reflection favorably against the existing state-of-the-art method on all imaging settings.
arXiv Detail & Related papers (2021-03-29T19:29:29Z)
Location-aware Single Image Reflection Removal [54.93808224890273]
This paper proposes a novel location-aware deep learning-based single image reflection removal method. We use a reflection confidence map as the cues for the network to learn how to encode the reflection information adaptively. The integration of location information into the network significantly improves the quality of reflection removal results.
arXiv Detail & Related papers (2020-12-13T19:34:35Z)
Unsupervised Single-Image Reflection Separation Using Perceptual Deep Image Priors [6.333390830515411]
We propose a novel unsupervised framework for single-image reflection separation. We optimize the parameters of two cross-coupled deep convolutional networks on a target image to generate two exclusive background and reflection layers. Our results show that our method significantly outperforms the closest unsupervised method in the literature for removing reflections from single images.
arXiv Detail & Related papers (2020-09-01T21:08:30Z)
Learning to See Through Obstructions [117.77024641706451]
We present a learning-based approach for removing unwanted obstructions from a short sequence of images captured by a moving camera. Our method leverages the motion differences between the background and the obstructing elements to recover both layers. We show that training on synthetically generated data transfers well to real images.
arXiv Detail & Related papers (2020-04-02T17:59:12Z)
Polarized Reflection Removal with Perfect Alignment in the Wild [66.48211204364142]
We present a novel formulation to removing reflection from polarized images in the wild. We first identify the misalignment issues of existing reflection removal datasets. We build a new dataset with more than 100 types of glass in which obtained transmission images are perfectly aligned with input mixed images.
arXiv Detail & Related papers (2020-03-28T13:29:31Z)
Single image reflection removal via learning with multi-image constraints [50.54095311597466]
We propose a novel learning-based solution that combines the advantages of the aforementioned approaches and overcomes their drawbacks. Our algorithm works by learning a deep neural network to optimize the target with joint constraints enhanced among multiple input images. Our algorithm runs in real-time and state-of-the-art reflection removal performance on real images.
arXiv Detail & Related papers (2019-12-08T06:10:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.