Highlight Specular Reflection Separation based on Tensor Low-rank and Sparse Decomposition Using Polarimetric Cues

• URL: http://arxiv.org/abs/2207.03543v1
• Date: Thu, 7 Jul 2022 19:28:46 GMT
• Title: Highlight Specular Reflection Separation based on Tensor Low-rank and Sparse Decomposition Using Polarimetric Cues
• Authors: Moein Shakeri, Hong Zhang
• Abstract summary: Method is motivated by the observation that the specular highlight of an image is sparsely distributed. We define and impose a new polarization regularization term as constraint on color channels. We demonstrate that our method is able to significantly improve the accuracy of highlight specular removal.
• Score: 7.109720146155956
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper is concerned with specular reflection removal based on tensor low-rank decomposition framework with the help of polarization information. Our method is motivated by the observation that the specular highlight of an image is sparsely distributed while the remaining diffuse reflection can be well approximated by a linear combination of several distinct colors using a low-rank and sparse decomposition framework. Unlike current solutions, our tensor low-rank decomposition keeps the spatial structure of specular and diffuse information which enables us to recover the diffuse image under strong specular reflection or in saturated regions. We further define and impose a new polarization regularization term as constraint on color channels. This regularization boosts the performance of the method to recover an accurate diffuse image by handling the color distortion, a common problem of chromaticity-based methods, especially in case of strong specular reflection. Through comprehensive experiments on both synthetic and real polarization images, we demonstrate that our method is able to significantly improve the accuracy of highlight specular removal, and outperform the competitive methods to recover the diffuse image, especially in regions of strong specular reflection or in saturated areas.

Related papers

• Photometric Inverse Rendering: Shading Cues Modeling and Surface Reflectance Regularization [46.146783750386994]
  We propose a new method for neural inverse rendering. Our method jointly optimize the light source position to account for the self-shadows in images. To enhance surface reflectance decomposition, we introduce a new regularization.
  arXiv  Detail & Related papers  (2024-08-13T11:39:14Z)
• SS-SfP:Neural Inverse Rendering for Self Supervised Shape from (Mixed) Polarization [21.377923666134116]
  Shape from Polarization (SfP) is the problem popularly known as Shape from Polarization (SfP) We present a novel inverse rendering-based framework to estimate the 3D shape (per-pixel surface normals and depth) of objects and scenes from single-view polarization images.
  arXiv  Detail & Related papers  (2024-07-12T14:29:00Z)
• LightenDiffusion: Unsupervised Low-Light Image Enhancement with Latent-Retinex Diffusion Models [39.28266945709169]
  We propose a diffusion-based unsupervised framework that incorporates physically explainable Retinex theory with diffusion models for low-light image enhancement. Experiments on publicly available real-world benchmarks show that the proposed LightenDiffusion outperforms state-of-the-art unsupervised competitors.
  arXiv  Detail & Related papers  (2024-07-12T02:54:43Z)
• Global Structure-Aware Diffusion Process for Low-Light Image Enhancement [64.69154776202694]
  This paper studies a diffusion-based framework to address the low-light image enhancement problem. We advocate for the regularization of its inherent ODE-trajectory. Experimental evaluations reveal that the proposed framework attains distinguished performance in low-light enhancement.
  arXiv  Detail & Related papers  (2023-10-26T17:01:52Z)
• Fearless Luminance Adaptation: A Macro-Micro-Hierarchical Transformer for Exposure Correction [65.5397271106534]
  A single neural network is difficult to handle all exposure problems. In particular, convolutions hinder the ability to restore faithful color or details on extremely over-/under- exposed regions. We propose a Macro-Micro-Hierarchical transformer, which consists of a macro attention to capture long-range dependencies, a micro attention to extract local features, and a hierarchical structure for coarse-to-fine correction.
  arXiv  Detail & Related papers  (2023-09-02T09:07:36Z)
• TensoIR: Tensorial Inverse Rendering [51.57268311847087]
  TensoIR is a novel inverse rendering approach based on tensor factorization and neural fields. TensoRF is a state-of-the-art approach for radiance field modeling.
  arXiv  Detail & Related papers  (2023-04-24T21:39:13Z)
• Invariant Descriptors for Intrinsic Reflectance Optimization [15.638996731038231]
  Intrinsic image decomposition aims to factorize an image into albedo (reflectance) and shading (illumination) sub-components. Being ill-posed and under-constrained, it is a very challenging computer vision problem. Our approach is physics-based, learning-free and leads to more accurate and robust reflectance decompositions.
  arXiv  Detail & Related papers  (2022-04-08T13:52:13Z)
• Low-Light Image Enhancement with Normalizing Flow [92.52290821418778]
  In this paper, we investigate to model this one-to-many relationship via a proposed normalizing flow model. An invertible network that takes the low-light images/features as the condition and learns to map the distribution of normally exposed images into a Gaussian distribution. The experimental results on the existing benchmark datasets show our method achieves better quantitative and qualitative results, obtaining better-exposed illumination, less noise and artifact, and richer colors.
  arXiv  Detail & Related papers  (2021-09-13T12:45:08Z)
• Polarization Guided Specular Reflection Separation [8.280173807482901]
  We derive a polarization guided model to incorporate the polarization information into a designed optimization separation strategy. We show that our method can faithfully separate the specular reflection, especially in some challenging scenarios.
  arXiv  Detail & Related papers  (2021-03-22T08:22:28Z)
• Polarized Reflection Removal with Perfect Alignment in the Wild [66.48211204364142]
  We present a novel formulation to removing reflection from polarized images in the wild. We first identify the misalignment issues of existing reflection removal datasets. We build a new dataset with more than 100 types of glass in which obtained transmission images are perfectly aligned with input mixed images.
  arXiv  Detail & Related papers  (2020-03-28T13:29:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.