Colorful Diffuse Intrinsic Image Decomposition in the Wild

• URL: http://arxiv.org/abs/2409.13690v1
• Date: Fri, 20 Sep 2024 17:59:40 GMT
• Title: Colorful Diffuse Intrinsic Image Decomposition in the Wild
• Authors: Chris Careaga, Yağız Aksoy,
• Abstract summary: Intrinsic image decomposition aims to separate the surface reflectance and the effects from the illumination given a single photograph. In this work, we separate an input image into its diffuse albedo, colorful diffuse shading, and specular residual components. Our extended intrinsic model enables illumination-aware analysis of photographs and can be used for image editing applications.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Intrinsic image decomposition aims to separate the surface reflectance and the effects from the illumination given a single photograph. Due to the complexity of the problem, most prior works assume a single-color illumination and a Lambertian world, which limits their use in illumination-aware image editing applications. In this work, we separate an input image into its diffuse albedo, colorful diffuse shading, and specular residual components. We arrive at our result by gradually removing first the single-color illumination and then the Lambertian-world assumptions. We show that by dividing the problem into easier sub-problems, in-the-wild colorful diffuse shading estimation can be achieved despite the limited ground-truth datasets. Our extended intrinsic model enables illumination-aware analysis of photographs and can be used for image editing applications such as specularity removal and per-pixel white balancing.

Related papers

• Attentive Illumination Decomposition Model for Multi-Illuminant White Balancing [27.950125640986805]
  White balance (WB) algorithms in many commercial cameras assume single and uniform illumination. We present a deep white balancing model that leverages the slot attention, where each slot is in charge of representing individual illuminants. This design enables the model to generate chromaticities and weight maps for individual illuminants, which are then fused to compose the final illumination map.
  arXiv  Detail & Related papers  (2024-02-28T12:15:29Z)
• Diffusion Posterior Illumination for Ambiguity-aware Inverse Rendering [63.24476194987721]
  Inverse rendering, the process of inferring scene properties from images, is a challenging inverse problem. Most existing solutions incorporate priors into the inverse-rendering pipeline to encourage plausible solutions. We propose a novel scheme that integrates a denoising probabilistic diffusion model pre-trained on natural illumination maps into an optimization framework.
  arXiv  Detail & Related papers  (2023-09-30T12:39:28Z)
• Towards High-Quality Specular Highlight Removal by Leveraging Large-Scale Synthetic Data [45.30068102110486]
  This paper aims to remove specular highlights from a single object-level image. We propose a three-stage network to address them. We present a large-scale synthetic dataset of object-level images.
  arXiv  Detail & Related papers  (2023-09-12T15:10:23Z)
• Physically-Based Editing of Indoor Scene Lighting from a Single Image [106.60252793395104]
  We present a method to edit complex indoor lighting from a single image with its predicted depth and light source segmentation masks. We tackle this problem using two novel components: 1) a holistic scene reconstruction method that estimates scene reflectance and parametric 3D lighting, and 2) a neural rendering framework that re-renders the scene from our predictions.
  arXiv  Detail & Related papers  (2022-05-19T06:44:37Z)
• Invariant Descriptors for Intrinsic Reflectance Optimization [15.638996731038231]
  Intrinsic image decomposition aims to factorize an image into albedo (reflectance) and shading (illumination) sub-components. Being ill-posed and under-constrained, it is a very challenging computer vision problem. Our approach is physics-based, learning-free and leads to more accurate and robust reflectance decompositions.
  arXiv  Detail & Related papers  (2022-04-08T13:52:13Z)
• Self-Supervised Intrinsic Image Decomposition Network Considering Reflectance Consistency [23.300763504208593]
  Intrinsic image decomposition aims to decompose an image into illumination-invariant and illumination-variant components. Our network can decompose images into reflectance and shading components.
  arXiv  Detail & Related papers  (2021-11-05T07:52:06Z)
• DIB-R++: Learning to Predict Lighting and Material with a Hybrid Differentiable Renderer [78.91753256634453]
  We consider the challenging problem of predicting intrinsic object properties from a single image by exploiting differentiables. In this work, we propose DIBR++, a hybrid differentiable which supports these effects by combining specularization and ray-tracing. Compared to more advanced physics-based differentiables, DIBR++ is highly performant due to its compact and expressive model.
  arXiv  Detail & Related papers  (2021-10-30T01:59:39Z)
• Auto White-Balance Correction for Mixed-Illuminant Scenes [52.641704254001844]
  Auto white balance (AWB) is applied by camera hardware to remove color cast caused by scene illumination. This paper presents an effective AWB method to deal with such mixed-illuminant scenes. Our method does not require illuminant estimation, as is the case in traditional camera AWB modules.
  arXiv  Detail & Related papers  (2021-09-17T20:13:31Z)
• Intrinsic Image Transfer for Illumination Manipulation [1.2387676601792899]
  This paper presents a novel intrinsic image transfer (IIT) algorithm for illumination manipulation. It creates a local image translation between two illumination surfaces. We illustrate that all losses can be reduced without the necessity of taking an intrinsic image decomposition.
  arXiv  Detail & Related papers  (2021-07-01T19:12:24Z)
• The Cube++ Illumination Estimation Dataset [50.58610459038332]
  A new illumination estimation dataset is proposed in this paper. It consists of 4890 images with known illumination colors as well as with additional semantic data. The dataset can be used for training and testing of methods that perform single or two-illuminant estimation.
  arXiv  Detail & Related papers  (2020-11-19T18:50:08Z)
• Shape, Illumination, and Reflectance from Shading [86.71603503678216]
  A fundamental problem in computer vision is that of inferring the intrinsic, 3D structure of the world from flat, 2D images. We find that certain explanations are more likely than others: surfaces tend to be smooth, paint tends to be uniform, and illumination tends to be natural. Our technique can be viewed as a superset of several classic computer vision problems.
  arXiv  Detail & Related papers  (2020-10-07T18:14:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.