LIPIDS: Learning-based Illumination Planning In Discretized (Light) Space for Photometric Stereo

• URL: http://arxiv.org/abs/2409.02716v1
• Date: Sun, 1 Sep 2024 09:54:16 GMT
• Title: LIPIDS: Learning-based Illumination Planning In Discretized (Light) Space for Photometric Stereo
• Authors: Ashish Tiwari, Mihir Sutariya, Shanmuganathan Raman,
• Abstract summary: Photometric stereo is a powerful method for obtaining per-pixel surface normals from differently illuminated images of an object. Finding an optimal configuration is challenging due to the vast number of possible lighting directions. We introduce LIPIDS - Learning-based Illumination Planning In Discretized light Space.
• Score: 19.021200954913475
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Photometric stereo is a powerful method for obtaining per-pixel surface normals from differently illuminated images of an object. While several methods address photometric stereo with different image (or light) counts ranging from one to two to a hundred, very few focus on learning optimal lighting configuration. Finding an optimal configuration is challenging due to the vast number of possible lighting directions. Moreover, exhaustively sampling all possibilities is impractical due to time and resource constraints. Photometric stereo methods have demonstrated promising performance on existing datasets, which feature limited light directions sparsely sampled from the light space. Therefore, can we optimally utilize these datasets for illumination planning? In this work, we introduce LIPIDS - Learning-based Illumination Planning In Discretized light Space to achieve minimal and optimal lighting configurations for photometric stereo under arbitrary light distribution. We propose a Light Sampling Network (LSNet) that optimizes lighting direction for a fixed number of lights by minimizing the normal loss through a normal regression network. The learned light configurations can directly estimate surface normals during inference, even using an off-the-shelf photometric stereo method. Extensive qualitative and quantitative analyses on synthetic and real-world datasets show that photometric stereo under learned lighting configurations through LIPIDS either surpasses or is nearly comparable to existing illumination planning methods across different photometric stereo backbones.

Related papers

• MERLiN: Single-Shot Material Estimation and Relighting for Photometric Stereo [26.032964551717548]
  Photometric stereo typically demands intricate data acquisition setups involving multiple light sources to recover surface normals accurately. We propose MERLiN, an attention-based hourglass network that integrates single image-based inverse rendering and relighting within a single unified framework.
  arXiv  Detail & Related papers  (2024-09-01T09:32:03Z)
• MixLight: Borrowing the Best of both Spherical Harmonics and Gaussian Models [69.39388799906409]
  Existing works estimate illumination by generating illumination maps or regressing illumination parameters. This paper presents MixLight, a joint model that utilizes the complementary characteristics of SH and SG to achieve a more complete illumination representation.
  arXiv  Detail & Related papers  (2024-04-19T10:17:10Z)
• A CNN Based Approach for the Point-Light Photometric Stereo Problem [26.958763133729846]
  We propose a CNN-based approach capable of handling realistic assumptions by leveraging recent improvements of deep neural networks for far-field Photometric Stereo. Our approach outperforms the state-of-the-art on the DiLiGenT real world dataset. In order to measure the performance of our approach for near-field point-light source PS data, we introduce LUCES the first real-world 'dataset for near-fieLd point light soUrCe photomEtric Stereo'
  arXiv  Detail & Related papers  (2022-10-10T12:57:12Z)
• Self-calibrating Photometric Stereo by Neural Inverse Rendering [88.67603644930466]
  This paper tackles the task of uncalibrated photometric stereo for 3D object reconstruction. We propose a new method that jointly optimize object shape, light directions, and light intensities. Our method demonstrates state-of-the-art accuracy in light estimation and shape recovery on real-world datasets.
  arXiv  Detail & Related papers  (2022-07-16T02:46:15Z)
• Universal Photometric Stereo Network using Global Lighting Contexts [4.822598110892846]
  This paper tackles a new photometric stereo task, named universal photometric stereo. It is supposed to work for objects with diverse shapes and materials under arbitrary lighting variations without assuming any specific models.
  arXiv  Detail & Related papers  (2022-06-06T09:32:06Z)
• Sparse Needlets for Lighting Estimation with Spherical Transport Loss [89.52531416604774]
  NeedleLight is a new lighting estimation model that represents illumination with needlets and allows lighting estimation in both frequency domain and spatial domain jointly. Extensive experiments show that NeedleLight achieves superior lighting estimation consistently across multiple evaluation metrics as compared with state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-06-24T15:19:42Z)
• LUCES: A Dataset for Near-Field Point Light Source Photometric Stereo [30.31403197697561]
  We introduce LUCES, the first real-world 'dataset for near-fieLd point light soUrCe photomEtric Stereo' of 14 objects of a varying of materials. A device counting 52 LEDs has been designed to lit each object positioned 10 to 30 centimeters away from the camera. We evaluate the performance of the latest near-field Photometric Stereo algorithms on the proposed dataset.
  arXiv  Detail & Related papers  (2021-04-27T12:30:42Z)
• GMLight: Lighting Estimation via Geometric Distribution Approximation [86.95367898017358]
  This paper presents a lighting estimation framework that employs a regression network and a generative projector for effective illumination estimation. We parameterize illumination scenes in terms of the geometric light distribution, light intensity, ambient term, and auxiliary depth, and estimate them as a pure regression task. With the estimated lighting parameters, the generative projector synthesizes panoramic illumination maps with realistic appearance and frequency.
  arXiv  Detail & Related papers  (2021-02-20T03:31:52Z)
• Light Stage Super-Resolution: Continuous High-Frequency Relighting [58.09243542908402]
  We propose a learning-based solution for the "super-resolution" of scans of human faces taken from a light stage. Our method aggregates the captured images corresponding to neighboring lights in the stage, and uses a neural network to synthesize a rendering of the face. Our learned model is able to produce renderings for arbitrary light directions that exhibit realistic shadows and specular highlights.
  arXiv  Detail & Related papers  (2020-10-17T23:40:43Z)
• Multi-View Photometric Stereo: A Robust Solution and Benchmark Dataset for Spatially Varying Isotropic Materials [65.95928593628128]
  We present a method to capture both 3D shape and spatially varying reflectance with a multi-view photometric stereo technique. Our algorithm is suitable for perspective cameras and nearby point light sources.
  arXiv  Detail & Related papers  (2020-01-18T12:26:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.