NeuralMPS: Non-Lambertian Multispectral Photometric Stereo via Spectral Reflectance Decomposition

• URL: http://arxiv.org/abs/2211.15311v1
• Date: Mon, 28 Nov 2022 13:46:17 GMT
• Title: NeuralMPS: Non-Lambertian Multispectral Photometric Stereo via Spectral Reflectance Decomposition
• Authors: Jipeng Lv, Heng Guo, Guanying Chen, Jinxiu Liang and Boxin Shi
• Abstract summary: We propose a deep neural network named NeuralMPS to solve the MPS problem under general non-Lambertian spectral reflectances. We show that the MPS problem for surfaces with a uniform material is equivalent to the conventional photometric stereo(CPS) with unknown light intensities.
• Score: 47.5182946590776
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multispectral photometric stereo(MPS) aims at recovering the surface normal of a scene from a single-shot multispectral image captured under multispectral illuminations. Existing MPS methods adopt the Lambertian reflectance model to make the problem tractable, but it greatly limits their application to real-world surfaces. In this paper, we propose a deep neural network named NeuralMPS to solve the MPS problem under general non-Lambertian spectral reflectances. Specifically, we present a spectral reflectance decomposition(SRD) model to disentangle the spectral reflectance into geometric components and spectral components. With this decomposition, we show that the MPS problem for surfaces with a uniform material is equivalent to the conventional photometric stereo(CPS) with unknown light intensities. In this way, NeuralMPS reduces the difficulty of the non-Lambertian MPS problem by leveraging the well-studied non-Lambertian CPS methods. Experiments on both synthetic and real-world scenes demonstrate the effectiveness of our method.

Related papers

• Adaptive Stereo Depth Estimation with Multi-Spectral Images Across All Lighting Conditions [58.88917836512819]
  We propose a novel framework incorporating stereo depth estimation to enforce accurate geometric constraints. To mitigate the effects of poor lighting on stereo matching, we introduce Degradation Masking. Our method achieves state-of-the-art (SOTA) performance on the Multi-Spectral Stereo (MS2) dataset.
  arXiv  Detail & Related papers  (2024-11-06T03:30:46Z)
• Physics-Free Spectrally Multiplexed Photometric Stereo under Unknown Spectral Composition [10.620997969499205]
  We present a groundbreaking spectrally multiplexed photometric stereo approach for recovering surface normals without the need for calibrated lighting or sensors. By embracing spectral ambiguity as an advantage, our technique enables the generation of training data without specialized multispectral rendering frameworks. We introduce a unique, physics-free network architecture, SpectraM-PS, that effectively processes multiplexed images to determine surface normals.
  arXiv  Detail & Related papers  (2024-10-28T04:03:37Z)
• NeISF: Neural Incident Stokes Field for Geometry and Material Estimation [50.588983686271284]
  Multi-view inverse rendering is the problem of estimating the scene parameters such as shapes, materials, or illuminations from a sequence of images captured under different viewpoints. We propose Neural Incident Stokes Fields (NeISF), a multi-view inverse framework that reduces ambiguities using polarization cues.
  arXiv  Detail & Related papers  (2023-11-22T06:28:30Z)
• Unsupervised Spectral Demosaicing with Lightweight Spectral Attention Networks [6.7433262627741914]
  This paper presents a deep learning-based spectral demosaicing technique trained in an unsupervised manner. The proposed method outperforms conventional unsupervised methods in terms of spatial distortion suppression, spectral fidelity, robustness, and computational cost.
  arXiv  Detail & Related papers  (2023-07-05T02:45:44Z)
• Deep Learning Methods for Calibrated Photometric Stereo and Beyond [86.57469194387264]
  Photometric stereo recovers the surface normals of an object from multiple images with varying shading cues. Deep learning methods have shown a powerful ability in the context of photometric stereo against non-Lambertian surfaces.
  arXiv  Detail & Related papers  (2022-12-16T11:27:44Z)
• Uncertainty-Aware Deep Multi-View Photometric Stereo [100.97116470055273]
  Photometric stereo (PS) is excellent at recovering high-frequency surface details, whereas multi-view stereo (MVS) can help remove the low-frequency distortion due to PS and retain the global shape. This paper proposes an approach that can effectively utilize such complementary strengths of PS and MVS. We estimate per-pixel surface normals and depth using an uncertainty-aware deep-PS network and deep-MVS network, respectively.
  arXiv  Detail & Related papers  (2022-02-26T05:45:52Z)
• Spectral Splitting and Aggregation Network for Hyperspectral Face Super-Resolution [82.59267937569213]
  High-resolution (HR) hyperspectral face image plays an important role in face related computer vision tasks under uncontrolled conditions. In this paper, we investigate how to adapt the deep learning techniques to hyperspectral face image super-resolution. We present a spectral splitting and aggregation network (SSANet) for HFSR with limited training samples.
  arXiv  Detail & Related papers  (2021-08-31T02:13:00Z)
• LADMM-Net: An Unrolled Deep Network For Spectral Image Fusion From Compressive Data [6.230751621285322]
  Hyperspectral (HS) and multispectral (MS) image fusion aims at estimating a high-resolution spectral image from a low-spatial-resolution HS image and a low-spectral-resolution MS image. In this work, a deep learning architecture under the algorithm unrolling approach is proposed for solving the fusion problem from HS and MS compressive measurements.
  arXiv  Detail & Related papers  (2021-03-01T12:04:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.