Physics-Free Spectrally Multiplexed Photometric Stereo under Unknown Spectral Composition

• URL: http://arxiv.org/abs/2410.20716v1
• Date: Mon, 28 Oct 2024 04:03:37 GMT
• Title: Physics-Free Spectrally Multiplexed Photometric Stereo under Unknown Spectral Composition
• Authors: Satoshi Ikehata, Yuta Asano,
• Abstract summary: We present a groundbreaking spectrally multiplexed photometric stereo approach for recovering surface normals without the need for calibrated lighting or sensors. By embracing spectral ambiguity as an advantage, our technique enables the generation of training data without specialized multispectral rendering frameworks. We introduce a unique, physics-free network architecture, SpectraM-PS, that effectively processes multiplexed images to determine surface normals.
• Score: 10.620997969499205
• License:
• Abstract: In this paper, we present a groundbreaking spectrally multiplexed photometric stereo approach for recovering surface normals of dynamic surfaces without the need for calibrated lighting or sensors, a notable advancement in the field traditionally hindered by stringent prerequisites and spectral ambiguity. By embracing spectral ambiguity as an advantage, our technique enables the generation of training data without specialized multispectral rendering frameworks. We introduce a unique, physics-free network architecture, SpectraM-PS, that effectively processes multiplexed images to determine surface normals across a wide range of conditions and material types, without relying on specific physically-based knowledge. Additionally, we establish the first benchmark dataset, SpectraM14, for spectrally multiplexed photometric stereo, facilitating comprehensive evaluations against existing calibrated methods. Our contributions significantly enhance the capabilities for dynamic surface recovery, particularly in uncalibrated setups, marking a pivotal step forward in the application of photometric stereo across various domains.

Related papers

• Adaptive Stereo Depth Estimation with Multi-Spectral Images Across All Lighting Conditions [58.88917836512819]
  We propose a novel framework incorporating stereo depth estimation to enforce accurate geometric constraints. To mitigate the effects of poor lighting on stereo matching, we introduce Degradation Masking. Our method achieves state-of-the-art (SOTA) performance on the Multi-Spectral Stereo (MS2) dataset.
  arXiv  Detail & Related papers  (2024-11-06T03:30:46Z)
• Correlation Hyperspectral Imaging [0.0]
  Hyperspectral imaging aims at providing information on both the spatial and the spectral distribution of light, with high resolution. We address this limitation by exploiting light intensity correlations, which are shown to enable overcoming the typical downsides of traditional hyperspectral imaging techniques. The enabled combination of high spatial and spectral resolution, high speed, and insensitivity to undesired spectral features shall lead to a paradigm change in hyperspectral imaging devices.
  arXiv  Detail & Related papers  (2024-07-18T19:54:44Z)
• RMAFF-PSN: A Residual Multi-Scale Attention Feature Fusion Photometric Stereo Network [37.759675702107586]
  Predicting accurate maps of objects from two-dimensional images in regions of complex structure spatial material variations is challenging. We propose a method of calibrated feature information from different resolution stages and scales of the image. This approach preserves more physical information, such as texture and geometry of the object in complex regions.
  arXiv  Detail & Related papers  (2024-04-11T14:05:37Z)
• SpectralGPT: Spectral Remote Sensing Foundation Model [60.023956954916414]
  A universal RS foundation model, named SpectralGPT, is purpose-built to handle spectral RS images using a novel 3D generative pretrained transformer (GPT) Compared to existing foundation models, SpectralGPT accommodates input images with varying sizes, resolutions, time series, and regions in a progressive training fashion, enabling full utilization of extensive RS big data. Our evaluation highlights significant performance improvements with pretrained SpectralGPT models, signifying substantial potential in advancing spectral RS big data applications within the field of geoscience.
  arXiv  Detail & Related papers  (2023-11-13T07:09:30Z)
• Unsupervised Spectral Demosaicing with Lightweight Spectral Attention Networks [6.7433262627741914]
  This paper presents a deep learning-based spectral demosaicing technique trained in an unsupervised manner. The proposed method outperforms conventional unsupervised methods in terms of spatial distortion suppression, spectral fidelity, robustness, and computational cost.
  arXiv  Detail & Related papers  (2023-07-05T02:45:44Z)
• Deep Learning Methods for Calibrated Photometric Stereo and Beyond [86.57469194387264]
  Photometric stereo recovers the surface normals of an object from multiple images with varying shading cues. Deep learning methods have shown a powerful ability in the context of photometric stereo against non-Lambertian surfaces.
  arXiv  Detail & Related papers  (2022-12-16T11:27:44Z)
• NeuralMPS: Non-Lambertian Multispectral Photometric Stereo via Spectral Reflectance Decomposition [47.5182946590776]
  We propose a deep neural network named NeuralMPS to solve the MPS problem under general non-Lambertian spectral reflectances. We show that the MPS problem for surfaces with a uniform material is equivalent to the conventional photometric stereo(CPS) with unknown light intensities.
  arXiv  Detail & Related papers  (2022-11-28T13:46:17Z)
• Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction [127.20208645280438]
  Hyperspectral image (HSI) reconstruction aims to recover the 3D spatial-spectral signal from a 2D measurement. Modeling the inter-spectra interactions is beneficial for HSI reconstruction. Mask-guided Spectral-wise Transformer (MST) proposes a novel framework for HSI reconstruction.
  arXiv  Detail & Related papers  (2021-11-15T16:59:48Z)
• Non-Photorealistic Rendering of Layered Materials: A Multispectral Approach [0.0]
  We present multispectral rendering techniques for visualizing layered materials found in biological specimens. We are the first to use acquired data from the near-infrared and ultraviolet spectra for non-photorealistic rendering (NPR)
  arXiv  Detail & Related papers  (2021-09-02T08:40:05Z)
• Learning Inter- and Intra-frame Representations for Non-Lambertian Photometric Stereo [14.5172791293107]
  We build a two-stage Convolutional Neural Network (CNN) architecture to construct inter- and intra-frame representations. We experimentally investigate numerous network design alternatives for identifying the optimal scheme to deploy inter-frame and intra-frame feature extraction modules.
  arXiv  Detail & Related papers  (2020-12-26T11:22:56Z)
• Hyperspectral-Multispectral Image Fusion with Weighted LASSO [68.04032419397677]
  We propose an approach for fusing hyperspectral and multispectral images to provide high-quality hyperspectral output. We demonstrate that the proposed sparse fusion and reconstruction provides quantitatively superior results when compared to existing methods on publicly available images.
  arXiv  Detail & Related papers  (2020-03-15T23:07:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.