Dense Dispersed Structured Light for Hyperspectral 3D Imaging of Dynamic Scenes

• URL: http://arxiv.org/abs/2412.01140v1
• Date: Mon, 02 Dec 2024 05:30:18 GMT
• Title: Dense Dispersed Structured Light for Hyperspectral 3D Imaging of Dynamic Scenes
• Authors: Suhyun Shin, Seungwoo Yoon, Ryota Maeda, Seung-Hwan Baek,
• Abstract summary: Hyperspectral 3D imaging captures both depth maps and hyperspectral images, enabling geometric and material analysis.<n>Recent methods achieve high spectral and depth accuracy; however, they require long acquisition times often over several minutes or rely on large, expensive systems.<n>We present an accurate hyperspectral 3D imaging method for dynamic scenes that utilizes stereo RGB cameras and an affordable diffraction grating film.
• Score: 9.050557698554696
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Hyperspectral 3D imaging captures both depth maps and hyperspectral images, enabling comprehensive geometric and material analysis. Recent methods achieve high spectral and depth accuracy; however, they require long acquisition times often over several minutes or rely on large, expensive systems, restricting their use to static scenes. We present Dense Dispersed Structured Light (DDSL), an accurate hyperspectral 3D imaging method for dynamic scenes that utilizes stereo RGB cameras and an RGB projector equipped with an affordable diffraction grating film. We design spectrally multiplexed DDSL patterns that significantly reduce the number of required projector patterns, thereby accelerating acquisition speed. Additionally, we formulate an image formation model and a reconstruction method to estimate a hyperspectral image and depth map from captured stereo images. As the first practical and accurate hyperspectral 3D imaging method for dynamic scenes, we experimentally demonstrate that DDSL achieves a spectral resolution of 15.5 nm full width at half maximum (FWHM), a depth error of 4 mm, and a frame rate of 6.6 fps.

Related papers

• High-Speed Dynamic 3D Imaging with Sensor Fusion Splatting [15.309934457166394]
  Capturing and reconstructing high-speed dynamic 3D scenes has numerous applications in computer graphics, vision, and interdisciplinary fields such as robotics, aerodynamics, and evolutionary biology. Traditional RGB cameras suffer from low frame rates, limited exposure times, and narrow baselines. We propose a novel sensor fusion approach using Gaussian splatting, which combines RGB, depth, and event cameras to capture and reconstruct scenes at high speeds.
  arXiv  Detail & Related papers  (2025-02-07T03:17:31Z)
• Stereo4D: Learning How Things Move in 3D from Internet Stereo Videos [76.07894127235058]
  We present a system for mining high-quality 4D reconstructions from internet stereoscopic, wide-angle videos. We use this method to generate large-scale data in the form of world-consistent, pseudo-metric 3D point clouds. We demonstrate the utility of this data by training a variant of DUSt3R to predict structure and 3D motion from real-world image pairs.
  arXiv  Detail & Related papers  (2024-12-12T18:59:54Z)
• MM3DGS SLAM: Multi-modal 3D Gaussian Splatting for SLAM Using Vision, Depth, and Inertial Measurements [59.70107451308687]
  We show for the first time that using 3D Gaussians for map representation with unposed camera images and inertial measurements can enable accurate SLAM. Our method, MM3DGS, addresses the limitations of prior rendering by enabling faster scale awareness, and improved trajectory tracking. We also release a multi-modal dataset, UT-MM, collected from a mobile robot equipped with a camera and an inertial measurement unit.
  arXiv  Detail & Related papers  (2024-04-01T04:57:41Z)
• Dispersed Structured Light for Hyperspectral 3D Imaging [39.23448863407426]
  Dispersed Structured Light (DSL) is a cost-effective and compact method for accurate hyperspectral 3D imaging. DSL achieves spectral accuracy of 18.8nm full-width half-millimeter and depth error of 1mm.
  arXiv  Detail & Related papers  (2023-11-30T06:45:52Z)
• R3D3: Dense 3D Reconstruction of Dynamic Scenes from Multiple Cameras [106.52409577316389]
  R3D3 is a multi-camera system for dense 3D reconstruction and ego-motion estimation. Our approach exploits spatial-temporal information from multiple cameras, and monocular depth refinement. We show that this design enables a dense, consistent 3D reconstruction of challenging, dynamic outdoor environments.
  arXiv  Detail & Related papers  (2023-08-28T17:13:49Z)
• Shakes on a Plane: Unsupervised Depth Estimation from Unstabilized Photography [54.36608424943729]
  We show that in a ''long-burst'', forty-two 12-megapixel RAW frames captured in a two-second sequence, there is enough parallax information from natural hand tremor alone to recover high-quality scene depth. We devise a test-time optimization approach that fits a neural RGB-D representation to long-burst data and simultaneously estimates scene depth and camera motion.
  arXiv  Detail & Related papers  (2022-12-22T18:54:34Z)
• Hyperspectral 3D Mapping of Underwater Environments [0.7087237546722617]
  We present an initial method for creating hyperspectral 3D reconstructions of underwater environments. By fusing the data gathered by a classical RGB camera, an inertial navigation system and a hyperspectral push-broom camera, we show that the proposed method creates highly accurate 3D reconstructions with hyperspectral textures.
  arXiv  Detail & Related papers  (2021-10-13T08:37:22Z)
• Neural Radiance Fields Approach to Deep Multi-View Photometric Stereo [103.08512487830669]
  We present a modern solution to the multi-view photometric stereo problem (MVPS) We procure the surface orientation using a photometric stereo (PS) image formation model and blend it with a multi-view neural radiance field representation to recover the object's surface geometry. Our method performs neural rendering of multi-view images while utilizing surface normals estimated by a deep photometric stereo network.
  arXiv  Detail & Related papers  (2021-10-11T20:20:03Z)
• Lightweight Multi-View 3D Pose Estimation through Camera-Disentangled Representation [57.11299763566534]
  We present a solution to recover 3D pose from multi-view images captured with spatially calibrated cameras. We exploit 3D geometry to fuse input images into a unified latent representation of pose, which is disentangled from camera view-points. Our architecture then conditions the learned representation on camera projection operators to produce accurate per-view 2d detections.
  arXiv  Detail & Related papers  (2020-04-05T12:52:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.