Spectral 3D Computer Vision -- A Review

• URL: http://arxiv.org/abs/2302.08054v1
• Date: Thu, 16 Feb 2023 03:29:40 GMT
• Title: Spectral 3D Computer Vision -- A Review
• Authors: Yajie Sun and Ali Zia and Vivien Rolland and Charissa Yu and Jun Zhou
• Abstract summary: Spectral 3D computer vision examines both the geometric and spectral properties of objects. This emerging paradigm advances traditional computer vision and opens new avenues of research in 3D structure, depth estimation, motion analysis, and more. It has found applications in areas such as smart agriculture, environment monitoring, building inspection, geological exploration, and digital cultural heritage records.
• Score: 5.385154980085054
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Spectral 3D computer vision examines both the geometric and spectral properties of objects. It provides a deeper understanding of an object's physical properties by providing information from narrow bands in various regions of the electromagnetic spectrum. Mapping the spectral information onto the 3D model reveals changes in the spectra-structure space or enhances 3D representations with properties such as reflectance, chromatic aberration, and varying defocus blur. This emerging paradigm advances traditional computer vision and opens new avenues of research in 3D structure, depth estimation, motion analysis, and more. It has found applications in areas such as smart agriculture, environment monitoring, building inspection, geological exploration, and digital cultural heritage records. This survey offers a comprehensive overview of spectral 3D computer vision, including a unified taxonomy of methods, key application areas, and future challenges and prospects.

Related papers

• Advances in Feed-Forward 3D Reconstruction and View Synthesis: A Survey [154.50661618628433]
  3D reconstruction and view synthesis are foundational problems in computer vision, graphics, and immersive technologies such as augmented reality (AR), virtual reality (VR), and digital twins.<n>Recent advances in feed-forward approaches, driven by deep learning, have revolutionized this field by enabling fast and generalizable 3D reconstruction and view synthesis.
  arXiv  Detail & Related papers  (2025-07-19T06:13:25Z)
• The Role of Cyclopean-Eye in Stereo Vision [3.336618863186337]
  This work investigates how 3D structure and human-inspired perception contribute to accurate depth reconstruction.<n>We revisit the Cyclopean Eye model and propose novel geometric constraints that account for occlusions and depth discontinuities.
  arXiv  Detail & Related papers  (2025-06-26T00:11:26Z)
• VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstruction [86.82819259860186]
  We introduce VLM-3R, a unified framework for Vision-Language Models (VLMs) that incorporates 3D Reconstructive instruction tuning.<n>VLM-3R processes monocular video frames by employing a geometry encoder to derive implicit 3D tokens that represent spatial understanding.
  arXiv  Detail & Related papers  (2025-05-26T17:56:30Z)
• GAA-TSO: Geometry-Aware Assisted Depth Completion for Transparent and Specular Objects [5.227530976208222]
  We propose a geometry-aware assisted depth completion method for transparent and specular objects. Specifically, we back-project the input depth to a point cloud and build the 3D branch to extract hierarchical scene-level 3D structural features. Our method significantly enhances the performance of downstream robotic grasping tasks.
  arXiv  Detail & Related papers  (2025-03-21T12:46:38Z)
• Back to the Future Cyclopean Stereo: a human perception approach combining deep and geometric constraints [3.336618863186337]
  We provide analytical 3D surface models as viewed by a cyclopean eye model. This geometrical foundation combined with learned stereo features allows our system to benefit from the strengths of both approaches. Our approach aims to demonstrate that understanding and modeling geometrical properties of 3D surfaces is beneficial to computer vision research.
  arXiv  Detail & Related papers  (2025-02-28T17:58:20Z)
• Textured Mesh Saliency: Bridging Geometry and Texture for Human Perception in 3D Graphics [50.23625950905638]
  We present a new dataset for textured mesh saliency, created through an innovative eye-tracking experiment in a six degrees of freedom (6-DOF) VR environment. Our proposed model predicts saliency maps for textured mesh surfaces by treating each triangular face as an individual unit and assigning a saliency density value to reflect the importance of each local surface region.
  arXiv  Detail & Related papers  (2024-12-11T08:27:33Z)
• 3D Representation Methods: A Survey [0.0]
  3D representation has experienced significant advancements, driven by the increasing demand for high-fidelity 3D models in various applications. This review examines the development and current state of 3D representation methods, highlighting their research trajectories, innovations, strength and weakness.
  arXiv  Detail & Related papers  (2024-10-09T02:01:05Z)
• SpectralGaussians: Semantic, spectral 3D Gaussian splatting for multi-spectral scene representation, visualization and analysis [3.8834382997684087]
  We propose a novel cross-spectral rendering framework based on 3D Gaussian Splatting (3DGS) This framework generates realistic and semantically meaningful splats from registered multi-view spectrum and segmentation maps. Our contributions address challenges in multi-spectral scene representation, rendering, and editing, offering new possibilities for diverse applications.
  arXiv  Detail & Related papers  (2024-08-13T15:32:54Z)
• Survey on Modeling of Human-made Articulated Objects [15.296495542958496]
  3D modeling of articulated objects is a research problem within computer vision, graphics, and robotics. This survey provides a comprehensive overview of the current state-of-the-art in 3D modeling of articulated objects.
  arXiv  Detail & Related papers  (2024-03-22T03:47:02Z)
• Towards Unified 3D Object Detection via Algorithm and Data Unification [70.27631528933482]
  We build the first unified multi-modal 3D object detection benchmark MM- Omni3D and extend the aforementioned monocular detector to its multi-modal version. We name the designed monocular and multi-modal detectors as UniMODE and MM-UniMODE, respectively.
  arXiv  Detail & Related papers  (2024-02-28T18:59:31Z)
• Semantically-aware Neural Radiance Fields for Visual Scene Understanding: A Comprehensive Review [26.436253160392123]
  Review thoroughly examines the role of semantically-aware Neural Radiance Fields (NeRFs) in visual scene understanding. NeRFs adeptly infer 3D representations for both stationary and dynamic objects in a scene.
  arXiv  Detail & Related papers  (2024-02-17T00:15:09Z)
• A Survey on 3D Gaussian Splatting [51.96747208581275]
  3D Gaussian splatting (GS) has emerged as a transformative technique in the realm of explicit radiance field and computer graphics. We provide the first systematic overview of the recent developments and critical contributions in the domain of 3D GS. By enabling unprecedented rendering speed, 3D GS opens up a plethora of applications, ranging from virtual reality to interactive media and beyond.
  arXiv  Detail & Related papers  (2024-01-08T13:42:59Z)
• MinD-3D: Reconstruct High-quality 3D objects in Human Brain [50.534007259536715]
  Recon3DMind is an innovative task aimed at reconstructing 3D visuals from Functional Magnetic Resonance Imaging (fMRI) signals. We present the fMRI-Shape dataset, which includes data from 14 participants and features 360-degree videos of 3D objects. We propose MinD-3D, a novel and effective three-stage framework specifically designed to decode the brain's 3D visual information from fMRI signals.
  arXiv  Detail & Related papers  (2023-12-12T18:21:36Z)
• SpectralGPT: Spectral Remote Sensing Foundation Model [60.023956954916414]
  A universal RS foundation model, named SpectralGPT, is purpose-built to handle spectral RS images using a novel 3D generative pretrained transformer (GPT) Compared to existing foundation models, SpectralGPT accommodates input images with varying sizes, resolutions, time series, and regions in a progressive training fashion, enabling full utilization of extensive RS big data. Our evaluation highlights significant performance improvements with pretrained SpectralGPT models, signifying substantial potential in advancing spectral RS big data applications within the field of geoscience.
  arXiv  Detail & Related papers  (2023-11-13T07:09:30Z)
• MagicDrive: Street View Generation with Diverse 3D Geometry Control [82.69871576797166]
  We introduce MagicDrive, a novel street view generation framework, offering diverse 3D geometry controls. Our design incorporates a cross-view attention module, ensuring consistency across multiple camera views.
  arXiv  Detail & Related papers  (2023-10-04T06:14:06Z)
• CMR3D: Contextualized Multi-Stage Refinement for 3D Object Detection [57.44434974289945]
  We propose Contextualized Multi-Stage Refinement for 3D Object Detection (CMR3D) framework. Our framework takes a 3D scene as input and strives to explicitly integrate useful contextual information of the scene. In addition to 3D object detection, we investigate the effectiveness of our framework for the problem of 3D object counting.
  arXiv  Detail & Related papers  (2022-09-13T05:26:09Z)
• Spectral MVIR: Joint Reconstruction of 3D Shape and Spectral Reflectance [15.600451871592107]
  We present a rendering model that considers both geometric and photometric principles in the image formation. We build a cost-optimization MVIR framework for the joint reconstruction of the 3D shape and the per-vertex spectral reflectance.
  arXiv  Detail & Related papers  (2021-04-15T08:36:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.