3D-VCD: Hallucination Mitigation in 3D-LLM Embodied Agents through Visual Contrastive Decoding

• URL: http://arxiv.org/abs/2604.08645v1
• Date: Thu, 09 Apr 2026 17:57:35 GMT
• Title: 3D-VCD: Hallucination Mitigation in 3D-LLM Embodied Agents through Visual Contrastive Decoding
• Abstract summary: We introduce 3D-VCD, the first inference-time visual contrastive decoding framework for hallucination mitigation in 3D embodied agents.<n>3D-VCD constructs a distorted 3D scene graph by applying semantic and geometric perturbations to object-centric representations.<n>We show that it consistently improves grounded reasoning without any retraining.
• Score: 4.6658306510829
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large multimodal models are increasingly used as the reasoning core of embodied agents operating in 3D environments, yet they remain prone to hallucinations that can produce unsafe and ungrounded decisions. Existing inference-time hallucination mitigation methods largely target 2D vision-language settings and do not transfer to embodied 3D reasoning, where failures arise from object presence, spatial layout, and geometric grounding rather than pixel-level inconsistencies. We introduce 3D-VCD, the first inference-time visual contrastive decoding framework for hallucination mitigation in 3D embodied agents. 3D-VCD constructs a distorted 3D scene graph by applying semantic and geometric perturbations to object-centric representations, such as category substitutions and coordinate or extent corruption. By contrasting predictions under the original and distorted 3D contexts, our method suppresses tokens that are insensitive to grounded scene evidence and are therefore likely driven by language priors. We evaluate 3D-VCD on the 3D-POPE and HEAL benchmarks and show that it consistently improves grounded reasoning without any retraining, establishing inference-time contrastive decoding over structured 3D representations as an effective and practical route to more reliable embodied intelligence.

Related papers

• 3DWay: Generalizing Robot Manipulation via 3D Consistent Waypoints [92.67359514804048]
  Intermediate representations are key to bridging the modality gap between generalizable manipulation policies and large-scale pretrained vision-language models.<n>We propose predicting 3D consistent waypoints (3DWay) from multi-view images.<n>By reformulating 3D waypoints prediction as generating multi-view consistent 2D waypoints followed by geometric triangulation, we enable explicit 3D motion specification.
  arXiv  Detail & Related papers  (2026-09-08T04:18:46Z)
• Map-Det3D: Metric Feed-Forward 3D Reconstruction Prior for Multi-view 3D Object Detection from Streaming Inputs [62.865932175477035]
  Metric 3D object detection is a core capability for embodied agents, yet most reliable systems lean on depth sensors.<n>We propose Map-Det3D, an online multi-view 3D object detection model that brings detection directly into a 3D space reconstructed from RGB.
  arXiv  Detail & Related papers  (2026-08-12T15:33:42Z)
• Occ-VLM: Occupancy Grounded Vision Language Model for Indoor Scene Understanding [31.76161781831759]
  We propose Occ-VLM, a novel framework for 3D scene understanding that operates purely on posed RGB images and employs a single 2D vision encoder.<n>Occ-VLM achieves both accurate geometric perception and robust vision-language reasoning.
  arXiv  Detail & Related papers  (2026-06-18T04:24:28Z)
• 3DThinkVLA: Endowing Vision-Language-Action Models with Latent 3D Priors via 3D-Thinking-Guided Co-training [19.5808550016589]
  3D geometry perception and 3D spatial reasoning are distinct capabilities that can be disentangled and injected at different feature hierarchies.<n>We propose a 3D-thinking-guided co-training framework that enables vision-language-action (VLA) models to perform 3D spatial reasoning implicitly during action prediction.
  arXiv  Detail & Related papers  (2026-06-03T04:34:07Z)
• 3D-IDE: 3D Implicit Depth Emergent [45.72771473431863]
  3D-Implicit Depth Emergence is a method that reframes 3D perception as an emergent property derived from geometric self-vision.<n>Our method enables 3D perception to emerge implicitly, disentangling features in dense regions.<n>Our approach achieves a 55% reduction in inference latency while maintaining strong performance across diverse downstream tasks.
  arXiv  Detail & Related papers  (2026-03-28T00:54:19Z)
• OneWorld: Taming Scene Generation with 3D Unified Representation Autoencoder [90.8453349494245]
  We present OneWorld, a framework that performs diffusion directly within a coherent 3D representation space.<n>OneWorld generates high-quality 3D scenes with superior cross-view consistency compared to state-of-the-art 2D-based methods.
  arXiv  Detail & Related papers  (2026-03-17T03:43:37Z)
• 3D CoCa v2: Contrastive Learners with Test-Time Search for Generalizable Spatial Intelligence [15.064925965953122]
  3D captioning aims to describe 3D scenes in natural language.<n>We propose 3D CoCa v2, a generalizable 3D captioning framework.<n>We show improvements over 3D CoCa of +1.50 CIDEr@0.5IoU on ScanRefer and +1.61 CIDEr@0.5IoU on Nr3D.
  arXiv  Detail & Related papers  (2026-01-10T09:13:10Z)
• Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited Views [41.05815610513033]
  3DThinker is a framework that exploits the rich geometric information embedded within images while reasoning, like humans do.<n>Our framework is the first to enable 3D mentaling during reasoning without any 3D prior input, and it does not rely on explicitly labeled 3D data for training.
  arXiv  Detail & Related papers  (2025-10-21T13:36:58Z)
• Semantic Causality-Aware Vision-Based 3D Occupancy Prediction [63.752869043357585]
  Vision-based 3D semantic occupancy prediction is a critical task in 3D vision.<n>Existing methods, however, often rely on modular pipelines.<n>We propose a novel causal loss that enables holistic, end-to-end supervision of the modular 2D-to-3D transformation pipeline.
  arXiv  Detail & Related papers  (2025-09-10T08:29:22Z)
• Does Your 3D Encoder Really Work? When Pretrain-SFT from 2D VLMs Meets 3D VLMs [72.11701578308804]
  This paper categorizes recent 3D Vision-Language Models into 3D object-centric, 2D image-based, and 3D scene-centric approaches.<n>Despite the architectural similarity of 3D scene-centric VLMs to their 2D counterparts, they have exhibited comparatively lower performance compared with the latest 3D object-centric and 2D image-based approaches.<n>Our investigation suggests that while these models possess cross-modal alignment capabilities, they tend to over-rely on linguistic cues and overfit to frequent answer distributions.
  arXiv  Detail & Related papers  (2025-06-05T17:56:12Z)
• PointAD: Comprehending 3D Anomalies from Points and Pixels for Zero-shot 3D Anomaly Detection [13.60524473223155]
  This paper introduces PointAD, a novel approach that transfers the strong generalization capabilities of CLIP for recognizing 3D anomalies on unseen objects.<n>PointAD renders 3D anomalies into multiple 2D renderings and projects them back into 3D space.<n>Our model can directly integrate RGB information, further enhancing the understanding of 3D anomalies in a plug-and-play manner.
  arXiv  Detail & Related papers  (2024-10-01T01:40:22Z)
• 3D-Aware Indoor Scene Synthesis with Depth Priors [62.82867334012399]
  Existing methods fail to model indoor scenes due to the large diversity of room layouts and the objects inside. We argue that indoor scenes do not have a shared intrinsic structure, and hence only using 2D images cannot adequately guide the model with the 3D geometry.
  arXiv  Detail & Related papers  (2022-02-17T09:54:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.