Uncertainty-Aware Gaussian Map for Vision-Language Navigation

• URL: http://arxiv.org/abs/2605.26503v1
• Date: Tue, 26 May 2026 03:33:47 GMT
• Title: Uncertainty-Aware Gaussian Map for Vision-Language Navigation
• Abstract summary: Vision-Language Navigation (VLN) requires an agent to navigate 3D environments following natural language instructions.<n>In this work, we explicitly model three forms of perceptual uncertainty (i.e., geometric, semantic, and appearance uncertainty) and integrate them into the agent's observation space to enable informed decision-making.
• Score: 63.97713877754199
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-Language Navigation (VLN) requires an agent to navigate 3D environments following natural language instructions. During navigation, existing agents commonly encounter perceptual uncertainty, such as insufficient evidence for reliable grounding or ambiguity in interpreting spatial cues, yet they typically ignore such information when predicting actions. In this work, we explicitly model three forms of perceptual uncertainty (i.e., geometric, semantic, and appearance uncertainty) and integrate them into the agent's observation space to enable informed decision-making. Concretely, our agent first constructs a Semantic Gaussian Map (SGM), composed of differentiable 3D Gaussian primitives initialized from panoramic observations, that encodes both the geometric structure and semantic content of the environment. On top of SGM, geometric uncertainty is estimated through variational perturbations of Gaussian position and scale to assess structural reliability; semantic uncertainty is captured by perturbing Gaussian semantic attributes to reveal ambiguous interpretations; and appearance uncertainty is characterized by Fisher Information, which measures the sensitivity of rendered observations to Gaussian-level variations. These uncertainties are incorporated into SGM, extending it into a unified 3D Value Map, which grounds them as affordances and constraints that support reliable navigation. Comprehensive evaluations across multiple VLN benchmarks show the effectiveness of our agent.

Related papers

• Estimating Semantic Ambiguity via Gaussian Context Distributions for VLM-Driven Traversability Analysis [14.31446990918297]
  We present a novel pipeline for vision-based traversability estimation that explicitly models contextual uncertainty.<n>Our approach utilizes Conceptual Anchoring to ground open-vocabulary VLM predictions onto a continuous physical traversability scale.<n>We derive both a dense traversability map and a dense uncertainty map based on the statistical properties of the distribution.
  arXiv  Detail & Related papers  (2026-09-08T11:18:32Z)
• DOC-GS: Dual-Domain Observation and Calibration for Reliable Sparse-View Gaussian Splatting [80.43237927269575]
  In this paper, we revisit sparse-view 3DGS reconstruction from a new perspective.<n>We identify the core challenge as the unobservability of Gaussian primitive reliability.<n>Motivated by this observation, we propose a unified Dual-domain Observation and geometrics in rendered images framework.
  arXiv  Detail & Related papers  (2026-04-08T07:01:24Z)
• Counterfactual Explanations on Robust Perceptual Geodesics [13.054357482525505]
  We introduce Perceptual Counterfactual Geodesics (PCG), a method that constructs counterfactuals by tracing geodesics under a metric induced from robust vision features.<n>This geometry aligns with human perception and penalizes brittle directions, enabling smooth, on-manifold, semantically valid transitions.<n>Experiments on three vision datasets show that PCG outperforms baselines and reveals failure modes hidden under standard metrics.
  arXiv  Detail & Related papers  (2026-01-26T16:52:54Z)
• Improving Semantic Uncertainty Quantification in LVLMs with Semantic Gaussian Processes [60.75226150503949]
  We propose a Bayesian framework that quantifies semantic uncertainty by analyzing the geometric structure of answer embeddings.<n>S GPU maps generated answers into a dense semantic space, computes the Gram matrix of their semantic embeddings, and summarizes their semantic configuration.<n>We show that S GPU transfers across models and modalities, indicating that its spectral representation captures general patterns of semantic uncertainty.
  arXiv  Detail & Related papers  (2025-12-16T08:15:24Z)
• Adaptive Dual Uncertainty Optimization: Boosting Monocular 3D Object Detection under Test-Time Shifts [80.32933059529135]
  Test-Time Adaptation (TTA) methods have emerged to adapt to target distributions during inference.<n>We propose Dual Uncertainty Optimization (DUO), the first TTA framework designed to jointly minimize both uncertainties for robust M3OD.<n>In parallel, we design a semantic-aware normal field constraint that preserves geometric coherence in regions with clear semantic cues.
  arXiv  Detail & Related papers  (2025-08-28T07:09:21Z)
• EvidMTL: Evidential Multi-Task Learning for Uncertainty-Aware Semantic Surface Mapping from Monocular RGB Images [7.069718718698565]
  Existing mapping methods often suffer from overconfident semantic predictions, and sparse and noisy depth sensing.<n>We introduce EvidMTL, a multi-task learning framework that uses evidential heads for depth estimation and semantic segmentation.<n>We present EvidKimera, an uncertainty-aware semantic surface mapping framework, which uses evidential depth and semantics prediction for improved 3D metric-semantic consistency.
  arXiv  Detail & Related papers  (2025-03-06T13:56:48Z)
• Evidential Semantic Mapping in Off-road Environments with Uncertainty-aware Bayesian Kernel Inference [5.120567378386614]
  We propose an evidential semantic mapping framework, which can enhance reliability in perceptually challenging off-road environments. By adaptively handling semantic uncertainties, the proposed framework constructs robust representations of the surroundings even in previously unseen environments.
  arXiv  Detail & Related papers  (2024-03-21T05:13:34Z)
• Uncertainty-Aware AB3DMOT by Variational 3D Object Detection [74.8441634948334]
  Uncertainty estimation is an effective tool to provide statistically accurate predictions. In this paper, we propose a Variational Neural Network-based TANet 3D object detector to generate 3D object detections with uncertainty.
  arXiv  Detail & Related papers  (2023-02-12T14:30:03Z)
• Robust Self-Supervised LiDAR Odometry via Representative Structure Discovery and 3D Inherent Error Modeling [67.75095378830694]
  We develop a two-stage odometry estimation network, where we obtain the ego-motion by estimating a set of sub-region transformations. In this paper, we aim to alleviate the influence of unreliable structures in training, inference and mapping phases. Our two-frame odometry outperforms the previous state of the arts by 16%/12% in terms of translational/rotational errors.
  arXiv  Detail & Related papers  (2022-02-27T12:52:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.