From General Vision to Reliable Traversability Estimation: Adapting Vision Foundation Models for Unstructured Outdoor Environments

• URL: http://arxiv.org/abs/2605.29565v1
• Date: Thu, 28 May 2026 08:16:20 GMT
• Title: From General Vision to Reliable Traversability Estimation: Adapting Vision Foundation Models for Unstructured Outdoor Environments
• Abstract summary: We propose Vision-to-Traversability Adaptation (ViTA), a framework that adapts vision-based approaches for reliable traversability estimation.<n>ViTA injects task-specific knowledge through learnable traversability prompts while preserving the VFM's cross-domain generalization.<n>We show that ViTA achieves state-of-the-art IoU and Precision with substantial false-positive reduction and strong cross-domain generalization.
• Score: 12.275095764403432
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Vision-based approaches have become the dominant paradigm for traversability estimation in unstructured outdoor environments, typically adapting vision foundation models (VFMs) via semantic segmentation supervision. However, this paradigm faces three fundamental challenges that undermine its reliability: the task-agnostic design of VFMs, the ambiguity of traversability annotations, and the discrepancy between semantic labels and physical safety. We propose Vision-to-Traversability Adaptation (ViTA), a framework that adapts VFMs for reliable traversability estimation, instantiated on SAM2. ViTA injects task-specific knowledge through learnable traversability prompts while preserving the VFM's cross-domain generalization. To handle annotation ambiguity, we introduce Perspective-Diversified Training, which estimates semantic uncertainty to suppress confident predictions at ambiguous boundaries. To bridge the semantic-traversability discrepancy, we distill geometric knowledge during training, enabling slope and elevation reasoning from RGB images alone at inference. The semantic and geometric outputs are fused into a continuous traversability score that reflects both semantic uncertainty and geometric risk. Evaluations across diverse domains, including challenging real-world off-road datasets, demonstrate that ViTA achieves state-of-the-art IoU and Precision with substantial false-positive reduction and strong cross-domain generalization.

Related papers

• Estimating Semantic Ambiguity via Gaussian Context Distributions for VLM-Driven Traversability Analysis [14.31446990918297]
  We present a novel pipeline for vision-based traversability estimation that explicitly models contextual uncertainty.<n>Our approach utilizes Conceptual Anchoring to ground open-vocabulary VLM predictions onto a continuous physical traversability scale.<n>We derive both a dense traversability map and a dense uncertainty map based on the statistical properties of the distribution.
  arXiv  Detail & Related papers  (2026-09-08T11:18:32Z)
• Uncertainty-Aware Gaussian Map for Vision-Language Navigation [63.97713877754199]
  Vision-Language Navigation (VLN) requires an agent to navigate 3D environments following natural language instructions.<n>In this work, we explicitly model three forms of perceptual uncertainty (i.e., geometric, semantic, and appearance uncertainty) and integrate them into the agent's observation space to enable informed decision-making.
  arXiv  Detail & Related papers  (2026-05-26T03:33:47Z)
• From Local to Global to Mechanistic: An iERF-Centered Unified Framework for Interpreting Vision Models [29.908678230814015]
  We introduce an iERF-centric framework that unifies local, global, and mechanistic interpretability around a single analysis unit.<n>On the local side, Sharing Ratio Decomposition (SRD) expresses each PFV as a mixture of upstream PFVs via sharing ratios and propagates iERFs to construct class-discriminative saliency maps.<n>For the global view, we introduce Concept-Anchored Feature Explanation (CAFE), which utilizes the iERF as a semantic label.
  arXiv  Detail & Related papers  (2026-05-01T07:25:49Z)
• Towards Domain-Generalized Open-Vocabulary Object Detection: A Progressive Domain-invariant Cross-modal Alignment Method [59.30562121800656]
  Open-Vocabulary Object Detection has achieved remarkable success in generalizing to novel categories.<n>We provide a principled revisit of the OVOD paradigm, uncovering a fundamental vulnerability.<n>We propose Progressive Domain-invariant Cross-modal Alignment (PICA)
  arXiv  Detail & Related papers  (2026-03-29T07:39:31Z)
• Perception-Aware Multimodal Spatial Reasoning from Monocular Images [57.42071289037214]
  spatial reasoning from monocular images is essential for autonomous driving.<n>Current Vision-Language Models (VLMs) still struggle with fine-grained geometric perception.<n>We propose a perception-aware multimodal reasoning framework that equipsVLMs with explicit object-centric grounding ability.
  arXiv  Detail & Related papers  (2026-03-07T02:05:12Z)
• Reasoning-Driven Amodal Completion: Collaborative Agents and Perceptual Evaluation [17.405818788700234]
  We present a Collaborative Multi-Agent Reasoning Framework that explicitly decouples Semantic Planning from Visual Synthesis.<n>Our method generates a structured, explicit plan before pixel generation, enabling visually and semantically coherent single-pass synthesis.<n>Addressing the limitations of traditional metrics in assessing inferred invisible content, we introduce the MAC-Score, a novel human-aligned evaluation metric.
  arXiv  Detail & Related papers  (2025-12-24T04:39:45Z)
• TransFIRA: Transfer Learning for Face Image Recognizability Assessment [73.61309363885552]
  TransFIRA is a lightweight and annotation-free framework that grounds recognizability directly in embedding space.<n>New extensions beyond faces include encoder-grounded explainability that reveals how degradations and subject-specific factors affect recognizability.<n> Experiments confirm state-of-the-art results on faces, strong robustness on body recognition, and under cross-dataset shifts.
  arXiv  Detail & Related papers  (2025-10-07T18:16:21Z)
• Adaptive Dual Uncertainty Optimization: Boosting Monocular 3D Object Detection under Test-Time Shifts [80.32933059529135]
  Test-Time Adaptation (TTA) methods have emerged to adapt to target distributions during inference.<n>We propose Dual Uncertainty Optimization (DUO), the first TTA framework designed to jointly minimize both uncertainties for robust M3OD.<n>In parallel, we design a semantic-aware normal field constraint that preserves geometric coherence in regions with clear semantic cues.
  arXiv  Detail & Related papers  (2025-08-28T07:09:21Z)
• Exploiting the Asymmetric Uncertainty Structure of Pre-trained VLMs on the Unit Hypersphere [2.803487857262511]
  We propose AsymVLM to build probabilistic embeddings from pre-trained vision-language models on the unit hypersphere, enabling uncertainty quantification.<n>We validate the effectiveness of the probabilistic embeddings on established benchmarks, and present comprehensive ablation studies demonstrating the inherent nature of asymmetry in the uncertainty structure of textual and visual data.
  arXiv  Detail & Related papers  (2025-05-16T09:24:29Z)
• Evidential Semantic Mapping in Off-road Environments with Uncertainty-aware Bayesian Kernel Inference [5.120567378386614]
  We propose an evidential semantic mapping framework, which can enhance reliability in perceptually challenging off-road environments. By adaptively handling semantic uncertainties, the proposed framework constructs robust representations of the surroundings even in previously unseen environments.
  arXiv  Detail & Related papers  (2024-03-21T05:13:34Z)
• Dive into Ambiguity: Latent Distribution Mining and Pairwise Uncertainty Estimation for Facial Expression Recognition [59.52434325897716]
  We propose a solution, named DMUE, to address the problem of annotation ambiguity from two perspectives. For the former, an auxiliary multi-branch learning framework is introduced to better mine and describe the latent distribution in the label space. For the latter, the pairwise relationship of semantic feature between instances are fully exploited to estimate the ambiguity extent in the instance space.
  arXiv  Detail & Related papers  (2021-04-01T03:21:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.