AnyTraverse: An off-road traversability framework with VLM and human operator in the loop

• URL: http://arxiv.org/abs/2506.16826v1
• Date: Fri, 20 Jun 2025 08:31:13 GMT
• Title: AnyTraverse: An off-road traversability framework with VLM and human operator in the loop
• Abstract summary: Off-road traversability segmentation enables autonomous navigation with applications in search-and-rescue, military operations, wildlife exploration, and agriculture.<n>We present AnyTraverse, a framework combining natural language-based prompts with human-operator assistance to determine navigable regions for diverse robotic vehicles.
• Score: 4.870043547158868
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Off-road traversability segmentation enables autonomous navigation with applications in search-and-rescue, military operations, wildlife exploration, and agriculture. Current frameworks struggle due to significant variations in unstructured environments and uncertain scene changes, and are not adaptive to be used for different robot types. We present AnyTraverse, a framework combining natural language-based prompts with human-operator assistance to determine navigable regions for diverse robotic vehicles. The system segments scenes for a given set of prompts and calls the operator only when encountering previously unexplored scenery or unknown class not part of the prompt in its region-of-interest, thus reducing active supervision load while adapting to varying outdoor scenes. Our zero-shot learning approach eliminates the need for extensive data collection or retraining. Our experimental validation includes testing on RELLIS-3D, Freiburg Forest, and RUGD datasets and demonstrate real-world deployment on multiple robot platforms. The results show that AnyTraverse performs better than GA-NAV and Off-seg while offering a vehicle-agnostic approach to off-road traversability that balances automation with targeted human supervision.

Related papers

• TANGO: Humanoid Navigation in Cluttered Environments with a Whole-Body Vision-Language-Action Model [49.03909082056646]
  We study the problem of navigating cluttered indoor environments with a humanoid robot.<n>We introduce TANGO, the first whole-body vision-language navigation framework.<n>In simulation experiments, TANGO demonstrates state-of-the-art performance in vision-language navigation.
  arXiv  Detail & Related papers  (2026-09-08T17:59:55Z)
• Qwen-RobotNav Technical Report: A Scalable Navigation Model Designed for an Agentic Navigation System [96.69286664036143]
  We present Qwen-RobotNav, a scalable navigation model built on Qwen-RobotNav.<n>We train Qwen-RobotNav on 15.6M samples; co-training with vision-language data prevents the collapse into reactive action-sequence mappers.<n>Qwen-RobotNav sets new state-of-the-art results across major navigation benchmarks.
  arXiv  Detail & Related papers  (2026-06-16T16:17:44Z)
• Self-Supervised Online Robot-Agnostic Traversability Estimation for Open-World Environments [14.96664579110115]
  Self-supervised online traversability estimation enables robots to continuously learn from unlabeled open-world experiences.<n>We introduce COTRATE, an online learning framework for continuous traversability estimation from multimodal, unlabeled robot experience.
  arXiv  Detail & Related papers  (2026-05-27T13:08:07Z)
• AutoFly: Vision-Language-Action Model for UAV Autonomous Navigation in the Wild [62.47761809929869]
  Vision-language navigation (VLN) requires intelligent agents to navigate environments by interpreting linguistic instructions alongside visual observations.<n>Current VLN research for unmanned aerial vehicles (UAVs) relies on detailed, pre-specified instructions to guide the UAV along predetermined routes.<n>We propose AutoFly, an end-to-end Vision-Language-Action model for autonomous UAV navigation.
  arXiv  Detail & Related papers  (2026-02-10T11:08:07Z)
• CHORAL: Traversal-Aware Planning for Safe and Efficient Heterogeneous Multi-Robot Routing [4.3992789649203194]
  Monitoring large, unknown, and complex environments with autonomous robots poses significant navigation challenges.<n>We propose an integrated semantic-aware framework for coordinating heterogeneous robots.<n> Experiments in simulation and in a real inspection mission with three robotic platforms demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2026-01-15T12:34:22Z)
• Heterogeneous Robot Collaboration in Unstructured Environments with Grounded Generative Intelligence [54.91177026001217]
  Large language model (LLM)-enabled teaming methods typically assume well-structured and known environments.<n>We present SPINE-HT, a framework that addresses these limitations by grounding the reasoning abilities of LLMs in the context of a heterogeneous robot team.<n>Our framework achieves nearly twice the success rate compared to prior LLM-enabled heterogeneous teaming approaches.
  arXiv  Detail & Related papers  (2025-10-30T18:24:38Z)
• VENTURA: Adapting Image Diffusion Models for Unified Task Conditioned Navigation [15.811034169990423]
  VENTURA is a vision-supervised navigation system that finetunes internet-pretrained image diffusion models for path planning.<n>A lightweight behavior-cloning policy grounds these visual plans into executable trajectories, yielding an interface that follows natural language instructions.<n>In extensive real-world evaluations, VENTURA outperforms state-of-the-art foundation model baselines on object reaching, obstacle avoidance, and terrain preference tasks.
  arXiv  Detail & Related papers  (2025-10-01T19:21:28Z)
• Watch Your STEPP: Semantic Traversability Estimation using Pose Projected Features [4.392942391043664]
  We propose a method for estimating terrain traversability by learning from demonstrations of human walking.<n>Our approach leverages dense, pixel-wise feature embeddings generated using the DINOv2 vision Transformer model.<n>By minimizing loss, the network distinguishes between familiar terrain with a low reconstruction error and unfamiliar or hazardous terrain with a higher reconstruction error.
  arXiv  Detail & Related papers  (2025-01-29T11:53:58Z)
• IN-Sight: Interactive Navigation through Sight [20.184155117341497]
  IN-Sight is a novel approach to self-supervised path planning. It calculates traversability scores and incorporates them into a semantic map. To precisely navigate around obstacles, IN-Sight employs a local planner.
  arXiv  Detail & Related papers  (2024-08-01T07:27:54Z)
• Commonsense Reasoning for Legged Robot Adaptation with Vision-Language Models [81.55156507635286]
  Legged robots are physically capable of navigating a diverse variety of environments and overcoming a wide range of obstructions. Current learning methods often struggle with generalization to the long tail of unexpected situations without heavy human supervision. We propose a system, VLM-Predictive Control (VLM-PC), combining two key components that we find to be crucial for eliciting on-the-fly, adaptive behavior selection.
  arXiv  Detail & Related papers  (2024-07-02T21:00:30Z)
• Active Visual Localization for Multi-Agent Collaboration: A Data-Driven Approach [47.373245682678515]
  This work investigates how active visual localization can be used to overcome challenges of viewpoint changes. Specifically, we focus on the problem of selecting the optimal viewpoint at a given location. The result demonstrates the superior performance of the data-driven approach when compared to existing methods.
  arXiv  Detail & Related papers  (2023-10-04T08:18:30Z)
• STERLING: Self-Supervised Terrain Representation Learning from Unconstrained Robot Experience [43.49602846732077]
  We introduce Self-supervised TErrain Representation LearnING (STERLING) STERLING is a novel approach for learning terrain representations that relies solely on easy-to-collect, unconstrained (e.g., non-expert) and unlabelled robot experience. We evaluate STERLING features on the task of preference-aligned visual navigation and find that STERLING features perform on par with fully supervised approaches.
  arXiv  Detail & Related papers  (2023-09-26T22:55:32Z)
• Fast Traversability Estimation for Wild Visual Navigation [17.015268056925745]
  We propose Wild Visual Navigation (WVN), an online self-supervised learning system for traversability estimation. The system is able to continuously adapt from a short human demonstration in the field. We demonstrate the advantages of our approach with experiments and ablation studies in challenging environments in forests, parks, and grasslands.
  arXiv  Detail & Related papers  (2023-05-15T10:19:30Z)
• Offline Reinforcement Learning for Visual Navigation [66.88830049694457]
  ReViND is the first offline RL system for robotic navigation that can leverage previously collected data to optimize user-specified reward functions in the real-world. We show that ReViND can navigate to distant goals using only offline training from this dataset, and exhibit behaviors that qualitatively differ based on the user-specified reward function.
  arXiv  Detail & Related papers  (2022-12-16T02:23:50Z)
• Control Transformer: Robot Navigation in Unknown Environments through PRM-Guided Return-Conditioned Sequence Modeling [0.0]
  We propose Control Transformer that models return-conditioned sequences from low-level policies guided by a sampling-based Probabilistic Roadmap planner. We show that Control Transformer can successfully navigate through mazes and transfer to unknown environments.
  arXiv  Detail & Related papers  (2022-11-11T18:44:41Z)
• GNM: A General Navigation Model to Drive Any Robot [67.40225397212717]
  General goal-conditioned model for vision-based navigation can be trained on data obtained from many distinct but structurally similar robots. We analyze the necessary design decisions for effective data sharing across robots. We deploy the trained GNM on a range of new robots, including an under quadrotor.
  arXiv  Detail & Related papers  (2022-10-07T07:26:41Z)
• Polyline Based Generative Navigable Space Segmentation for Autonomous Visual Navigation [57.3062528453841]
  We propose a representation-learning-based framework to enable robots to learn the navigable space segmentation in an unsupervised manner. We show that the proposed PSV-Nets can learn the visual navigable space with high accuracy, even without any single label.
  arXiv  Detail & Related papers  (2021-10-29T19:50:48Z)
• A Multi-UAV System for Exploration and Target Finding in Cluttered and GPS-Denied Environments [68.31522961125589]
  We propose a framework for a team of UAVs to cooperatively explore and find a target in complex GPS-denied environments with obstacles. The team of UAVs autonomously navigates, explores, detects, and finds the target in a cluttered environment with a known map. Results indicate that the proposed multi-UAV system has improvements in terms of time-cost, the proportion of search area surveyed, as well as successful rates for search and rescue missions.
  arXiv  Detail & Related papers  (2021-07-19T12:54:04Z)
• ReLMoGen: Leveraging Motion Generation in Reinforcement Learning for Mobile Manipulation [99.2543521972137]
  ReLMoGen is a framework that combines a learned policy to predict subgoals and a motion generator to plan and execute the motion needed to reach these subgoals. Our method is benchmarked on a diverse set of seven robotics tasks in photo-realistic simulation environments. ReLMoGen shows outstanding transferability between different motion generators at test time, indicating a great potential to transfer to real robots.
  arXiv  Detail & Related papers  (2020-08-18T08:05:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.