Robostral Navigate

• URL: http://arxiv.org/abs/2607.20785v2
• Date: Fri, 24 Jul 2026 21:13:04 GMT
• Title: Robostral Navigate
• Abstract summary: Robostral Navigate is a vision-language model for navigation systems at scale.<n>It consumes only a stream of monocular RGB images and predicts waypoints by pointing to the next target location in the current camera view.<n>We generate 2.4 million trajectories across 350k simulated scenes to reduce the reliance on real-world data collection.
• Score: 138.74472687550374
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deploying navigation systems at scale requires a recipe that minimizes sensor assumptions, generalizes across robot embodiments, and trains efficiently. Yet, today's best systems depend on depth sensors, multi-camera rigs, or pre-built maps, limiting the hardware they support and increasing deployment cost. We introduce Robostral Navigate, an 8B vision-language model built around this scalability objective. The model consumes only a stream of monocular RGB images - the most ubiquitous sensor across robotic platforms and predicts waypoints by pointing to the next target location in the current camera view. Operating purely in image space, rather than robot-specific coordinates, makes the policy naturally robust to changes in camera intrinsics and scene scale, enabling deployment across wheeled, legged, and aerial robots without recalibration. We generate 2.4 million trajectories across 350k simulated scenes to reduce the reliance on real-world data collection and scale easily. We further introduce a prefix-caching training recipe that packs entire episodes into single training sequences, reducing training tokens by 22x and cutting training time from months to days. A tree-based attention mask prevents conditioning on previous ground-truth actions, encouraging visually grounded action prediction, and reinforcement learning is used to further improve exploration and recovery capabilities. On the Room-to-Room and Room-Across-Room in Continuous Environments (R2R-CE and RxR-CE) benchmarks, Robostral Navigate sets a new state of the art. On R2R-CE, it achieves a 77.4% success rate, surpassing the best monocular method by 10.5 points and the strongest depth- or multi-camera system by 5.3 points despite using only a single RGB camera. On RxR-CE, it reaches 75.1% success rate, outperforming all monocular baselines.

Related papers

• Revisiting Articulated Parts Perception in Robot Manipulation [53.043391395657835]
  Geometric Primary Structure (GPS) is an abstraction of the part geometry structure to balance scalability and quality.<n>GPS is integrated with a portable Virtual Reality (VR) device and requires only one minute to annotate one object sequence.<n>With this efficient VR-GPS system, we collect 41K frames for 234 objects across six part classes, and train a generalizable GPS model with a single RGB-D object image as input.
  arXiv  Detail & Related papers  (2026-06-06T11:07:29Z)
• Learning to Localize Reference Trajectories in Image-Space for Visual Navigation [11.2075405205209]
  We present LoTIS, a model for visual navigation that provides robot-agnostic image-space guidance.<n>Instead of predicting actions tied to specific robots, we predict the image-space coordinates of the reference trajectory as they would appear in the robot's current view.<n>Our model's predictions provide guidance zero-shot across diverse embodiments.
  arXiv  Detail & Related papers  (2026-02-21T11:33:00Z)
• NOVA: Navigation via Object-Centric Visual Autonomy for High-Speed Target Tracking in Unstructured GPS-Denied Environments [56.35569661650558]
  We introduce NOVA, a fully onboard, object-centric framework that enables robust target tracking and collision-aware navigation.<n>Rather than constructing a global map, NOVA formulates perception, estimation, and control entirely in the target's reference frame.<n>We validate NOVA across challenging real-world scenarios, including urban mazes, forest trails, and repeated transitions through buildings with intermittent GPS loss.
  arXiv  Detail & Related papers  (2025-06-23T14:28:30Z)
• MTevent: A Multi-Task Event Camera Dataset for 6D Pose Estimation and Moving Object Detection [1.1083289076967895]
  MTevent is a dataset designed for 6D pose estimation and moving object detection in highly dynamic environments.<n>Our setup consists of a stereo-event camera and an RGB camera, capturing 75 scenes, each on average 16 seconds.<n>We evaluate the task of 6D pose estimation using NVIDIA's FoundationPose on RGB images, achieving an Average Recall of 0.22 with ground-truth masks.
  arXiv  Detail & Related papers  (2025-05-16T14:18:21Z)
• CabiNet: Scaling Neural Collision Detection for Object Rearrangement with Procedural Scene Generation [54.68738348071891]
  We first generate over 650K cluttered scenes - orders of magnitude more than prior work - in diverse everyday environments. We render synthetic partial point clouds from this data and use it to train our CabiNet model architecture. CabiNet is a collision model that accepts object and scene point clouds, captured from a single-view depth observation.
  arXiv  Detail & Related papers  (2023-04-18T21:09:55Z)
• CRN: Camera Radar Net for Accurate, Robust, Efficient 3D Perception [20.824179713013734]
  We propose Camera Radar Net (CRN), a novel camera-radar fusion framework. CRN generates semantically rich and spatially accurate bird's-eye-view (BEV) feature map for various tasks. CRN with real-time setting operates at 20 FPS while achieving comparable performance to LiDAR detectors on nuScenes.
  arXiv  Detail & Related papers  (2023-04-03T00:47:37Z)
• AZTR: Aerial Video Action Recognition with Auto Zoom and Temporal Reasoning [63.628195002143734]
  We propose a novel approach for aerial video action recognition. Our method is designed for videos captured using UAVs and can run on edge or mobile devices. We present a learning-based approach that uses customized auto zoom to automatically identify the human target and scale it appropriately.
  arXiv  Detail & Related papers  (2023-03-02T21:24:19Z)
• Neural Scene Representation for Locomotion on Structured Terrain [56.48607865960868]
  We propose a learning-based method to reconstruct the local terrain for a mobile robot traversing urban environments. Using a stream of depth measurements from the onboard cameras and the robot's trajectory, the estimates the topography in the robot's vicinity. We propose a 3D reconstruction model that faithfully reconstructs the scene, despite the noisy measurements and large amounts of missing data coming from the blind spots of the camera arrangement.
  arXiv  Detail & Related papers  (2022-06-16T10:45:17Z)
• Rapid Exploration for Open-World Navigation with Latent Goal Models [78.45339342966196]
  We describe a robotic learning system for autonomous exploration and navigation in diverse, open-world environments. At the core of our method is a learned latent variable model of distances and actions, along with a non-parametric topological memory of images. We use an information bottleneck to regularize the learned policy, giving us (i) a compact visual representation of goals, (ii) improved generalization capabilities, and (iii) a mechanism for sampling feasible goals for exploration.
  arXiv  Detail & Related papers  (2021-04-12T23:14:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.