Learning to Localize Reference Trajectories in Image-Space for Visual Navigation

• URL: http://arxiv.org/abs/2602.18803v1
• Date: Sat, 21 Feb 2026 11:33:00 GMT
• Title: Learning to Localize Reference Trajectories in Image-Space for Visual Navigation
• Abstract summary: We present LoTIS, a model for visual navigation that provides robot-agnostic image-space guidance.<n>Instead of predicting actions tied to specific robots, we predict the image-space coordinates of the reference trajectory as they would appear in the robot's current view.<n>Our model's predictions provide guidance zero-shot across diverse embodiments.
• Score: 11.2075405205209
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present LoTIS, a model for visual navigation that provides robot-agnostic image-space guidance by localizing a reference RGB trajectory in the robot's current view, without requiring camera calibration, poses, or robot-specific training. Instead of predicting actions tied to specific robots, we predict the image-space coordinates of the reference trajectory as they would appear in the robot's current view. This creates robot-agnostic visual guidance that easily integrates with local planning. Consequently, our model's predictions provide guidance zero-shot across diverse embodiments. By decoupling perception from action and learning to localize trajectory points rather than imitate behavioral priors, we enable a cross-trajectory training strategy for robustness to viewpoint and camera changes. We outperform state-of-the-art methods by 20-50 percentage points in success rate on conventional forward navigation, achieving 94-98% success rate across diverse sim and real environments. Furthermore, we achieve over 5x improvements on challenging tasks where baselines fail, such as backward traversal. The system is straightforward to use: we show how even a video from a phone camera directly enables different robots to navigate to any point on the trajectory. Videos, demo, and code are available at https://finnbusch.com/lotis.

Related papers

• ImagiNav: Scalable Embodied Navigation via Generative Visual Prediction and Inverse Dynamics [13.382453086651019]
  Vision-Language Navigation has relied on end-to-end policies trained on expensive, embodiment-specific robot data.<n>We propose ImagiNav, a novel modular paradigm that decouples visual planning from robot actuation.<n>ImagiNav demonstrates strong zero-shot transfer to robot navigation without requiring robot demonstrations.
  arXiv  Detail & Related papers  (2026-03-14T08:34:02Z)
• DreamToNav: Generalizable Navigation for Robots via Generative Video Planning [1.964570633684439]
  We present DreamToNav, a novel autonomous robot framework that uses generative video models to enable intuitive, human-in-the-loop control.<n>We evaluate the approach on both a wheeled mobile robot and a quadruped robot in indoor navigation tasks.<n>DreamToNav achieves a success rate of 76.7%, with final goal errors typically within 0.05-0.10 m and trajectory tracking errors below 0.15 m.
  arXiv  Detail & Related papers  (2026-03-06T11:57:10Z)
• YOPO-Nav: Visual Navigation using 3DGS Graphs from One-Pass Videos [5.7772802828964664]
  We propose YOPO-Nav, which encodes an environment into a compact spatial representation composed of interconnected local 3D Gaussian Splatting (3DGS) models.<n>During navigation, the framework aligns the robot's current visual observation with this representation and predicts actions that guide it back toward the demonstrated trajectory.<n>We benchmark recent visual navigation methods on trajectories from YOPO-Campus using a Clearpath Jackal robot. Experimental results show YOPO-Nav provides excellent performance in image-goal navigation for real-world scenes on a physical robot.
  arXiv  Detail & Related papers  (2025-12-10T18:32:38Z)
• Sight Over Site: Perception-Aware Reinforcement Learning for Efficient Robotic Inspection [57.37596278863949]
  In this work, we revisit inspection from a perception-aware perspective.<n>We propose an end-to-end reinforcement learning framework that explicitly incorporates target visibility as the primary objective.<n>We show that our method outperforms existing classical and learning-based navigation approaches.
  arXiv  Detail & Related papers  (2025-09-22T15:14:02Z)
• Track2Act: Predicting Point Tracks from Internet Videos enables Generalizable Robot Manipulation [65.46610405509338]
  We seek to learn a generalizable goal-conditioned policy that enables zero-shot robot manipulation. Our framework,Track2Act predicts tracks of how points in an image should move in future time-steps based on a goal. We show that this approach of combining scalably learned track prediction with a residual policy enables diverse generalizable robot manipulation.
  arXiv  Detail & Related papers  (2024-05-02T17:56:55Z)
• Gesture2Path: Imitation Learning for Gesture-aware Navigation [54.570943577423094]
  We present Gesture2Path, a novel social navigation approach that combines image-based imitation learning with model-predictive control. We deploy our method on real robots and showcase the effectiveness of our approach for the four gestures-navigation scenarios.
  arXiv  Detail & Related papers  (2022-09-19T23:05:36Z)
• See What the Robot Can't See: Learning Cooperative Perception for Visual Navigation [11.943412856714154]
  We train the sensors to encode and communicate relevant viewpoint information to the mobile robot. We overcome the challenge of enabling all the sensors to predict the direction along the shortest path to the target. Our results show that by using communication between the sensors and the robot, we achieve up to 2.0x improvement in SPL.
  arXiv  Detail & Related papers  (2022-08-01T11:37:01Z)
• Neural Scene Representation for Locomotion on Structured Terrain [56.48607865960868]
  We propose a learning-based method to reconstruct the local terrain for a mobile robot traversing urban environments. Using a stream of depth measurements from the onboard cameras and the robot's trajectory, the estimates the topography in the robot's vicinity. We propose a 3D reconstruction model that faithfully reconstructs the scene, despite the noisy measurements and large amounts of missing data coming from the blind spots of the camera arrangement.
  arXiv  Detail & Related papers  (2022-06-16T10:45:17Z)
• Sparse Image based Navigation Architecture to Mitigate the need of precise Localization in Mobile Robots [3.1556608426768324]
  This paper focuses on mitigating the need for exact localization of a mobile robot to pursue autonomous navigation using a sparse set of images. The proposed method consists of a model architecture - RoomNet, for unsupervised learning resulting in a coarse identification of the environment. The latter uses sparse image matching to characterise the similarity of frames achieved vis-a-vis the frames viewed by the robot during the mapping and training stage.
  arXiv  Detail & Related papers  (2022-03-29T06:38:18Z)
• Rapid Exploration for Open-World Navigation with Latent Goal Models [78.45339342966196]
  We describe a robotic learning system for autonomous exploration and navigation in diverse, open-world environments. At the core of our method is a learned latent variable model of distances and actions, along with a non-parametric topological memory of images. We use an information bottleneck to regularize the learned policy, giving us (i) a compact visual representation of goals, (ii) improved generalization capabilities, and (iii) a mechanism for sampling feasible goals for exploration.
  arXiv  Detail & Related papers  (2021-04-12T23:14:41Z)
• ViNG: Learning Open-World Navigation with Visual Goals [82.84193221280216]
  We propose a learning-based navigation system for reaching visually indicated goals. We show that our system, which we call ViNG, outperforms previously-proposed methods for goal-conditioned reinforcement learning. We demonstrate ViNG on a number of real-world applications, such as last-mile delivery and warehouse inspection.
  arXiv  Detail & Related papers  (2020-12-17T18:22:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.