Robust Scene Transfer for PointGoal Navigation via Privileged Sensor Guided Contrastive Learning

• URL: http://arxiv.org/abs/2606.05506v1
• Date: Wed, 03 Jun 2026 23:15:50 GMT
• Title: Robust Scene Transfer for PointGoal Navigation via Privileged Sensor Guided Contrastive Learning
• Abstract summary: We propose a sensor-guided adaptive contrastive learning framework for visual representation learning in PointGoal navigation.<n>During training, privileged LiDAR sensing guides the contrastive objective through a geometry-aware similarity metric and adaptive temperature scaling.<n>The resulting encoder is pretrained independently, frozen, and used as the backbone for reinforcement learning.
• Score: 45.2217589644315
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a sensor-guided adaptive contrastive learning framework for visual representation learning in PointGoal navigation. During training, privileged LiDAR sensing guides the contrastive objective through a geometry-aware similarity metric and adaptive temperature scaling, encouraging visual embeddings to capture navigation-relevant structure rather than scene-specific appearance. The resulting encoder is pretrained independently, frozen, and used as the perceptual backbone for reinforcement learning, decoupling representation learning from policy optimization. We further introduce a cross-stage domain mismatch between representation pretraining and policy learning to suppress environment-specific shortcuts and promote reliance on task-relevant features. Extensive experiments in high-fidelity simulation demonstrate that our approach significantly improves policy-level scene transfer across diverse indoor and outdoor environments. At deployment, the agent relies only on monocular RGB observations together with standard task-related inputs such as goal position and proprioceptive signals, without access to LiDAR or other privileged sensors. Our method outperforms large pretrained vision models and standard contrastive baselines under severe appearance and semantic shifts. We also release a multimodal dataset to support future research on privileged-guided visual representation learning for navigation. The code is available at:

Related papers

• Ordinal Neural Collapse as a Representation Prior for Visual Navigation [14.612259909790454]
  We present ORION, a method that organizes the encoder's representation space according to the ordinal structure of navigation actions.<n>Experiments show that ORION consistently outperforms end-to-end and neural collapse baselines in navigation success rate and goal progress.
  arXiv  Detail & Related papers  (2026-06-25T10:25:25Z)
• Cross-Modal Reinforcement Learning for Navigation with Degraded Depth Measurements [9.487836148196713]
  We introduce a Cross-Modal Wasserstein Autoencoder that learns shared latent representations by enforcing cross-modal consistency.<n>The learned representations are integrated with a Reinforcement Learning-based policy for collision-free navigation in unstructured environments.
  arXiv  Detail & Related papers  (2026-03-23T16:45:03Z)
• VLD: Visual Language Goal Distance for Reinforcement Learning Navigation [5.225089020389076]
  We introduce Vision-Language Distance (VLD) learning, a framework for goal-conditioned navigation.<n>We first train a self-supervised distance-to-goal predictor on internet-scale video data.<n>This predictor generalizes across both image- and text-based goals, providing a distance signal that can be minimized by a reinforcement learning policy.
  arXiv  Detail & Related papers  (2025-12-08T19:05:51Z)
• Few-Shot Remote Sensing Image Scene Classification with CLIP and Prompt Learning [0.9558392439655014]
  We explore prompt learning as a lightweight and efficient adaptation strategy for few-shot remote sensing image scene classification.<n>We benchmark these prompt-learning methods against two standard baselines: zero-shot CLIP with hand-crafted prompts and a linear probe trained on frozen CLIP features.<n>Our findings underscore prompt learning as a scalable and efficient solution for bridging the domain gap in satellite and aerial imagery.
  arXiv  Detail & Related papers  (2025-10-28T11:39:22Z)
• ZoRI: Towards Discriminative Zero-Shot Remote Sensing Instance Segmentation [23.40908829241552]
  We propose a novel task called zero-shot remote sensing instance segmentation, aimed at identifying aerial objects that are absent from training data.<n>We introduce a knowledge-injected adaptation strategy that decouples semantic-related information to preserve the pretrained vision-language alignment.<n>We establish new experimental protocols and benchmarks, and extensive experiments convincingly demonstrate that ZoRI achieves the state-of-art performance.
  arXiv  Detail & Related papers  (2024-12-17T11:00:56Z)
• Interactive Semantic Map Representation for Skill-based Visual Object Navigation [43.71312386938849]
  This paper introduces a new representation of a scene semantic map formed during the embodied agent interaction with the indoor environment. We have implemented this representation into a full-fledged navigation approach called SkillTron. The proposed approach makes it possible to form both intermediate goals for robot exploration and the final goal for object navigation.
  arXiv  Detail & Related papers  (2023-11-07T16:30:12Z)
• Learning Navigational Visual Representations with Semantic Map Supervision [85.91625020847358]
  We propose a navigational-specific visual representation learning method by contrasting the agent's egocentric views and semantic maps. Ego$2$-Map learning transfers the compact and rich information from a map, such as objects, structure and transition, to the agent's egocentric representations for navigation.
  arXiv  Detail & Related papers  (2023-07-23T14:01:05Z)
• Bilevel Fast Scene Adaptation for Low-Light Image Enhancement [50.639332885989255]
  Enhancing images in low-light scenes is a challenging but widely concerned task in the computer vision. Main obstacle lies in the modeling conundrum from distribution discrepancy across different scenes. We introduce the bilevel paradigm to model the above latent correspondence. A bilevel learning framework is constructed to endow the scene-irrelevant generality of the encoder towards diverse scenes.
  arXiv  Detail & Related papers  (2023-06-02T08:16:21Z)
• Anticipating the Unseen Discrepancy for Vision and Language Navigation [63.399180481818405]
  Vision-Language Navigation requires the agent to follow natural language instructions to reach a specific target. The large discrepancy between seen and unseen environments makes it challenging for the agent to generalize well. We propose Unseen Discrepancy Anticipating Vision and Language Navigation (DAVIS) that learns to generalize to unseen environments via encouraging test-time visual consistency.
  arXiv  Detail & Related papers  (2022-09-10T19:04:40Z)
• Contrastive Instruction-Trajectory Learning for Vision-Language Navigation [66.16980504844233]
  A vision-language navigation (VLN) task requires an agent to reach a target with the guidance of natural language instruction. Previous works fail to discriminate the similarities and discrepancies across instruction-trajectory pairs and ignore the temporal continuity of sub-instructions. We propose a Contrastive Instruction-Trajectory Learning framework that explores invariance across similar data samples and variance across different ones to learn distinctive representations for robust navigation.
  arXiv  Detail & Related papers  (2021-12-08T06:32:52Z)
• Semantics-aware Adaptive Knowledge Distillation for Sensor-to-Vision Action Recognition [131.6328804788164]
  We propose a framework, named Semantics-aware Adaptive Knowledge Distillation Networks (SAKDN), to enhance action recognition in vision-sensor modality (videos) The SAKDN uses multiple wearable-sensors as teacher modalities and uses RGB videos as student modality.
  arXiv  Detail & Related papers  (2020-09-01T03:38:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.