Difference-Based Relational Learning for Zero-Shot Object-Goal Visual Navigation With Direct Sim-to-Real Transfer

• URL: http://arxiv.org/abs/2607.15642v1
• Date: Fri, 17 Jul 2026 05:35:40 GMT
• Title: Difference-Based Relational Learning for Zero-Shot Object-Goal Visual Navigation With Direct Sim-to-Real Transfer
• Abstract summary: Temporal Difference-Relational Network (T-DRN) for robust zero-shot sim-to-real transfer.<n>Experiments in AI2-THOR demonstrate that T-DRN improves zero-shot generalization in terms of success rates over strong baselines.<n>T-DRN is systematically validated on a physical wheeled robot, demonstrating robust performance under real sensing and actuation constraints.
• Score: 0.5371337604556311
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: End-to-end deep reinforcement learning (DRL) for zero-shot object-goal visual navigation remains challenged by the sim-to-real gap, particularly variations in object appearance and restricted camera field-of-view (FoV). This letter proposes a Temporal Difference-Relational Network (T-DRN) for robust zero-shot sim-to-real transfer. T-DRN combines a Siamese difference-based feature extractor, which computes relational difference between the target and observed objects to produce domain-independent representations, with a dual-frame temporal buffer that preserves short-term object continuity under narrow FoV. Extensive experiments in AI2-THOR demonstrate that T-DRN improves zero-shot generalization in terms of success rates over strong baselines. Furthermore, T-DRN is systematically validated on a physical wheeled robot, demonstrating robust performance under real sensing and actuation constraints and supporting the feasibility of direct sim-to-real transfer.

Related papers

• REFINE: Trajectory Representation Learning via Closed-Loop Transcription -- Extended Version [22.872723951282282]
  REFINE is a simple yet effective lEarning Framework vIa closed-loop trajector rEfinement for trajectory data.<n>We provide a control-theoretic analysis that establishes guarantees for the proposed closed-loop optimization.<n>Experiments on four real-world datasets demonstrate that REFINE consistently outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-09-07T08:26:49Z)
• Task-Relevant Feature-Dynamics Fidelity Enables Zero-Shot Sim-to-Real Transfer for Robotic Ultrasound Scanning [58.25007580703646]
  We develop a TR-oriented ultrasound simulator that combines a shared structural intermediate domain, trajectory-level fixed noise, and few-step conditional flow generation.<n>In phantom experiments, a policy trained exclusively in simulation succeeded in 390 of 400 zero-shot deployments across four target planes.
  arXiv  Detail & Related papers  (2026-08-30T02:37:00Z)
• Causal Bootstrapped Alignment for Unsupervised Video-Based Visible-Infrared Person Re-Identification [52.784239635604735]
  VVI-ReID is a critical technique for all-day surveillance, where temporal information provides additional cues beyond static images.<n>Existing approaches rely heavily on fully supervised learning with expensive cross-modality annotations, limiting scalability.<n>We propose a Causal Bootstrapped Alignment framework that explicitly exploits inherent video priors.
  arXiv  Detail & Related papers  (2026-04-17T02:15:59Z)
• SDF-Net: Structure-Aware Disentangled Feature Learning for Opticall-SAR Ship Re-identification [10.512241383910721]
  Cross-modal ship re-identification (ReID) between optical and synthetic aperture radar (SAR) imagery is fundamentally challenged.<n>We propose SDF-Net, a Structure-Aware Disentangled Feature Learning Network that systematically incorporates geometric consistency into optical--SAR ship ReID.<n> experiments on the HOSS-ReID dataset demonstrate that SDF-Net consistently outperforms existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-03-13T02:41:43Z)
• Perception-Aware Multimodal Spatial Reasoning from Monocular Images [57.42071289037214]
  spatial reasoning from monocular images is essential for autonomous driving.<n>Current Vision-Language Models (VLMs) still struggle with fine-grained geometric perception.<n>We propose a perception-aware multimodal reasoning framework that equipsVLMs with explicit object-centric grounding ability.
  arXiv  Detail & Related papers  (2026-03-07T02:05:12Z)
• RSGround-R1: Rethinking Remote Sensing Visual Grounding through Spatial Reasoning [61.84363374647606]
  Remote Sensing Visual Grounding (RSVG) aims to localize target objects in large-scale aerial imagery based on natural language descriptions.<n>These descriptions often rely heavily on positional cues, posing unique challenges for Multimodal Large Language Models (MLLMs) in spatial reasoning.<n>We propose a reasoning-guided, position-aware post-training framework, dubbed textbfRSGround-R1, to progressively enhance spatial understanding.
  arXiv  Detail & Related papers  (2026-01-29T12:35:57Z)
• OptiCorNet: Optimizing Sequence-Based Context Correlation for Visual Place Recognition [2.3093110834423616]
  This paper presents OptiCorNet, a novel sequence modeling framework.<n>It unifies spatial feature extraction and temporal differencing into a differentiable, end-to-end trainable module.<n>Our approach outperforms state-of-the-art baselines under challenging seasonal and viewpoint variations.
  arXiv  Detail & Related papers  (2025-07-19T04:29:43Z)
• Learning Scene-Level Signed Directional Distance Function with Ellipsoidal Priors and Neural Residuals [17.51160139116379]
  Recent work shows that implicit continuous representations of occupancy, signed distance, or radiance learned using neural networks offer advantages in reconstruction fidelity, efficiency, and differentiability.<n>In this work, we explore a directional formulation of signed distance, called signed directional distance function (SDDF)<n>We show that SDDF is competitive with the state-of-the-art neural implicit scene models in terms of reconstruction accuracy and rendering efficiency.
  arXiv  Detail & Related papers  (2025-03-25T21:01:05Z)
• Multi-Modality Driven LoRA for Adverse Condition Depth Estimation [61.525312117638116]
  We propose Multi-Modality Driven LoRA (MMD-LoRA) for Adverse Condition Depth Estimation.<n>It consists of two core components: Prompt Driven Domain Alignment (PDDA) and Visual-Text Consistent Contrastive Learning (VTCCL)<n>It achieves state-of-the-art performance on the nuScenes and Oxford RobotCar datasets.
  arXiv  Detail & Related papers  (2024-12-28T14:23:58Z)
• Continuity-Discrimination Convolutional Neural Network for Visual Object Tracking [150.51667609413312]
  This paper proposes a novel model, named Continuity-Discrimination Convolutional Neural Network (CD-CNN) for visual object tracking. To address this problem, CD-CNN models temporal appearance continuity based on the idea of temporal slowness. In order to alleviate inaccurate target localization and drifting, we propose a novel notion, object-centroid.
  arXiv  Detail & Related papers  (2021-04-18T06:35:03Z)
• Pose Discrepancy Spatial Transformer Based Feature Disentangling for Partial Aspect Angles SAR Target Recognition [11.552273102567048]
  This letter presents a novel framework termed DistSTN for the task of synthetic aperture radar (SAR) automatic target recognition (ATR) In contrast to the conventional SAR ATR algorithms, DistSTN considers a more challenging practical scenario for non-cooperative targets. We develop an amortized inference scheme that enables efficient feature extraction and recognition using an encoder-decoder mechanism.
  arXiv  Detail & Related papers  (2021-03-07T11:47:34Z)
• Domain Adaptive Robotic Gesture Recognition with Unsupervised Kinematic-Visual Data Alignment [60.31418655784291]
  We propose a novel unsupervised domain adaptation framework which can simultaneously transfer multi-modality knowledge, i.e., both kinematic and visual data, from simulator to real robot. It remedies the domain gap with enhanced transferable features by using temporal cues in videos, and inherent correlations in multi-modal towards recognizing gesture. Results show that our approach recovers the performance with great improvement gains, up to 12.91% in ACC and 20.16% in F1score without using any annotations in real robot.
  arXiv  Detail & Related papers  (2021-03-06T09:10:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.