SOPD-SocialNav: Selective On-Policy Distillation for Vision-Language Social Navigation

• URL: http://arxiv.org/abs/2607.19850v1
• Date: Wed, 22 Jul 2026 07:35:35 GMT
• Title: SOPD-SocialNav: Selective On-Policy Distillation for Vision-Language Social Navigation
• Abstract summary: We propose SOPD-SocialNav, a method that transfers social navigation knowledge from a large teacher VLM to a lightweight student VLM.<n> SOPD consistently outperforms supervised fine-tuning, off-policy distillation, and standard on-policy distillation baselines in action prediction, perception consistency, and reasoning consistency.<n>These results suggest that SOPD is an effective strategy for building lightweight yet socially aware VLM-based navigation systems.
• Score: 8.30043700809655
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-language models have shown strong potential for social robot navigation by leveraging rich semantic understanding of complex environments and human behaviors. However, large scale VLMs are difficult to deploy on resource-constrained robotic platforms, while lightweight VLMs often lack sufficient social reasoning capability. To address this problem, we propose SOPD-SocialNav, a selective on-policy distillation (SOPD) method that transfers social navigation knowledge from a large teacher VLM to a lightweight student VLM. SOPD introduces an entropy-based token selection mechanism that uses teacher uncertainty to identify socially informative decision tokens, while suppressing gradients from low-entropy tokens corresponding to trivial navigation states. A temperature-controlled Jensen-Shannon divergence objective is then used to align the student and teacher distributions on the selected tokens. Experiments on the SNEI and MUSON benchmarks demonstrate that SOPD consistently outperforms supervised fine-tuning, off-policy distillation, and standard on-policy distillation baselines in action prediction, perception consistency, and reasoning consistency. Real-world deployment on a Scout Mini robot further shows that the distilled model can generate more socially appropriate navigation behaviors in conversational and queuing scenarios. These results suggest that SOPD is an effective strategy for building lightweight yet socially aware VLM-based navigation systems.

Related papers

• Vision-Language Models for Deployable Social Robot Navigation: Bridging Semantic Reasoning and Low-Level Control [27.669648439283936]
  Social robot navigation (SRN) requires understanding human intentions, social norms, and contextual cues to generate socially compliant behaviors.<n>Recent advances in Vision-Language Models (VLMs) have opened new opportunities for SRN.<n>Survey aims to provide a foundation for building reliable, socially compliant, and deployable VLM-enabled navigation systems.
  arXiv  Detail & Related papers  (2026-06-27T06:32:47Z)
• E-SocialNav: Efficient Socially Compliant Navigation with Language Models [33.58546605940754]
  Language models (LMs) are increasingly applied to robotic navigation.<n>Existing benchmarks primarily emphasize navigation success rates while paying limited attention to social compliance.<n>We propose E-SocialNav, an efficient LM designed for socially compliant navigation.
  arXiv  Detail & Related papers  (2026-03-21T05:39:50Z)
• Interpretable Debiasing of Vision-Language Models for Social Fairness [55.85977929985967]
  We introduce an interpretable, model-agnostic bias mitigation framework, DeBiasLens, that localizes social attribute neurons in Vision-Language models.<n>We train SAEs on facial image or caption datasets without corresponding social attribute labels to uncover neurons highly responsive to specific demographics.<n>Our research lays the groundwork for future auditing tools, prioritizing social fairness in emerging real-world AI systems.
  arXiv  Detail & Related papers  (2026-02-27T13:37:11Z)
• From Obstacles to Etiquette: Robot Social Navigation with VLM-Informed Path Selection [57.74400052368147]
  This paper presents a social robot navigation framework that integrates geometric planning with contextual social reasoning.<n>The system first extracts obstacles and human dynamics to generate geometrically feasible candidate paths, then leverages a fine-tuned vision-language model (VLM) to evaluate these paths.<n>Experiments in four social navigation contexts demonstrate that our method achieves the best overall performance with the lowest personal space violation duration, the minimal pedestrian-facing time, and no social zone intrusions.
  arXiv  Detail & Related papers  (2026-02-09T18:46:12Z)
• SocialNav-MoE: A Mixture-of-Experts Vision Language Model for Socially Compliant Navigation with Reinforcement Fine-Tuning [6.245382633570723]
  Socially compliant navigation that accounts for human comfort, social norms, and contextual appropriateness remains underexplored.<n>We propose SocialNav-MoE, an efficient Mixture-of-Experts vision language model for socially compliant navigation with reinforcement fine-tuning.<n>Experiments on the SNEI dataset demonstrate that SocialNav-MoE achieves an excellent balance between navigation accuracy and efficiency.
  arXiv  Detail & Related papers  (2025-12-15T14:21:15Z)
• LISN: Language-Instructed Social Navigation with VLM-based Controller Modulating [47.62872797480247]
  We present LISN-Bench, the first simulation-based benchmark for language-instructed social navigation.<n>We propose Social-Nav-Modulator, a fast-slow hierarchical system where a VLM agent modulates costmaps and controller parameters.<n>Our method achieves an average success rate of 91.3%, which is greater than 63% than the most competitive baseline.
  arXiv  Detail & Related papers  (2025-12-10T18:54:30Z)
• EvolveNav: Empowering LLM-Based Vision-Language Navigation via Self-Improving Embodied Reasoning [145.32076310071434]
  We propose EvolveNav, a novel embodied reasoning paradigm that realizes adaptable and generalizable navigational reasoning.<n>EvolveNav involves a two-stage training process: (1) Formalized CoT Supervised Fine-Tuning, where we train the model with curated formalized CoT labels to first activate the model's navigational reasoning capabilities, and simultaneously increase the reasoning speed; (2) Self-Reflective Post-Training, where the model is iteratively trained with its own reasoning outputs as self-enriched CoT labels to enhance the supervision diversity.
  arXiv  Detail & Related papers  (2025-06-02T11:28:32Z)
• GSON: A Group-based Social Navigation Framework with Large Multimodal Model [9.94576166903495]
  This paper introduces GSON, a novel group-based social navigation framework.<n>GSON uses visual prompting to enable zero-shot extraction of social relationships among pedestrians.<n>We validate GSON through extensive real-world mobile robot navigation experiments.
  arXiv  Detail & Related papers  (2024-09-26T17:27:15Z)
• Distilling Reinforcement Learning Policies for Interpretable Robot Locomotion: Gradient Boosting Machines and Symbolic Regression [53.33734159983431]
  This paper introduces a novel approach to distill neural RL policies into more interpretable forms. We train expert neural network policies using RL and distill them into (i) GBMs, (ii) EBMs, and (iii) symbolic policies.
  arXiv  Detail & Related papers  (2024-03-21T11:54:45Z)
• Multi-Agent Dynamic Relational Reasoning for Social Robot Navigation [50.01551945190676]
  Social robot navigation can be helpful in various contexts of daily life but requires safe human-robot interactions and efficient trajectory planning. We propose a systematic relational reasoning approach with explicit inference of the underlying dynamically evolving relational structures. We demonstrate its effectiveness for multi-agent trajectory prediction and social robot navigation.
  arXiv  Detail & Related papers  (2024-01-22T18:58:22Z)
• SocNavGym: A Reinforcement Learning Gym for Social Navigation [0.0]
  SocNavGym is an advanced simulation environment for social navigation. It can generate different types of social navigation scenarios. It can also be configured to work with different hand-crafted and data-driven social reward signals.
  arXiv  Detail & Related papers  (2023-04-27T11:29:02Z)
• Counterfactual Vision-and-Language Navigation via Adversarial Path Sampling [65.99956848461915]
  Vision-and-Language Navigation (VLN) is a task where agents must decide how to move through a 3D environment to reach a goal.<n>One of the problems of the VLN task is data scarcity since it is difficult to collect enough navigation paths with human-annotated instructions for interactive environments.<n>We propose an adversarial-driven counterfactual reasoning model that can consider effective conditions instead of low-quality augmented data.
  arXiv  Detail & Related papers  (2019-11-17T18:02:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.