Interpreting Context-Aware Human Preferences for Multi-Objective Robot Navigation

• URL: http://arxiv.org/abs/2603.17510v1
• Date: Wed, 18 Mar 2026 09:14:05 GMT
• Title: Interpreting Context-Aware Human Preferences for Multi-Objective Robot Navigation
• Abstract summary: We present a pipeline that enables robots to understand and apply context-dependent navigation preferences.<n>A Vision-Language Model (VLM) extracts structured environmental context from onboard visual observations.<n>A Large Language Models (LLM) convert natural language user feedback into interpretable, context-dependent behavioral rules.<n>A preference translation module then maps contextual information and stored rules into numerical preference vectors.
• Score: 6.15445073348032
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Robots operating in human-shared environments must not only achieve task-level navigation objectives such as safety and efficiency, but also adapt their behavior to human preferences. However, as human preferences are typically expressed in natural language and depend on environmental context, it is difficult to directly integrate them into low-level robot control policies. In this work, we present a pipeline that enables robots to understand and apply context-dependent navigation preferences by combining foundational models with a Multi-Objective Reinforcement Learning (MORL) navigation policy. Thus, our approach integrates high-level semantic reasoning with low-level motion control. A Vision-Language Model (VLM) extracts structured environmental context from onboard visual observations, while Large Language Models (LLM) convert natural language user feedback into interpretable, context-dependent behavioral rules stored in a persistent but updatable rule memory. A preference translation module then maps contextual information and stored rules into numerical preference vectors that parameterize a pretrained MORL policy for real-time navigation adaptation. We evaluate the proposed framework through quantitative component-level evaluations, a user study, and real-world robot deployments in various indoor environments. Our results demonstrate that the system reliably captures user intent, generates consistent preference vectors, and enables controllable behavior adaptation across diverse contexts. Overall, the proposed pipeline improves the adaptability, transparency, and usability of robots operating in shared human environments, while maintaining safe and responsive real-time control.

Related papers

• In-Context Model Predictive Generation: Open-Vocabulary Motion Synthesis from Language Models to Physics [109.99871326497333]
  In-Context Model Predictive Generation (ICMPG) is a framework that integrates language-model planning with inference-time physical feedback.<n>ICMPG reformulates motion synthesis as a Model Predictive Control (MPC)-like process with two modules.
  arXiv  Detail & Related papers  (2026-06-25T12:50:33Z)
• VAIC: Vision-Guided Humanoid Agile Object Interaction Control via Decoupled Commands [56.029505206531155]
  Vision Guided Agile Interaction Control (VAIC) is a unified framework that bridges the gap between current controllers and humanoid robots.<n>First, a privileged teacher policy masters diverse interaction skills using precise object kinematics and exact environmental states.<n>Second, a deployable student policy distills these capabilities by replacing full body tracking with velocity targets across multiple axes.
  arXiv  Detail & Related papers  (2026-06-08T09:52:55Z)
• APPLV: Adaptive Planner Parameter Learning from Vision-Language-Action Model [21.492007331670376]
  We propose an Adaptive Planner Learning from Vision-Language-Action Model (textscapplv) for autonomous navigation.<n>textscapplv outperforms existing methods in both navigation performance and generalization to unseen environments.<n>We develop two training strategies: supervised learning fine-tuning from collected navigation trajectories and reinforcement learning fine-tuning to further optimize navigation performance.
  arXiv  Detail & Related papers  (2026-03-09T19:23:09Z)
• OpenFrontier: General Navigation with Visual-Language Grounded Frontiers [54.661157616245966]
  Open-world navigation requires robots to make decisions in complex everyday environments.<n>Recent advances in vision--language navigation (VLN) and vision--language--action (VLA) models enable end-to-end policies conditioned on natural language.<n>We propose OpenFrontier, a training-free navigation framework that seamlessly integrates diverse vision--language prior models.
  arXiv  Detail & Related papers  (2026-03-05T17:02:22Z)
• Interpretable Robot Control via Structured Behavior Trees and Large Language Models [0.14990005092937678]
  This paper presents a novel framework that bridges natural language understanding and robotic execution.<n>The proposed approach is practical in real-world scenarios, with an average cognition-to-execution accuracy of approximately 94%.
  arXiv  Detail & Related papers  (2025-08-13T08:53:13Z)
• Learning to Tune Like an Expert: Interpretable and Scene-Aware Navigation via MLLM Reasoning and CVAE-Based Adaptation [12.561993540768729]
  We present LE-Nav, an interpretable and scene-aware navigation framework for service robots.<n>To achieve zero-shot scene understanding, we utilize one-shot exemplars and chain-of-thought prompting strategies.<n>Experiments show that LE-Nav can generate hyperparameters achieving human-level tuning across diverse planners and scenarios.
  arXiv  Detail & Related papers  (2025-07-15T05:37:24Z)
• General-Purpose Robotic Navigation via LVLM-Orchestrated Perception, Reasoning, and Acting [5.291702442384798]
  Agentic Robotic Navigation Architecture (ARNA) is a general-purpose framework that equips an LVLM-based agent with a library of perception, reasoning, and navigation tools.<n>At runtime, the agent autonomously defines and executes task-specific navigation tools drawn from modern robotic stacks.<n>ARNA outperforms state-of-the-art EQA-specific approaches.
  arXiv  Detail & Related papers  (2025-06-20T20:06:14Z)
• LANGTRAJ: Diffusion Model and Dataset for Language-Conditioned Trajectory Simulation [102.1527101235251]
  LangTraj is a language-conditioned scene-diffusion model that simulates the joint behavior of all agents in traffic scenarios.<n>By conditioning on natural language inputs, LangTraj provides flexible and intuitive control over interactive behaviors.<n>LangTraj demonstrates strong performance in realism, language controllability, and language-conditioned safety-critical simulation.
  arXiv  Detail & Related papers  (2025-04-15T17:14:06Z)
• Collaborative Instance Object Navigation: Leveraging Uncertainty-Awareness to Minimize Human-Agent Dialogues [54.81155589931697]
  Collaborative Instance object Navigation (CoIN) is a new task setting where the agent actively resolve uncertainties about the target instance.<n>We propose a novel training-free method, Agent-user Interaction with UncerTainty Awareness (AIUTA)<n>First, upon object detection, a Self-Questioner model initiates a self-dialogue within the agent to obtain a complete and accurate observation description.<n>An Interaction Trigger module determines whether to ask a question to the human, continue or halt navigation.
  arXiv  Detail & Related papers  (2024-12-02T08:16:38Z)
• Unifying Large Language Model and Deep Reinforcement Learning for Human-in-Loop Interactive Socially-aware Navigation [16.789333617628138]
  Social robot navigation planners face two major challenges: managing real-time user inputs and ensuring socially compliant behaviors.<n>We introduce SALM, an interactive, human-in-loop Socially-Aware navigation Large Language Model framework.<n>A memory mechanism archives temporal data for continuous refinement, while a multi-step graph-of-thoughts inference-based large language feedback model adaptively fuses the strengths of both planning approaches.
  arXiv  Detail & Related papers  (2024-03-22T23:12:28Z)
• CorNav: Autonomous Agent with Self-Corrected Planning for Zero-Shot Vision-and-Language Navigation [73.78984332354636]
  CorNav is a novel zero-shot framework for vision-and-language navigation. It incorporates environmental feedback for refining future plans and adjusting its actions. It consistently outperforms all baselines in a zero-shot multi-task setting.
  arXiv  Detail & Related papers  (2023-06-17T11:44:04Z)
• LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action [76.71101507291473]
  We present a system, LM-Nav, for robotic navigation that enjoys the benefits of training on unannotated large datasets of trajectories. We show that such a system can be constructed entirely out of pre-trained models for navigation (ViNG), image-language association (CLIP), and language modeling (GPT-3), without requiring any fine-tuning or language-annotated robot data.
  arXiv  Detail & Related papers  (2022-07-10T10:41:50Z)
• Visual-Language Navigation Pretraining via Prompt-based Environmental Self-exploration [83.96729205383501]
  We introduce prompt-based learning to achieve fast adaptation for language embeddings. Our model can adapt to diverse vision-language navigation tasks, including VLN and REVERIE.
  arXiv  Detail & Related papers  (2022-03-08T11:01:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.