NavThinker: Action-Conditioned World Models for Coupled Prediction and Planning in Social Navigation

• URL: http://arxiv.org/abs/2603.15359v2
• Date: Wed, 18 Mar 2026 15:08:12 GMT
• Title: NavThinker: Action-Conditioned World Models for Coupled Prediction and Planning in Social Navigation
• Abstract summary: Social navigation requires robots to act safely in dynamic human environments.<n>We propose NavThinker, a future-aware framework that couples an action-conditioned world model with on-policy reinforcement learning.<n> Experiments on single- and multi-robot Social-HM3D show state-of-the-art navigation success, with zero-shot transfer to Social-MP3D and real-world deployment on a Unitree Go2.
• Score: 20.161788094382526
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Social navigation requires robots to act safely in dynamic human environments. Effective behavior demands thinking ahead: reasoning about how the scene and pedestrians evolve under different robot actions rather than reacting to current observations alone. This creates a coupled prediction-planning challenge, where robot actions and human motion mutually influence each other. To address this challenge, we propose NavThinker, a future-aware framework that couples an action-conditioned world model with on-policy reinforcement learning. The world model operates in the Depth Anything V2 patch feature space and performs autoregressive prediction of future scene geometry and human motion; multi-head decoders then produce future depth maps and human trajectories, yielding a future-aware state aligned with traversability and interaction risk. Crucially, we train the policy with DD-PPO while injecting world-model think-ahead signals via: (i) action-conditioned future features fused into the current observation embedding and (ii) social reward shaping from predicted human trajectories. Experiments on single- and multi-robot Social-HM3D show state-of-the-art navigation success, with zero-shot transfer to Social-MP3D and real-world deployment on a Unitree Go2, validating generalization and practical applicability. Webpage: https://hutslib.github.io/NavThinker.

Related papers

• Social-Pose: Enhancing Trajectory Prediction with Human Body Pose [70.59399670794171]
  We study the benefits of predicting human trajectories using human body poses instead of solely their Cartesian space locations in time.<n>We propose Social-pose', an attention-based pose encoder that effectively captures the poses of all humans in a scene and their social relations.
  arXiv  Detail & Related papers  (2025-07-30T14:58:48Z)
• From Cognition to Precognition: A Future-Aware Framework for Social Navigation [1.9094009409000596]
  We propose a reinforcement learning architecture, namely Falcon, to tackle socially-aware navigation.<n>We introduce a novel SocialNav benchmark containing two new datasets, Social-HM3D and Social-MP3D.<n>We conduct a detailed experimental analysis with the state-of-the-art learning-based method and two classic rule-based path-planning algorithms.
  arXiv  Detail & Related papers  (2024-09-20T06:08:24Z)
• Following the Human Thread in Social Navigation [10.384957135051119]
  Social Navigation relies on the robot's real-time adaptation to human motion.<n>Human trajectories emerge as crucial cues in Social Navigation, but they are partially observable from the robot's egocentric view.<n>We present the first Social Dynamics Adaptation model based on the robot's state-action history to infer the social dynamics.
  arXiv  Detail & Related papers  (2024-04-17T12:39:48Z)
• Robot Interaction Behavior Generation based on Social Motion Forecasting for Human-Robot Interaction [9.806227900768926]
  We propose to model social motion forecasting in a shared human-robot representation space. ECHO operates in the aforementioned shared space to predict the future motions of the agents encountered in social scenarios. We evaluate our model in multi-person and human-robot motion forecasting tasks and obtain state-of-the-art performance by a large margin.
  arXiv  Detail & Related papers  (2024-02-07T11:37:14Z)
• Social-Transmotion: Promptable Human Trajectory Prediction [65.80068316170613]
  Social-Transmotion is a generic Transformer-based model that exploits diverse and numerous visual cues to predict human behavior.<n>Our approach is validated on multiple datasets, including JTA, JRDB, Pedestrians and Cyclists in Road Traffic, and ETH-UCY.
  arXiv  Detail & Related papers  (2023-12-26T18:56:49Z)
• Learning Vision-based Pursuit-Evasion Robot Policies [54.52536214251999]
  We develop a fully-observable robot policy that generates supervision for a partially-observable one. We deploy our policy on a physical quadruped robot with an RGB-D camera on pursuit-evasion interactions in the wild.
  arXiv  Detail & Related papers  (2023-08-30T17:59:05Z)
• Socially Compliant Navigation Dataset (SCAND): A Large-Scale Dataset of Demonstrations for Social Navigation [92.66286342108934]
  Social navigation is the capability of an autonomous agent, such as a robot, to navigate in a'socially compliant' manner in the presence of other intelligent agents such as humans. Our dataset contains 8.7 hours, 138 trajectories, 25 miles of socially compliant, human teleoperated driving demonstrations.
  arXiv  Detail & Related papers  (2022-03-28T19:09:11Z)
• Intention Aware Robot Crowd Navigation with Attention-Based Interaction Graph [3.8461692052415137]
  We study the problem of safe and intention-aware robot navigation in dense and interactive crowds. We propose a novel recurrent graph neural network with attention mechanisms to capture heterogeneous interactions among agents. We demonstrate that our method enables the robot to achieve good navigation performance and non-invasiveness in challenging crowd navigation scenarios.
  arXiv  Detail & Related papers  (2022-03-03T16:26:36Z)
• TRiPOD: Human Trajectory and Pose Dynamics Forecasting in the Wild [77.59069361196404]
  TRiPOD is a novel method for predicting body dynamics based on graph attentional networks. To incorporate a real-world challenge, we learn an indicator representing whether an estimated body joint is visible/invisible at each frame. Our evaluation shows that TRiPOD outperforms all prior work and state-of-the-art specifically designed for each of the trajectory and pose forecasting tasks.
  arXiv  Detail & Related papers  (2021-04-08T20:01:00Z)
• Visual Navigation Among Humans with Optimal Control as a Supervisor [72.5188978268463]
  We propose an approach that combines learning-based perception with model-based optimal control to navigate among humans. Our approach is enabled by our novel data-generation tool, HumANav. We demonstrate that the learned navigation policies can anticipate and react to humans without explicitly predicting future human motion.
  arXiv  Detail & Related papers  (2020-03-20T16:13:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.