Incremental Residual Reinforcement Learning Toward Real-World Learning for Social Navigation

• URL: http://arxiv.org/abs/2604.07945v1
• Date: Thu, 09 Apr 2026 08:08:37 GMT
• Title: Incremental Residual Reinforcement Learning Toward Real-World Learning for Social Navigation
• Abstract summary: Social navigation has emerged as a critical task, driving research into deep reinforcement learning (RL) approaches.<n>Real-world RL, in which agents learn while operating directly in physical environments, presents a promising solution to this issue.<n>In this study, we propose incremental residual RL (IRRL), which enhances learning efficiency by training only on the residuals relative to a base policy.
• Score: 2.1204495827342438
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As the demand for mobile robots continues to increase, social navigation has emerged as a critical task, driving active research into deep reinforcement learning (RL) approaches. However, because pedestrian dynamics and social conventions vary widely across different regions, simulations cannot easily encompass all possible real-world scenarios. Real-world RL, in which agents learn while operating directly in physical environments, presents a promising solution to this issue. Nevertheless, this approach faces significant challenges, particularly regarding constrained computational resources on edge devices and learning efficiency. In this study, we propose incremental residual RL (IRRL). This method integrates incremental learning, which is a lightweight process that operates without a replay buffer or batch updates, with residual RL, which enhances learning efficiency by training only on the residuals relative to a base policy. Through the simulation experiments, we demonstrated that, despite lacking a replay buffer, IRRL achieved performance comparable to those of conventional replay buffer-based methods and outperformed existing incremental learning approaches. Furthermore, the real-world experiments confirmed that IRRL can enable robots to effectively adapt to previously unseen environments through the real-world learning.

Related papers

• Reset-Free Reinforcement Learning for Real-World Agile Driving: An Empirical Study [3.038642416291856]
  This paper presents an empirical study of reset-free reinforcement learning (RL) for real-world agile driving.<n>We employ Model Predictive Path Integral control (MPPI) as both the reset policy and the base policy for residual learning.<n>Our results reveal a clear gap between simulation and real-world: SAC with residual learning achieves the highest returns in simulation, yet only TD-MPC2 consistently outperforms the MPPI baseline on the physical platform.
  arXiv  Detail & Related papers  (2026-04-09T00:28:54Z)
• Sim2Real Reinforcement Learning for Soccer skills [0.0]
  This thesis work presents a more efficient and effective approach to training control-related tasks for humanoid robots using Reinforcement Learning (RL)<n>The traditional RL methods are limited in adapting to real-world environments, complexity, and natural motions.<n>The transfer of the learned policy from simulation to the real world was unsuccessful.
  arXiv  Detail & Related papers  (2025-12-13T19:29:35Z)
• Improving RL Exploration for LLM Reasoning through Retrospective Replay [45.00643118030677]
  We propose a novel algorithm named Retrospective Replay-based Reinforcement Learning (RRL), which introduces a dynamic replay mechanism throughout the training process.<n>RRL enables the model to revisit promising states identified in the early stages, thereby improving its efficiency and effectiveness in exploration.
  arXiv  Detail & Related papers  (2025-04-19T17:40:04Z)
• Grow Your Limits: Continuous Improvement with Real-World RL for Robotic Locomotion [66.69666636971922]
  We present APRL, a policy regularization framework that modulates the robot's exploration over the course of training. APRL enables a quadrupedal robot to efficiently learn to walk entirely in the real world within minutes.
  arXiv  Detail & Related papers  (2023-10-26T17:51:46Z)
• Hybrid Reinforcement Learning for Optimizing Pump Sustainability in Real-World Water Distribution Networks [55.591662978280894]
  This article addresses the pump-scheduling optimization problem to enhance real-time control of real-world water distribution networks (WDNs) Our primary objectives are to adhere to physical operational constraints while reducing energy consumption and operational costs. Traditional optimization techniques, such as evolution-based and genetic algorithms, often fall short due to their lack of convergence guarantees.
  arXiv  Detail & Related papers  (2023-10-13T21:26:16Z)
• REBOOT: Reuse Data for Bootstrapping Efficient Real-World Dexterous Manipulation [61.7171775202833]
  We introduce an efficient system for learning dexterous manipulation skills withReinforcement learning. The main idea of our approach is the integration of recent advances in sample-efficient RL and replay buffer bootstrapping. Our system completes the real-world training cycle by incorporating learned resets via an imitation-based pickup policy.
  arXiv  Detail & Related papers  (2023-09-06T19:05:31Z)
• End-to-end Lidar-Driven Reinforcement Learning for Autonomous Racing [0.0]
  Reinforcement Learning (RL) has emerged as a transformative approach in the domains of automation and robotics. This study develops and trains an RL agent to navigate a racing environment solely using feedforward raw lidar and velocity data. The agent's performance is then experimentally evaluated in a real-world racing scenario.
  arXiv  Detail & Related papers  (2023-09-01T07:03:05Z)
• Actively Learning Costly Reward Functions for Reinforcement Learning [56.34005280792013]
  We show that it is possible to train agents in complex real-world environments orders of magnitudes faster. By enabling the application of reinforcement learning methods to new domains, we show that we can find interesting and non-trivial solutions.
  arXiv  Detail & Related papers  (2022-11-23T19:17:20Z)
• Don't Start From Scratch: Leveraging Prior Data to Automate Robotic Reinforcement Learning [70.70104870417784]
  Reinforcement learning (RL) algorithms hold the promise of enabling autonomous skill acquisition for robotic systems. In practice, real-world robotic RL typically requires time consuming data collection and frequent human intervention to reset the environment. In this work, we study how these challenges can be tackled by effective utilization of diverse offline datasets collected from previously seen tasks.
  arXiv  Detail & Related papers  (2022-07-11T08:31:22Z)
• Influence-Augmented Local Simulators: A Scalable Solution for Fast Deep RL in Large Networked Systems [18.281902746944525]
  In this paper, we study how to build lightweight simulators of complicated systems that can run sufficiently fast for deep RL. We focus on domains where agents interact with a reduced portion of a larger environment while still being affected by the global dynamics. Our method combines the use of local simulators with learned models that mimic the influence of the global system.
  arXiv  Detail & Related papers  (2022-02-03T11:33:58Z)
• Accelerating Robotic Reinforcement Learning via Parameterized Action Primitives [92.0321404272942]
  Reinforcement learning can be used to build general-purpose robotic systems. However, training RL agents to solve robotics tasks still remains challenging. In this work, we manually specify a library of robot action primitives (RAPS), parameterized with arguments that are learned by an RL policy. We find that our simple change to the action interface substantially improves both the learning efficiency and task performance.
  arXiv  Detail & Related papers  (2021-10-28T17:59:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.