Improving Generalization in Reinforcement Learning Training Regimes for Social Robot Navigation

• URL: http://arxiv.org/abs/2308.14947v2
• Date: Wed, 28 Feb 2024 21:59:22 GMT
• Title: Improving Generalization in Reinforcement Learning Training Regimes for Social Robot Navigation
• Authors: Adam Sigal, Hsiu-Chin Lin, AJung Moon
• Abstract summary: We propose a method to improve the generalization performance of RL social navigation methods using curriculum learning. Our results show that the use of curriculum learning in training can be used to achieve better generalization performance than previous training methods.
• Score: 5.475804640008192
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In order for autonomous mobile robots to navigate in human spaces, they must abide by our social norms. Reinforcement learning (RL) has emerged as an effective method to train sequential decision-making policies that are able to respect these norms. However, a large portion of existing work in the field conducts both RL training and testing in simplistic environments. This limits the generalization potential of these models to unseen environments, and the meaningfulness of their reported results. We propose a method to improve the generalization performance of RL social navigation methods using curriculum learning. By employing multiple environment types and by modeling pedestrians using multiple dynamics models, we are able to progressively diversify and escalate difficulty in training. Our results show that the use of curriculum learning in training can be used to achieve better generalization performance than previous training methods. We also show that results presented in many existing state-of-the-art RL social navigation works do not evaluate their methods outside of their training environments, and thus do not reflect their policies' failure to adequately generalize to out-of-distribution scenarios. In response, we validate our training approach on larger and more crowded testing environments than those used in training, allowing for more meaningful measurements of model performance.

Related papers

• Importance Sampling-Guided Meta-Training for Intelligent Agents in Highly Interactive Environments [43.144056801987595]
  This study introduces a novel training framework that integrates guided meta RL with importance sampling (IS) to optimize training distributions. By estimating a naturalistic distribution from real-world datasets, the framework ensures a balanced focus across common and extreme driving scenarios.
  arXiv  Detail & Related papers  (2024-07-22T17:57:12Z)
• Unlearning with Control: Assessing Real-world Utility for Large Language Model Unlearning [97.2995389188179]
  Recent research has begun to approach large language models (LLMs) unlearning via gradient ascent (GA) Despite their simplicity and efficiency, we suggest that GA-based methods face the propensity towards excessive unlearning. We propose several controlling methods that can regulate the extent of excessive unlearning.
  arXiv  Detail & Related papers  (2024-06-13T14:41:00Z)
• Aquatic Navigation: A Challenging Benchmark for Deep Reinforcement Learning [53.3760591018817]
  We propose a new benchmarking environment for aquatic navigation using recent advances in the integration between game engines and Deep Reinforcement Learning. Specifically, we focus on PPO, one of the most widely accepted algorithms, and we propose advanced training techniques. Our empirical evaluation shows that a well-designed combination of these ingredients can achieve promising results.
  arXiv  Detail & Related papers  (2024-05-30T23:20:23Z)
• Avoidance Navigation Based on Offline Pre-Training Reinforcement Learning [0.0]
  This paper presents a Pre-Training Deep Reinforcement Learning(DRL) for avoidance navigation without map for mobile robots. The efficient offline training strategy is proposed to speed up the inefficient random explorations in early stage. It was demonstrated that our DRL model have universal general capacity in different environment.
  arXiv  Detail & Related papers  (2023-08-03T06:19:46Z)
• Rethinking Closed-loop Training for Autonomous Driving [82.61418945804544]
  We present the first empirical study which analyzes the effects of different training benchmark designs on the success of learning agents. We propose trajectory value learning (TRAVL), an RL-based driving agent that performs planning with multistep look-ahead. Our experiments show that TRAVL can learn much faster and produce safer maneuvers compared to all the baselines.
  arXiv  Detail & Related papers  (2023-06-27T17:58:39Z)
• Representation Learning for Continuous Action Spaces is Beneficial for Efficient Policy Learning [64.14557731665577]
  Deep reinforcement learning (DRL) breaks through the bottlenecks of traditional reinforcement learning (RL) In this paper, we propose an efficient policy learning method in latent state and action spaces. The effectiveness of the proposed method is demonstrated by MountainCar,CarRacing and Cheetah experiments.
  arXiv  Detail & Related papers  (2022-11-23T19:09:37Z)
• Training and Evaluation of Deep Policies using Reinforcement Learning and Generative Models [67.78935378952146]
  GenRL is a framework for solving sequential decision-making problems. It exploits the combination of reinforcement learning and latent variable generative models. We experimentally determine the characteristics of generative models that have most influence on the performance of the final policy training.
  arXiv  Detail & Related papers  (2022-04-18T22:02:32Z)
• Instance based Generalization in Reinforcement Learning [24.485597364200824]
  We analyze policy learning in the context of Partially Observable Markov Decision Processes (POMDPs) We prove that, independently of the exploration strategy, reusing instances introduces significant changes on the effective Markov dynamics the agent observes during training. We propose training a shared belief representation over an ensemble of specialized policies, from which we compute a consensus policy that is used for data collection, disallowing instance specific exploitation.
  arXiv  Detail & Related papers  (2020-11-02T16:19:44Z)
• Dynamics Generalization via Information Bottleneck in Deep Reinforcement Learning [90.93035276307239]
  We propose an information theoretic regularization objective and an annealing-based optimization method to achieve better generalization ability in RL agents. We demonstrate the extreme generalization benefits of our approach in different domains ranging from maze navigation to robotic tasks. This work provides a principled way to improve generalization in RL by gradually removing information that is redundant for task-solving.
  arXiv  Detail & Related papers  (2020-08-03T02:24:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.