Multi-Agent Inverse Reinforcement Learning in Real World Unstructured Pedestrian Crowds

• URL: http://arxiv.org/abs/2405.16439v3
• Date: Wed, 26 Mar 2025 21:19:58 GMT
• Title: Multi-Agent Inverse Reinforcement Learning in Real World Unstructured Pedestrian Crowds
• Authors: Rohan Chandra, Haresh Karnan, Negar Mehr, Peter Stone, Joydeep Biswas,
• Abstract summary: We present a new multi-agent maximum entropy inverse reinforcement learning algorithm for real world unstructured pedestrian crowds.<n>Key to our approach is a simple, but effective, mathematical trick which we name the so-called tractability-rationality trade-off trick.<n>Our key findings show that, on the dense Speedway dataset, our approach ranks 1st among top 7 baselines with >2X improvement over single-agent IRL.
• Score: 40.244244289229925
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Social robot navigation in crowded public spaces such as university campuses, restaurants, grocery stores, and hospitals, is an increasingly important area of research. One of the core strategies for achieving this goal is to understand humans' intent--underlying psychological factors that govern their motion--by learning their reward functions, typically via inverse reinforcement learning (IRL). Despite significant progress in IRL, learning reward functions of multiple agents simultaneously in dense unstructured pedestrian crowds has remained intractable due to the nature of the tightly coupled social interactions that occur in these scenarios \textit{e.g.} passing, intersections, swerving, weaving, etc. In this paper, we present a new multi-agent maximum entropy inverse reinforcement learning algorithm for real world unstructured pedestrian crowds. Key to our approach is a simple, but effective, mathematical trick which we name the so-called tractability-rationality trade-off trick that achieves tractability at the cost of a slight reduction in accuracy. We compare our approach to the classical single-agent MaxEnt IRL as well as state-of-the-art trajectory prediction methods on several datasets including the ETH, UCY, SCAND, JRDB, and a new dataset, called Speedway, collected at a busy intersection on a University campus focusing on dense, complex agent interactions. Our key findings show that, on the dense Speedway dataset, our approach ranks 1st among top 7 baselines with >2X improvement over single-agent IRL, and is competitive with state-of-the-art large transformer-based encoder-decoder models on sparser datasets such as ETH/UCY (ranks 3rd among top 7 baselines).

Related papers

• Gen-C: Populating Virtual Worlds with Generative Crowds [1.5293427903448022]
  We introduce Gen-C, a generative model to automate the task of authoring high-level crowd behaviors. Gen-C bypasses the labor-intensive and challenging task of collecting and annotating real crowd video data. We demonstrate the effectiveness of our approach in two scenarios, a University Campus and a Train Station.
  arXiv  Detail & Related papers  (2025-04-02T17:33:53Z)
• HEIGHT: Heterogeneous Interaction Graph Transformer for Robot Navigation in Crowded and Constrained Environments [8.974071308749007]
  We study the problem of robot navigation in dense and interactive crowds with environmental constraints such as corridors and furniture. Previous methods fail to consider all types of interactions among agents and obstacles, leading to unsafe and inefficient robot paths. We propose a structured framework to learn robot navigation policies with reinforcement learning.
  arXiv  Detail & Related papers  (2024-11-19T00:56:35Z)
• VITAL: Visual Teleoperation to Enhance Robot Learning through Human-in-the-Loop Corrections [10.49712834719005]
  We propose a low-cost visual teleoperation system for bimanual manipulation tasks, called VITAL. Our approach leverages affordable hardware and visual processing techniques to collect demonstrations. We enhance the generalizability and robustness of the learned policies by utilizing both real and simulated environments.
  arXiv  Detail & Related papers  (2024-07-30T23:29:47Z)
• Online Context Learning for Socially-compliant Navigation [49.609656402450746]
  This letter introduces an online context learning method that aims to empower robots to adapt to new social environments online. Experiments using a community-wide simulator show that our method outperforms the state-of-the-art ones.
  arXiv  Detail & Related papers  (2024-06-17T12:59:13Z)
• TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correction [25.36756787147331]
  Learning in simulation and transferring the learned policy to the real world has the potential to enable generalist robots. We propose a data-driven approach to enable successful sim-to-real transfer based on a human-in-the-loop framework. We show that our approach can achieve successful sim-to-real transfer in complex and contact-rich manipulation tasks such as furniture assembly.
  arXiv  Detail & Related papers  (2024-05-16T17:59:07Z)
• Attention Graph for Multi-Robot Social Navigation with Deep Reinforcement Learning [0.0]
  We present MultiSoc, a new method for learning multi-agent socially aware navigation strategies using deep reinforcement learning (RL) Inspired by recent works on multi-agent deep RL, our method leverages graph-based representation of agent interactions, combining the positions and fields of view of entities (pedestrians and agents) Our method learns faster than social navigation deep RL mono-agent techniques, and enables efficient multi-agent implicit coordination in challenging crowd navigation with multiple heterogeneous humans.
  arXiv  Detail & Related papers  (2024-01-31T15:24:13Z)
• Multi-Agent Dynamic Relational Reasoning for Social Robot Navigation [50.01551945190676]
  Social robot navigation can be helpful in various contexts of daily life but requires safe human-robot interactions and efficient trajectory planning. We propose a systematic relational reasoning approach with explicit inference of the underlying dynamically evolving relational structures. We demonstrate its effectiveness for multi-agent trajectory prediction and social robot navigation.
  arXiv  Detail & Related papers  (2024-01-22T18:58:22Z)
• A Study on Learning Social Robot Navigation with Multimodal Perception [6.052803245103173]
  We present a study on learning social robot navigation with multimodal perception using a large-scale real-world dataset. We compare unimodal and multimodal learning approaches against a set of classical navigation approaches in different social scenarios. The results show that multimodal learning has a clear advantage over unimodal learning in both dataset and human studies.
  arXiv  Detail & Related papers  (2023-09-22T01:47:47Z)
• Learning Human-to-Robot Handovers from Point Clouds [63.18127198174958]
  We propose the first framework to learn control policies for vision-based human-to-robot handovers. We show significant performance gains over baselines on a simulation benchmark, sim-to-sim transfer and sim-to-real transfer.
  arXiv  Detail & Related papers  (2023-03-30T17:58:36Z)
• Navigating to Objects in the Real World [76.1517654037993]
  We present a large-scale empirical study of semantic visual navigation methods comparing methods from classical, modular, and end-to-end learning approaches. We find that modular learning works well in the real world, attaining a 90% success rate. In contrast, end-to-end learning does not, dropping from 77% simulation to 23% real-world success rate due to a large image domain gap between simulation and reality.
  arXiv  Detail & Related papers  (2022-12-02T01:10:47Z)
• Learning From Good Trajectories in Offline Multi-Agent Reinforcement Learning [98.07495732562654]
  offline multi-agent reinforcement learning (MARL) aims to learn effective multi-agent policies from pre-collected datasets. One agent learned by offline MARL often inherits this random policy, jeopardizing the performance of the entire team. We propose a novel framework called Shared Individual Trajectories (SIT) to address this problem.
  arXiv  Detail & Related papers  (2022-11-28T18:11:26Z)
• DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Reality [64.51295032956118]
  We train a policy that can perform robust dexterous manipulation on an anthropomorphic robot hand. Our work reaffirms the possibilities of sim-to-real transfer for dexterous manipulation in diverse kinds of hardware and simulator setups.
  arXiv  Detail & Related papers  (2022-10-25T01:51:36Z)
• Decentralized Cooperative Multi-Agent Reinforcement Learning with Exploration [35.75029940279768]
  We study multi-agent reinforcement learning in the most basic cooperative setting -- Markov teams. We propose an algorithm in which each agent independently runs a stage-based V-learning style algorithm. We show that the agents can learn an $epsilon$-approximate Nash equilibrium policy in at most $proptowidetildeO (1/epsilon4)$ episodes.
  arXiv  Detail & Related papers  (2021-10-12T02:45:12Z)
• Zero-Shot Reinforcement Learning on Graphs for Autonomous Exploration Under Uncertainty [6.42522897323111]
  We present a framework for self-learning a high-performance exploration policy in a single simulation environment. We propose a novel approach that uses graph neural networks in conjunction with deep reinforcement learning.
  arXiv  Detail & Related papers  (2021-05-11T02:42:17Z)
• PHASE: PHysically-grounded Abstract Social Events for Machine Social Perception [50.551003004553806]
  We create a dataset of physically-grounded abstract social events, PHASE, that resemble a wide range of real-life social interactions. Phase is validated with human experiments demonstrating that humans perceive rich interactions in the social events. As a baseline model, we introduce a Bayesian inverse planning approach, SIMPLE, which outperforms state-of-the-art feed-forward neural networks.
  arXiv  Detail & Related papers  (2021-03-02T18:44:57Z)
• Robot Navigation in a Crowd by Integrating Deep Reinforcement Learning and Online Planning [8.211771115758381]
  It is still an open and challenging problem for mobile robots navigating along time-efficient and collision-free paths in a crowd. Deep reinforcement learning is a promising solution to this problem. We propose a graph-based deep reinforcement learning method, SG-DQN. Our model can help the robot better understand the crowd and achieve a high success rate of more than 0.99 in the crowd navigation task.
  arXiv  Detail & Related papers  (2021-02-26T02:17:13Z)
• PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse Temporal Difference Learning [102.36450942613091]
  We propose an inverse reinforcement learning algorithm, called emphinverse temporal difference learning (ITD) We show how to seamlessly integrate ITD with learning from online environment interactions, arriving at a novel algorithm for reinforcement learning with demonstrations, called $Psi Phi$-learning.
  arXiv  Detail & Related papers  (2021-02-24T21:12:09Z)
• Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey [0.07366405857677225]
  We cover the background behind sim-to-real transfer in deep reinforcement learning. We overview the main methods being utilized at the moment: domain randomization, domain adaptation, imitation learning, meta-learning and knowledge distillation.
  arXiv  Detail & Related papers  (2020-09-24T21:05:46Z)
• End-to-end Contextual Perception and Prediction with Interaction Transformer [79.14001602890417]
  We tackle the problem of detecting objects in 3D and forecasting their future motion in the context of self-driving. To capture their spatial-temporal dependencies, we propose a recurrent neural network with a novel Transformer architecture. Our model can be trained end-to-end, and runs in real-time.
  arXiv  Detail & Related papers  (2020-08-13T14:30:12Z)
• Human Trajectory Forecasting in Crowds: A Deep Learning Perspective [89.4600982169]
  We present an in-depth analysis of existing deep learning-based methods for modelling social interactions. We propose two knowledge-based data-driven methods to effectively capture these social interactions. We develop a large scale interaction-centric benchmark TrajNet++, a significant yet missing component in the field of human trajectory forecasting.
  arXiv  Detail & Related papers  (2020-07-07T17:19:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.