Understanding reinforcement learned crowds
- URL: http://arxiv.org/abs/2209.09344v1
- Date: Mon, 19 Sep 2022 20:47:49 GMT
- Title: Understanding reinforcement learned crowds
- Authors: Ariel Kwiatkowski, Vicky Kalogeiton, Julien Pettr\'e, Marie-Paule Cani
- Abstract summary: Reinforcement Learning methods are used to animate virtual agents.
It is not obvious what is their real impact, and how they affect the results.
We analyze some of these arbitrary choices in terms of their impact on the learning performance.
- Score: 9.358303424584902
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Simulating trajectories of virtual crowds is a commonly encountered task in
Computer Graphics. Several recent works have applied Reinforcement Learning
methods to animate virtual agents, however they often make different design
choices when it comes to the fundamental simulation setup. Each of these
choices comes with a reasonable justification for its use, so it is not obvious
what is their real impact, and how they affect the results. In this work, we
analyze some of these arbitrary choices in terms of their impact on the
learning performance, as well as the quality of the resulting simulation
measured in terms of the energy efficiency. We perform a theoretical analysis
of the properties of the reward function design, and empirically evaluate the
impact of using certain observation and action spaces on a variety of
scenarios, with the reward function and energy usage as metrics. We show that
directly using the neighboring agents' information as observation generally
outperforms the more widely used raycasting. Similarly, using nonholonomic
controls with egocentric observations tends to produce more efficient behaviors
than holonomic controls with absolute observations. Each of these choices has a
significant, and potentially nontrivial impact on the results, and so
researchers should be mindful about choosing and reporting them in their work.
Related papers
- What Makes Pre-Trained Visual Representations Successful for Robust
Manipulation? [57.92924256181857]
We find that visual representations designed for manipulation and control tasks do not necessarily generalize under subtle changes in lighting and scene texture.
We find that emergent segmentation ability is a strong predictor of out-of-distribution generalization among ViT models.
arXiv Detail & Related papers (2023-11-03T18:09:08Z) - Reward Function Design for Crowd Simulation via Reinforcement Learning [12.449513548800466]
Reinforcement learning has shown great potential in simulating virtual crowds, but the design of the reward function is critical to achieving effective and efficient results.
We provide theoretical insights on the validity of certain reward functions according to their analytical properties, and evaluate them empirically using a range of scenarios.
Our findings can inform the development of new crowd simulation techniques, and contribute to the wider study of human-like navigation.
arXiv Detail & Related papers (2023-09-22T12:55:30Z) - Dynamic-Resolution Model Learning for Object Pile Manipulation [33.05246884209322]
We investigate how to learn dynamic and adaptive representations at different levels of abstraction to achieve the optimal trade-off between efficiency and effectiveness.
Specifically, we construct dynamic-resolution particle representations of the environment and learn a unified dynamics model using graph neural networks (GNNs)
We show that our method achieves significantly better performance than state-of-the-art fixed-resolution baselines at the gathering, sorting, and redistribution of granular object piles.
arXiv Detail & Related papers (2023-06-29T05:51:44Z) - DOMINO: Visual Causal Reasoning with Time-Dependent Phenomena [59.291745595756346]
We propose a set of visual analytics methods that allow humans to participate in the discovery of causal relations associated with windows of time delay.
Specifically, we leverage a well-established method, logic-based causality, to enable analysts to test the significance of potential causes.
Since an effect can be a cause of other effects, we allow users to aggregate different temporal cause-effect relations found with our method into a visual flow diagram.
arXiv Detail & Related papers (2023-03-12T03:40:21Z) - The Influences of Color and Shape Features in Visual Contrastive
Learning [0.0]
This paper investigates the influences of individual image features (e.g., color and shape) to model performance remain ambiguous.
Experimental results show that compared with supervised representations, contrastive representations tend to cluster with objects of similar color.
arXiv Detail & Related papers (2023-01-29T15:10:14Z) - Task Formulation Matters When Learning Continually: A Case Study in
Visual Question Answering [58.82325933356066]
Continual learning aims to train a model incrementally on a sequence of tasks without forgetting previous knowledge.
We present a detailed study of how different settings affect performance for Visual Question Answering.
arXiv Detail & Related papers (2022-09-30T19:12:58Z) - Supervised Contrastive Learning for Affect Modelling [2.570570340104555]
We introduce three different supervised contrastive learning approaches for training representations that consider affect information.
Results demonstrate the representation capacity of contrastive learning and its efficiency in boosting the accuracy of affect models.
arXiv Detail & Related papers (2022-08-25T17:40:19Z) - An Empirical Investigation of Commonsense Self-Supervision with
Knowledge Graphs [67.23285413610243]
Self-supervision based on the information extracted from large knowledge graphs has been shown to improve the generalization of language models.
We study the effect of knowledge sampling strategies and sizes that can be used to generate synthetic data for adapting language models.
arXiv Detail & Related papers (2022-05-21T19:49:04Z) - Models, Pixels, and Rewards: Evaluating Design Trade-offs in Visual
Model-Based Reinforcement Learning [109.74041512359476]
We study a number of design decisions for the predictive model in visual MBRL algorithms.
We find that a range of design decisions that are often considered crucial, such as the use of latent spaces, have little effect on task performance.
We show how this phenomenon is related to exploration and how some of the lower-scoring models on standard benchmarks will perform the same as the best-performing models when trained on the same training data.
arXiv Detail & Related papers (2020-12-08T18:03:21Z) - Point Cloud Based Reinforcement Learning for Sim-to-Real and Partial
Observability in Visual Navigation [62.22058066456076]
Reinforcement Learning (RL) represents powerful tools to solve complex robotic tasks.
RL does not work directly in the real-world, which is known as the sim-to-real transfer problem.
We propose a method that learns on an observation space constructed by point clouds and environment randomization.
arXiv Detail & Related papers (2020-07-27T17:46:59Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.