Leveraging Human Feedback to Evolve and Discover Novel Emergent
Behaviors in Robot Swarms
- URL: http://arxiv.org/abs/2305.16148v2
- Date: Sun, 16 Jul 2023 20:05:40 GMT
- Title: Leveraging Human Feedback to Evolve and Discover Novel Emergent
Behaviors in Robot Swarms
- Authors: Connor Mattson, Daniel S. Brown
- Abstract summary: We seek to leverage human input to automatically discover a taxonomy of collective behaviors that can emerge from a particular multi-agent system.
Our proposed approach adapts to user preferences by learning a similarity space over swarm collective behaviors.
We test our approach in simulation on two robot capability models and show that our methods consistently discover a richer set of emergent behaviors than prior work.
- Score: 14.404339094377319
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Robot swarms often exhibit emergent behaviors that are fascinating to
observe; however, it is often difficult to predict what swarm behaviors can
emerge under a given set of agent capabilities. We seek to efficiently leverage
human input to automatically discover a taxonomy of collective behaviors that
can emerge from a particular multi-agent system, without requiring the human to
know beforehand what behaviors are interesting or even possible. Our proposed
approach adapts to user preferences by learning a similarity space over swarm
collective behaviors using self-supervised learning and human-in-the-loop
queries. We combine our learned similarity metric with novelty search and
clustering to explore and categorize the space of possible swarm behaviors. We
also propose several general-purpose heuristics that improve the efficiency of
our novelty search by prioritizing robot controllers that are likely to lead to
interesting emergent behaviors. We test our approach in simulation on two robot
capability models and show that our methods consistently discover a richer set
of emergent behaviors than prior work. Code, videos, and datasets are available
at https://sites.google.com/view/evolving-novel-swarms.
Related papers
- Innate Motivation for Robot Swarms by Minimizing Surprise: From Simple Simulations to Real-World Experiments [6.21540494241516]
Large-scale mobile multi-robot systems can be beneficial over monolithic robots because of higher potential for robustness and scalability.
Developing controllers for multi-robot systems is challenging because the multitude of interactions is hard to anticipate and difficult to model.
Innate motivation tries to avoid the specific formulation of rewards and work instead with different drivers, such as curiosity.
A unique advantage of the swarm robot case is that swarm members populate the robot's environment and can trigger more active behaviors in a self-referential loop.
arXiv Detail & Related papers (2024-05-04T06:25:58Z) - Promptable Behaviors: Personalizing Multi-Objective Rewards from Human
Preferences [53.353022588751585]
We present Promptable Behaviors, a novel framework that facilitates efficient personalization of robotic agents to diverse human preferences.
We introduce three distinct methods to infer human preferences by leveraging different types of interactions.
We evaluate the proposed method in personalized object-goal navigation and flee navigation tasks in ProcTHOR and RoboTHOR.
arXiv Detail & Related papers (2023-12-14T21:00:56Z) - Exploring Behavior Discovery Methods for Heterogeneous Swarms of
Limited-Capability Robots [9.525230669966415]
We study the problem of determining the emergent behaviors that are possible given a functionally heterogeneous swarm of robots.
To the best of our knowledge, these are the first known emergent behaviors for heterogeneous swarms of computation-free agents.
arXiv Detail & Related papers (2023-10-25T19:20:32Z) - Learning NEAT Emergent Behaviors in Robot Swarms [1.0958014189747356]
We present a method of training distributed robotic swarm algorithms to produce emergent behavior.
Inspired by the biological evolution of emergent behavior in animals, we use an evolutionary algorithm to train a population of individual behaviors.
We evaluate our algorithm on various tasks where a somewhat complex group behavior is required for success.
arXiv Detail & Related papers (2023-09-26T04:40:52Z) - SACSoN: Scalable Autonomous Control for Social Navigation [62.59274275261392]
We develop methods for training policies for socially unobtrusive navigation.
By minimizing this counterfactual perturbation, we can induce robots to behave in ways that do not alter the natural behavior of humans in the shared space.
We collect a large dataset where an indoor mobile robot interacts with human bystanders.
arXiv Detail & Related papers (2023-06-02T19:07:52Z) - Incremental procedural and sensorimotor learning in cognitive humanoid
robots [52.77024349608834]
This work presents a cognitive agent that can learn procedures incrementally.
We show the cognitive functions required in each substage and how adding new functions helps address tasks previously unsolved by the agent.
Results show that this approach is capable of solving complex tasks incrementally.
arXiv Detail & Related papers (2023-04-30T22:51:31Z) - Inferring Versatile Behavior from Demonstrations by Matching Geometric
Descriptors [72.62423312645953]
Humans intuitively solve tasks in versatile ways, varying their behavior in terms of trajectory-based planning and for individual steps.
Current Imitation Learning algorithms often only consider unimodal expert demonstrations and act in a state-action-based setting.
Instead, we combine a mixture of movement primitives with a distribution matching objective to learn versatile behaviors that match the expert's behavior and versatility.
arXiv Detail & Related papers (2022-10-17T16:42:59Z) - A-ACT: Action Anticipation through Cycle Transformations [89.83027919085289]
We take a step back to analyze how the human capability to anticipate the future can be transferred to machine learning algorithms.
A recent study on human psychology explains that, in anticipating an occurrence, the human brain counts on both systems.
In this work, we study the impact of each system for the task of action anticipation and introduce a paradigm to integrate them in a learning framework.
arXiv Detail & Related papers (2022-04-02T21:50:45Z) - Collective motion emerging from evolving swarm controllers in different
environments using gradient following task [2.7402733069181]
We consider a challenging task where robots with limited sensing and communication abilities must follow the gradient of an environmental feature.
We use Differential Evolution to evolve a neural network controller for simulated Thymio II robots.
Experiments confirm the feasibility of our approach, the evolved robot controllers induced swarm behaviour that solved the task.
arXiv Detail & Related papers (2022-03-22T10:08:50Z) - Beyond Tracking: Using Deep Learning to Discover Novel Interactions in
Biological Swarms [3.441021278275805]
We propose training deep network models to predict system-level states directly from generic graphical features from the entire view.
Because the resulting predictive models are not based on human-understood predictors, we use explanatory modules.
This represents an example of augmented intelligence in behavioral ecology -- knowledge co-creation in a human-AI team.
arXiv Detail & Related papers (2021-08-20T22:50:41Z) - Learning Predictive Models From Observation and Interaction [137.77887825854768]
Learning predictive models from interaction with the world allows an agent, such as a robot, to learn about how the world works.
However, learning a model that captures the dynamics of complex skills represents a major challenge.
We propose a method to augment the training set with observational data of other agents, such as humans.
arXiv Detail & Related papers (2019-12-30T01:10:41Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.