Eco-evolutionary Dynamics of Non-episodic Neuroevolution in Large Multi-agent Environments

• URL: http://arxiv.org/abs/2302.09334v3
• Date: Fri, 4 Aug 2023 12:08:41 GMT
• Title: Eco-evolutionary Dynamics of Non-episodic Neuroevolution in Large Multi-agent Environments
• Authors: Gautier Hamon and Eleni Nisioti and Cl\'ement Moulin-Frier
• Abstract summary: We present a method for continuously evolving adaptive agents without any environment or population reset. We show that NE can operate in an ecologically non-episodic multi-agent setting, finding sustainable collective foraging strategies.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Neuroevolution (NE) has recently proven a competitive alternative to learning by gradient descent in reinforcement learning tasks. However, the majority of NE methods and associated simulation environments differ crucially from biological evolution: the environment is reset to initial conditions at the end of each generation, whereas natural environments are continuously modified by their inhabitants; agents reproduce based on their ability to maximize rewards within a population, while biological organisms reproduce and die based on internal physiological variables that depend on their resource consumption; simulation environments are primarily single-agent while the biological world is inherently multi-agent and evolves alongside the population. In this work we present a method for continuously evolving adaptive agents without any environment or population reset. The environment is a large grid world with complex spatiotemporal resource generation, containing many agents that are each controlled by an evolvable recurrent neural network and locally reproduce based on their internal physiology. The entire system is implemented in JAX, allowing very fast simulation on a GPU. We show that NE can operate in an ecologically-valid non-episodic multi-agent setting, finding sustainable collective foraging strategies in the presence of a complex interplay between ecological and evolutionary dynamics.

Related papers

• Meta Flow Matching: Integrating Vector Fields on the Wasserstein Manifold [83.18058549195855]
  We argue that multiple processes in natural sciences have to be represented as vector fields on the Wasserstein manifold of probability densities. In particular, this is crucial for personalized medicine where the development of diseases and their respective treatment response depends on the microenvironment of cells specific to each patient. We propose Meta Flow Matching (MFM), a practical approach to integrating along these vector fields on the Wasserstein manifold by amortizing the flow model over the initial populations.
  arXiv  Detail & Related papers  (2024-08-26T20:05:31Z)
• Agent Alignment in Evolving Social Norms [65.45423591744434]
  We propose an evolutionary framework for agent evolution and alignment, named EvolutionaryAgent. In an environment where social norms continuously evolve, agents better adapted to the current social norms will have a higher probability of survival and proliferation. We show that EvolutionaryAgent can align progressively better with the evolving social norms while maintaining its proficiency in general tasks.
  arXiv  Detail & Related papers  (2024-01-09T15:44:44Z)
• DARLEI: Deep Accelerated Reinforcement Learning with Evolutionary Intelligence [77.78795329701367]
  We present DARLEI, a framework that combines evolutionary algorithms with parallelized reinforcement learning. We characterize DARLEI's performance under various conditions, revealing factors impacting diversity of evolved morphologies. We hope to extend DARLEI in future work to include interactions between diverse morphologies in richer environments.
  arXiv  Detail & Related papers  (2023-12-08T16:51:10Z)
• Dynamics of niche construction in adaptable populations evolving in diverse environments [0.0]
  niche construction (NC) is the reciprocal process to natural selection where individuals inheritable changes to their environment. We study NC in simulation environments that consist of multiple, diverse niches and populations that evolve their plasticity, evolvability and niche-constructing behaviors. Our study suggests that complexifying the simulation environments studying NC, by considering multiple and diverse niches, is necessary for understanding its dynamics.
  arXiv  Detail & Related papers  (2023-05-16T11:52:14Z)
• Using Genetic Algorithms to Simulate Evolution [0.0]
  We are able to predict future changes as well as simulate the process using genetic algorithms. By optimizing genetic algorithms to hold entities in an environment, we are able to assign varying characteristics such as speed, size, and cloning probability. Learning about how species grow and evolve allows us to find ways to improve technology, help animals going extinct to survive, and figure* out how diseases spread.
  arXiv  Detail & Related papers  (2022-09-14T00:23:06Z)
• Towards the Neuroevolution of Low-level Artificial General Intelligence [5.2611228017034435]
  We argue that the search for Artificial General Intelligence (AGI) should start from a much lower level than human-level intelligence. Our hypothesis is that learning occurs through sensory feedback when an agent acts in an environment. We evaluate a method to evolve a biologically-inspired artificial neural network that learns from environment reactions.
  arXiv  Detail & Related papers  (2022-07-27T15:30:50Z)
• The Introspective Agent: Interdependence of Strategy, Physiology, and Sensing for Embodied Agents [51.94554095091305]
  We argue for an introspective agent, which considers its own abilities in the context of its environment. Just as in nature, we hope to reframe strategy as one tool, among many, to succeed in an environment.
  arXiv  Detail & Related papers  (2022-01-02T20:14:01Z)
• Continuous Learning and Adaptation with Membrane Potential and Activation Threshold Homeostasis [91.3755431537592]
  This paper presents the Membrane Potential and Activation Threshold Homeostasis (MPATH) neuron model. The model allows neurons to maintain a form of dynamic equilibrium by automatically regulating their activity when presented with input. Experiments demonstrate the model's ability to adapt to and continually learn from its input.
  arXiv  Detail & Related papers  (2021-04-22T04:01:32Z)
• Co-optimising Robot Morphology and Controller in a Simulated Open-Ended Environment [1.4502611532302039]
  We show how changing the environment, where the agent locomotes, affects the convergence of morphologies. We show that agent-populations evolving in open-endedly evolving environments exhibit larger morphological diversity than agent-populations evolving in hand crafted curricula of environments.
  arXiv  Detail & Related papers  (2021-04-07T11:28:23Z)
• Ecological Reinforcement Learning [76.9893572776141]
  We study the kinds of environment properties that can make learning under such conditions easier. understanding how properties of the environment impact the performance of reinforcement learning agents can help us to structure our tasks in ways that make learning tractable.
  arXiv  Detail & Related papers  (2020-06-22T17:55:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.