Towards self-organized control: Using neural cellular automata to robustly control a cart-pole agent

• URL: http://arxiv.org/abs/2106.15240v2
• Date: Mon, 12 Jul 2021 08:40:56 GMT
• Title: Towards self-organized control: Using neural cellular automata to robustly control a cart-pole agent
• Authors: Alexandre Variengien, Stefano Nichele, Tom Glover and Sidney Pontes-Filho
• Abstract summary: We use neural cellular automata to control a cart-pole agent. We trained the model using deep-Q learning, where the states of the output cells were used as the Q-value estimates to be optimized.
• Score: 62.997667081978825
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Neural cellular automata (Neural CA) are a recent framework used to model biological phenomena emerging from multicellular organisms. In these systems, artificial neural networks are used as update rules for cellular automata. Neural CA are end-to-end differentiable systems where the parameters of the neural network can be learned to achieve a particular task. In this work, we used neural CA to control a cart-pole agent. The observations of the environment are transmitted in input cells, while the values of output cells are used as a readout of the system. We trained the model using deep-Q learning, where the states of the output cells were used as the Q-value estimates to be optimized. We found that the computing abilities of the cellular automata were maintained over several hundreds of thousands of iterations, producing an emergent stable behavior in the environment it controls for thousands of steps. Moreover, the system demonstrated life-like phenomena such as a developmental phase, regeneration after damage, stability despite a noisy environment, and robustness to unseen disruption such as input deletion.

Related papers

• Locally adaptive cellular automata for goal-oriented self-organization [14.059479351946386]
  We propose a new model class of adaptive cellular automata that allows for the generation of scalable and expressive models. We show how to implement adaptation by coupling the update rule of the cellular automaton with itself and the system state in a localized way.
  arXiv  Detail & Related papers  (2023-06-12T12:32:23Z)
• A Bio-Inspired Chaos Sensor Model Based on the Perceptron Neural Network: Machine Learning Concept and Application for Computational Neuro-Science [0.0]
  The study presents a bio-inspired chaos sensor model based on the perceptron neural network for the estimation of entropy of spike train in neurodynamic systems. The model is able to dynamically track the chaotic behavior of a spike signal and transmit this information to other parts of the neurodynamic model for further processing.
  arXiv  Detail & Related papers  (2023-06-03T03:36:47Z)
• Game of Intelligent Life [0.0]
  Recent advances in the field have combined CA with convolutional neural networks to achieve self-regenerating images. The goal of this project is to use the idea of idea of neural cellular automata to grow prediction machines.
  arXiv  Detail & Related papers  (2023-01-02T23:06:26Z)
• Growing Isotropic Neural Cellular Automata [63.91346650159648]
  We argue that the original Growing NCA model has an important limitation: anisotropy of the learned update rule. We demonstrate that cell systems can be trained to grow accurate asymmetrical patterns through either of two methods.
  arXiv  Detail & Related papers  (2022-05-03T11:34:22Z)
• Goal-Guided Neural Cellular Automata: Learning to Control Self-Organising Systems [10.524752369156339]
  We present an approach to control these type of systems called Goal-Guided Neural Cellular Automata (GoalNCA) GoalNCA uses goal encodings to control cell behavior dynamically at every step of cellular growth. We also demonstrate the robustness of the NCA with its ability to preserve task performance, even when only a portion of cells receive goal information.
  arXiv  Detail & Related papers  (2022-04-25T23:11:51Z)
• POPPINS : A Population-Based Digital Spiking Neuromorphic Processor with Integer Quadratic Integrate-and-Fire Neurons [50.591267188664666]
  We propose a population-based digital spiking neuromorphic processor in 180nm process technology with two hierarchy populations. The proposed approach enables the developments of biomimetic neuromorphic system and various low-power, and low-latency inference processing applications.
  arXiv  Detail & Related papers  (2022-01-19T09:26:34Z)
• Continuous Learning and Adaptation with Membrane Potential and Activation Threshold Homeostasis [91.3755431537592]
  This paper presents the Membrane Potential and Activation Threshold Homeostasis (MPATH) neuron model. The model allows neurons to maintain a form of dynamic equilibrium by automatically regulating their activity when presented with input. Experiments demonstrate the model's ability to adapt to and continually learn from its input.
  arXiv  Detail & Related papers  (2021-04-22T04:01:32Z)
• Constrained plasticity reserve as a natural way to control frequency and weights in spiking neural networks [0.0]
  We show how cellular dynamics help neurons to filter out the intense signals to help neurons keep a stable firing rate. Such an approach might be used in the machine learning domain to improve the robustness of AI systems.
  arXiv  Detail & Related papers  (2021-03-15T05:22:14Z)
• Neuromorphic adaptive spiking CPG towards bio-inspired locomotion of legged robots [58.720142291102135]
  Spiking Central Pattern Generator generates different locomotion patterns driven by an external stimulus. The locomotion of the end robotic platform (any-legged robot) can be adapted to the terrain by using any sensor as input.
  arXiv  Detail & Related papers  (2021-01-24T12:44:38Z)
• Non-linear Neurons with Human-like Apical Dendrite Activations [81.18416067005538]
  We show that a standard neuron followed by our novel apical dendrite activation (ADA) can learn the XOR logical function with 100% accuracy. We conduct experiments on six benchmark data sets from computer vision, signal processing and natural language processing.
  arXiv  Detail & Related papers  (2020-02-02T21:09:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.