Dynamical Equations With Bottom-up Self-Organizing Properties Learn Accurate Dynamical Hierarchies Without Any Loss Function

• URL: http://arxiv.org/abs/2302.02140v1
• Date: Sat, 4 Feb 2023 10:00:14 GMT
• Title: Dynamical Equations With Bottom-up Self-Organizing Properties Learn Accurate Dynamical Hierarchies Without Any Loss Function
• Authors: Danilo Vasconcellos Vargas, Tham Yik Foong, Heng Zhang
• Abstract summary: We propose a learning system where patterns are defined within the realm of nonlinear dynamics with positive and negative feedback loops. Experiments reveal that such a system can map temporal to spatial correlation, enabling hierarchical structures to be learned from sequential data.
• Score: 15.122944754472435
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Self-organization is ubiquitous in nature and mind. However, machine learning and theories of cognition still barely touch the subject. The hurdle is that general patterns are difficult to define in terms of dynamical equations and designing a system that could learn by reordering itself is still to be seen. Here, we propose a learning system, where patterns are defined within the realm of nonlinear dynamics with positive and negative feedback loops, allowing attractor-repeller pairs to emerge for each pattern observed. Experiments reveal that such a system can map temporal to spatial correlation, enabling hierarchical structures to be learned from sequential data. The results are accurate enough to surpass state-of-the-art unsupervised learning algorithms in seven out of eight experiments as well as two real-world problems. Interestingly, the dynamic nature of the system makes it inherently adaptive, giving rise to phenomena similar to phase transitions in chemistry/thermodynamics when the input structure changes. Thus, the work here sheds light on how self-organization can allow for pattern recognition and hints at how intelligent behavior might emerge from simple dynamic equations without any objective/loss function.

Related papers

• Learning System Dynamics without Forgetting [60.08612207170659]
  Predicting trajectories of systems with unknown dynamics is crucial in various research fields, including physics and biology. We present a novel framework of Mode-switching Graph ODE (MS-GODE), which can continually learn varying dynamics. We construct a novel benchmark of biological dynamic systems, featuring diverse systems with disparate dynamics.
  arXiv  Detail & Related papers  (2024-06-30T14:55:18Z)
• AI-Lorenz: A physics-data-driven framework for black-box and gray-box identification of chaotic systems with symbolic regression [2.07180164747172]
  We develop a framework that learns mathematical expressions modeling complex dynamical behaviors. We train a small neural network to learn the dynamics of a system, its rate of change in time, and missing model terms. This, in turn, enables us to predict the future evolution of the dynamical behavior.
  arXiv  Detail & Related papers  (2023-12-21T18:58:41Z)
• A Waddington landscape for prototype learning in generalized Hopfield networks [0.0]
  We study the learning dynamics of Generalized Hopfield networks. We observe a strong resemblance to the canalized, or low-dimensional, dynamics of cells as they differentiate.
  arXiv  Detail & Related papers  (2023-12-04T21:28:14Z)
• Latent Traversals in Generative Models as Potential Flows [113.4232528843775]
  We propose to model latent structures with a learned dynamic potential landscape. Inspired by physics, optimal transport, and neuroscience, these potential landscapes are learned as physically realistic partial differential equations. Our method achieves both more qualitatively and quantitatively disentangled trajectories than state-of-the-art baselines.
  arXiv  Detail & Related papers  (2023-04-25T15:53:45Z)
• Learning Individual Interactions from Population Dynamics with Discrete-Event Simulation Model [9.827590402695341]
  We will explore the possibility of learning a discrete-event simulation representation of complex system dynamics. Our results show that the algorithm can data-efficiently capture complex network dynamics in several fields with meaningful events.
  arXiv  Detail & Related papers  (2022-05-04T21:33:56Z)
• Capturing Actionable Dynamics with Structured Latent Ordinary Differential Equations [68.62843292346813]
  We propose a structured latent ODE model that captures system input variations within its latent representation. Building on a static variable specification, our model learns factors of variation for each input to the system, thus separating the effects of the system inputs in the latent space.
  arXiv  Detail & Related papers  (2022-02-25T20:00:56Z)
• Improving Coherence and Consistency in Neural Sequence Models with Dual-System, Neuro-Symbolic Reasoning [49.6928533575956]
  We use neural inference to mediate between the neural System 1 and the logical System 2. Results in robust story generation and grounded instruction-following show that this approach can increase the coherence and accuracy of neurally-based generations.
  arXiv  Detail & Related papers  (2021-07-06T17:59:49Z)
• Learning Continuous System Dynamics from Irregularly-Sampled Partial Observations [33.63818978256567]
  We present LG-ODE, a latent ordinary differential equation generative model for modeling multi-agent dynamic system with known graph structure. It can simultaneously learn the embedding of high dimensional trajectories and infer continuous latent system dynamics. Our model employs a novel encoder parameterized by a graph neural network that can infer initial states in an unsupervised way.
  arXiv  Detail & Related papers  (2020-11-08T01:02:22Z)
• Hierarchically Organized Latent Modules for Exploratory Search in Morphogenetic Systems [21.23182328329019]
  We introduce a novel dynamic and modular architecture that enables unsupervised learning of a hierarchy of diverse representations. We show that this system forms a discovery assistant that can efficiently adapt its diversity search towards preferences of a user.
  arXiv  Detail & Related papers  (2020-07-02T15:28:27Z)
• Euclideanizing Flows: Diffeomorphic Reduction for Learning Stable Dynamical Systems [74.80320120264459]
  We present an approach to learn such motions from a limited number of human demonstrations. The complex motions are encoded as rollouts of a stable dynamical system. The efficacy of this approach is demonstrated through validation on an established benchmark as well demonstrations collected on a real-world robotic system.
  arXiv  Detail & Related papers  (2020-05-27T03:51:57Z)
• Learning Stable Deep Dynamics Models [91.90131512825504]
  We propose an approach for learning dynamical systems that are guaranteed to be stable over the entire state space. We show that such learning systems are able to model simple dynamical systems and can be combined with additional deep generative models to learn complex dynamics.
  arXiv  Detail & Related papers  (2020-01-17T00:04:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.