Predictive Learning in Energy-based Models with Attractor Structures

• URL: http://arxiv.org/abs/2501.13997v1
• Date: Thu, 23 Jan 2025 11:04:25 GMT
• Title: Predictive Learning in Energy-based Models with Attractor Structures
• Authors: Xingsi Dong, Pengxiang Yuan, Si Wu,
• Abstract summary: We introduce a framework that employs an energy-based model (EBM) to capture the nuanced processes of predicting observation after action within the neural system. In experimental evaluations, our model demonstrates efficacy across diverse scenarios.
• Score: 5.542697199599134
• License:
• Abstract: Predictive models are highly advanced in understanding the mechanisms of brain function. Recent advances in machine learning further underscore the power of prediction for optimal representation in learning. However, there remains a gap in creating a biologically plausible model that explains how the neural system achieves prediction. In this paper, we introduce a framework that employs an energy-based model (EBM) to capture the nuanced processes of predicting observation after action within the neural system, encompassing prediction, learning, and inference. We implement the EBM with a hierarchical structure and integrate a continuous attractor neural network for memory, constructing a biologically plausible model. In experimental evaluations, our model demonstrates efficacy across diverse scenarios. The range of actions includes eye movement, motion in environments, head turning, and static observation while the environment changes. Our model not only makes accurate predictions for environments it was trained on, but also provides reasonable predictions for unseen environments, matching the performances of machine learning methods in multiple tasks. We hope that this study contributes to a deep understanding of how the neural system performs prediction.

Related papers

• Scientific machine learning in ecological systems: A study on the predator-prey dynamics [1.4633779950109127]
  We aim to uncover the underlying differential equations without prior knowledge of the system, relying solely on training data and neural networks. We demonstrate that both Neural ODEs and UDEs can be effectively utilized for prediction and forecasting the LotkaVolterra system. We observe how UDEs outperform Neural ODEs by effectively recovering the underlying dynamics and achieving accurate forecasting with significantly less training data.
  arXiv  Detail & Related papers  (2024-11-11T10:40:45Z)
• BLEND: Behavior-guided Neural Population Dynamics Modeling via Privileged Knowledge Distillation [6.3559178227943764]
  We propose BLEND, a behavior-guided neural population dynamics modeling framework via privileged knowledge distillation. By considering behavior as privileged information, we train a teacher model that takes both behavior observations (privileged features) and neural activities (regular features) as inputs. A student model is then distilled using only neural activity.
  arXiv  Detail & Related papers  (2024-10-02T12:45:59Z)
• CogDPM: Diffusion Probabilistic Models via Cognitive Predictive Coding [62.075029712357]
  This work introduces the Cognitive Diffusion Probabilistic Models (CogDPM) CogDPM features a precision estimation method based on the hierarchical sampling capabilities of diffusion models and weight the guidance with precision weights estimated by the inherent property of diffusion models. We apply CogDPM to real-world prediction tasks using the United Kindom precipitation and surface wind datasets.
  arXiv  Detail & Related papers  (2024-05-03T15:54:50Z)
• Neural Foundations of Mental Simulation: Future Prediction of Latent Representations on Dynamic Scenes [3.2744507958793143]
  We combine a goal-driven modeling approach with dense neurophysiological data and human behavioral readouts to impinge on this question. Specifically, we construct and evaluate several classes of sensory-cognitive networks to predict the future state of rich, ethologically-relevant environments. We find strong differentiation across these model classes in their ability to predict neural and behavioral data both within and across diverse environments.
  arXiv  Detail & Related papers  (2023-05-19T15:56:06Z)
• Learning Theory of Mind via Dynamic Traits Attribution [59.9781556714202]
  We propose a new neural ToM architecture that learns to generate a latent trait vector of an actor from the past trajectories. This trait vector then multiplicatively modulates the prediction mechanism via a fast weights' scheme in the prediction neural network. We empirically show that the fast weights provide a good inductive bias to model the character traits of agents and hence improves mindreading ability.
  arXiv  Detail & Related papers  (2022-04-17T11:21:18Z)
• EINNs: Epidemiologically-Informed Neural Networks [75.34199997857341]
  We introduce a new class of physics-informed neural networks-EINN-crafted for epidemic forecasting. We investigate how to leverage both the theoretical flexibility provided by mechanistic models as well as the data-driven expressability afforded by AI models.
  arXiv  Detail & Related papers  (2022-02-21T18:59:03Z)
• Convolutional Motif Kernel Networks [1.104960878651584]
  We show that our model is able to robustly learn on small datasets and reaches state-of-the-art performance on relevant healthcare prediction tasks. Our proposed method can be utilized on DNA and protein sequences.
  arXiv  Detail & Related papers  (2021-11-03T15:06:09Z)
• The Neural Coding Framework for Learning Generative Models [91.0357317238509]
  We propose a novel neural generative model inspired by the theory of predictive processing in the brain. In a similar way, artificial neurons in our generative model predict what neighboring neurons will do, and adjust their parameters based on how well the predictions matched reality.
  arXiv  Detail & Related papers  (2020-12-07T01:20:38Z)
• Learning to Abstract and Predict Human Actions [60.85905430007731]
  We model the hierarchical structure of human activities in videos and demonstrate the power of such structure in action prediction. We propose Hierarchical-Refresher-Anticipator, a multi-level neural machine that can learn the structure of human activities by observing a partial hierarchy of events and roll-out such structure into a future prediction in multiple levels of abstraction.
  arXiv  Detail & Related papers  (2020-08-20T23:57:58Z)
• The large learning rate phase of deep learning: the catapult mechanism [50.23041928811575]
  We present a class of neural networks with solvable training dynamics. We find good agreement between our model's predictions and training dynamics in realistic deep learning settings. We believe our results shed light on characteristics of models trained at different learning rates.
  arXiv  Detail & Related papers  (2020-03-04T17:52:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.