Learning Successor Features with Distributed Hebbian Temporal Memory

• URL: http://arxiv.org/abs/2310.13391v4
• Date: Sun, 01 Jun 2025 10:53:35 GMT
• Title: Learning Successor Features with Distributed Hebbian Temporal Memory
• Authors: Evgenii Dzhivelikian, Petr Kuderov, Aleksandr I. Panov,
• Abstract summary: This paper presents a novel approach to address the challenge of online sequence learning for decision making under uncertainty.<n>The proposed algorithm, Distributed Hebbian Temporal Memory (DHTM), is based on the factor graph formalism and a multi-component neuron model.<n> Experimental results show that DHTM outperforms LSTM, RWKV and a biologically inspired HMM-like algorithm, CSCG, on non-stationary data sets.
• Score: 44.99833362998488
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper presents a novel approach to address the challenge of online sequence learning for decision making under uncertainty in non-stationary, partially observable environments. The proposed algorithm, Distributed Hebbian Temporal Memory (DHTM), is based on the factor graph formalism and a multi-component neuron model. DHTM aims to capture sequential data relationships and make cumulative predictions about future observations, forming Successor Features (SFs). Inspired by neurophysiological models of the neocortex, the algorithm uses distributed representations, sparse transition matrices, and local Hebbian-like learning rules to overcome the instability and slow learning of traditional temporal memory algorithms such as RNN and HMM. Experimental results show that DHTM outperforms LSTM, RWKV and a biologically inspired HMM-like algorithm, CSCG, on non-stationary data sets. Our results suggest that DHTM is a promising approach to address the challenges of online sequence learning and planning in dynamic environments.

Related papers

• Deep-Unrolling Multidimensional Harmonic Retrieval Algorithms on Neuromorphic Hardware [78.17783007774295]
  This paper explores the potential of conversion-based neuromorphic algorithms for highly accurate and energy-efficient single-snapshot multidimensional harmonic retrieval.<n>A novel method for converting the complex-valued convolutional layers and activations into spiking neural networks (SNNs) is developed.<n>The converted SNNs achieve almost five-fold power efficiency at moderate performance loss compared to the original CNNs.
  arXiv  Detail & Related papers  (2024-12-05T09:41:33Z)
• Generative forecasting of brain activity enhances Alzheimer's classification and interpretation [16.09844316281377]
  Resting-state functional magnetic resonance imaging (rs-fMRI) offers a non-invasive method to monitor neural activity. Deep learning has shown promise in capturing these representations. In this study, we focus on time series forecasting of independent component networks derived from rs-fMRI as a form of data augmentation.
  arXiv  Detail & Related papers  (2024-10-30T23:51:31Z)
• Trajectory Flow Matching with Applications to Clinical Time Series Modeling [77.58277281319253]
  Trajectory Flow Matching (TFM) trains a Neural SDE in a simulation-free manner, bypassing backpropagation through the dynamics. We demonstrate improved performance on three clinical time series datasets in terms of absolute performance and uncertainty prediction.
  arXiv  Detail & Related papers  (2024-10-28T15:54:50Z)
• SINDyG: Sparse Identification of Nonlinear Dynamical Systems from Graph-Structured Data, with Applications to Stuart-Landau Oscillator Networks [0.27624021966289597]
  We develop a new method called Sparse Identification of Dynamical Systems from Graph-structured data (SINDyG)<n>SINDyG incorporates the network structure into sparse regression to identify model parameters that explain the underlying network dynamics.<n>Our experiments validate the improved accuracy and simplicity of discovered network dynamics.
  arXiv  Detail & Related papers  (2024-09-02T17:51:37Z)
• GRSN: Gated Recurrent Spiking Neurons for POMDPs and MARL [28.948871773551854]
  Spiking neural networks (SNNs) are widely applied in various fields due to their energy-efficient and fast-inference capabilities. In current spiking reinforcement learning (SRL) algorithms, the simulation results of multiple time steps can only correspond to a single-step decision in RL. We propose a novel temporal alignment paradigm (TAP) that leverages the single-step update of spiking neurons to accumulate historical state information in RL.
  arXiv  Detail & Related papers  (2024-04-24T02:20:50Z)
• Spatial-Temporal DAG Convolutional Networks for End-to-End Joint Effective Connectivity Learning and Resting-State fMRI Classification [42.82118108887965]
  Building comprehensive brain connectomes has proved to be fundamental importance in resting-state fMRI (rs-fMRI) analysis. We model the brain network as a directed acyclic graph (DAG) to discover direct causal connections between brain regions. We propose Spatial-Temporal DAG Convolutional Network (ST-DAGCN) to jointly infer effective connectivity and classify rs-fMRI time series.
  arXiv  Detail & Related papers  (2023-12-16T04:31:51Z)
• Brain-Inspired Spiking Neural Network for Online Unsupervised Time Series Prediction [13.521272923545409]
  We present a novel Continuous Learning-based Unsupervised Recurrent Spiking Neural Network Model (CLURSNN) CLURSNN makes online predictions by reconstructing the underlying dynamical system using Random Delay Embedding. We show that the proposed online time series prediction methodology outperforms state-of-the-art DNN models when predicting an evolving Lorenz63 dynamical system.
  arXiv  Detail & Related papers  (2023-04-10T16:18:37Z)
• Composite FORCE learning of chaotic echo state networks for time-series prediction [7.650966670809372]
  This paper proposes a composite FORCE learning method to train ESNs whose initial activity is spontaneously chaotic. numerical results have shown that it significantly improves learning and prediction performances compared with existing methods.
  arXiv  Detail & Related papers  (2022-07-06T03:44:09Z)
• Axonal Delay As a Short-Term Memory for Feed Forward Deep Spiking Neural Networks [3.985532502580783]
  Recent studies have found that the time delay of neurons plays an important role in the learning process. configuring the precise timing of the spike is a promising direction for understanding and improving the transmission process of temporal information in SNNs. In this paper, we verify the effectiveness of integrating time delay into supervised learning and propose a module that modulates the axonal delay through short-term memory.
  arXiv  Detail & Related papers  (2022-04-20T16:56:42Z)
• Predicting the temporal dynamics of turbulent channels through deep learning [0.0]
  We aim to assess the capability of neural networks to reproduce the temporal evolution of a minimal turbulent channel flow. Long-short-term-memory (LSTM) networks and a Koopman-based framework (KNF) are trained to predict the temporal dynamics of the minimal-channel-flow modes.
  arXiv  Detail & Related papers  (2022-03-02T09:31:03Z)
• Convolutional generative adversarial imputation networks for spatio-temporal missing data in storm surge simulations [86.5302150777089]
  Generative Adversarial Imputation Nets (GANs) and GAN-based techniques have attracted attention as unsupervised machine learning methods. We name our proposed method as Con Conval Generative Adversarial Imputation Nets (Conv-GAIN)
  arXiv  Detail & Related papers  (2021-11-03T03:50:48Z)
• Cloud Failure Prediction with Hierarchical Temporary Memory: An Empirical Assessment [64.73243241568555]
  Hierarchical Temporary Memory (HTM) is an unsupervised learning algorithm inspired by the features of the neocortex. This paper presents the first systematic study that assesses HTM in the context of failure prediction.
  arXiv  Detail & Related papers  (2021-10-06T07:09:45Z)
• Deep Bayesian Active Learning for Accelerating Stochastic Simulation [74.58219903138301]
  Interactive Neural Process (INP) is a deep active learning framework for simulations and with active learning approaches. For active learning, we propose a novel acquisition function, Latent Information Gain (LIG), calculated in the latent space of NP based models. The results demonstrate STNP outperforms the baselines in the learning setting and LIG achieves the state-of-the-art for active learning.
  arXiv  Detail & Related papers  (2021-06-05T01:31:51Z)
• Influence Estimation and Maximization via Neural Mean-Field Dynamics [60.91291234832546]
  We propose a novel learning framework using neural mean-field (NMF) dynamics for inference and estimation problems. Our framework can simultaneously learn the structure of the diffusion network and the evolution of node infection probabilities.
  arXiv  Detail & Related papers  (2021-06-03T00:02:05Z)
• Dynamic Mode Decomposition in Adaptive Mesh Refinement and Coarsening Simulations [58.720142291102135]
  Dynamic Mode Decomposition (DMD) is a powerful data-driven method used to extract coherent schemes. This paper proposes a strategy to enable DMD to extract from observations with different mesh topologies and dimensions.
  arXiv  Detail & Related papers  (2021-04-28T22:14:25Z)
• Rectified Linear Postsynaptic Potential Function for Backpropagation in Deep Spiking Neural Networks [55.0627904986664]
  Spiking Neural Networks (SNNs) usetemporal spike patterns to represent and transmit information, which is not only biologically realistic but also suitable for ultra-low-power event-driven neuromorphic implementation. This paper investigates the contribution of spike timing dynamics to information encoding, synaptic plasticity and decision making, providing a new perspective to design of future DeepSNNs and neuromorphic hardware systems.
  arXiv  Detail & Related papers  (2020-03-26T11:13:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.