Emergence of the Primacy Effect in Structured State-Space Models

• URL: http://arxiv.org/abs/2502.13729v2
• Date: Thu, 20 Feb 2025 07:51:22 GMT
• Title: Emergence of the Primacy Effect in Structured State-Space Models
• Authors: Takashi Morita,
• Abstract summary: artificial neural network (ANN) models are typically designed with a memory that decays monotonically over time. This study reveals a counterintuitive finding: a recently developed ANN architecture, called structured state-space models, exhibits the primacy effect when trained and evaluated.
• Score: 1.4594704809280983
• License:
• Abstract: Human and animal memory for sequentially presented items is well-documented to be more accurate for those at the beginning and end of the sequence, phenomena known as the primacy and recency effects, respectively. By contrast, artificial neural network (ANN) models are typically designed with a memory that decays monotonically over time. Accordingly, ANNs are expected to show the recency effect but not the primacy effect. Contrary to this theoretical expectation, however, the present study reveals a counterintuitive finding: a recently developed ANN architecture, called structured state-space models, exhibits the primacy effect when trained and evaluated on a synthetic task that mirrors psychological memory experiments. Given that this model was originally designed for recovering neuronal activity patterns observed in biological brains, this result provides a novel perspective on the psychological primacy effect while also posing a non-trivial puzzle for the current theories in machine learning.

Related papers

• Artificial Kuramoto Oscillatory Neurons [65.16453738828672]
  It has long been known in both neuroscience and AI that ''binding'' between neurons leads to a form of competitive learning. We introduce Artificial rethinking together with arbitrary connectivity designs such as fully connected convolutional, or attentive mechanisms. We show that this idea provides performance improvements across a wide spectrum of tasks such as unsupervised object discovery, adversarial robustness, uncertainty, and reasoning.
  arXiv  Detail & Related papers  (2024-10-17T17:47:54Z)
• Neural Dynamics Model of Visual Decision-Making: Learning from Human Experts [28.340344705437758]
  We implement a comprehensive visual decision-making model that spans from visual input to behavioral output. Our model aligns closely with human behavior and reflects neural activities in primates. A neuroimaging-informed fine-tuning approach was introduced and applied to the model, leading to performance improvements.
  arXiv  Detail & Related papers  (2024-09-04T02:38:52Z)
• Brain-inspired Computational Modeling of Action Recognition with Recurrent Spiking Neural Networks Equipped with Reinforcement Delay Learning [4.9798155883849935]
  Action recognition has received significant attention due to its intricate nature and the brain's exceptional performance in this area. Current solutions for action recognition either exhibit limitations in effectively addressing the problem or lack the necessary biological plausibility. This article presents an effective brain-inspired computational model for action recognition.
  arXiv  Detail & Related papers  (2024-06-17T17:34:16Z)
• The Impact of Geometric Complexity on Neural Collapse in Transfer Learning [6.554326244334867]
  Flatness of the loss surface and neural collapse have recently emerged as useful pre-training metrics. We show through experiments and theory that mechanisms which affect the geometric complexity of the pre-trained network also influence the neural collapse.
  arXiv  Detail & Related papers  (2024-05-24T16:52:09Z)
• A Neuro-mimetic Realization of the Common Model of Cognition via Hebbian Learning and Free Energy Minimization [55.11642177631929]
  Large neural generative models are capable of synthesizing semantically rich passages of text or producing complex images. We discuss the COGnitive Neural GENerative system, such an architecture that casts the Common Model of Cognition.
  arXiv  Detail & Related papers  (2023-10-14T23:28:48Z)
• Contrastive-Signal-Dependent Plasticity: Self-Supervised Learning in Spiking Neural Circuits [61.94533459151743]
  This work addresses the challenge of designing neurobiologically-motivated schemes for adjusting the synapses of spiking networks. Our experimental simulations demonstrate a consistent advantage over other biologically-plausible approaches when training recurrent spiking networks.
  arXiv  Detail & Related papers  (2023-03-30T02:40:28Z)
• Plasticity Neural Network Based on Astrocytic Influence at Critical Periods, Synaptic Competition and Compensation by Current and Mnemonic Brain Plasticity and Synapse Formation [7.8787868286474]
  Based on the RNN frame, we accomplished the model construction, formula derivation and algorithm testing for PNN. The question we proposed is whether the promotion of neuroscience and brain cognition was achieved by model construction, formula derivation or algorithm testing.
  arXiv  Detail & Related papers  (2022-03-19T14:38:54Z)
• Mapping and Validating a Point Neuron Model on Intel's Neuromorphic Hardware Loihi [77.34726150561087]
  We investigate the potential of Intel's fifth generation neuromorphic chip - Loihi' Loihi is based on the novel idea of Spiking Neural Networks (SNNs) emulating the neurons in the brain. We find that Loihi replicates classical simulations very efficiently and scales notably well in terms of both time and energy performance as the networks get larger.
  arXiv  Detail & Related papers  (2021-09-22T16:52:51Z)
• Towards a Predictive Processing Implementation of the Common Model of Cognition [79.63867412771461]
  We describe an implementation of the common model of cognition grounded in neural generative coding and holographic associative memory. The proposed system creates the groundwork for developing agents that learn continually from diverse tasks as well as model human performance at larger scales.
  arXiv  Detail & Related papers  (2021-05-15T22:55:23Z)
• Object-based attention for spatio-temporal reasoning: Outperforming neuro-symbolic models with flexible distributed architectures [15.946511512356878]
  We show that a fully-learned neural network with the right inductive biases can perform substantially better than all previous neural-symbolic models. Our model makes critical use of both self-attention and learned "soft" object-centric representations.
  arXiv  Detail & Related papers  (2020-12-15T18:57:40Z)
• Fooling the primate brain with minimal, targeted image manipulation [67.78919304747498]
  We propose an array of methods for creating minimal, targeted image perturbations that lead to changes in both neuronal activity and perception as reflected in behavior. Our work shares the same goal with adversarial attack, namely the manipulation of images with minimal, targeted noise that leads ANN models to misclassify the images.
  arXiv  Detail & Related papers  (2020-11-11T08:30:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.