Hierarchical Working Memory and a New Magic Number

• URL: http://arxiv.org/abs/2408.07637v1
• Date: Wed, 14 Aug 2024 16:03:47 GMT
• Title: Hierarchical Working Memory and a New Magic Number
• Authors: Weishun Zhong, Mikhail Katkov, Misha Tsodyks,
• Abstract summary: We propose a recurrent neural network model for chunking within the framework of the synaptic theory of working memory. Our work provides a novel conceptual and analytical framework for understanding the on-the-fly organization of information in the brain that is crucial for cognition.
• Score: 1.024113475677323
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The extremely limited working memory span, typically around four items, contrasts sharply with our everyday experience of processing much larger streams of sensory information concurrently. This disparity suggests that working memory can organize information into compact representations such as chunks, yet the underlying neural mechanisms remain largely unknown. Here, we propose a recurrent neural network model for chunking within the framework of the synaptic theory of working memory. We showed that by selectively suppressing groups of stimuli, the network can maintain and retrieve the stimuli in chunks, hence exceeding the basic capacity. Moreover, we show that our model can dynamically construct hierarchical representations within working memory through hierarchical chunking. A consequence of this proposed mechanism is a new limit on the number of items that can be stored and subsequently retrieved from working memory, depending only on the basic working memory capacity when chunking is not invoked. Predictions from our model were confirmed by analyzing single-unit responses in epileptic patients and memory experiments with verbal material. Our work provides a novel conceptual and analytical framework for understanding the on-the-fly organization of information in the brain that is crucial for cognition.

Related papers

• Storing overlapping associative memories on latent manifolds in low-rank spiking networks [5.041384008847852]
  We revisit the associative memory problem in light of advances in understanding spike-based computation. We show that the spiking activity for a large class of all-inhibitory networks is situated on a low-dimensional, convex, and piecewise-linear manifold. We propose several learning rules, and demonstrate a linear scaling of the storage capacity with the number of neurons, as well as robust pattern completion abilities.
  arXiv  Detail & Related papers  (2024-11-26T14:48:25Z)
• MindBridge: A Cross-Subject Brain Decoding Framework [60.58552697067837]
  Brain decoding aims to reconstruct stimuli from acquired brain signals. Currently, brain decoding is confined to a per-subject-per-model paradigm. We present MindBridge, that achieves cross-subject brain decoding by employing only one model.
  arXiv  Detail & Related papers  (2024-04-11T15:46:42Z)
• Sequential Memory with Temporal Predictive Coding [6.228559238589584]
  We propose a PC-based model for emphsequential memory, called emphtemporal predictive coding (tPC) We show that our tPC models can memorize and retrieve sequential inputs accurately with a biologically plausible neural implementation.
  arXiv  Detail & Related papers  (2023-05-19T20:03:31Z)
• Measures of Information Reflect Memorization Patterns [53.71420125627608]
  We show that the diversity in the activation patterns of different neurons is reflective of model generalization and memorization. Importantly, we discover that information organization points to the two forms of memorization, even for neural activations computed on unlabelled in-distribution examples.
  arXiv  Detail & Related papers  (2022-10-17T20:15:24Z)
• CogNGen: Constructing the Kernel of a Hyperdimensional Predictive Processing Cognitive Architecture [79.07468367923619]
  We present a new cognitive architecture that combines two neurobiologically plausible, computational models. We aim to develop a cognitive architecture that has the power of modern machine learning techniques.
  arXiv  Detail & Related papers  (2022-03-31T04:44:28Z)
• Associative Memories via Predictive Coding [37.59398215921529]
  Associative memories in the brain receive and store patterns of activity registered by the sensory neurons. We present a novel neural model for realizing associative memories based on a hierarchical generative network that receives external stimuli via sensory neurons.
  arXiv  Detail & Related papers  (2021-09-16T15:46:26Z)
• Hierarchical Associative Memory [2.66512000865131]
  Associative Memories or Modern Hopfield Networks have many appealing properties. They can do pattern completion, store a large number of memories, and can be described using a recurrent neural network. This paper tackles a gap and describes a fully recurrent model of associative memory with an arbitrary large number of layers.
  arXiv  Detail & Related papers  (2021-07-14T01:38:40Z)
• Kanerva++: extending The Kanerva Machine with differentiable, locally block allocated latent memory [75.65949969000596]
  Episodic and semantic memory are critical components of the human memory model. We develop a new principled Bayesian memory allocation scheme that bridges the gap between episodic and semantic memory. We demonstrate that this allocation scheme improves performance in memory conditional image generation.
  arXiv  Detail & Related papers  (2021-02-20T18:40:40Z)
• Slow manifolds in recurrent networks encode working memory efficiently and robustly [0.0]
  Working memory is a cognitive function involving the storage and manipulation of latent information over brief intervals of time. We use a top-down modeling approach to examine network-level mechanisms of working memory.
  arXiv  Detail & Related papers  (2021-01-08T18:47:02Z)
• Towards a Neural Model for Serial Order in Frontal Cortex: a Brain Theory from Memory Development to Higher-Level Cognition [53.816853325427424]
  We propose that the immature prefrontal cortex (PFC) use its primary functionality of detecting hierarchical patterns in temporal signals. Our hypothesis is that the PFC detects the hierarchical structure in temporal sequences in the form of ordinal patterns and use them to index information hierarchically in different parts of the brain. By doing so, it gives the tools to the language-ready brain for manipulating abstract knowledge and planning temporally ordered information.
  arXiv  Detail & Related papers  (2020-05-22T14:29:51Z)
• Self-Attentive Associative Memory [69.40038844695917]
  We propose to separate the storage of individual experiences (item memory) and their occurring relationships (relational memory) We achieve competitive results with our proposed two-memory model in a diversity of machine learning tasks.
  arXiv  Detail & Related papers  (2020-02-10T03:27:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.