Graceful forgetting: Memory as a process

• URL: http://arxiv.org/abs/2502.11105v1
• Date: Sun, 16 Feb 2025 12:46:34 GMT
• Title: Graceful forgetting: Memory as a process
• Authors: Alain de Cheveigné,
• Abstract summary: A rational theory of memory is proposed to explain how we can accommodate input within bounded storage space. The theory is intended as an aid to make sense of our extensive knowledge of memory, and bring us closer to an understanding of memory in functional and mechanistic terms.
• Score: 0.0
• License:
• Abstract: A rational theory of memory is proposed to explain how we can accommodate unbounded sensory input within bounded storage space. Memory is stored as statistics, organized into complex structures that are constantly summarized and compressed to make room for new input. This process, driven by space constraints, is guided by heuristics that optimize the memory for future needs. Sensory input is rapidly encoded as simple statistics that are more slowly elaborated into more abstract constructs. This theory differs from previous accounts of memory by (a) its reliance on statistics, (b) its use of heuristics to guide the choice of statistics, and (c) the emphasis on memory as a process that is intensive, complex, and expensive. The theory is intended as an aid to make sense of our extensive knowledge of memory, and bring us closer to an understanding of memory in functional and mechanistic terms.

Related papers

• Memory-Driven Metaheuristics: Improving Optimization Performance [0.0]
  This chapter explores the significance of memory in metaheuristic algorithms. The key factors influencing the effectiveness of memory mechanisms are discussed. A comprehensive analysis of how memory mechanisms are incorporated into popular metaheuristic algorithms is presented.
  arXiv  Detail & Related papers  (2024-11-07T13:27:03Z)
• Hierarchical Working Memory and a New Magic Number [1.024113475677323]
  We propose a recurrent neural network model for chunking within the framework of the synaptic theory of working memory. Our work provides a novel conceptual and analytical framework for understanding the on-the-fly organization of information in the brain that is crucial for cognition.
  arXiv  Detail & Related papers  (2024-08-14T16:03:47Z)
• Constant Memory Attention Block [74.38724530521277]
  Constant Memory Attention Block (CMAB) is a novel general-purpose attention block that computes its output in constant memory and performs updates in constant computation. We show our proposed methods achieve results competitive with state-of-the-art while being significantly more memory efficient.
  arXiv  Detail & Related papers  (2023-06-21T22:41:58Z)
• Sequential Memory with Temporal Predictive Coding [6.228559238589584]
  We propose a PC-based model for emphsequential memory, called emphtemporal predictive coding (tPC) We show that our tPC models can memorize and retrieve sequential inputs accurately with a biologically plausible neural implementation.
  arXiv  Detail & Related papers  (2023-05-19T20:03:31Z)
• ABC: Attention with Bounded-memory Control [67.40631793251997]
  We show that bounded-memory control (ABC) can be subsumed into one abstraction, attention with bounded-memory control (ABC) ABC reveals new, unexplored possibilities. First, it connects several efficient attention variants that would otherwise seem apart. Last, we present a new instance of ABC, which draws inspiration from existing ABC approaches, but replaces their memory-organizing functions with a learned, contextualized one.
  arXiv  Detail & Related papers  (2021-10-06T03:53:25Z)
• Memory and attention in deep learning [19.70919701635945]
  Memory construction for machine is inevitable. Recent progresses on modeling memory in deep learning have revolved around external memory constructions. The aim of this thesis is to advance the understanding on memory and attention in deep learning.
  arXiv  Detail & Related papers  (2021-07-03T09:21:13Z)
• Learning to Rehearse in Long Sequence Memorization [107.14601197043308]
  Existing reasoning tasks often have an important assumption that the input contents can be always accessed while reasoning. Memory augmented neural networks introduce a human-like write-read memory to compress and memorize the long input sequence in one pass. But they have two serious drawbacks: 1) they continually update the memory from current information and inevitably forget the early contents; 2) they do not distinguish what information is important and treat all contents equally. We propose the Rehearsal Memory to enhance long-sequence memorization by self-supervised rehearsal with a history sampler.
  arXiv  Detail & Related papers  (2021-06-02T11:58:30Z)
• Kanerva++: extending The Kanerva Machine with differentiable, locally block allocated latent memory [75.65949969000596]
  Episodic and semantic memory are critical components of the human memory model. We develop a new principled Bayesian memory allocation scheme that bridges the gap between episodic and semantic memory. We demonstrate that this allocation scheme improves performance in memory conditional image generation.
  arXiv  Detail & Related papers  (2021-02-20T18:40:40Z)
• Memformer: A Memory-Augmented Transformer for Sequence Modeling [55.780849185884996]
  We present Memformer, an efficient neural network for sequence modeling. Our model achieves linear time complexity and constant memory space complexity when processing long sequences.
  arXiv  Detail & Related papers  (2020-10-14T09:03:36Z)
• Self-Attentive Associative Memory [69.40038844695917]
  We propose to separate the storage of individual experiences (item memory) and their occurring relationships (relational memory) We achieve competitive results with our proposed two-memory model in a diversity of machine learning tasks.
  arXiv  Detail & Related papers  (2020-02-10T03:27:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.