Temporal Context Reinstatement Drives Episodic-Like Order Memory in Long-Context Language Models

• URL: http://arxiv.org/abs/2607.22575v1
• Date: Fri, 05 Jun 2026 14:19:10 GMT
• Title: Temporal Context Reinstatement Drives Episodic-Like Order Memory in Long-Context Language Models
• Abstract summary: Long-term memory experiments offer new insights into how temporal aspects of long-term episodic memory may be instantiated in artificial and biological systems.<n>We show that models exhibit the same characteristic distance effect observed in humans on a temporal order memory task.<n>These findings support temporal context reinstatement as an important mechanism for episodic-like temporal-order memory in LLMs.
• Score: 39.75150245096903
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Human episodic memory supports the retrieval of experiences that unfold over extended timescales, yet the computational mechanisms underlying this ability remain debated due to the limited mechanistic accessibility in long-term memory experiments in humans. Long-context LLMs may offer promising ways to reveal plausible computational mechanisms that drive this type of retrieval. Here, we investigate whether and how LLMs capture the core behavioral signatures of episodic memory via a temporal order memory task. Using a new dataset of human behavior based on memory of a full-length novel, we show that models exhibit the same characteristic distance effect observed in humans on this task. We next apply long-context mechanistic interpretability analyses to uncover how models solve this task, and find that model performance relies on a one-dimensional temporal code that is reinstated during retrieval by a single time-reinstatement attention head. These findings support temporal context reinstatement as an important mechanism for episodic-like temporal-order memory in LLMs, offering new insights into how temporal aspects of long-term episodic memory may be instantiated in both artificial and biological systems.

Related papers

• MemoryWAM: Efficient World Action Modeling with Persistent Memory [80.90899269062128]
  We introduce MemoryWAM, a world action model with efficient persistent memory.<n>A tailored attention mechanism enables retrieval of both detailed short-term context and compressed long-term context.<n>MemoryWAM outperforms strong vision-language-action (VLA) and WAM baselines in both simulation and the real world.
  arXiv  Detail & Related papers  (2026-06-18T17:59:51Z)
• MemGround: Long-Term Memory Evaluation Kit for Large Language Models in Gamified Scenarios [33.8882826707344]
  MemGround is a rigorous long-term memory benchmark grounded in rich, gamified interactive scenarios.<n>To comprehensively quantify both memory utilization and behavioral trajectories, we propose a multi-dimensional metric suite.
  arXiv  Detail & Related papers  (2026-03-23T02:57:39Z)
• Beyond Dialogue Time: Temporal Semantic Memory for Personalized LLM Agents [68.84161689205779]
  Temporal Semantic Memory (TSM) is a memory framework that models semantic time for point-wise memory.<n>TSM consistently outperforms existing methods and achieves up to 12.2% absolute improvement in accuracy.
  arXiv  Detail & Related papers  (2026-01-12T12:24:44Z)
• Amory: Building Coherent Narrative-Driven Agent Memory through Agentic Reasoning [14.368376032599437]
  Amory is a working memory framework that actively constructs structured memory representations during offline time.<n>Amory organizes conversational fragments into episodic narratives, consolidates memories with momentum, and semanticizes peripheral facts into semantic memory.<n>Amory achieves considerable improvements over previous state-of-the-art, with performance comparable to full context reasoning while reducing response time by 50%.
  arXiv  Detail & Related papers  (2026-01-09T19:51:11Z)
• Evaluating Long-Term Memory for Long-Context Question Answering [100.1267054069757]
  We present a systematic evaluation of memory-augmented methods using LoCoMo, a benchmark of synthetic long-context dialogues annotated for question-answering tasks.<n>Our findings show that memory-augmented approaches reduce token usage by over 90% while maintaining competitive accuracy.
  arXiv  Detail & Related papers  (2025-10-27T18:03:50Z)
• Mnemosyne: An Unsupervised, Human-Inspired Long-Term Memory Architecture for Edge-Based LLMs [1.2582867366903179]
  We introduce Mnemosyne, an unsupervised, human-inspired long-term memory architecture for edge-based large language models (LLM)<n>Our approach uses graph-structured storage, modular substance and redundancy filters, memory committing and pruning mechanisms, and probabilistic recall with temporal decay and refresh processes modeled after human memory.<n>Mnemosyne is designed for use in longitudinal healthcare assistants, where repetitive and semantically similar but temporally distinct conversations are limited by naive retrieval.
  arXiv  Detail & Related papers  (2025-10-07T03:32:53Z)
• Position: Episodic Memory is the Missing Piece for Long-Term LLM Agents [43.94686139164999]
  We present an episodic memory framework for Large Language Models (LLMs) agents, centered around five key properties of episodic memory.<n>This position paper argues that now is the right time for an explicit, integrated focus on episodic memory to catalyze the development of long-term agents.
  arXiv  Detail & Related papers  (2025-02-10T19:14:51Z)
• Long Short-term Memory with Two-Compartment Spiking Neuron [64.02161577259426]
  We propose a novel biologically inspired Long Short-Term Memory Leaky Integrate-and-Fire spiking neuron model, dubbed LSTM-LIF. Our experimental results, on a diverse range of temporal classification tasks, demonstrate superior temporal classification capability, rapid training convergence, strong network generalizability, and high energy efficiency of the proposed LSTM-LIF model. This work, therefore, opens up a myriad of opportunities for resolving challenging temporal processing tasks on emerging neuromorphic computing machines.
  arXiv  Detail & Related papers  (2023-07-14T08:51:03Z)
• Sequential Memory with Temporal Predictive Coding [6.228559238589584]
  We propose a PC-based model for emphsequential memory, called emphtemporal predictive coding (tPC) We show that our tPC models can memorize and retrieve sequential inputs accurately with a biologically plausible neural implementation.
  arXiv  Detail & Related papers  (2023-05-19T20:03:31Z)
• Memory in humans and deep language models: Linking hypotheses for model augmentation [1.0485739694839669]
  We argue that memory-augmented Transformers can benefit substantially from considering insights from the memory literature in humans. We detail an approach to integrating evidence from the human memory system through the specification of cross-domain linking hypotheses.
  arXiv  Detail & Related papers  (2022-10-04T19:35:11Z)
• Temporal Memory Relation Network for Workflow Recognition from Surgical Video [53.20825496640025]
  We propose a novel end-to-end temporal memory relation network (TMNet) for relating long-range and multi-scale temporal patterns. We have extensively validated our approach on two benchmark surgical video datasets.
  arXiv  Detail & Related papers  (2021-03-30T13:20:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.