A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts

• URL: http://arxiv.org/abs/2402.09727v3
• Date: Mon, 22 Jul 2024 05:33:51 GMT
• Title: A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts
• Abstract summary: We propose ReadAgent, an agent system that increases effective context length up to 20x in our experiments. Inspired by how humans interactively read long documents, we implement ReadAgent as a simple prompting system. We evaluate ReadAgent against baselines using retrieval methods, using the original long contexts, and using the gist memories.
• Score: 35.68159165639245
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Current Large Language Models (LLMs) are not only limited to some maximum context length, but also are not able to robustly consume long inputs. To address these limitations, we propose ReadAgent, an LLM agent system that increases effective context length up to 20x in our experiments. Inspired by how humans interactively read long documents, we implement ReadAgent as a simple prompting system that uses the advanced language capabilities of LLMs to (1) decide what content to store together in a memory episode, (2) compress those memory episodes into short episodic memories called gist memories, and (3) take actions to look up passages in the original text if ReadAgent needs to remind itself of relevant details to complete a task. We evaluate ReadAgent against baselines using retrieval methods, using the original long contexts, and using the gist memories. These evaluations are performed on three long-document reading comprehension tasks: QuALITY, NarrativeQA, and QMSum. ReadAgent outperforms the baselines on all three tasks while extending the effective context window by 3.5-20x.

Related papers

• When Memory Takes Gradients: Collaborative Vector Memory for Agentic Recommender Systems [6.12907469209873]
  Agentic recommender systems ground each decision of a large language model in a persistent memory of the user.<n>We propose CoVeMem, which vectorizes the collaborative core of the agent's memory.
  arXiv  Detail & Related papers  (2026-08-27T09:51:18Z)
• GenericAgent: A Token-Efficient Self-Evolving LLM Agent via Contextual Information Density Maximization (V1.0) [56.81743709880371]
  Long-horizon large language model (LLM) agents are fundamentally limited by context.<n>We argue that long-horizon performance is determined not by context length, but by how much decision-relevant information is maintained within a finite context budget.<n>We present GenericAgent (GA), a general-purpose, self-evolving LLM agent system built around a single principle: information density and context.
  arXiv  Detail & Related papers  (2026-04-18T17:59:15Z)
• Coding Agents are Effective Long-Context Processors [32.55917012704666]
  Long-context processing can be externalized from latent attention into explicit, executable interactions.<n> coding agents outperform published state-of-the-art by 17.3% on average.
  arXiv  Detail & Related papers  (2026-03-20T19:03:20Z)
• Agentic Memory: Learning Unified Long-Term and Short-Term Memory Management for Large Language Model Agents [57.38404718635204]
  Large language model (LLM) agents face fundamental limitations in long-horizon reasoning due to finite context windows.<n>Existing methods typically handle long-term memory (LTM) and short-term memory (STM) as separate components.<n>We propose Agentic Memory (AgeMem), a unified framework that integrates LTM and STM management directly into the agent's policy.
  arXiv  Detail & Related papers  (2026-01-05T08:24:16Z)
• MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent [53.82053723030023]
  We introduce a novel agent workflow, MemAgent, which reads text in segments and updates the memory using an overwrite strategy.<n>MemAgent has demonstrated superb long-context capabilities, being able to extrapolate from an 8K context trained on 32K text to a 3.5M QA task with performance loss 5% and achieves 95%+ in 512K RULER test.
  arXiv  Detail & Related papers  (2025-07-03T03:11:50Z)
• MDocAgent: A Multi-Modal Multi-Agent Framework for Document Understanding [40.52017994491893]
  MDocAgent is a novel RAG and multi-agent framework that leverages both text and image. Our system employs five specialized agents: a general agent, a critical agent, a text agent, an image agent and a summarizing agent. Preliminary experiments on five benchmarks demonstrate the effectiveness of our MDocAgent, achieve an average improvement of 12.1%.
  arXiv  Detail & Related papers  (2025-03-18T06:57:21Z)
• GuardAgent: Safeguard LLM Agents by a Guard Agent via Knowledge-Enabled Reasoning [79.07152553060601]
  Existing methods for enhancing the safety of large language models (LLMs) are not directly transferable to LLM-powered agents. We propose GuardAgent, the first LLM agent as a guardrail to other LLM agents. GuardAgent comprises two steps: 1) creating a task plan by analyzing the provided guard requests, and 2) generating guardrail code based on the task plan and executing the code by calling APIs or using external engines.
  arXiv  Detail & Related papers  (2024-06-13T14:49:26Z)
• SirLLM: Streaming Infinite Retentive LLM [74.40196814292426]
  Large Language Models (LLMs) process inputs of any length and maintain a degree of memory. Recent efforts have employed streaming inputs to alleviate the pressure of excessively long text inputs. We introduce Streaming Infinite Retentive LLM (SirLLM), which allows LLMs to maintain longer memory during infinite-length dialogues.
  arXiv  Detail & Related papers  (2024-05-21T06:37:03Z)
• LongAgent: Scaling Language Models to 128k Context through Multi-Agent Collaboration [47.861310541425766]
  textscLongAgent is based on multi-agent collaboration and scales to a context of 128K. Agent team instantiated with LLaMA-7B achieves significant improvements in tasks such as 128k-long text retrieval, multi-hop question answering, compared to GPT-4.
  arXiv  Detail & Related papers  (2024-02-18T11:46:52Z)
• Walking Down the Memory Maze: Beyond Context Limit through Interactive Reading [63.93888816206071]
  We introduce MemWalker, a method that processes the long context into a tree of summary nodes. Upon receiving a query, the model navigates this tree in search of relevant information, and responds once it gathers sufficient information. We show that, beyond effective reading, MemWalker enhances explainability by highlighting the reasoning steps as it interactively reads the text; pinpointing the relevant text segments related to the query.
  arXiv  Detail & Related papers  (2023-10-08T06:18:14Z)
• RET-LLM: Towards a General Read-Write Memory for Large Language Models [53.288356721954514]
  RET-LLM is a novel framework that equips large language models with a general write-read memory unit. Inspired by Davidsonian semantics theory, we extract and save knowledge in the form of triplets. Our framework exhibits robust performance in handling temporal-based question answering tasks.
  arXiv  Detail & Related papers  (2023-05-23T17:53:38Z)
• Enhancing Large Language Model with Self-Controlled Memory Framework [56.38025154501917]
  Large Language Models (LLMs) are constrained by their inability to process lengthy inputs, resulting in the loss of critical historical information. We propose the Self-Controlled Memory (SCM) framework to enhance the ability of LLMs to maintain long-term memory and recall relevant information.
  arXiv  Detail & Related papers  (2023-04-26T07:25:31Z)
• Generate rather than Retrieve: Large Language Models are Strong Context Generators [74.87021992611672]
  We present a novel perspective for solving knowledge-intensive tasks by replacing document retrievers with large language model generators. We call our method generate-then-read (GenRead), which first prompts a large language model to generate contextutal documents based on a given question, and then reads the generated documents to produce the final answer.
  arXiv  Detail & Related papers  (2022-09-21T01:30:59Z)
• ReadTwice: Reading Very Large Documents with Memories [19.45538971299312]
  We propose ReadTwice, a technique that combines several strengths of prior approaches to model long-range dependencies with Transformers. The main idea is to read text in small segments, in parallel, summarizing each segment into a memory table to be used in a second read of the text. We show that the method outperforms models of comparable size on several question answering (QA) datasets and sets a new state of the art on the challenging NarrativeQA task.
  arXiv  Detail & Related papers  (2021-05-10T10:13:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.