OCR-Memory: Optical Context Retrieval for Long-Horizon Agent Memory

• URL: http://arxiv.org/abs/2604.26622v1
• Date: Wed, 29 Apr 2026 12:49:30 GMT
• Title: OCR-Memory: Optical Context Retrieval for Long-Horizon Agent Memory
• Abstract summary: OCR-Memory is a memory framework that leverages the visual modality as a high-density representation of agent experience.<n> OCR-Memory retrieves stored experience via a emphlocate-and-transcribe paradigm that selects relevant regions through visual anchors and retrieves the corresponding verbatim text.
• Score: 12.240259398680259
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Autonomous LLM agents increasingly operate in long-horizon, interactive settings where success depends on reusing experience accumulated over extended histories. However, existing agent memory systems are fundamentally constrained by text-context budgets: storing or revisiting raw trajectories is prohibitively token-expensive, while summarization and text-only retrieval trade token savings for information loss and fragmented evidence. To address this limitation, we propose Optical Context Retrieval Memory (OCR-Memory), a memory framework that leverages the visual modality as a high-density representation of agent experience, enabling retention of arbitrarily long histories with minimal prompt overhead at retrieval time. Specifically, OCR-Memory renders historical trajectories into images annotated with unique visual identifiers. OCR-Memory retrieves stored experience via a \emph{locate-and-transcribe} paradigm that selects relevant regions through visual anchors and retrieves the corresponding verbatim text, avoiding free-form generation and reducing hallucination. Experiments on long-horizon agent benchmarks show consistent gains under strict context limits, demonstrating that optical encoding increases effective memory capacity while preserving faithful evidence recovery.

Related papers

• CueMem: Cue-Guided Context Reconstruction for Long-Term Conversational Memory [38.76918926805768]
  CueMem is a cue-guided framework that treats extracted memory records as retrieval cues.<n>It reconstructs query-relevant dialogue context from their source turns.<n>Experiments on LoCoMo and LongMemEval show that CueMem consistently outperforms representative long-term memory baselines.
  arXiv  Detail & Related papers  (2026-09-11T02:27:22Z)
• Beyond Retrieval: Progressive Latent Memory Evolution for Streaming Video Understanding [64.34948094316685]
  We introduce LatentStream, a progressive latent working memory framework that shifts streaming memory from store-and-retrieve to retrieve-and-internalize.<n>LatentStream achieves new state-of-the-art results on existing online and offline video benchmarks.
  arXiv  Detail & Related papers  (2026-09-03T17:28:14Z)
• Harness the Memory: A Holistic Evaluation of Memory Substrates in Memory Agents [97.19033877543522]
  We present a controlled harness evaluation of memory substrates for memory-augmented agents.<n>We instrument 26 performance and efficiency metrics under a unified harness.<n>These findings motivate substrate routing as a necessary component of adaptive agent memory systems.
  arXiv  Detail & Related papers  (2026-08-15T03:29:48Z)
• RRM: Experience-Driven Reflective Retrieval Memory for Long-Horizon Multimodal Reasoning [11.148362747759917]
  We introduce Reflective Retrieval Memory (RRM), a reflective memory framework for long-horizon multimodal reasoning.<n>RRM augments an entity-centric multimodal memory graph with reflective experience memory, which distills transferable procedural retrieval knowledge.<n>RRM consistently outperforms previous state-of-the-art approaches on M3-Bench-Robot, M3-Bench-Web, and Video-MME-Long.
  arXiv  Detail & Related papers  (2026-07-30T12:58:57Z)
• Infini Memory: Maintainable Topic Documents for Long-Term LLM Agent Memory [19.802226925602927]
  Long-term LLM agents need persistent memory that can track changing facts and provide relevant evidence across sessions.<n>We propose Infini Memory, a maintainable text-based persistent memory architecture that treats agent memory as topic-structured documents.
  arXiv  Detail & Related papers  (2026-06-09T10:31:51Z)
• Memory is Reconstructed, Not Retrieved: Graph Memory for LLM Agents [51.30250860677378]
  We propose MRAgent, a framework that combines an associative memory graph with an active reconstruction mechanism.<n>Running on a Cue-Tag-Content graph, our active reconstruction mechanism integrates reasoning directly into memory access.<n>Experiments on the LoCoMo benchmark and LongMemEval benchmark demonstrate significant improvements over strong baselines.
  arXiv  Detail & Related papers  (2026-06-04T11:29:46Z)
• MemReread: Enhancing Agentic Long-Context Reasoning via Memory-Guided Rereading [46.00471665436759]
  We propose MemReread to tackle long-context reasoning tasks without the quadratic complexity of standard attention mechanisms.<n>Built upon streaming reading, MemReread circumvents intermediate retrieval. It triggers question decomposition and rereading when the final memory is insufficient.<n>This design supports non-linear reasoning while preserving the inherent logical flow of document comprehension.
  arXiv  Detail & Related papers  (2026-05-11T09:30:59Z)
• All-Mem: Agentic Lifelong Memory via Dynamic Topology Evolution [20.68235182577703]
  All-Mem maintains a structured memory bank via explicit, non destructive consolidation.<n>Periodically offline, an LLM diagnoser proposes confidence scored topology edits executed with three operators: SPLIT, MERGE, and UPDATE.<n>At query time, typed links enable hop bounded, budgeted expansion from active anchors to archived evidence when needed.
  arXiv  Detail & Related papers  (2026-03-20T03:14:40Z)
• Evoking User Memory: Personalizing LLM via Recollection-Familiarity Adaptive Retrieval [59.295767860331004]
  RF-Mem is a familiarity uncertainty-guided dual-path memory retriever.<n>It embeds human-like dual-process recognition into the retriever.<n>It consistently outperforms one-shot retrieval and full-context reasoning under fixed budget and latency constraints.
  arXiv  Detail & Related papers  (2026-03-10T06:31:44Z)
• MemOCR: Layout-Aware Visual Memory for Efficient Long-Horizon Reasoning [36.52465672754168]
  We introduce MemOCR, a multimodal memory agent that improves long-horizon reasoning under tight context budgets.<n>MemOCR allocates memory space with adaptive information density through visual layout.<n>We train MemOCR with reinforcement learning under budget-aware objectives that expose the agent to diverse compression levels.
  arXiv  Detail & Related papers  (2026-01-29T09:47:17Z)
• AMA: Adaptive Memory via Multi-Agent Collaboration [54.490349689939166]
  We propose Adaptive Memory via Multi-Agent Collaboration (AMA), a novel framework that leverages coordinated agents to manage memory across multiple granularities.<n>AMA significantly outperforms state-of-the-art baselines while reducing token consumption by approximately 80% compared to full-context methods.
  arXiv  Detail & Related papers  (2026-01-28T08:09:49Z)
• WorldMM: Dynamic Multimodal Memory Agent for Long Video Reasoning [66.24870234484668]
  We introduce WorldMM, a novel multimodal memory agent that constructs and retrieves from multiple complementary memories.<n>WorldMM significantly outperforms existing baselines across five long video question-answering benchmarks.
  arXiv  Detail & Related papers  (2025-12-02T05:14:52Z)
• Look Back to Reason Forward: Revisitable Memory for Long-Context LLM Agents [33.617262543252494]
  We present ReMemR1, a memory-augmented agent with callback-enhanced memory that allows selective retrieval from the entire memory history.<n>We also propose Reinforcement Learning with Multi-Level Rewards (RLMLR), which combines final-answer rewards with dense, step-level signals that guide effective memory use.
  arXiv  Detail & Related papers  (2025-09-27T01:36:46Z)
• In Prospect and Retrospect: Reflective Memory Management for Long-term Personalized Dialogue Agents [70.12342024019044]
  Large Language Models (LLMs) have made significant progress in open-ended dialogue, yet their inability to retain and retrieve relevant information limits their effectiveness.<n>We propose Reflective Memory Management (RMM), a novel mechanism for long-term dialogue agents, integrating forward- and backward-looking reflections.<n>RMM shows more than 10% accuracy improvement over the baseline without memory management on the LongMemEval dataset.
  arXiv  Detail & Related papers  (2025-03-11T04:15:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.