Keep It InMind: Benchmarking the Implicit-Association Blind Spot in Agent Memory

• URL: http://arxiv.org/abs/2607.24368v1
• Date: Mon, 27 Jul 2026 12:42:12 GMT
• Title: Keep It InMind: Benchmarking the Implicit-Association Blind Spot in Agent Memory
• Abstract summary: Long-term memory systems store what a user says in an external store and retrieve it when a related query arrives.<n>This interface rests on an assumption so natural that it is rarely stated: a memory that is needed will resemble the query that needs it.<n>We call this failure mode the implicit-association blind spot and introduce InMind, a benchmark spanning ten life domains.
• Score: 32.094723684433895
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Long-term memory systems store what a user says in an external store and retrieve it when a related query arrives. This interface rests on an assumption so natural that it is rarely stated: a memory that is needed will resemble the query that needs it. World knowledge breaks the assumption. A tree-nut allergy should change the answer to a macaron request through their almond-flour ingredient, yet the two texts share no cue a retriever can see. We call this failure mode the implicit-association blind spot and introduce InMind, a 125-task, expert-verified benchmark spanning ten life domains, with 113 tasks grounded in citable public sources. Its paired controls separate three explanations that existing evaluations conflate: the fact was never stored, the model lacks the bridging knowledge, or the fact was stored and never surfaced. The verdict is clean. With the decisive memory placed in context, the backbone answers 84.0 percent of indirect queries; when the same memory must be retrieved, six vector, graph, and agentic memory systems reach at most 14.4 percent, even though they recall the same facts on demand at up to 100 percent. An embedding with eight times the dimensionality raises answer-blind target recall for every system yet leaves the gap essentially intact. A minimal diagnostic probe that keeps memory visible before the query arrives recovers most of the gap, locating the failure in the query-conditioned interface itself and pointing to routing, deciding which facts must stay visible, as the open problem InMind is built to score.

Related papers

• Agent Zero Memory: Provenance-Aware Long-Term Memory for LLM Agents [3.6666917625778193]
  Agent Zero Memory is a provenance-aware long-term memory system.<n>It distils a user's conversations, files, and connected sources into three parallel memory systems.
  arXiv  Detail & Related papers  (2026-08-30T06:55:59Z)
• Towards a Formal Definition of Agent Memory: Basis, Span, Optimality, and the Sequential Memory Problem [12.078734827417685]
  There is no unified formal account of what a memory is or when it is optimal.<n>The central idea is that memory is a basis, knowledge is its span, and answerability is a coverage problem.<n>The optimal memory is then the capacity-constrained maximizer of expected coverage.
  arXiv  Detail & Related papers  (2026-08-12T04:54:26Z)
• MEMPROBE: Probing Long-Term Agent Memory via Hidden User-State Recovery [58.64329475683699]
  We argue that long-term memory should be evaluated as an auditable post-interaction artifact.<n>We instantiate this view in MEMPROBE, a benchmark in which a memory-equipped agent assists simulated users.<n>We find that successful assistance and recoverable memory behave as distinct capabilities.
  arXiv  Detail & Related papers  (2026-06-23T13:52:46Z)
• Eywa: Provenance-Grounded Long-Term Memory for AI Agents [0.0]
  Existing memory systems collapse source evidence, extracted facts, retrieved context, and answer policy into one opaque prompt path.<n>We present Eywa, a provenance-grounded memory architecture built around evidence before belief.
  arXiv  Detail & Related papers  (2026-05-29T02:56:35Z)
• WorldMemArena: Evaluating Multimodal Agent Memory Through Action-World Interaction [72.1620416874118]
  Multimodal large language models are increasingly deployed as long-horizon agents.<n>Existing benchmarks measure recall over static dialogue, collapse memory into a single end-of-task accuracy, and reduce visual observations to captions.<n>We formulate multimodal agent memory as an Action-World Interaction Loop with an observable four-stage lifecycle.
  arXiv  Detail & Related papers  (2026-05-28T04:27:20Z)
• MemEye: A Visual-Centric Evaluation Framework for Multimodal Agent Memory [80.97855900579512]
  Existing evaluations rarely test whether agents preserve the visual evidence needed for later reasoning.<n>We introduce MemEye, a framework that evaluates memory capabilities from two dimensions.<n>Under this framework, we construct a new benchmark across 8 life-scenario tasks.
  arXiv  Detail & Related papers  (2026-05-14T17:37:52Z)
• From Unstructured Recall to Schema-Grounded Memory: Reliable AI Memory via Iterative, Schema-Aware Extraction [39.146761527401424]
  Persistent AI memory is often reduced to a retrieval problem.<n>This paper argues that reliable external AI memory must be schema-grounded harnesses.<n>We present an iterative, schema-aware write path that decomposes memory ingestion into object detection, field detection, and field-value extraction.
  arXiv  Detail & Related papers  (2026-04-30T14:14:02Z)
• MemSearch-o1: Empowering Large Language Models with Reasoning-Aligned Memory Growth in Agentic Search [59.98962867649009]
  We propose MemSearch-o1, an agentic search framework built on reasoning-aligned memory growth and retracing.<n>MemSearch-o1 grows fine-grained memory fragments from memory seed tokens from queries, then retraces and deeply refines the memory via a contribution function, and finally reorganizes a globally connected memory path.
  arXiv  Detail & Related papers  (2026-04-19T05:35:06Z)
• Synthius-Mem: Brain-Inspired Hallucination-Resistant Persona Memory Achieving 94.4% Memory Accuracy and 99.6% Adversarial Robustness on LoCoMo [0.0]
  We present Synthius-Mem, a brain-inspired structured persona memory system that takes a fundamentally different approach.<n>On the LoCoMo benchmark, Synthius-Mem achieves 94.37% accuracy, exceeding all published systems.<n> Synthius-Mem reduces token consumption by 5x compared to full-context replay while achieving higher accuracy.
  arXiv  Detail & Related papers  (2026-04-13T14:47:48Z)
• AdaMem: Adaptive User-Centric Memory for Long-Horizon Dialogue Agents [49.63422082885992]
  We propose AdaMem, an adaptive user-centric memory framework for long-horizon dialogue agents.<n>AdaMem organizes dialogue history into working, episodic, persona, and graph memories.<n>We evaluate AdaMem on the LoCoMo and PERSONAMEM benchmarks for long-horizon reasoning and user modeling.
  arXiv  Detail & Related papers  (2026-03-17T13:22:54Z)
• MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks [55.145729491377374]
  Existing evaluations of agents with memory typically assess memorization and action in isolation.<n>We introduce MemoryArena, a unified evaluation gym for benchmarking agent memory in multi-session Memory-Agent-Environment loops.<n> MemoryArena supports evaluation across web navigation, preference-constrained planning, progressive information search, and sequential formal reasoning.
  arXiv  Detail & Related papers  (2026-02-18T09:49:14Z)
• Hindsight is 20/20: Building Agent Memory that Retains, Recalls, and Reflects [11.300084544174894]
  We present Hindsight, a memory architecture that treats agent memory as a structured, first-class substrate for reasoning.<n>It supports three core operations -- retain, recall, and reflect -- that govern how information is added, accessed, and updated.<n>Hindsight with an open-source 20B model lifts overall accuracy from 39% to 83.6% over a full-context baseline.
  arXiv  Detail & Related papers  (2025-12-14T19:47:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.