Evaluating Memory Capability in Continuous Lifelog Scenario

• URL: http://arxiv.org/abs/2604.11182v2
• Date: Fri, 17 Apr 2026 06:09:59 GMT
• Title: Evaluating Memory Capability in Continuous Lifelog Scenario
• Abstract summary: Existing benchmarks primarily focus on online one-on-one chatting or human-AI interactions.<n>We propose a hierarchical synthesis framework to curate textbftextscLifeDialBench<n>We also propose an textbfOnline Evaluation protocol that strictly adheres to temporal causality.
• Score: 18.42798691109198
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Nowadays, wearable devices can continuously lifelog ambient conversations, creating substantial opportunities for memory systems. However, existing benchmarks primarily focus on online one-on-one chatting or human-AI interactions, thus neglecting the unique demands of real-world scenarios. Given the scarcity of public lifelogging audio datasets, we propose a hierarchical synthesis framework to curate \textbf{\textsc{LifeDialBench}}, a novel benchmark comprising two complementary subsets: \textbf{EgoMem}, built on real-world egocentric videos, and \textbf{LifeMem}, constructed using simulated virtual community. Crucially, to address the issue of temporal leakage in traditional offline settings, we propose an \textbf{Online Evaluation} protocol that strictly adheres to temporal causality, ensuring systems are evaluated in a realistic streaming fashion. Our experimental results reveal a counterintuitive finding: current sophisticated memory systems fail to outperform a simple RAG-based baseline. This highlights the detrimental impact of over-designed structures and lossy compression in current approaches, emphasizing the necessity of high-fidelity context preservation for lifelog scenarios.

Related papers

• ThinkFlow: Self-Evolving Probabilistic Latent Memory for Lifelong Conversational Agents [30.033481918192987]
  ThinkFlow is a novel end-to-end latent memory framework for lifelong conversational agents.<n>It bypasses the text bottleneck by compressing conversational flows into probabilistic latent memory skills.<n>It provides highly personalized and contextually accurate responses over extended multi-session interactions.
  arXiv  Detail & Related papers  (2026-09-15T11:21:39Z)
• Beyond Episodic Evaluation: Memory Architectural Bottlenecks in Sequential Embodied Question Answering [93.80043825360514]
  Embodied question answering (EQA) is traditionally evaluated under an episodic formulation, where agents solve each task independently and reset internal state between episodes.<n>We investigate how different memory architectures behave when EQA agents are evaluated sequentially, with multiple questions answered in the same scene while memory is carried forward across queries.<n>We find that simply preserving existing memory is often insufficient.<n>We highlight the necessity of structured, spatially grounded memory: architectures that map persistent visual observations onto metric 3D geometry preserve visual-semantic evidence in a coherent scene representation.
  arXiv  Detail & Related papers  (2026-07-23T17:50:45Z)
• EventVLA: Event-Driven Visual Evidence Memory for Long-Horizon Vision-Language-Action Policies [68.812675280427]
  EventVLA is an end-to-end framework founded on the concept of sparse visual evidence memory.<n>KEM directly predicts future probabilities from the VLA's latent embeddings to autonomously capture and store sparse, task-critical visual events.<n>We propose RoboTwin-MeM, a diagnostic benchmark designed to evaluate non-Markovian manipulation tasks with interactive visual evidence.
  arXiv  Detail & Related papers  (2026-06-18T11:11:37Z)
• LifeSide: Benchmarking Agents as Lifelong Digital Companions [47.43438363687746]
  benchmark is a benchmark centered on multi-session textitMemory-Emotion-Environment loops.<n>By modeling users as persistent worlds with layered profiles and event trajectories, benchmark uses multi-agent simulation to project environmental dynamics into dialogue.<n>Our experiment results reveal a stark reality: even models that saturate current memory benchmarks fail to sustain accurate user understanding and true companionship over long horizons.
  arXiv  Detail & Related papers  (2026-06-03T09:37:25Z)
• Beyond Static Dialogues: Benchmarking Realistic, Heterogeneous, and Evolving Long-Term Memory [54.947805187562274]
  We introduce realistic dialogues across diverse interaction scenarios that exhibit dynamic temporal evolution and long-term coherence.<n>The resulting benchmark encompasses challenging question-answer pairs spanning seven inquiry types.<n>We identify at least one of 27 critical memory characteristics that we identify as essential yet underexplored in current research.
  arXiv  Detail & Related papers  (2026-05-29T09:51:22Z)
• Towards Real-world Human Behavior Simulation: Benchmarking Large Language Models on Long-horizon, Cross-scenario, Heterogeneous Behavior Traces [81.41397370235102]
  We introduce OmniBehavior, the first user simulation benchmark constructed entirely from real-world data.<n>We show that current models struggle to accurately simulate complex behaviors, with performance plateauing even as context windows expand.<n>This results in the loss of individual differences and long-tail behaviors, highlighting critical directions for future high-fidelity simulation research.
  arXiv  Detail & Related papers  (2026-04-09T15:26:21Z)
• AdaMem: Adaptive User-Centric Memory for Long-Horizon Dialogue Agents [49.63422082885992]
  We propose AdaMem, an adaptive user-centric memory framework for long-horizon dialogue agents.<n>AdaMem organizes dialogue history into working, episodic, persona, and graph memories.<n>We evaluate AdaMem on the LoCoMo and PERSONAMEM benchmarks for long-horizon reasoning and user modeling.
  arXiv  Detail & Related papers  (2026-03-17T13:22:54Z)
• Beyond Short-Horizon: VQ-Memory for Robust Long-Horizon Manipulation in Non-Markovian Simulation Benchmarks [96.60530830276281]
  RuleSafe is a new articulated manipulation benchmark built upon a scalable LLM-aided simulation framework.<n>VQ-Memory is a compact and structured temporal representation that uses vector-quantized variational autoencoders.
  arXiv  Detail & Related papers  (2026-03-10T11:13:54Z)
• Bounded State in an Infinite Horizon: Proactive Hierarchical Memory for Ad-Hoc Recall over Streaming Dialogues [27.69314585543646]
  We present textbfProStream, a proactive hierarchical memory framework for streaming dialogues.<n>It enables ad-hoc memory recall on demand by reasoning over continuous streams with multi-granular distillation.<n>Experiments show that ProStream outperforms baselines in both accuracy and efficiency.
  arXiv  Detail & Related papers  (2026-03-05T07:25:25Z)
• LifeBench: A Benchmark for Long-Horizon Multi-Source Memory [22.24847456134897]
  We introduce Lifebench, which features densely connected, long-horizon event simulation.<n>Lifebench pushes AI agents beyond simple recall, requiring the integration of declarative and non-declarative memory reasoning.<n>Performance results show that top-tier, state-of-the-art memory systems reach just 55.2% accuracy.
  arXiv  Detail & Related papers  (2026-03-04T06:42:17Z)
• From Verbatim to Gist: Distilling Pyramidal Multimodal Memory via Semantic Information Bottleneck for Long-Horizon Video Agents [78.30630000529133]
  We propose MM-Mem, a pyramidal multimodal memory architecture grounded in Fuzzy-Trace Theory.<n> MM-Mem memory structures hierarchically into a Sensory Buffer, Episodic Stream, and Symbolic.<n>Experiments confirm the effectiveness of MM-Mem on both offline and streaming tasks.
  arXiv  Detail & Related papers  (2026-03-02T05:12:45Z)
• Locomo-Plus: Beyond-Factual Cognitive Memory Evaluation Framework for LLM Agents [19.76627324918285]
  We introduce textbfLoCoMo-Plus, a benchmark for assessing cognitive memory under cue--trigger semantic disconnect.<n>We show that conventional string-matching metrics and explicit task-type prompting are misaligned with such scenarios.<n> Experiments across diverse backbone models, retrieval-based methods, and memory systems demonstrate that cognitive memory remains challenging.
  arXiv  Detail & Related papers  (2026-02-11T10:22:35Z)
• FlashMem: Distilling Intrinsic Latent Memory via Computation Reuse [4.210760734549566]
  FlashMem is a framework that distills intrinsic memory directly from transient reasoning states via computation reuse.<n>Experiments demonstrate that FlashMem matches the performance of heavy baselines while reducing inference latency by 5 times.
  arXiv  Detail & Related papers  (2026-01-09T03:27:43Z)
• SYNAPSE: Empowering LLM Agents with Episodic-Semantic Memory via Spreading Activation [29.545442480332515]
  We introduce Synapse, a unified memory architecture that transcends static rather than pre-computed links.<n>We show that Synapse significantly outperforms state-of-the-art methods in complex temporal and multi-hop reasoning tasks.<n>Our code and data will be made publicly available upon acceptance.
  arXiv  Detail & Related papers  (2026-01-06T06:19:58Z)
• OST-Bench: Evaluating the Capabilities of MLLMs in Online Spatio-temporal Scene Understanding [50.72259772580637]
  We introduce OST-Bench, a benchmark designed to evaluate Online Spatio-Temporal understanding from the perspective of an agent.<n>Built on an efficient data collection pipeline, OST-Bench consists of 1.4k scenes and 10k question-answer pairs collected from ScanNet, Matterport3D, and ARKitScenes.<n>We find that both complex-based spatial reasoning demands and long-term memory retrieval requirements significantly drop model performance along two separate axes.
  arXiv  Detail & Related papers  (2025-07-10T17:56:07Z)
• Hierarchical Temporal Context Learning for Camera-based Semantic Scene Completion [57.232688209606515]
  We present HTCL, a novel Temporal Temporal Context Learning paradigm for improving camera-based semantic scene completion. Our method ranks $1st$ on the Semantic KITTI benchmark and even surpasses LiDAR-based methods in terms of mIoU.
  arXiv  Detail & Related papers  (2024-07-02T09:11:17Z)
• Analysis of the Memorization and Generalization Capabilities of AI Agents: Are Continual Learners Robust? [91.682459306359]
  In continual learning (CL), an AI agent learns from non-stationary data streams under dynamic environments. In this paper, a novel CL framework is proposed to achieve robust generalization to dynamic environments while retaining past knowledge. The generalization and memorization performance of the proposed framework are theoretically analyzed.
  arXiv  Detail & Related papers  (2023-09-18T21:00:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.