EgoCITE: Context-Augmented Indexing and Time-Aware Retrieval for Long-Horizon Egocentric Memory

• URL: http://arxiv.org/abs/2608.12627v3
• Date: Tue, 18 Aug 2026 22:47:18 GMT
• Title: EgoCITE: Context-Augmented Indexing and Time-Aware Retrieval for Long-Horizon Egocentric Memory
• Abstract summary: EgoCITE is a long-horizon agentic memory framework for egocentric QA.<n>EgoIndex organizes complementary action, activity, utterance, and conversation representations into searchable memory indices.<n>EgoRetrv combines semantic search with question-conditioned temporal relevance scoring and curation of retrieved evidence.
• Score: 10.606058056542734
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Long-horizon egocentric memory transforms continuous first-person video and audio into a searchable record of past experiences. We demonstrate two bottlenecks in existing systems: indices built from context-poor captions are unreliable for agentic search, while retrieval ignores a question's temporal intent. To address both bottlenecks, we introduce EgoCITE (Egocentric Context-augmented Indexing and Time-aware Evidence retrieval), a long-horizon agentic memory framework for egocentric QA. EgoCITE comprises three components. EgoScheme uses local multimodal context to turn fragmentary video captions and speech transcripts into self-contained atomic memory indices. EgoIndex organizes complementary action, activity, utterance, and conversation representations into searchable multi-view memory indices at multiple granularities. EgoRetrv combines semantic search with question-conditioned temporal relevance scoring and curation of retrieved evidence. We evaluate EgoCITE on EgoLifeQA, EgoMem, and EgoR1-Bench in terms of answer accuracy and target-event retrieval alignment. EgoCITE improves accuracy over agentic memory baselines by at least 4.4--14.2% while achieving 36$\times$ lower cost than long-context LLM agents.

Related papers

• Vinci2: Providing Proactive Assistance in Continuous Egocentric Videos [74.71281081304846]
  Continuous egocentric video offers rich, evolving context that enables a new form of assistance: one that is proactive rather than merely reactive.<n>We present Vinci2, a proactive egocentric assistance system that advances the on-device assistant Vinci from reactive response toward proactivity.<n>EgoMemo is a training-free, memory-augmented agent that maintains three complementary memory representations.
  arXiv  Detail & Related papers  (2026-07-13T13:09:45Z)
• Imprint: Online Memory Compression for Long-Horizon Egocentric QA [3.944710336786154]
  Long-horizon egocentric question answering involves answering about events that have occurred hours or days in the past.<n>Existing long-horizon egocentric QA methods construct memory as hierarchical textual summaries of observations.<n>We propose Imprint, an interaction-centric memory framework that formulates long-horizon egocentric memory as an online memory compression problem.
  arXiv  Detail & Related papers  (2026-07-01T09:45:30Z)
• VL-MemKnG: Hybrid Memory with a Spatio-Temporal Knowledge Graph for Question Answering over Long Egocentric Navigation Trajectories [6.008251172997241]
  We present a hybrid memory framework that extends VL-MemKnG by combining atemporal knowledge graph with persistent segment-level contextual memory.<n>We also introduce WalkieKnowledgeT+, an extension of WalkieKnowledge for long-horizon navigation-oriented video question answering.
  arXiv  Detail & Related papers  (2026-06-15T18:21:14Z)
• Keep It in Mind: User Centric Continual Spatial Intelligence Reasoning in Egocentric Video Streams [58.77207336324662]
  UCS-Bench is a dataset spanning 170+ hours of egocentric visual observations with 8.1K+ timestamped questions.<n>We propose DirectMe, a framework that incrementally constructs and maintains a structured spatial memory from streaming egocentric observations.
  arXiv  Detail & Related papers  (2026-06-13T08:50:49Z)
• SuperMemory-VQA: An Egocentric Visual Question-Answering Benchmark for Long-Horizon Memory [14.96864764261795]
  SuperMemory-VQA is a dataset for evaluating AI assistants on practical, long-horizon memory tasks.<n>It contains 52.9 hours of everyday activities recorded with AI glasses, including synchronized RGB video, audio transcription, eye gaze, IMU, and SLAM trajectories.<n>We construct grounded 4,853 question-answer pairs that span object and location memory, intent recall, visual scene recall, timeline reconstruction, conversational memory, and in-context retrieval.
  arXiv  Detail & Related papers  (2026-05-30T17:53:10Z)
• EgoIntrospect: An Egocentric Dataset and Benchmark for User-Centric Internal State Reasoning [47.853306116245484]
  EgoIntrospect is the first egocentric dataset captured in user-driven scenarios with self-annotations.<n>It consists of 180 hours of recordings from 60 subjects, with an average recording duration of 3 hours per subject.<n>We formalize a suite of tasks centered on user internal states, including affective experience, interactive intent, and cognitive memory.
  arXiv  Detail & Related papers  (2026-05-17T05:05:29Z)
• EgoMemReason: A Memory-Driven Reasoning Benchmark for Long-Horizon Egocentric Video Understanding [89.26501160264199]
  EgoMemReason systematically evaluates week-long egocentric video understanding through memory-driven reasoning.<n>EgoMemReason comprises 500 questions across three memory types and six core challenges.<n>We evaluate EgoMemReason on 17 methods across MLLMs and agentic frameworks.
  arXiv  Detail & Related papers  (2026-05-11T01:59:59Z)
• TeleEgo: Benchmarking Egocentric AI Assistants in the Wild [55.53194302888826]
  Egocentric AI assistants in real-world settings must process multi-modal inputs (video, audio, text)<n>We introduce textbfTeleEgo, a long-duration, streaming, omni-modal benchmark for evaluating egocentric AI assistants.<n>The dataset features over 14 hours per participant of synchronized egocentric video, audio, and text across four domains.
  arXiv  Detail & Related papers  (2025-10-28T01:24:24Z)
• Omnia de EgoTempo: Benchmarking Temporal Understanding of Multi-Modal LLMs in Egocentric Videos [51.8995932557911]
  EgoTempo is a dataset designed to evaluate temporal understanding in the egocentric domain.<n>We show that state-of-the-art Multi-Modal Large Language Models (MLLMs) on benchmarks achieve remarkably high performance using just text or a single frame as input.<n>We hope EgoTempo will catalyze new research in the field and inspire models that better capture the complexity of temporal dynamics.
  arXiv  Detail & Related papers  (2025-03-17T18:50:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.