Personal AI Agent for Camera Roll VQA

• URL: http://arxiv.org/abs/2606.05275v1
• Date: Wed, 03 Jun 2026 17:59:30 GMT
• Title: Personal AI Agent for Camera Roll VQA
• Abstract summary: We study the personal camera roll visual question answering setting.<n>In this setting, a conversational AI assistant can access a user's personal camera roll and retrieve relevant photos to answer queries.<n>To support this, we collect and manually annotate questions that mimic real-world usage.
• Score: 61.305160563081614
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We study the personal camera roll visual question answering setting. In this setting, a conversational AI assistant can access a user's personal camera roll and retrieve relevant photos to answer queries, ranging from simple factual questions (e.g., ``Name of the food I tried yesterday?'') to more open-ended ones (e.g., ``Recommend some dishes I have never eaten before''). Given the vast nature of the personal camera roll (i.e., multiple years, hundreds to thousands of photos), a successful AI assistant needs to understand a long-horizon, highly personalized visual content stream in order to navigate and locate the correct and/or relevant information. To support this, we collect and manually annotate questions that mimic real-world usage. The final dataset, camroll, contains 50 users, 31,476 images, and 2,500 QA pairs. We further design camroll-agent, a conversational AI agent equipped with hierarchical memory and a minimal set of tools for efficient navigation over large, personalized visual memory. Experimental results show that camroll-agent outperforms numerous baselines and methods for long-context understanding AI agents system. Together, the camroll dataset and camroll-agent highlight the gap in AI agents' long-context reasoning: personalized visual memory requires different approaches from standard long-context textual memory, especially when consistency, visual details, and user-specific context are present.

Related papers

• Companion-style QA Assistance in Ego-Vision [55.53073147495282]
  We introduce BuddyVQA, a benchmark for companion-style question answering (QA) on egocentric streaming video.<n>It features two key characteristics that are common in daily first-person QA assistance but are largely overlooked in existing VideoQA benchmarks.<n>We propose MyBuddy, a companion-style QA assistant that highlights a multimodal chain-of-thought reasoning mechanism.
  arXiv  Detail & Related papers  (2026-09-06T16:49:45Z)
• SuperMemory-VQA: An Egocentric Visual Question-Answering Benchmark for Long-Horizon Memory [14.96864764261795]
  SuperMemory-VQA is a dataset for evaluating AI assistants on practical, long-horizon memory tasks.<n>It contains 52.9 hours of everyday activities recorded with AI glasses, including synchronized RGB video, audio transcription, eye gaze, IMU, and SLAM trajectories.<n>We construct grounded 4,853 question-answer pairs that span object and location memory, intent recall, visual scene recall, timeline reconstruction, conversational memory, and in-context retrieval.
  arXiv  Detail & Related papers  (2026-05-30T17:53:10Z)
• Personal Visual Memory from Explicit and Implicit Evidence [64.95933636732191]
  Long-term memory is increasingly important for personalized AI agents.<n>Existing benchmarks and methods remain largely text-centric.<n>VisualMem is a hybrid visual-text architecture that augments a text-memory backend with a structured personal visual memory module.
  arXiv  Detail & Related papers  (2026-05-27T17:56:11Z)
• Agentic Very Long Video Understanding [39.34545320553102]
  EGAgent is an enhanced agentic framework centered on entity scene graphs, which represent people, places, objects, and their relationships over time.<n>Our system equips a planning agent with tools for structured search and reasoning over these graphs, as well as hybrid visual and audio search capabilities, enabling detailed, cross-modal, and temporally coherent reasoning.<n>EgoLifeQA and Video-MME (Long) datasets show that our method achieves state-of-the-art performance on EgoLifeQA (57.5%) and competitive performance on Video-MME (Long) (74.1%) for complex longitudinal video understanding tasks.
  arXiv  Detail & Related papers  (2026-01-26T05:20:47Z)
• Can Vision-Language Models Answer Face to Face Questions in the Real-World? [8.006029714081997]
  We introduce a new dataset and benchmark, the Qualcomm Interactive Video dataset (IVD)<n>The dataset is based on a simple question-answering setup, where users ask questions that the system has to answer, in real-time, based on the camera and audio input.<n>We show that existing models fall far behind human performance on this task, and we identify the main sources for the performance gap.
  arXiv  Detail & Related papers  (2025-03-25T05:13:12Z)
• OmniQuery: Contextually Augmenting Captured Multimodal Memory to Enable Personal Question Answering [3.8233569758620063]
  We introduce OmniQuery, a novel system that is able to answer complex personal memory-related questions.<n>Given a question, OmniQuery retrieves relevant augmented memories and uses a large language model (LLM) to generate answers with references.<n>In human evaluations, we show the effectiveness of OmniQuery with an accuracy of 71.5%, outperforming a conventional RAG system by winning or tying for 74.5% of the time.
  arXiv  Detail & Related papers  (2024-09-12T17:48:08Z)
• Quriosity: Analyzing Human Questioning Behavior and Causal Inquiry through Curiosity-Driven Queries [92.1651731484397]
  We present Quriosity, a collection of 13.5K naturally occurring questions from three diverse sources.<n>Our analysis reveals a significant presence of causal questions (up to 42%) in the dataset.
  arXiv  Detail & Related papers  (2024-05-30T17:55:28Z)
• MuMuQA: Multimedia Multi-Hop News Question Answering via Cross-Media Knowledge Extraction and Grounding [131.8797942031366]
  We present a new QA evaluation benchmark with 1,384 questions over news articles that require cross-media grounding of objects in images onto text. Specifically, the task involves multi-hop questions that require reasoning over image-caption pairs to identify the grounded visual object being referred to and then predicting a span from the news body text to answer the question. We introduce a novel multimedia data augmentation framework, based on cross-media knowledge extraction and synthetic question-answer generation, to automatically augment data that can provide weak supervision for this task.
  arXiv  Detail & Related papers  (2021-12-20T18:23:30Z)
• Generating Natural Questions from Images for Multimodal Assistants [4.930442416763205]
  We present an approach for generating diverse and meaningful questions that consider image content and metadata of image. We evaluate our approach using standard evaluation metrics such as BLEU, METEOR, ROUGE, and CIDEr.
  arXiv  Detail & Related papers  (2020-11-17T19:12:23Z)
• CapWAP: Captioning with a Purpose [56.99405135645775]
  We propose a new task, Captioning with a Purpose (CapWAP) Our goal is to develop systems that can be tailored to be useful for the information needs of an intended population. We show that it is possible to use reinforcement learning to directly optimize for the intended information need.
  arXiv  Detail & Related papers  (2020-11-09T09:23:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.