Ego-Grounding for Personalized Question-Answering in Egocentric Videos

• URL: http://arxiv.org/abs/2604.01966v1
• Date: Thu, 02 Apr 2026 12:29:23 GMT
• Title: Ego-Grounding for Personalized Question-Answering in Egocentric Videos
• Abstract summary: We present the first systematic analysis of multimodal large language models (MLLMs) in personalized question-answering requiring ego-grounding.<n>MyEgo is the first egocentric VideoQA dataset designed to evaluate MLLMs' ability to understand, remember, and reason about the camera wearer.
• Score: 54.479709790133946
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We present the first systematic analysis of multimodal large language models (MLLMs) in personalized question-answering requiring ego-grounding - the ability to understand the camera-wearer in egocentric videos. To this end, we introduce MyEgo, the first egocentric VideoQA dataset designed to evaluate MLLMs' ability to understand, remember, and reason about the camera wearer. MyEgo comprises 541 long videos and 5K personalized questions asking about "my things", "my activities", and "my past". Benchmarking reveals that competitive MLLMs across variants, including open-source vs. proprietary, thinking vs. non-thinking, small vs. large scales all struggle on MyEgo. Top closed- and open-source models (e.g., GPT-5 and Qwen3-VL) achieve only~46% and 36% accuracy, trailing human performance by near 40% and 50% respectively. Surprisingly, neither explicit reasoning nor model scaling yield consistent improvements. Models improve when relevant evidence is explicitly provided, but gains drop over time, indicating limitations in tracking and remembering "me" and "my past". These findings collectively highlight the crucial role of ego-grounding and long-range memory in enabling personalized QA in egocentric videos. We hope MyEgo and our analyses catalyze further progress in these areas for egocentric personalized assistance. Data and code are available at https://github.com/Ryougetsu3606/MyEgo

Related papers

• Do You See What I Am Pointing At? Gesture-Based Egocentric Video Question Answering [81.15678825882692]
  EgoPointVQA is a dataset and benchmark for gesture-grounded egocentric question answering.<n>Hand Intent Tokens (HINT) encodes tokens derived from 3D hand keypoints using an off-the-shelf reconstruction model.<n>We show that our model outperforms others in different backbones and model sizes.
  arXiv  Detail & Related papers  (2026-03-13T00:23:38Z)
• EgoThinker: Unveiling Egocentric Reasoning with Spatio-Temporal CoT [56.24624833924252]
  EgoThinker is a framework that endows MLs with robust egocentric reasoning capabilities through-temporal chain-of-thought supervision and a two-stage learning curriculum.<n>EgoThinker outperforms existing methods across multiple egocentric benchmarks, while achieving substantial improvements in fine-grained-temporal localization tasks.
  arXiv  Detail & Related papers  (2025-10-27T17:38:17Z)
• EgoToM: Benchmarking Theory of Mind Reasoning from Egocentric Videos [26.930652137352197]
  We introduce EgoToM, a new video question-answering benchmark that extends Theory-of-Mind evaluation to egocentric domains.<n>Using a causal ToM model, we generate multi-choice video QA instances for the Ego4D dataset to benchmark the ability to predict a camera wearer's goals, beliefs, and next actions.<n>We study the performance of both humans and state of the art multimodal large language models (MLLMs) on these three interconnected inference problems.
  arXiv  Detail & Related papers  (2025-03-28T05:10:59Z)
• Omnia de EgoTempo: Benchmarking Temporal Understanding of Multi-Modal LLMs in Egocentric Videos [51.8995932557911]
  EgoTempo is a dataset designed to evaluate temporal understanding in the egocentric domain.<n>We show that state-of-the-art Multi-Modal Large Language Models (MLLMs) on benchmarks achieve remarkably high performance using just text or a single frame as input.<n>We hope EgoTempo will catalyze new research in the field and inspire models that better capture the complexity of temporal dynamics.
  arXiv  Detail & Related papers  (2025-03-17T18:50:36Z)
• MM-Ego: Towards Building Egocentric Multimodal LLMs for Video QA [72.47344411599322]
  This research aims to explore building a multimodal foundation model for egocentric video understanding.<n>We automatically generate 7M high-quality QA samples for egocentric videos ranging from 30 seconds to one hour long in Ego4D based on human-annotated data.<n>We contribute a challenging egocentric QA benchmark with 629 videos and 7,026 questions to evaluate the models' ability in recognizing and memorizing visual details across videos of varying lengths.
  arXiv  Detail & Related papers  (2024-10-09T17:59:59Z)
• Egocentric Video-Language Pretraining [74.04740069230692]
  Video-Language Pretraining aims to learn transferable representation to advance a wide range of video-text downstream tasks. We exploit the recently released Ego4D dataset to pioneer Egocentric training along three directions. We demonstrate strong performance on five egocentric downstream tasks across three datasets.
  arXiv  Detail & Related papers  (2022-06-03T16:28:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.