Companion-style QA Assistance in Ego-Vision

• URL: http://arxiv.org/abs/2609.06721v1
• Date: Sun, 06 Sep 2026 16:49:45 GMT
• Title: Companion-style QA Assistance in Ego-Vision
• Abstract summary: We introduce BuddyVQA, a benchmark for companion-style question answering (QA) on egocentric streaming video.<n>It features two key characteristics that are common in daily first-person QA assistance but are largely overlooked in existing VideoQA benchmarks.<n>We propose MyBuddy, a companion-style QA assistant that highlights a multimodal chain-of-thought reasoning mechanism.
• Score: 55.53073147495282
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: AI companions are envisioned as always-on assistants that support users in daily life. With this regard, we introduce BuddyVQA, a benchmark for companion-style question answering (QA) on egocentric streaming video. BuddyVQA contains 21.6K questions linked to 6K highlight moments across 1,012 long, egocentric videos. It features two key characteristics that are common in daily first-person QA assistance but are largely overlooked in existing VideoQA benchmarks: ego-deictic expressions and interactively chained questions (e.g., "Where is it?", "How to get there?"). These require models to infer a user's in-situation intent by resolving visual pronouns in the context of egocentric visual and QA contents, with both grounded in a long-form streaming setting. To tackle the challenges, we propose MyBuddy, a companion-style QA assistant that highlights a multimodal chain-of-thought reasoning mechanism to infer the final answer based on the historical QA and visual content. An additional question filter and multi-level memory are designed to facilitate efficient QA and visual information retrieval under streaming QA settings. Experiments show that MyBuddy significantly enhances the performance of foundation models on BuddyVQA. Moreover, these gains generalize to other streaming and common video QA benchmarks, demonstrating the applicability and effectiveness of our approach. Our code and dataset are available at https://github.com/QHUni/BuddyVQA

Related papers

• MovieCORE: COgnitive REasoning in Movies [39.759698550568174]
  MovieCORE is a novel dataset designed to probe deeper cognitive understanding of movie content.<n>We present an innovative agentic brainstorming approach, utilizing multiple large language models (LLMs) as thought agents.<n>To address the limitations of existing video-language models (VLMs), we introduce an agentic enhancement module, Agentic Choice Enhancement (ACE)
  arXiv  Detail & Related papers  (2025-08-26T13:43:45Z)
• FIQ: Fundamental Question Generation with the Integration of Question Embeddings for Video Question Answering [26.585985828583304]
  Video question of answering (VQA) is a task that requires the interpretation of a video to answer a given question.<n>We propose a novel approach designed to strengthen the reasoning ability of model by enhancing the fundamental understanding of videos.
  arXiv  Detail & Related papers  (2025-07-17T06:19:38Z)
• ImplicitQA: Going beyond frames towards Implicit Video Reasoning [39.63171940350552]
  ImplicitQA is a novel benchmark designed to test VideoQA models on human-like implicit reasoning.<n>ImplicitQA comprises 1K meticulously annotated QA pairs drawn from 1K high-quality creative video clips.
  arXiv  Detail & Related papers  (2025-06-26T19:53:54Z)
• VQA$^2$: Visual Question Answering for Video Quality Assessment [76.81110038738699]
  Video Quality Assessment (VQA) is a classic field in low-level visual perception.<n>Recent studies in the image domain have demonstrated that Visual Question Answering (VQA) can enhance markedly low-level visual quality evaluation.<n>We introduce the VQA2 Instruction dataset - the first visual question answering instruction dataset that focuses on video quality assessment.<n>The VQA2 series models interleave visual and motion tokens to enhance the perception of spatial-temporal quality details in videos.
  arXiv  Detail & Related papers  (2024-11-06T09:39:52Z)
• FunQA: Towards Surprising Video Comprehension [64.58663825184958]
  We introduce FunQA, a challenging video question-answering dataset. FunQA covers three previously unexplored types of surprising videos: HumorQA, CreativeQA, and MagicQA. In total, the FunQA benchmark consists of 312K free-text QA pairs derived from 4.3K video clips.
  arXiv  Detail & Related papers  (2023-06-26T17:59:55Z)
• NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenario [77.14723238359318]
  NuScenesQA is the first benchmark for VQA in the autonomous driving scenario, encompassing 34K visual scenes and 460K question-answer pairs. We leverage existing 3D detection annotations to generate scene graphs and design question templates manually. We develop a series of baselines that employ advanced 3D detection and VQA techniques.
  arXiv  Detail & Related papers  (2023-05-24T07:40:50Z)
• Structured Two-stream Attention Network for Video Question Answering [168.95603875458113]
  We propose a Structured Two-stream Attention network, namely STA, to answer a free-form or open-ended natural language question. First, we infer rich long-range temporal structures in videos using our structured segment component and encode text features. Then, our structured two-stream attention component simultaneously localizes important visual instance, reduces the influence of background video and focuses on the relevant text.
  arXiv  Detail & Related papers  (2022-06-02T12:25:52Z)
• NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actions [80.60423934589515]
  We introduce NExT-QA, a rigorously designed video question answering (VideoQA) benchmark. We set up multi-choice and open-ended QA tasks targeting causal action reasoning, temporal action reasoning, and common scene comprehension. We find that top-performing methods excel at shallow scene descriptions but are weak in causal and temporal action reasoning.
  arXiv  Detail & Related papers  (2021-05-18T04:56:46Z)
• End-to-End Video Question-Answer Generation with Generator-Pretester Network [27.31969951281815]
  We study a novel task, Video Question-Answer Generation (VQAG) for challenging Video Question Answering (Video QA) task in multimedia. As captions neither fully represent a video, nor are they always practically available, it is crucial to generate question-answer pairs based on a video via Video Question-Answer Generation (VQAG) We evaluate our system with the only two available large-scale human-annotated Video QA datasets and achieves state-of-the-art question generation performances.
  arXiv  Detail & Related papers  (2021-01-05T10:46:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.