FIQ: Fundamental Question Generation with the Integration of Question Embeddings for Video Question Answering

• URL: http://arxiv.org/abs/2507.12816v1
• Date: Thu, 17 Jul 2025 06:19:38 GMT
• Title: FIQ: Fundamental Question Generation with the Integration of Question Embeddings for Video Question Answering
• Authors: Ju-Young Oh, Ho-Joong Kim, Seong-Whan Lee,
• Abstract summary: Video question of answering (VQA) is a task that requires the interpretation of a video to answer a given question.<n>We propose a novel approach designed to strengthen the reasoning ability of model by enhancing the fundamental understanding of videos.
• Score: 26.585985828583304
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Video question answering (VQA) is a multimodal task that requires the interpretation of a video to answer a given question. Existing VQA methods primarily utilize question and answer (Q&A) pairs to learn the spatio-temporal characteristics of video content. However, these annotations are typically event-centric, which is not enough to capture the broader context of each video. The absence of essential details such as object types, spatial layouts, and descriptive attributes restricts the model to learning only a fragmented scene representation. This issue limits the model's capacity for generalization and higher-level reasoning. In this paper, we propose a fundamental question generation with the integration of question embeddings for video question answering (FIQ), a novel approach designed to strengthen the reasoning ability of the model by enhancing the fundamental understanding of videos. FIQ generates Q&A pairs based on descriptions extracted from videos, enriching the training data with fundamental scene information. Generated Q&A pairs enable the model to understand the primary context, leading to enhanced generalizability and reasoning ability. Furthermore, we incorporate a VQ-CAlign module that assists task-specific question embeddings with visual features, ensuring that essential domain-specific details are preserved to increase the adaptability of downstream tasks. Experiments on SUTD-TrafficQA demonstrate that our FIQ achieves state-of-the-art performance compared to existing baseline methods.

Related papers

• ImplicitQA: Going beyond frames towards Implicit Video Reasoning [36.65883181090953]
  ImplicitQA is a novel benchmark designed to test models on implicit reasoning.<n>It comprises 1K meticulously annotated QA pairs derived from 320+ high-quality creative video clips.
  arXiv  Detail & Related papers  (2025-06-26T19:53:54Z)
• Perceive, Query & Reason: Enhancing Video QA with Question-Guided Temporal Queries [50.47265863322891]
  Video Question Answering (Video QA) is a challenging video understanding task that requires models to comprehend entire videos.<n>Recent advancements in Multimodal Large Language Models (MLLMs) have transformed video QA by leveraging their exceptional commonsense reasoning capabilities.<n>We propose T-Former, a novel temporal modeling method that creates a question-guided temporal bridge between frame-wise visual perception and the reasoning capabilities of LLMs.
  arXiv  Detail & Related papers  (2024-12-26T17:53:14Z)
• VideoEspresso: A Large-Scale Chain-of-Thought Dataset for Fine-Grained Video Reasoning via Core Frame Selection [61.54044967253421]
  We introduce VideoEspresso, a novel dataset that features VideoQA pairs preserving essential spatial details and temporal coherence. Our construction pipeline employs a semantic-aware method to reduce redundancy, followed by generating QA pairs using GPT-4o. We propose a Hybrid LVLMs Collaboration framework, featuring a Frame Selector and a two-stage instruction fine-tuned reasoning LVLM.
  arXiv  Detail & Related papers  (2024-11-22T08:33:36Z)
• VQA$^2$: Visual Question Answering for Video Quality Assessment [76.81110038738699]
  Video Quality Assessment (VQA) is a classic field in low-level visual perception.<n>Recent studies in the image domain have demonstrated that Visual Question Answering (VQA) can enhance markedly low-level visual quality evaluation.<n>We introduce the VQA2 Instruction dataset - the first visual question answering instruction dataset that focuses on video quality assessment.<n>The VQA2 series models interleave visual and motion tokens to enhance the perception of spatial-temporal quality details in videos.
  arXiv  Detail & Related papers  (2024-11-06T09:39:52Z)
• Prompting Video-Language Foundation Models with Domain-specific Fine-grained Heuristics for Video Question Answering [71.62961521518731]
  HeurVidQA is a framework that leverages domain-specific entity-actions to refine pre-trained video-language foundation models. Our approach treats these models as implicit knowledge engines, employing domain-specific entity-action prompters to direct the model's focus toward precise cues that enhance reasoning.
  arXiv  Detail & Related papers  (2024-10-12T06:22:23Z)
• Question-Instructed Visual Descriptions for Zero-Shot Video Question Answering [7.429984955853609]
  We present Q-ViD, a simple approach for video question answering (video QA) Q-ViD relies on a single instruction-aware open vision-language model (InstructBLIP) to tackle videoQA using frame descriptions.
  arXiv  Detail & Related papers  (2024-02-16T13:59:07Z)
• NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actions [80.60423934589515]
  We introduce NExT-QA, a rigorously designed video question answering (VideoQA) benchmark. We set up multi-choice and open-ended QA tasks targeting causal action reasoning, temporal action reasoning, and common scene comprehension. We find that top-performing methods excel at shallow scene descriptions but are weak in causal and temporal action reasoning.
  arXiv  Detail & Related papers  (2021-05-18T04:56:46Z)
• HySTER: A Hybrid Spatio-Temporal Event Reasoner [75.41988728376081]
  We present the HySTER: a Hybrid Spatio-Temporal Event Reasoner to reason over physical events in videos. We define a method based on general temporal, causal and physics rules which can be transferred across tasks. This work sets the foundations for the incorporation of inductive logic programming in the field of VideoQA.
  arXiv  Detail & Related papers  (2021-01-17T11:07:17Z)
• End-to-End Video Question-Answer Generation with Generator-Pretester Network [27.31969951281815]
  We study a novel task, Video Question-Answer Generation (VQAG) for challenging Video Question Answering (Video QA) task in multimedia. As captions neither fully represent a video, nor are they always practically available, it is crucial to generate question-answer pairs based on a video via Video Question-Answer Generation (VQAG) We evaluate our system with the only two available large-scale human-annotated Video QA datasets and achieves state-of-the-art question generation performances.
  arXiv  Detail & Related papers  (2021-01-05T10:46:06Z)
• Hierarchical Conditional Relation Networks for Multimodal Video Question Answering [67.85579756590478]
  Video QA adds at least two more layers of complexity - selecting relevant content for each channel in the context of a linguistic query. Conditional Relation Network (CRN) takes as input a set of tensorial objects translating into a new set of objects that encode relations of the inputs. CRN is then applied for Video QA in two forms, short-form where answers are reasoned solely from the visual content, and long-form where associated information, such as subtitles, is presented.
  arXiv  Detail & Related papers  (2020-10-18T02:31:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.