Reflective Dialogue between Teacher and Solver Agents for Video Question Answering

• URL: http://arxiv.org/abs/2605.27885v1
• Date: Wed, 27 May 2026 03:08:12 GMT
• Title: Reflective Dialogue between Teacher and Solver Agents for Video Question Answering
• Abstract summary: We propose a method that achieves adaptation solely through inference-time context injection.<n> Experiments on the EgoCross benchmark demonstrate that our method outperforms both a baseline zero-shot setting and a standard in-context learning approach.
• Score: 1.0742675209112622
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Various approaches have been proposed to adapt Vision-Language Models (VLMs) to specialized domains for Video Question Answering, including fine-tuning and in-context learning. However, acquiring task-specific knowledge at the inference phase from only a small labeled support set without fine-tuning remains a challenge. In this paper, we propose a method that achieves adaptation solely through inference-time context injection. Our method first constructs a Reflective Dialogue (RD) -- a multi-turn conversation between two agents, in which Teacher poses each support question and delivers correctness feedback, and Solver answers and provides visual grounding explanations (or reflections) for both correct and incorrect answers. This dialogue history is then used as context at the inference phase. Experiments on the EgoCross benchmark demonstrate that our method outperforms both a baseline zero-shot setting and a standard in-context learning approach that passes support set examples directly, achieving 3rd place in the Open-source Track of the 1st Cross-Domain EgoCross Challenge at the CVPR 2026 EgoVis Workshop, for which this paper also serves as a technical report.

Related papers

• Contextual Speech Extraction: Leveraging Textual History as an Implicit Cue for Target Speech Extraction [50.630431647192054]
  This paper investigates a novel approach for Target Speech Extraction (TSE)<n>It relies solely on textual context to extract the target speech.<n>We present three CSE models and analyze their performances on three datasets.
  arXiv  Detail & Related papers  (2025-03-11T18:26:10Z)
• Interactive Text-to-Image Retrieval with Large Language Models: A Plug-and-Play Approach [33.231639257323536]
  In this paper, we address the issue of dialogue-form context query within the interactive text-to-image retrieval task. By reformulating the dialogue-form context, we eliminate the necessity of fine-tuning a retrieval model on existing visual dialogue data. We construct the LLM questioner to generate non-redundant questions about the attributes of the target image.
  arXiv  Detail & Related papers  (2024-06-05T16:09:01Z)
• Cue-CoT: Chain-of-thought Prompting for Responding to In-depth Dialogue Questions with LLMs [59.74002011562726]
  We propose a novel linguistic cue-based chain-of-thoughts (textitCue-CoT) to provide a more personalized and engaging response. We build a benchmark with in-depth dialogue questions, consisting of 6 datasets in both Chinese and English. Empirical results demonstrate our proposed textitCue-CoT method outperforms standard prompting methods in terms of both textithelpfulness and textitacceptability on all datasets.
  arXiv  Detail & Related papers  (2023-05-19T16:27:43Z)
• Collaborative Reasoning on Multi-Modal Semantic Graphs for Video-Grounded Dialogue Generation [53.87485260058957]
  We study video-grounded dialogue generation, where a response is generated based on the dialogue context and the associated video. The primary challenges of this task lie in (1) the difficulty of integrating video data into pre-trained language models (PLMs) We propose a multi-agent reinforcement learning method to collaboratively perform reasoning on different modalities.
  arXiv  Detail & Related papers  (2022-10-22T14:45:29Z)
• UTC: A Unified Transformer with Inter-Task Contrastive Learning for Visual Dialog [30.864390060624213]
  We propose a contrastive learning-based framework UTC to unify and facilitate both discriminative and generative tasks in visual dialog with a single model. We evaluate our proposed UTC on the VisDial v1.0 dataset, where our method outperforms the state-of-the-art on both discriminative and generative tasks.
  arXiv  Detail & Related papers  (2022-05-01T08:36:18Z)
• Utterance Rewriting with Contrastive Learning in Multi-turn Dialogue [22.103162555263143]
  We introduce contrastive learning and multi-task learning to jointly model the problem. Our proposed model achieves state-of-the-art performance on several public datasets.
  arXiv  Detail & Related papers  (2022-03-22T10:13:27Z)
• elBERto: Self-supervised Commonsense Learning for Question Answering [131.51059870970616]
  We propose a Self-supervised Bidirectional Representation Learning of Commonsense framework, which is compatible with off-the-shelf QA model architectures. The framework comprises five self-supervised tasks to force the model to fully exploit the additional training signals from contexts containing rich commonsense. elBERto achieves substantial improvements on out-of-paragraph and no-effect questions where simple lexical similarity comparison does not help.
  arXiv  Detail & Related papers  (2022-03-17T16:23:45Z)
• Self-supervised Contrastive Cross-Modality Representation Learning for Spoken Question Answering [29.545937716796082]
  Spoken question answering (SQA) requires fine-grained understanding of both spoken documents and questions. We propose novel training schemes for spoken question answering with a self-supervised training stage and a contrastive representation learning stage. Our model achieves state-of-the-art results on three SQA benchmarks.
  arXiv  Detail & Related papers  (2021-09-08T01:13:14Z)
• Structural Pre-training for Dialogue Comprehension [51.215629336320305]
  We present SPIDER, Structural Pre-traIned DialoguE Reader, to capture dialogue exclusive features. To simulate the dialogue-like features, we propose two training objectives in addition to the original LM objectives. Experimental results on widely used dialogue benchmarks verify the effectiveness of the newly introduced self-supervised tasks.
  arXiv  Detail & Related papers  (2021-05-23T15:16:54Z)
• Probing Task-Oriented Dialogue Representation from Language Models [106.02947285212132]
  This paper investigates pre-trained language models to find out which model intrinsically carries the most informative representation for task-oriented dialogue tasks. We fine-tune a feed-forward layer as the classifier probe on top of a fixed pre-trained language model with annotated labels in a supervised way.
  arXiv  Detail & Related papers  (2020-10-26T21:34:39Z)
• Learning an Effective Context-Response Matching Model with Self-Supervised Tasks for Retrieval-based Dialogues [88.73739515457116]
  We introduce four self-supervised tasks including next session prediction, utterance restoration, incoherence detection and consistency discrimination. We jointly train the PLM-based response selection model with these auxiliary tasks in a multi-task manner. Experiment results indicate that the proposed auxiliary self-supervised tasks bring significant improvement for multi-turn response selection.
  arXiv  Detail & Related papers  (2020-09-14T08:44:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.