Self-Evolving Visual Questioner

• URL: http://arxiv.org/abs/2606.13929v1
• Date: Thu, 11 Jun 2026 21:45:46 GMT
• Title: Self-Evolving Visual Questioner
• Abstract summary: Vision-language models (VLMs) are typically trained as passive answerers.<n>We show that a VLM can continuously improve itself as a visual questioner without any external supervision.<n>We propose a self-evolving framework that uses a VLM itself as both a proposer and a filter to produce harder, more informative, and visual-centric questions.
• Score: 53.949845616431624
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-language models (VLMs) are typically trained as passive answerers, while their ability to actively ask diverse, non-trivial, visual-centric and grounded questions remains underexplored. Existing visual questioners' performance is bottlenecked by the availability of high-quality training data or the cost of curating them. We show that a VLM can continuously improve itself as a visual questioner without any external supervision. We propose a self-evolving framework that uses a VLM itself as both a proposer and a filter to produce harder, more informative, and visual-centric questions, while maintaining their exploration diversity to avoid training collapse. These questions are then used to train the VLM in both questioner and answerer modes. To evaluate the questioner, we introduce an agentic protocol that assesses questions along perception, reasoning, and diversity dimensions. Experiments across various backbone VLMs show that our method substantially enhances the quality and substantially expands the difficulty boundary of autonomous question generation. Under the same budget, our self-supervision is more effective than training on the static source data. Moreover, the self-evolving questioner remains a competitive or even better answerer.

Related papers

• RISE: Reliable Improvement in Self-Evolving Vision-Language Models [22.16641554921884]
  Vision-language models (VLMs) have achieved strong multimodal reasoning capabilities.<n>VLMs still rely heavily on large-scale human-constructed supervision for post-training.<n>We propose textbfRISE, a reliable self-evolving framework for vision-language models.
  arXiv  Detail & Related papers  (2026-05-20T08:57:57Z)
• Right this way: Can VLMs Guide Us to See More to Answer Questions? [11.693356269848517]
  In question-answering scenarios, humans assess whether the available information is sufficient and seek additional information if necessary. In contrast, Vision Language Models (VLMs) typically generate direct, one-shot responses without evaluating the sufficiency of the information. This study demonstrates the potential to narrow the gap between information assessment and acquisition in VLMs, bringing their performance closer to humans.
  arXiv  Detail & Related papers  (2024-11-01T06:43:54Z)
• Uncertainty-Guided Self-Questioning and Answering for Video-Language Alignment [19.345829429613037]
  We propose a self-training framework that augments question samples during training process through self-questioning and answering.<n>Low-quality self-generated questions may contaminate the performance, especially in the early stages of training.<n>We introduce Evidential Deep Learning to estimate uncertainty and assess the quality of self-generated questions.
  arXiv  Detail & Related papers  (2024-09-17T05:17:37Z)
• Multimodal Reranking for Knowledge-Intensive Visual Question Answering [77.24401833951096]
  We introduce a multi-modal reranker to improve the ranking quality of knowledge candidates for answer generation. Experiments on OK-VQA and A-OKVQA show that multi-modal reranker from distant supervision provides consistent improvements.
  arXiv  Detail & Related papers  (2024-07-17T02:58:52Z)
• LOVA3: Learning to Visual Question Answering, Asking and Assessment [61.51687164769517]
  Question answering, asking, and assessment are three innate human traits crucial for understanding the world and acquiring knowledge.<n>Current Multimodal Large Language Models (MLLMs) primarily focus on question answering, often neglecting the full potential of questioning and assessment skills.<n>We introduce LOVA3, an innovative framework named "Learning tO Visual question Answering, Asking and Assessment"
  arXiv  Detail & Related papers  (2024-05-23T18:21:59Z)
• Don't Just Say "I don't know"! Self-aligning Large Language Models for Responding to Unknown Questions with Explanations [70.6395572287422]
  Self-alignment method is capable of not only refusing to answer but also providing explanation to the unanswerability of unknown questions. We conduct disparity-driven self-curation to select qualified data for fine-tuning the LLM itself for aligning the responses to unknown questions as desired.
  arXiv  Detail & Related papers  (2024-02-23T02:24:36Z)
• Towards Top-Down Reasoning: An Explainable Multi-Agent Approach for Visual Question Answering [47.668572102657684]
  This work introduces a novel, explainable multi-agent collaboration framework by leveraging the expansive knowledge of Large Language Models (LLMs) to enhance the capabilities of Vision Language Models (VLMs)
  arXiv  Detail & Related papers  (2023-11-29T03:10:42Z)
• UNK-VQA: A Dataset and a Probe into the Abstention Ability of Multi-modal Large Models [55.22048505787125]
  This paper contributes a comprehensive dataset, called UNK-VQA. We first augment the existing data via deliberate perturbations on either the image or question. We then extensively evaluate the zero- and few-shot performance of several emerging multi-modal large models.
  arXiv  Detail & Related papers  (2023-10-17T02:38:09Z)
• SC-ML: Self-supervised Counterfactual Metric Learning for Debiased Visual Question Answering [10.749155815447127]
  We propose a self-supervised counterfactual metric learning (SC-ML) method to focus the image features better. SC-ML can adaptively select the question-relevant visual features to answer the question, reducing the negative influence of question-irrelevant visual features on inferring answers.
  arXiv  Detail & Related papers  (2023-04-04T09:05:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.