WikiVQABench: A Knowledge-Grounded Visual Question Answering Benchmark from Wikipedia and Wikidata

• URL: http://arxiv.org/abs/2605.21479v1
• Date: Wed, 20 May 2026 17:58:24 GMT
• Title: WikiVQABench: A Knowledge-Grounded Visual Question Answering Benchmark from Wikipedia and Wikidata
• Abstract summary: We introduce WikiVQABench, a knowledge-grounded VQA benchmark constructed by combining Wikipedia images, their associated article captions, and structured knowledge from Wikidata.<n>Our pipeline uses large language models (LLMs) to generate candidate multiple-choice image-question-answer sets.<n>WikiVQABench comprises a substantial collection of Wikipedia images with curated multiple-choice questions designed to benchmark knowledge-aware vision-language models.
• Score: 0.806864870370307
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Visual Question Answering (VQA) benchmarks have largely emphasized perception-based tasks that can be solved from visual content alone. In contrast, many real-world scenarios require external knowledge that is not directly observable in the image to answer correctly. We introduce WikiVQABench, a human-curated knowledge-grounded VQA benchmark constructed by systematically combining Wikipedia images, their associated article captions, and structured knowledge from Wikidata. Our pipeline uses large language models (LLMs) to generate candidate multiple-choice image-question-answer sets. All generated instances are subsequently reviewed and curated by human annotators to ensure factual correctness, visual-text consistency, and that each question requires external knowledge in addition to visual evidence for correct resolution. WikiVQABench comprises a substantial collection of Wikipedia images with curated multiple-choice questions designed to benchmark knowledge-aware vision-language models (VLMs). Evaluation of fifteen VLMs (256M-90B parameters) reveals a wide performance range (24.7%-75.6% accuracy), demonstrating that the benchmark effectively discriminates model capabilities on knowledge-intensive reasoning. The dataset and benchmarking code are publicly available.

Related papers

• Knowledge-based Visual Question Answer with Multimodal Processing, Retrieval and Filtering [55.49652734090316]
  A knowledge-based visual question answering (KB-VQA) requires visual language models (VLMs) to integrate visual understanding with external knowledge retrieval.<n>We propose a novel three-stage method, termed Wiki-PRF, including Processing, Retrieval and Filtering stages.<n> Experiments on benchmark datasets (E-VQA and InfoSeek) show significant improvements in answer quality, achieving state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-10-16T12:10:00Z)
• VisualOverload: Probing Visual Understanding of VLMs in Really Dense Scenes [36.370533774426555]
  We present VisualOverload, a visual question answering (VQA) benchmark comprising 2,720 question-answer pairs.<n>Unlike prior VQA datasets that typically focus on near global image understanding, VisualOverload challenges models to perform simple, knowledge-free vision tasks in densely populated scenes.<n>We observe that even the best model (o3) out of 37 tested models only achieves 19.6% accuracy on our hardest test split and overall 69.5% accuracy on all questions.
  arXiv  Detail & Related papers  (2025-09-29T18:00:25Z)
• EchoSight: Advancing Visual-Language Models with Wiki Knowledge [39.02148880719576]
  We introduce EchoSight, a novel framework for knowledge-based Visual Question Answering.<n>To strive for high-performing retrieval, EchoSight first searches wiki articles by using visual-only information.<n>Our experimental results on the Encyclopedic VQA and InfoSeek datasets demonstrate that EchoSight establishes new state-of-the-art results in knowledge-based VQA.
  arXiv  Detail & Related papers  (2024-07-17T16:55:42Z)
• Self-Bootstrapped Visual-Language Model for Knowledge Selection and Question Answering [11.183845003492964]
  We use Passage Retrieval (DPR) to retrieve related knowledge to help the model answer questions. DPR conduct retrieving in natural language space, which may not ensure comprehensive acquisition of image information. We propose a novel framework that leverages the visual-language model to select the key knowledge retrieved by DPR and answer questions.
  arXiv  Detail & Related papers  (2024-04-22T07:44:20Z)
• Open-Set Knowledge-Based Visual Question Answering with Inference Paths [79.55742631375063]
  The purpose of Knowledge-Based Visual Question Answering (KB-VQA) is to provide a correct answer to the question with the aid of external knowledge bases. We propose a new retriever-ranker paradigm of KB-VQA, Graph pATH rankER (GATHER for brevity) Specifically, it contains graph constructing, pruning, and path-level ranking, which not only retrieves accurate answers but also provides inference paths that explain the reasoning process.
  arXiv  Detail & Related papers  (2023-10-12T09:12:50Z)
• Open-domain Visual Entity Recognition: Towards Recognizing Millions of Wikipedia Entities [54.26896306906937]
  We present OVEN-Wiki, where a model need to link an image onto a Wikipedia entity with respect to a text query. We show that a PaLI-based auto-regressive visual recognition model performs surprisingly well, even on Wikipedia entities that have never been seen during fine-tuning. While PaLI-based models obtain higher overall performance, CLIP-based models are better at recognizing tail entities.
  arXiv  Detail & Related papers  (2023-02-22T05:31:26Z)
• A-OKVQA: A Benchmark for Visual Question Answering using World Knowledge [39.788346536244504]
  A-OKVQA is a crowdsourced dataset composed of about 25K questions. We demonstrate the potential of this new dataset through a detailed analysis of its contents.
  arXiv  Detail & Related papers  (2022-06-03T17:52:27Z)
• REVIVE: Regional Visual Representation Matters in Knowledge-Based Visual Question Answering [75.53187719777812]
  This paper revisits visual representation in knowledge-based visual question answering (VQA) We propose a new knowledge-based VQA method REVIVE, which tries to utilize the explicit information of object regions. We achieve new state-of-the-art performance, i.e., 58.0% accuracy, surpassing previous state-of-the-art method by a large margin.
  arXiv  Detail & Related papers  (2022-06-02T17:59:56Z)
• Knowledge-Routed Visual Question Reasoning: Challenges for Deep Representation Embedding [140.5911760063681]
  We propose a novel dataset named Knowledge-Routed Visual Question Reasoning for VQA model evaluation. We generate the question-answer pair based on both the Visual Genome scene graph and an external knowledge base with controlled programs.
  arXiv  Detail & Related papers  (2020-12-14T00:33:44Z)
• KILT: a Benchmark for Knowledge Intensive Language Tasks [102.33046195554886]
  We present a benchmark for knowledge-intensive language tasks (KILT) All tasks in KILT are grounded in the same snapshot of Wikipedia. We find that a shared dense vector index coupled with a seq2seq model is a strong baseline.
  arXiv  Detail & Related papers  (2020-09-04T15:32:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.