MicroVQA++: High-Quality Microscopy Reasoning Dataset with Weakly Supervised Graphs for Multimodal Large Language Model

• URL: http://arxiv.org/abs/2511.11407v1
• Date: Fri, 14 Nov 2025 15:35:43 GMT
• Title: MicroVQA++: High-Quality Microscopy Reasoning Dataset with Weakly Supervised Graphs for Multimodal Large Language Model
• Authors: Manyu Li, Ruian He, Chenxi Ma, Weimin Tan, Bo Yan,
• Abstract summary: MicroVQA++ is a three-stage, large-scale and high-quality microscopy VQA corpus.<n>It bootstraps supervision from expert-validated figure-caption pairs sourced from peer-reviewed articles.<n>HiCQA-Graph is a novel heterogeneous graph over images, captions, and QAs that fuses NLI-based textual entailment, CLIP-based vision-language alignment, and agent signals.
• Score: 28.472848113791162
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Multimodal Large Language Models are increasingly applied to biomedical imaging, yet scientific reasoning for microscopy remains limited by the scarcity of large-scale, high-quality training data. We introduce MicroVQA++, a three-stage, large-scale and high-quality microscopy VQA corpus derived from the BIOMEDICA archive. Stage one bootstraps supervision from expert-validated figure-caption pairs sourced from peer-reviewed articles. Stage two applies HiCQA-Graph, a novel heterogeneous graph over images, captions, and QAs that fuses NLI-based textual entailment, CLIP-based vision-language alignment, and agent signals to identify and filter inconsistent samples. Stage three uses a MultiModal Large Language Model (MLLM) agent to generate multiple-choice questions (MCQ) followed by human screening. The resulting release comprises a large training split and a human-checked test split whose Bloom's level hard-sample distribution exceeds the MicroVQA benchmark. Our work delivers (i) a quality-controlled dataset that couples expert literature with graph-based filtering and human refinement; (ii) HiCQA-Graph, the first graph that jointly models (image, caption, QA) for cross-modal consistency filtering; (iii) evidence that careful data construction enables 4B-scale MLLMs to reach competitive microscopy reasoning performance (e.g., GPT-5) and achieve state-of-the-art performance among open-source MLLMs. Code and dataset will be released after the review process concludes.

Related papers

• More Images, More Problems? A Controlled Analysis of VLM Failure Modes [80.64323947730905]
  Large Vision Language Models (LVLMs) have demonstrated remarkable capabilities, yet their proficiency in understanding and reasoning over multiple images remains largely unexplored.<n>We introduce MIMIC, a new benchmark designed to rigorously evaluate the multi-image capabilities of LVLMs.
  arXiv  Detail & Related papers  (2026-01-12T18:45:13Z)
• Benchmarking Vision-Language and Multimodal Large Language Models in Zero-shot and Few-shot Scenarios: A study on Christian Iconography [0.764671395172401]
  This study evaluates the capabilities of Multimodal Large Language Models (LLMs) and Vision Language Models (VLMs) in single-label classification of Christian Iconography.
  arXiv  Detail & Related papers  (2025-09-23T09:23:31Z)
• MatQnA: A Benchmark Dataset for Multi-modal Large Language Models in Materials Characterization and Analysis [2.184404734602291]
  MatQnA is the first multi-modal benchmark dataset specifically designed for material characterization techniques.<n>We employ a hybrid approach combining LLMs with human-in-the-loop validation to construct high-quality question-answer pairs.<n>Preliminary evaluation results show that the most advanced multi-modal AI models have already achieved nearly 90% accuracy on objective questions.
  arXiv  Detail & Related papers  (2025-09-14T16:23:48Z)
• aLLoyM: A large language model for alloy phase diagram prediction [1.045661264013178]
  We introduce aLLoyM, a fine-tuned Large Language Models (LLMs) specifically trained on alloy compositions, temperatures, and their corresponding phase information.<n>We fine-tuned Mistral, an open-source pre-trained LLM, for two distinct Q&A formats: multiple-choice and short-answer.<n> Benchmark evaluations demonstrate that fine-tuning substantially enhances performance on multiple-choice phase diagram questions.
  arXiv  Detail & Related papers  (2025-07-30T10:32:39Z)
• MicroVQA: A Multimodal Reasoning Benchmark for Microscopy-Based Scientific Research [57.61445960384384]
  MicroVQA consists of 1,042 multiple-choice questions (MCQs) curated by biology experts across diverse microscopy modalities.<n> Benchmarking on state-of-the-art MLLMs reveal a peak performance of 53%.<n>Expert analysis of chain-of-thought responses shows perception errors are the most frequent, followed by knowledge errors and then overgeneralization errors.
  arXiv  Detail & Related papers  (2025-03-17T17:33:10Z)
• BioD2C: A Dual-level Semantic Consistency Constraint Framework for Biomedical VQA [5.840467499436581]
  We propose BioD2C: a novel Dual-level Semantic Consistency Constraint Framework for Biomedical VQA.<n>BioD2C achieves dual-level semantic interaction alignment at both the model and feature levels, enabling the model to adaptively learn visual features based on the question.<n>In this work, we establish a new dataset, BioVGQ, to address inherent biases in prior datasets by filtering manually-altered images and aligning question-answer pairs with multimodal context.
  arXiv  Detail & Related papers  (2025-03-04T10:39:42Z)
• VOILA: Evaluation of MLLMs For Perceptual Understanding and Analogical Reasoning [63.0285363282581]
  Multimodal Large Language Models (MLLMs) have become a powerful tool for integrating visual and textual information.<n>We introduce VOILA, a benchmark designed to evaluate MLLMs' perceptual understanding and abstract relational reasoning.<n>We reveal that current MLLMs struggle to comprehend inter-image relationships and exhibit limited capabilities in high-level relational reasoning.
  arXiv  Detail & Related papers  (2025-02-25T23:36:19Z)
• Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling [191.7830199016589]
  We introduce InternVL 2.5, an advanced multimodal large language model (MLLM) series that builds upon InternVL 2.0.<n>InternVL 2.5 exhibits competitive performance, rivaling leading commercial models such as GPT-4o and Claude-3.5-Sonnet.<n>We hope this model contributes to the open-source community by setting new standards for developing and applying multimodal AI systems.
  arXiv  Detail & Related papers  (2024-12-06T18:57:08Z)
• MMSci: A Dataset for Graduate-Level Multi-Discipline Multimodal Scientific Understanding [59.41495657570397]
  We present a comprehensive dataset compiled from Nature Communications articles covering 72 scientific fields.<n>We evaluated 19 proprietary and open-source models on two benchmark tasks, figure captioning and multiple-choice, and conducted human expert annotation.<n>Fine-tuning Qwen2-VL-7B with our task-specific data achieved better performance than GPT-4o and even human experts in multiple-choice evaluations.
  arXiv  Detail & Related papers  (2024-07-06T00:40:53Z)
• Enabling Multimodal Generation on CLIP via Vision-Language Knowledge Distillation [79.72299298976525]
  We propose to augment a vision-language pre-training model with a textual pre-trained language model (PLM) via vision-language knowledge distillation (VLKD) Experiments show that the resulting model has strong zero-shot performance on multimodal generation tasks, such as open-ended visual question answering and image captioning. The original textual language understanding and generation ability of the PLM is maintained after VLKD, which makes our model versatile for both multimodal and unimodal tasks.
  arXiv  Detail & Related papers  (2022-03-12T09:33:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.