Q-Mirror: Unlocking the Multi-Modal Potential of Scientific Text-Only QA Pairs

• URL: http://arxiv.org/abs/2509.24297v2
• Date: Tue, 30 Sep 2025 04:56:54 GMT
• Title: Q-Mirror: Unlocking the Multi-Modal Potential of Scientific Text-Only QA Pairs
• Authors: Junying Wang, Zicheng Zhang, Ye Shen, Yalun Wu, Yingji Liang, Yijin Guo, Farong Wen, Wenzhe Li, Xuezhi Zhao, Qi Jia, Guangtao Zhai,
• Abstract summary: We explore the potential for transforming Text-Only QA Pairs (TQAs) into high-quality Multi-Modal QA Pairs (MMQAs)<n>We develop a TQA-to-MMQA framework and establish a comprehensive, multi-dimensional MMQA quality that provides principles for the transformation.<n>We develop an agentic system (Q-Mirror) which operationalizes our framework by integrating MMQA generation and evaluation into a closed loop for iterative refinement.
• Score: 60.0988889107102
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: High-quality, multi-modal benchmarks are crucial for advancing scientific reasoning in large models yet their manual creation is costly and unscalable. To address this bottleneck, we explore the potential for transforming Text-Only QA Pairs (TQAs) into high-quality Multi-Modal QA Pairs (MMQAs), which include three parts: 1) Task Definition \& Evaluation Rubric: We develop a TQA-to-MMQA framework and establish a comprehensive, multi-dimensional MMQA quality rubric that provides principles for the transformation. 2) Benchmark Construction: Then we construct two extensive benchmarks to rigorously evaluate state-of-the-art generation \& understanding models on the distinct tasks of MMQA generation \& MMQA quality evaluation. 3) Preliminary Solution: We develop an agentic system (Q-Mirror), which operationalizes our framework by integrating MMQA generation and evaluation into a closed loop for iterative refinement. Our experiments show that while state-of-the-art models can generate MMQAs, their outputs still leave substantial gaps, underscoring the need for reliable evaluation. We further demonstrate that top-tier understanding models align closely with human judgment in MMQA quality assessment. Leveraging both insights, the Q-Mirror agent raises average scores from 78.90 to 85.22 and pass rates from 72\% to 95\%, offering a practical path to large-scale scientific benchmarks.

Related papers

• Enhancing Image Quality Assessment Ability of LMMs via Retrieval-Augmented Generation [102.10193318526137]
  Large Multimodal Models (LMMs) have recently shown remarkable promise in low-level visual perception tasks.<n>We introduce IQARAG, a training-free framework that enhances LMMs' Image Quality Assessment (IQA) ability.<n>IQARAG leverages Retrieval-Augmented Generation (RAG) to retrieve some semantically similar but quality-variant reference images with corresponding Mean Opinion Scores (MOSs) for input image.
  arXiv  Detail & Related papers  (2026-01-13T08:00:02Z)
• iDETEX: Empowering MLLMs for Intelligent DETailed EXplainable IQA [10.857047397246598]
  iDETEX is a unified multimodal large language model (MLLM) capable of simultaneously performing three key tasks: quality grounding, perception, and description.<n>We validate our approach on the large-scale ViDA-UGC benchmark, where iDETEX achieves state-of-the-art performance across all subtasks.
  arXiv  Detail & Related papers  (2025-10-20T09:26:12Z)
• Q-Ponder: A Unified Training Pipeline for Reasoning-based Visual Quality Assessment [10.701522670464463]
  multimodal large language models (MLLMs) can proficiently evaluate visual quality through interpretable assessments.<n>We propose a unified two-stage training framework comprising a cold-start stage and a reinforcement learning-based fine-tuning stage.<n>We designate the models derived from these two stages as Q-Ponder-CI and Q-Ponder.
  arXiv  Detail & Related papers  (2025-06-03T10:11:51Z)
• M3-AGIQA: Multimodal, Multi-Round, Multi-Aspect AI-Generated Image Quality Assessment [65.3860007085689]
  M3-AGIQA is a comprehensive framework that enables more human-aligned, holistic evaluation of AI-generated images.<n>By aligning model outputs more closely with human judgment, M3-AGIQA delivers robust and interpretable quality scores.
  arXiv  Detail & Related papers  (2025-02-21T03:05:45Z)
• Grounding-IQA: Multimodal Language Grounding Model for Image Quality Assessment [69.07445098168344]
  We introduce a new image quality assessment (IQA) task paradigm, grounding-IQA.<n>Grounding-IQA comprises two subtasks: grounding-IQA-description (GIQA-DES) and visual question answering (GIQA-VQA)<n>To realize grounding-IQA, we construct a corresponding dataset, GIQA-160K, through our proposed automated annotation pipeline.<n>Experiments demonstrate that our proposed task paradigm, dataset, and benchmark facilitate the more fine-grained IQA application.
  arXiv  Detail & Related papers  (2024-11-26T09:03:16Z)
• Few-Shot Image Quality Assessment via Adaptation of Vision-Language Models [93.91086467402323]
  Gradient-Regulated Meta-Prompt IQA Framework (GRMP-IQA) designed to efficiently adapt the visual-language pre-trained model, CLIP, to IQA tasks.<n> GRMP-IQA consists of two core modules: (i) Meta-Prompt Pre-training Module and (ii) Quality-Aware Gradient Regularization.
  arXiv  Detail & Related papers  (2024-09-09T07:26:21Z)
• 2AFC Prompting of Large Multimodal Models for Image Quality Assessment [38.86162365208038]
  Two-alternative forced choice (2AFC) prompting is widely regarded as the most reliable way of collecting human opinions of visual quality. Global quality score of each image estimated by a particular LMM can be efficiently aggregated using the maximum a posterior estimation.
  arXiv  Detail & Related papers  (2024-02-02T06:05:18Z)
• Q-Boost: On Visual Quality Assessment Ability of Low-level Multi-Modality Foundation Models [80.79438689784958]
  We introduce Q-Boost, a strategy designed to enhance low-level MLLMs in image quality assessment (IQA) and video quality assessment (VQA) tasks. Q-Boost innovates by incorporating a middle ground' approach through $neutral$ prompts, allowing for a more balanced and detailed assessment. The experimental results show that the low-level MLLMs exhibit outstanding zeros-shot performance on the IQA/VQA tasks equipped with the Q-Boost strategy.
  arXiv  Detail & Related papers  (2023-12-23T17:02:25Z)
• MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities [153.37868034779385]
  We propose MM-Vet, an evaluation benchmark that examines large multimodal models (LMMs) on complicated multimodal tasks.<n>Recent LMMs have shown various intriguing abilities, such as solving math problems written on the blackboard, reasoning about events and celebrities in news images, and explaining visual jokes.
  arXiv  Detail & Related papers  (2023-08-04T17:59:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.