ProIQA: A Process-Based Framework for Fine-Grained Math Item Quality Assessment

• URL: http://arxiv.org/abs/2609.15292v2
• Date: Tue, 15 Sep 2026 10:06:23 GMT
• Title: ProIQA: A Process-Based Framework for Fine-Grained Math Item Quality Assessment
• Abstract summary: This paper proposes Process-based Item Quality Assessment (ProIQA) for fine-grained quality assessment of math items.<n>We first formulate IQA across three dimensions, including knowledge concepts, difficulty, and disciplinary competencies, under a unified process-aware perspective.<n>We construct a process-enhanced IQA resource by augmenting original item data with structured reasoning trees derived from raw solutions.
• Score: 18.126940507683425
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automatic Item Generation (AIG) is pivotal for personalized education, yet guaranteeing the pedagogical value of generated items remains a bottleneck. Existing Item Quality Assessment (IQA) methods typically rely on unscalable manual reviews or shallow stem-based metrics, failing to capture the reasoning process required for mathematical problem-solving. To bridge this gap, this paper proposes Process-based Item Quality Assessment (ProIQA), a process-aware framework for fine-grained quality assessment of math items. We first formulate IQA across three heterogeneous dimensions, including knowledge concepts, difficulty, and disciplinary competencies, under a unified process-aware perspective. Based on this formulation, we construct a process-enhanced IQA resource by augmenting original item data with structured reasoning trees derived from raw solutions. Technically, ProIQA leverages Large Language Modelsto construct hierarchical reasoning trees and employs Graph Neural Networks (GNN) to encode their topological dependencies and procedural semantics. The resulting solving representation is fused with stem semantics through a dual-view (``Stem + Solving'') architecture, enabling comprehensive assessment across learning objectives. Extensive experiments on K12 mathematical datasets show that ProIQA effectively captures process-oriented features, offering a scalable data-driven solution for evaluating AIG outputs in intelligent education systems.

Related papers

• IQA-T1: Tool-based Visual Evidence Reasoning for Image Quality Assessment [59.83941105390897]
  We propose IQA-T1, a tool-based visual evidence reasoning framework.<n>During inference, the model autonomously invokes specialized analysis tools to generate structured visual evidence.<n>Experiments show that IQA-T1 achieves the best overall performance across datasets.
  arXiv  Detail & Related papers  (2026-07-14T05:47:32Z)
• Computability of Agentic Systems [0.0]
  The Quest Graph is a formal framework for analyzing the capabilities of agentic systems with finite context.<n>We show that reference-augmented (Turing-complete) systems can be exponentially more efficient at simulating complex graphs than their non-augmented (context-free) counterparts.
  arXiv  Detail & Related papers  (2026-01-26T16:06:15Z)
• AgenticIQA: An Agentic Framework for Adaptive and Interpretable Image Quality Assessment [69.06977852423564]
  Image quality assessment (IQA) reflects both the quantification and interpretation of perceptual quality rooted in the human visual system.<n>AgenticIQA decomposes IQA into four subtasks -- distortion detection, distortion analysis, tool selection, and tool execution.<n>To support training and evaluation, we introduce AgenticIQA-200K, a large-scale instruction dataset tailored for IQA agents, and AgenticIQA-Eval, the first benchmark for assessing the planning, execution, and summarization capabilities of VLM-based IQA agents.
  arXiv  Detail & Related papers  (2025-09-30T09:37:01Z)
• Towards the Next Generation of Software: Insights from Grey Literature on AI-Native Applications [13.876049229274114]
  AI-native applications are a new paradigm in software engineering that fundamentally redefines how software is designed, developed, and evolved.<n>Despite their growing prominence, AI-native applications still lack a unified engineering definition and architectural blueprint.<n>This study seeks to establish a comprehensive understanding of AI-native applications by identifying their defining characteristics, key quality attributes, and typical technology stacks.
  arXiv  Detail & Related papers  (2025-09-16T15:01:23Z)
• Large Language Models Meet Symbolic Provers for Logical Reasoning Evaluation [24.081573908824353]
  First-order logic (FOL) reasoning is pivotal for intelligent systems.<n>Existing benchmarks often rely on extensive human annotation or handcrafted templates.<n>We propose a novel framework called ProverGen that synergizes the generative strengths of Large Language Models with the rigor and precision of symbolic provers.
  arXiv  Detail & Related papers  (2025-02-10T15:31:54Z)
• Few-Shot Image Quality Assessment via Adaptation of Vision-Language Models [93.91086467402323]
  Gradient-Regulated Meta-Prompt IQA Framework (GRMP-IQA) designed to efficiently adapt the visual-language pre-trained model, CLIP, to IQA tasks.<n> GRMP-IQA consists of two core modules: (i) Meta-Prompt Pre-training Module and (ii) Quality-Aware Gradient Regularization.
  arXiv  Detail & Related papers  (2024-09-09T07:26:21Z)
• Large Multi-modality Model Assisted AI-Generated Image Quality Assessment [53.182136445844904]
  We introduce a large Multi-modality model Assisted AI-Generated Image Quality Assessment (MA-AGIQA) model. It uses semantically informed guidance to sense semantic information and extract semantic vectors through carefully designed text prompts. It achieves state-of-the-art performance, and demonstrates its superior generalization capabilities on assessing the quality of AI-generated images.
  arXiv  Detail & Related papers  (2024-04-27T02:40:36Z)
• Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspective [93.56647950778357]
  Blind image quality assessment (BIQA) predicts the human perception of image quality without any reference information. We develop a general and automated multitask learning scheme for BIQA to exploit auxiliary knowledge from other tasks.
  arXiv  Detail & Related papers  (2023-03-27T07:58:09Z)
• Generating Diverse and Consistent QA pairs from Contexts with Information-Maximizing Hierarchical Conditional VAEs [62.71505254770827]
  We propose a conditional variational autoencoder (HCVAE) for generating QA pairs given unstructured texts as contexts. Our model obtains impressive performance gains over all baselines on both tasks, using only a fraction of data for training.
  arXiv  Detail & Related papers  (2020-05-28T08:26:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.