Structure-Aware Modeling of Multiple-Choice Questions Improves Automatic Difficulty Estimation

• URL: http://arxiv.org/abs/2606.08988v1
• Date: Mon, 08 Jun 2026 03:35:49 GMT
• Title: Structure-Aware Modeling of Multiple-Choice Questions Improves Automatic Difficulty Estimation
• Abstract summary: AQDE holds growing promise for educational assessment because it has the potential to yield difficulty estimates competitive with expert judgment.<n>Prior AQDE studies report mixed evidence on whether adding distractors as additional text to the question stem and the correct key consistently improves difficulty prediction.<n>We hypothesize that the effectiveness of distractor information depends on its structural representation, and that explicitly modeling distractors as separate components improves difficulty estimation over baselines that omit this information.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automatic Question Difficulty Estimation (AQDE) holds growing promise for educational assessment because it has the potential to yield difficulty estimates that are competitive with expert judgment, while helping reduce the time and financial burden associated with pilot administrations and scaling to digital testing contexts. Prior AQDE studies report mixed evidence on whether adding distractors as additional text to the question stem and the correct key consistently improves difficulty prediction. We hypothesize that the effectiveness of distractor information depends on its structural representation, and that explicitly modeling distractors as separate components improves difficulty estimation over baselines that omit this information. To address this, we designed controlled architectures that model MCQ components as distinct inputs to isolate the contribution of distractor content and order. Specifically, we represented distractors by encoding each distractor as its own text input and aggregating their representations either with order-aware concatenation (with positional tags) or with an order-invariant summation. We evaluated these architectures using two Chilean datasets (Natural and Social Sciences, 2016-2020; 4,114 multiple-choice questions). Compared to a simpler model that only used the question stem and the key, our best distractor-aware architecture achieved higher predictive performance, reaching R^2 = 0.83 for Natural Sciences and R^2 = 0.71 for Social Sciences items. An order-invariant variant achieved nearly the same accuracy with approximately half as many parameters, offering a favorable accuracy-efficiency trade-off. These results show that structural information (especially distractor content) drives gains in predictive accuracy, supporting the development of efficient, structure-aware models that are computationally viable for large-scale educational applications.

Related papers

• Evian: Towards Explainable Visual Instruction-tuning Data Auditing [14.93566912726999]
  We introduce a novel "Decomposition-then-Evaluation" paradigm that breaks model responses into constituent cognitive components.<n>We instantiate this paradigm via EVIAN, an automated framework that evaluates these components along the axes of Image-Text Consistency, Logical Coherence, and Factual Accuracy.
  arXiv  Detail & Related papers  (2026-04-22T13:28:27Z)
• Adaptive Multi-Expert Reasoning via Difficulty-Aware Routing and Uncertainty-Guided Aggregation [0.2864713389096699]
  This paper describes Adaptive Multi-Expert Reasoning (AMR), a framework that focuses on problem complexity by reasoning with dynamically adapted strategies.<n>An agile routing system that focuses on problem text predicts problems' difficulty and uncertainty and guides a reconfigurable sampling mechanism to manage the breadth of generation.<n>When evaluated on the GSM8K dataset, AMR achieved 75.28% accuracy while only using the original training data.
  arXiv  Detail & Related papers  (2026-04-11T19:44:57Z)
• Enhancing the QA Model through a Multi-domain Debiasing Framework [1.7802147489386633]
  This study evaluates the ELECTRA-small model on the Stanford Question Answering dataset (SQuAD) v1.1 and adversarial datasets AddSent and AddOneSent.<n>We develop a multi-domain debiasing framework incorporating knowledge distillation, debiasing techniques, and domain expansion.
  arXiv  Detail & Related papers  (2026-01-01T08:39:07Z)
• MindGYM: What Matters in Question Synthesis for Thinking-Centric Fine-Tuning? [51.85759493254735]
  MindGYM is a structured and scalable framework for question synthesis.<n>It infuses high-level reasoning objectives to shape the model's synthesis behavior.<n>It composes more complex multi-hop questions based on QA seeds for deeper reasoning.
  arXiv  Detail & Related papers  (2025-03-12T16:03:03Z)
• Localized Gaussians as Self-Attention Weights for Point Clouds Correspondence [92.07601770031236]
  We investigate semantically meaningful patterns in the attention heads of an encoder-only Transformer architecture.<n>We find that fixing the attention weights not only accelerates the training process but also enhances the stability of the optimization.
  arXiv  Detail & Related papers  (2024-09-20T07:41:47Z)
• Self-Training with Pseudo-Label Scorer for Aspect Sentiment Quad Prediction [54.23208041792073]
  Aspect Sentiment Quad Prediction (ASQP) aims to predict all quads (aspect term, aspect category, opinion term, sentiment polarity) for a given review. A key challenge in the ASQP task is the scarcity of labeled data, which limits the performance of existing methods. We propose a self-training framework with a pseudo-label scorer, wherein a scorer assesses the match between reviews and their pseudo-labels.
  arXiv  Detail & Related papers  (2024-06-26T05:30:21Z)
• Improving Text Matching in E-Commerce Search with A Rationalizable, Intervenable and Fast Entity-Based Relevance Model [78.80174696043021]
  We propose a novel model called the Entity-Based Relevance Model (EBRM) The decomposition allows us to use a Cross-encoder QE relevance module for high accuracy. We also show that pretraining the QE module with auto-generated QE data from user logs can further improve the overall performance.
  arXiv  Detail & Related papers  (2023-07-01T15:44:53Z)
• Hawk: An Industrial-strength Multi-label Document Classifier [0.0]
  The paper describes the significance of these problems in detail and proposes a unique neural network architecture that addresses the above problems. A hydranet-like architecture is designed to have granular control over and improve the modularity, coupled with a weighted loss driving task-specific heads. The experimental results reveal that the proposed model outperforms the existing methods by a substantial margin.
  arXiv  Detail & Related papers  (2023-01-15T09:52:18Z)
• Amortized Inference for Causal Structure Learning [72.84105256353801]
  Learning causal structure poses a search problem that typically involves evaluating structures using a score or independence test. We train a variational inference model to predict the causal structure from observational/interventional data. Our models exhibit robust generalization capabilities under substantial distribution shift.
  arXiv  Detail & Related papers  (2022-05-25T17:37:08Z)
• MGA-VQA: Multi-Granularity Alignment for Visual Question Answering [75.55108621064726]
  Learning to answer visual questions is a challenging task since the multi-modal inputs are within two feature spaces. We propose Multi-Granularity Alignment architecture for Visual Question Answering task (MGA-VQA) Our model splits alignment into different levels to achieve learning better correlations without needing additional data and annotations.
  arXiv  Detail & Related papers  (2022-01-25T22:30:54Z)
• Contextualized Embeddings based Convolutional Neural Networks for Duplicate Question Identification [0.25782420501870296]
  Question Paraphrase Identification (QPI) is a critical task for large-scale Question-Answering forums. We propose a novel architecture combining a Bidirectional Transformer with Convolutional Neural Networks for the QPI task. Experimental results demonstrate that our model achieves state-of-the-art performance on the Quora Question Pairs dataset.
  arXiv  Detail & Related papers  (2021-09-03T14:30:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.