MCQ Difficulty Prediction via Modeling Learner Heterogeneity Using Data-Driven Cognitive Profiling

• URL: http://arxiv.org/abs/2605.16290v1
• Date: Mon, 13 Apr 2026 19:18:14 GMT
• Title: MCQ Difficulty Prediction via Modeling Learner Heterogeneity Using Data-Driven Cognitive Profiling
• Abstract summary: We propose a persona-driven framework that replaces theoretical ability sampling with data-driven cognitive profiling.<n>We identify behavioral personas via latent class analysis (LCA), then condition a large language model (LLM) to simulate response distributions for each persona.<n>The discovered personas are interpretable and offer insights into why items are difficult, with potential applications to diagnostic assessment design.
• Score: 0.11458853556386796
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Predicting the difficulty of multiple-choice questions (MCQs) is important for effective assessment, yet current methods typically assume a unimodal student ability distribution, overlooking the heterogeneous nature of student misconceptions. We propose a persona-driven framework that replaces theoretical ability sampling with data-driven cognitive profiling. Using student interactions from the EEDI dataset, we identify behavioral personas via latent class analysis (LCA), then condition a large language model (LLM) to simulate response distributions for each persona. These signals are aggregated with topic context and fed into a Ridge Regression model to predict the item response theory (IRT) difficulty parameter. With five-fold cross-validation, our method improves over a recent baseline (MSE: 0.367 to 0.274; R2: 0.525 to 0.686). The discovered personas are interpretable and offer insights into why items are difficult, with potential applications to diagnostic assessment design.

Related papers

• When Persona Simulations Are Informative: Graph-Structured Signals for Pluralistic Opinion Sensing [0.6627152091494143]
  Persona-Conditioned Informativeness ( PCI) is an unsupervised diagnostic metric.<n> PCI measures whether semantically similar personas deviate in concordant directions relative to item-level sample baselines.<n>These findings support PCI as a principled internal diagnostic for screening synthetic respondents in survey pipelines.
  arXiv  Detail & Related papers  (2026-08-23T14:33:35Z)
• Synthetic Student Responses: LLM-Extracted Features for IRT Difficulty Parameter Estimation [0.0]
  We investigate whether Item Response Theory difficulty parameters can be accurately estimated without student testing.<n>Our approach combines traditional linguistic features with pedagogical insights extracted using Large Language Models (LLMs)<n>Using a dataset of over 250,000 student responses to mathematics questions, our model achieves a Pearson correlation of approximately 0.78 between predicted and actual difficulty parameters on completely unseen questions.
  arXiv  Detail & Related papers  (2026-01-18T18:19:13Z)
• Reconstructing Item Characteristic Curves using Fine-Tuned Large Language Models [0.0]
  This study introduces a novel approach that implicitly models Item Response Theory (IRT) psychometric properties.<n>We train models to generate responses to multiple choice questions conditioned on discrete ability descriptors.<n>We reconstruct the probability of a correct response as a function of student ability, effectively generating synthetic Item Characteristic Curves (ICCs) to estimate IRT parameters.
  arXiv  Detail & Related papers  (2026-01-05T22:11:41Z)
• SMART: Simulated Students Aligned with Item Response Theory for Question Difficulty Prediction [38.7828715471869]
  We present SMART (Simulated Students Aligned with IRT), a novel method for aligning simulated students with instructed ability.<n>We show that SMART outperforms other item difficulty prediction methods by leveraging its improved ability alignment.
  arXiv  Detail & Related papers  (2025-07-07T15:41:38Z)
• Preference Learning for AI Alignment: a Causal Perspective [55.2480439325792]
  We frame this problem in a causal paradigm, providing the rich toolbox of causality to identify persistent challenges.<n>Inheriting from the literature of causal inference, we identify key assumptions necessary for reliable generalisation.<n>We illustrate failure modes of naive reward models and demonstrate how causally-inspired approaches can improve model robustness.
  arXiv  Detail & Related papers  (2025-06-06T10:45:42Z)
• Semi-supervised Clustering Through Representation Learning of Large-scale EHR Data [5.591260685112265]
  SCORE is a semi-supervised representation learning framework that captures multi-domain disease profiles through patient embeddings.<n>To handle the computational challenges of large-scale data, it introduces a hybrid Expectation-Maximization (EM) and Gaussian Variational Approximation (GVA) algorithm.<n>Our analysis shows that incorporating unlabeled data enhances accuracy and reduces sensitivity to label scarcity.
  arXiv  Detail & Related papers  (2025-05-27T05:20:17Z)
• Scalable Learning of Item Response Theory Models [48.91265296134559]
  Item Response Theory (IRT) models aim to assess latent abilities of $n$ examinees along with latent difficulty characteristics of $m$ test items from categorical data. We leverage the similarity of these models to logistic regression, which can be approximated accurately using small weighted subsets called coresets.
  arXiv  Detail & Related papers  (2024-03-01T17:12:53Z)
• Querying Easily Flip-flopped Samples for Deep Active Learning [63.62397322172216]
  Active learning is a machine learning paradigm that aims to improve the performance of a model by strategically selecting and querying unlabeled data. One effective selection strategy is to base it on the model's predictive uncertainty, which can be interpreted as a measure of how informative a sample is. This paper proposes the it least disagree metric (LDM) as the smallest probability of disagreement of the predicted label.
  arXiv  Detail & Related papers  (2024-01-18T08:12:23Z)
• TREEMENT: Interpretable Patient-Trial Matching via Personalized Dynamic Tree-Based Memory Network [54.332862955411656]
  Clinical trials are critical for drug development but often suffer from expensive and inefficient patient recruitment. In recent years, machine learning models have been proposed for speeding up patient recruitment via automatically matching patients with clinical trials. We introduce a dynamic tree-based memory network model named TREEMENT to provide accurate and interpretable patient trial matching.
  arXiv  Detail & Related papers  (2023-07-19T12:35:09Z)
• Differentiable Agent-based Epidemiology [71.81552021144589]
  We introduce GradABM: a scalable, differentiable design for agent-based modeling that is amenable to gradient-based learning with automatic differentiation. GradABM can quickly simulate million-size populations in few seconds on commodity hardware, integrate with deep neural networks and ingest heterogeneous data sources.
  arXiv  Detail & Related papers  (2022-07-20T07:32:02Z)
• Stateful Offline Contextual Policy Evaluation and Learning [88.9134799076718]
  We study off-policy evaluation and learning from sequential data. We formalize the relevant causal structure of problems such as dynamic personalized pricing. We show improved out-of-sample policy performance in this class of relevant problems.
  arXiv  Detail & Related papers  (2021-10-19T16:15:56Z)
• Bootstrapping Your Own Positive Sample: Contrastive Learning With Electronic Health Record Data [62.29031007761901]
  This paper proposes a novel contrastive regularized clinical classification model. We introduce two unique positive sampling strategies specifically tailored for EHR data. Our framework yields highly competitive experimental results in predicting the mortality risk on real-world COVID-19 EHR data.
  arXiv  Detail & Related papers  (2021-04-07T06:02:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.