Generating Multiple-Choice Knowledge Questions with Interpretable Difficulty Estimation using Knowledge Graphs and Large Language Models

• URL: http://arxiv.org/abs/2604.10748v1
• Date: Sun, 12 Apr 2026 17:42:24 GMT
• Title: Generating Multiple-Choice Knowledge Questions with Interpretable Difficulty Estimation using Knowledge Graphs and Large Language Models
• Abstract summary: This study proposes a novel methodology for generating multiple-choice questions (MCQs) with difficulty estimation from the input documents.<n>Our approach uses an LLM to construct a KG from input documents, from which MCQs are then systematically generated.<n>For each MCQ, nine difficulty signals are computed and combined into a unified difficulty score using a data-driven approach.
• Score: 1.1470070927586014
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generating multiple-choice questions (MCQs) with difficulty estimation remains challenging in automated MCQ-generation systems used in adaptive, AI-assisted education. This study proposes a novel methodology for generating MCQs with difficulty estimation from the input documents by utilizing knowledge graphs (KGs) and large language models (LLMs). Our approach uses an LLM to construct a KG from input documents, from which MCQs are then systematically generated. Each MCQ is generated by selecting a node from the KG as the key, sampling a related triple or quintuple -- optionally augmented with an extra triple -- and prompting an LLM to generate a corresponding stem from these graph components. Distractors are then selected from the KG. For each MCQ, nine difficulty signals are computed and combined into a unified difficulty score using a data-driven approach. Experimental results demonstrate that our method generates high-quality MCQs whose difficulty estimation is interpretable and aligns with human perceptions. Our approach improves automated MCQ generation by integrating structured knowledge representations with LLMs and a data-driven difficulty estimation model.

Related papers

• KG2Code: Bridging Knowledge Graphs and Large Language Models via Executable Code for Question Answering [64.63418702318029]
  KG2Code is a novel approach that transforms knowledge graphs into a code-based representation.<n> KG2Code-QA is introduced as a KGQA framework that formulates KGQA as a code generation task.
  arXiv  Detail & Related papers  (2026-06-26T11:27:55Z)
• HOME-KGQA: A Benchmark Dataset for Multimodal Knowledge Graph Question Answering on Household Daily Activities [11.096574335599092]
  We introduce HOME-KGQA, a novel KGQA benchmark built on a multimodal KG of daily household activities.<n>Home-KGQA consists of complex, multi-hop natural language questions paired with graph database query languages.<n>Results show that the LLM-based KGQA methods fail to achieve performance comparable to that on existing datasets.
  arXiv  Detail & Related papers  (2026-05-10T06:00:29Z)
• Orchestrating LLM Agents for Scientific Research: A Pilot Study of Multiple Choice Question (MCQ) Generation and Evaluation [0.0]
  Large language models (LLMs) are rapidly transforming scientific work, yet empirical evidence on how these systems reshape research activities remains limited.<n>We report a mixed-methods pilot evaluation of an AI-orchestrated research workflow in which a human researcher coordinated multiple LLM-based agents to perform data extraction, corpus construction, artifact generation, and artifact evaluation.
  arXiv  Detail & Related papers  (2026-02-21T16:20:07Z)
• FarsiMCQGen: a Persian Multiple-choice Question Generation Framework [2.026379197206863]
  This paper introduces FarsiMCQGen, an innovative approach for generating Persian-language multiple-choice questions (MCQs)<n>Our methodology combines candidate generation, filtering, and ranking techniques to build a model that generates answer choices resembling those in real MCQs.<n>We leverage advanced methods, including Transformers and knowledge graphs, integrated with rule-based approaches to craft credible distractors that challenge test-takers.
  arXiv  Detail & Related papers  (2025-10-16T20:52:07Z)
• Prompting is not Enough: Exploring Knowledge Integration and Controllable Generation [89.65955788873532]
  Open-domain question answering (OpenQA) represents a cornerstone in natural language processing (NLP)<n>We propose a novel framework named GenKI, which aims to improve the OpenQA performance by exploring Knowledge Integration and controllable Generation.
  arXiv  Detail & Related papers  (2025-05-26T08:18:33Z)
• Automatic Dataset Generation for Knowledge Intensive Question Answering Tasks [10.562940259841623]
  This paper presents a novel approach for enhancing Large Language Models (LLMs) in knowledge-intensive QA tasks.<n>The proposed system includes an automated QA generator and a model fine-tuner, evaluated using perplexity, ROUGE, BLEU, and BERTScore.<n>Experiments demonstrate improvements in logical coherence and factual accuracy, with implications for developing adaptable Artificial Intelligence (AI) systems.
  arXiv  Detail & Related papers  (2025-05-20T11:16:29Z)
• Question-Aware Knowledge Graph Prompting for Enhancing Large Language Models [51.47994645529258]
  We propose Question-Aware Knowledge Graph Prompting (QAP), which incorporates question embeddings into GNN aggregation to dynamically assess KG relevance.<n> Experimental results demonstrate that QAP outperforms state-of-the-art methods across multiple datasets, highlighting its effectiveness.
  arXiv  Detail & Related papers  (2025-03-30T17:09:11Z)
• Reasoning and Sampling-Augmented MCQ Difficulty Prediction via LLMs [1.749935196721634]
  We propose a novel, two-stage method to predict the difficulty of multiple-choice questions (MCQs)<n>First, to better estimate the complexity of each MCQ, we use large language models (LLMs) to augment the reasoning steps required to reach each option.<n>Second, to capture the plausibility of distractors, we sample knowledge levels from a distribution to account for variation among students responding to the MCQ.
  arXiv  Detail & Related papers  (2025-03-11T15:39:43Z)
• Token-Level Density-Based Uncertainty Quantification Methods for Eliciting Truthfulness of Large Language Models [76.17975723711886]
  Uncertainty quantification (UQ) is a prominent approach for eliciting truthful answers from large language models (LLMs)<n>In this work, we adapt Mahalanobis Distance (MD) - a well-established UQ technique in classification tasks - for text generation.<n>Our method extracts token embeddings from multiple layers of LLMs, computes MD scores for each token, and uses linear regression trained on these features to provide robust uncertainty scores.
  arXiv  Detail & Related papers  (2025-02-20T10:25:13Z)
• Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph [83.90988015005934]
  Uncertainty quantification is a key element of machine learning applications.<n>We introduce a novel benchmark that implements a collection of state-of-the-art UQ baselines.<n>We conduct a large-scale empirical investigation of UQ and normalization techniques across eleven tasks, identifying the most effective approaches.
  arXiv  Detail & Related papers  (2024-06-21T20:06:31Z)
• An Empirical Comparison of LM-based Question and Answer Generation Methods [79.31199020420827]
  Question and answer generation (QAG) consists of generating a set of question-answer pairs given a context. In this paper, we establish baselines with three different QAG methodologies that leverage sequence-to-sequence language model (LM) fine-tuning. Experiments show that an end-to-end QAG model, which is computationally light at both training and inference times, is generally robust and outperforms other more convoluted approaches.
  arXiv  Detail & Related papers  (2023-05-26T14:59:53Z)
• QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering [122.84513233992422]
  We propose a new model, QA-GNN, which addresses the problem of answering questions using knowledge from pre-trained language models (LMs) and knowledge graphs (KGs) We show its improvement over existing LM and LM+KG models, as well as its capability to perform interpretable and structured reasoning.
  arXiv  Detail & Related papers  (2021-04-13T17:32:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.