Generate-Then-Validate: A Novel Question Generation Approach Using Small Language Models

• URL: http://arxiv.org/abs/2512.10110v1
• Date: Wed, 10 Dec 2025 21:59:36 GMT
• Title: Generate-Then-Validate: A Novel Question Generation Approach Using Small Language Models
• Authors: Yumou Wei, John Stamper, Paulo F. Carvalho,
• Abstract summary: We present a novel question generation pipeline that leverages the text generation and the probabilistic reasoning abilities of SLMs to generate high-quality questions.<n>Our findings suggest that an SLM can effectively generate high-quality questions when guided by a well-designed pipeline.
• Score: 0.8602553195689513
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We explore the use of small language models (SLMs) for automatic question generation as a complement to the prevalent use of their large counterparts in learning analytics research. We present a novel question generation pipeline that leverages both the text generation and the probabilistic reasoning abilities of SLMs to generate high-quality questions. Adopting a "generate-then-validate" strategy, our pipeline first performs expansive generation to create an abundance of candidate questions and refine them through selective validation based on novel probabilistic reasoning. We conducted two evaluation studies, one with seven human experts and the other with a large language model (LLM), to assess the quality of the generated questions. Most judges (humans or LLMs) agreed that the generated questions had clear answers and generally aligned well with the intended learning objectives. Our findings suggest that an SLM can effectively generate high-quality questions when guided by a well-designed pipeline that leverages its strengths.

Related papers

• Learning to Refine: Self-Refinement of Parallel Reasoning in LLMs [102.48588475875749]
  We introduce Generative Self-Refinement (GSR), a novel parallel test-time scaling framework.<n>GSR generates a set of candidate responses in parallel and then performs self-refinement to synthesize a new superior solution.<n>We show that our method achieves state-of-the-art performance across five mathematical benchmarks.
  arXiv  Detail & Related papers  (2025-08-27T06:51:48Z)
• Leveraging Large Language Models to Generate Course-specific Semantically Annotated Learning Objects [2.1845291030915974]
  Recent progress in generative natural language models has opened up new potential in the generation of educational content.<n>This paper explores the potential of large language models for generating computer science questions that are sufficiently annotated for automatic learner model updates.
  arXiv  Detail & Related papers  (2024-12-05T14:24:07Z)
• The Future of Learning in the Age of Generative AI: Automated Question Generation and Assessment with Large Language Models [0.0]
  Large language models (LLMs) and generative AI have revolutionized natural language processing (NLP) This chapter explores the transformative potential of LLMs in automated question generation and answer assessment.
  arXiv  Detail & Related papers  (2024-10-12T15:54:53Z)
• Crafting Interpretable Embeddings by Asking LLMs Questions [89.49960984640363]
  Large language models (LLMs) have rapidly improved text embeddings for a growing array of natural-language processing tasks. We introduce question-answering embeddings (QA-Emb), embeddings where each feature represents an answer to a yes/no question asked to an LLM. We use QA-Emb to flexibly generate interpretable models for predicting fMRI voxel responses to language stimuli.
  arXiv  Detail & Related papers  (2024-05-26T22:30:29Z)
• A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models [71.25225058845324]
  Large Language Models (LLMs) have demonstrated revolutionary abilities in language understanding and generation. Retrieval-Augmented Generation (RAG) can offer reliable and up-to-date external knowledge. RA-LLMs have emerged to harness external and authoritative knowledge bases, rather than relying on the model's internal knowledge.
  arXiv  Detail & Related papers  (2024-05-10T02:48:45Z)
• Generative Judge for Evaluating Alignment [84.09815387884753]
  We propose a generative judge with 13B parameters, Auto-J, designed to address these challenges. Our model is trained on user queries and LLM-generated responses under massive real-world scenarios. Experimentally, Auto-J outperforms a series of strong competitors, including both open-source and closed-source models.
  arXiv  Detail & Related papers  (2023-10-09T07:27:15Z)
• Selecting Better Samples from Pre-trained LLMs: A Case Study on Question Generation [22.294762359009052]
  Large Language Models (LLMs) have in recent years demonstrated impressive prowess in natural language generation. We propose two prompt-based approaches to selecting high-quality questions from a set of LLM-generated candidates. Our method works under the constraints of 1) a black-box (non-modifiable) question generation model and 2) lack of access to human-annotated references.
  arXiv  Detail & Related papers  (2022-09-22T13:33:48Z)
• What should I Ask: A Knowledge-driven Approach for Follow-up Questions Generation in Conversational Surveys [63.51903260461746]
  We propose a novel task for knowledge-driven follow-up question generation in conversational surveys. We constructed a new human-annotated dataset of human-written follow-up questions with dialogue history and labeled knowledge. We then propose a two-staged knowledge-driven model for the task, which generates informative and coherent follow-up questions.
  arXiv  Detail & Related papers  (2022-05-23T00:57:33Z)
• Read before Generate! Faithful Long Form Question Answering with Machine Reading [77.17898499652306]
  Long-form question answering (LFQA) aims to generate a paragraph-length answer for a given question. We propose a new end-to-end framework that jointly models answer generation and machine reading.
  arXiv  Detail & Related papers  (2022-03-01T10:41:17Z)
• Reinforced Multi-task Approach for Multi-hop Question Generation [47.15108724294234]
  We take up Multi-hop question generation, which aims at generating relevant questions based on supporting facts in the context. We employ multitask learning with the auxiliary task of answer-aware supporting fact prediction to guide the question generator. We demonstrate the effectiveness of our approach through experiments on the multi-hop question answering dataset, HotPotQA.
  arXiv  Detail & Related papers  (2020-04-05T10:16:59Z)
• Asking Questions the Human Way: Scalable Question-Answer Generation from Text Corpus [23.676748207014903]
  We propose Answer-Clue-Style-aware Question Generation (ACS-QG) It aims at automatically generating high-quality and diverse question-answer pairs from unlabeled text corpus at scale. We can generate 2.8 million quality-assured question-answer pairs from a million sentences found in Wikipedia.
  arXiv  Detail & Related papers  (2020-01-27T05:27:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.