How Useful are Educational Questions Generated by Large Language Models?

• URL: http://arxiv.org/abs/2304.06638v1
• Date: Thu, 13 Apr 2023 16:05:25 GMT
• Title: How Useful are Educational Questions Generated by Large Language Models?
• Authors: Sabina Elkins, Ekaterina Kochmar, Jackie C.K. Cheung, Iulian Serban
• Abstract summary: High quality and diverse question generation can dramatically reduce the load on teachers and improve the quality of their educational content. Recent work in this domain has made progress with generation, but fails to show that real teachers judge the generated questions as sufficiently useful for the classroom setting.
• Score: 4.694536172504848
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Controllable text generation (CTG) by large language models has a huge potential to transform education for teachers and students alike. Specifically, high quality and diverse question generation can dramatically reduce the load on teachers and improve the quality of their educational content. Recent work in this domain has made progress with generation, but fails to show that real teachers judge the generated questions as sufficiently useful for the classroom setting; or if instead the questions have errors and/or pedagogically unhelpful content. We conduct a human evaluation with teachers to assess the quality and usefulness of outputs from combining CTG and question taxonomies (Bloom's and a difficulty taxonomy). The results demonstrate that the questions generated are high quality and sufficiently useful, showing their promise for widespread use in the classroom setting.

Related papers

• Research on the Application of Large Language Models in Automatic Question Generation: A Case Study of ChatGLM in the Context of High School Information Technology Curriculum [3.0753648264454547]
  The model is guided to generate diverse questions, which are then comprehensively evaluated by domain experts. The results indicate that ChatGLM outperforms human-generated questions in terms of clarity and teachers' willingness to use.
  arXiv  Detail & Related papers  (2024-08-21T11:38:32Z)
• Stepwise Verification and Remediation of Student Reasoning Errors with Large Language Model Tutors [78.53699244846285]
  Large language models (LLMs) present an opportunity to scale high-quality personalized education to all. LLMs struggle to precisely detect student's errors and tailor their feedback to these errors. Inspired by real-world teaching practice where teachers identify student errors and customize their response based on them, we focus on verifying student solutions.
  arXiv  Detail & Related papers  (2024-07-12T10:11:40Z)
• To what extent is ChatGPT useful for language teacher lesson plan creation? [0.0]
  This study examines trends in prompt specificity, variability, and weaknesses in foreign language teacher lesson plans generated by zero-shot prompting. Iterating a series of prompts that increased in complexity, we found that output lesson plans were generally high quality. Results suggest that the training of generative AI models on classic texts concerning pedagogical practices may represent a currently underexplored topic.
  arXiv  Detail & Related papers  (2024-04-25T12:00:03Z)
• YODA: Teacher-Student Progressive Learning for Language Models [82.0172215948963]
  This paper introduces YODA, a teacher-student progressive learning framework. It emulates the teacher-student education process to improve the efficacy of model fine-tuning. Experiments show that training LLaMA2 with data from YODA improves SFT with significant performance gain.
  arXiv  Detail & Related papers  (2024-01-28T14:32:15Z)
• How Teachers Can Use Large Language Models and Bloom's Taxonomy to Create Educational Quizzes [5.487297537295827]
  This paper applies a large language model-based QG approach where questions are generated with learning goals derived from Bloom's taxonomy. The results demonstrate that teachers prefer to write quizzes with automatically generated questions, and that such quizzes have no loss in quality compared to handwritten versions.
  arXiv  Detail & Related papers  (2024-01-11T13:47:13Z)
• Covering Uncommon Ground: Gap-Focused Question Generation for Answer Assessment [75.59538732476346]
  We focus on the problem of generating such gap-focused questions (GFQs) automatically. We define the task, highlight key desired aspects of a good GFQ, and propose a model that satisfies these.
  arXiv  Detail & Related papers  (2023-07-06T22:21:42Z)
• Is ChatGPT a Good Teacher Coach? Measuring Zero-Shot Performance For Scoring and Providing Actionable Insights on Classroom Instruction [5.948322127194399]
  We investigate whether generative AI could become a cost-effective complement to expert feedback by serving as an automated teacher coach. We propose three teacher coaching tasks for generative AI: (A) scoring transcript segments based on classroom observation instruments, (B) identifying highlights and missed opportunities for good instructional strategies, and (C) providing actionable suggestions for eliciting more student reasoning. We recruit expert math teachers to evaluate the zero-shot performance of ChatGPT on each of these tasks for elementary classroom math transcripts.
  arXiv  Detail & Related papers  (2023-06-05T17:59:21Z)
• MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problems [74.73881579517055]
  We propose a framework to generate such dialogues by pairing human teachers with a Large Language Model prompted to represent common student errors. We describe how we use this framework to collect MathDial, a dataset of 3k one-to-one teacher-student tutoring dialogues.
  arXiv  Detail & Related papers  (2023-05-23T21:44:56Z)
• Neural Multi-Task Learning for Teacher Question Detection in Online Classrooms [50.19997675066203]
  We build an end-to-end neural framework that automatically detects questions from teachers' audio recordings. By incorporating multi-task learning techniques, we are able to strengthen the understanding of semantic relations among different types of questions.
  arXiv  Detail & Related papers  (2020-05-16T02:17:04Z)
• Siamese Neural Networks for Class Activity Detection [49.320548570516124]
  We build a Siamese neural framework to automatically identify teacher and student utterances from classroom recordings. The proposed model is evaluated on real-world educational datasets.
  arXiv  Detail & Related papers  (2020-05-15T14:03:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.