Research on the Application of Large Language Models in Automatic Question Generation: A Case Study of ChatGLM in the Context of High School Information Technology Curriculum

• URL: http://arxiv.org/abs/2408.11539v1
• Date: Wed, 21 Aug 2024 11:38:32 GMT
• Title: Research on the Application of Large Language Models in Automatic Question Generation: A Case Study of ChatGLM in the Context of High School Information Technology Curriculum
• Authors: Yanxin Chen, Ling He,
• Abstract summary: The model is guided to generate diverse questions, which are then comprehensively evaluated by domain experts. The results indicate that ChatGLM outperforms human-generated questions in terms of clarity and teachers' willingness to use.
• Score: 3.0753648264454547
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This study investigates the application effectiveness of the Large Language Model (LLMs) ChatGLM in the automated generation of high school information technology exam questions. Through meticulously designed prompt engineering strategies, the model is guided to generate diverse questions, which are then comprehensively evaluated by domain experts. The evaluation dimensions include the Hitting(the degree of alignment with teaching content), Fitting (the degree of embodiment of core competencies), Clarity (the explicitness of question descriptions), and Willing to use (the teacher's willingness to use the question in teaching). The results indicate that ChatGLM outperforms human-generated questions in terms of clarity and teachers' willingness to use, although there is no significant difference in hit rate and fit. This finding suggests that ChatGLM has the potential to enhance the efficiency of question generation and alleviate the burden on teachers, providing a new perspective for the future development of educational assessment systems. Future research could explore further optimizations to the ChatGLM model to maintain high fit and hit rates while improving the clarity of questions and teachers' willingness to use them.

Related papers

• AGENT-CQ: Automatic Generation and Evaluation of Clarifying Questions for Conversational Search with LLMs [53.6200736559742]
  AGENT-CQ consists of two stages: a generation stage and an evaluation stage. CrowdLLM simulates human crowdsourcing judgments to assess generated questions and answers. Experiments on the ClariQ dataset demonstrate CrowdLLM's effectiveness in evaluating question and answer quality.
  arXiv  Detail & Related papers  (2024-10-25T17:06:27Z)
• The Future of Learning in the Age of Generative AI: Automated Question Generation and Assessment with Large Language Models [0.0]
  Large language models (LLMs) and generative AI have revolutionized natural language processing (NLP) This chapter explores the transformative potential of LLMs in automated question generation and answer assessment.
  arXiv  Detail & Related papers  (2024-10-12T15:54:53Z)
• Application of Large Language Models in Automated Question Generation: A Case Study on ChatGLM's Structured Questions for National Teacher Certification Exams [2.7363336723930756]
  This study explores the application potential of the large language models (LLMs) ChatGLM in the automatic generation of structured questions for National Teacher Certification Exams (NTCE) We guided ChatGLM to generate a series of simulated questions and conducted a comprehensive comparison with questions recollected from past examinees. The research results indicate that the questions generated by ChatGLM exhibit a high level of rationality, scientificity, and practicality similar to those of the real exam questions.
  arXiv  Detail & Related papers  (2024-08-19T13:32:14Z)
• Automated Educational Question Generation at Different Bloom's Skill Levels using Large Language Models: Strategies and Evaluation [0.0]
  We examine the ability of five state-of-the-art large language models to generate diverse and high-quality questions of different cognitive levels. Our findings suggest that LLms can generate relevant and high-quality educational questions of different cognitive levels when prompted with adequate information.
  arXiv  Detail & Related papers  (2024-08-08T11:56:57Z)
• Knowledge Tagging System on Math Questions via LLMs with Flexible Demonstration Retriever [48.5585921817745]
  Large Language Models (LLMs) are used to automate the knowledge tagging task. We show the strong performance of zero- and few-shot results over math questions knowledge tagging tasks. By proposing a reinforcement learning-based demonstration retriever, we successfully exploit the great potential of different-sized LLMs.
  arXiv  Detail & Related papers  (2024-06-19T23:30:01Z)
• A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models [71.25225058845324]
  Large Language Models (LLMs) have demonstrated revolutionary abilities in language understanding and generation. Retrieval-Augmented Generation (RAG) can offer reliable and up-to-date external knowledge. RA-LLMs have emerged to harness external and authoritative knowledge bases, rather than relying on the model's internal knowledge.
  arXiv  Detail & Related papers  (2024-05-10T02:48:45Z)
• Retrieval-augmented Generation to Improve Math Question-Answering: Trade-offs Between Groundedness and Human Preference [0.0]
  We design prompts that retrieve and use content from a high-quality open-source math textbook to generate responses to real student questions. We evaluate the efficacy of this RAG system for middle-school algebra and geometry QA by administering a multi-condition survey. We argue that while RAG is able to improve response quality, designers of math QA systems must consider trade-offs between generating responses preferred by students and responses closely matched to specific educational resources.
  arXiv  Detail & Related papers  (2023-10-04T22:09:28Z)
• Automating question generation from educational text [1.9325905076281444]
  The use of question-based activities (QBAs) is wide-spread in education, forming an integral part of the learning and assessment process. We design and evaluate an automated question generation tool for formative and summative assessment in schools.
  arXiv  Detail & Related papers  (2023-09-26T15:18:44Z)
• Covering Uncommon Ground: Gap-Focused Question Generation for Answer Assessment [75.59538732476346]
  We focus on the problem of generating such gap-focused questions (GFQs) automatically. We define the task, highlight key desired aspects of a good GFQ, and propose a model that satisfies these.
  arXiv  Detail & Related papers  (2023-07-06T22:21:42Z)
• Quiz Design Task: Helping Teachers Create Quizzes with Automated Question Generation [87.34509878569916]
  This paper focuses on the use case of helping teachers automate the generation of reading comprehension quizzes. In our study, teachers building a quiz receive question suggestions, which they can either accept or refuse with a reason.
  arXiv  Detail & Related papers  (2022-05-03T18:59:03Z)
• Neural Multi-Task Learning for Teacher Question Detection in Online Classrooms [50.19997675066203]
  We build an end-to-end neural framework that automatically detects questions from teachers' audio recordings. By incorporating multi-task learning techniques, we are able to strengthen the understanding of semantic relations among different types of questions.
  arXiv  Detail & Related papers  (2020-05-16T02:17:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.