Exploring Iterative Enhancement for Improving Learnersourced
Multiple-Choice Question Explanations with Large Language Models
- URL: http://arxiv.org/abs/2309.10444v4
- Date: Sun, 10 Mar 2024 13:48:41 GMT
- Title: Exploring Iterative Enhancement for Improving Learnersourced
Multiple-Choice Question Explanations with Large Language Models
- Authors: Qiming Bao, Juho Leinonen, Alex Yuxuan Peng, Wanjun Zhong, Ga\"el
Gendron, Timothy Pistotti, Alice Huang, Paul Denny, Michael Witbrock and
Jiamou Liu
- Abstract summary: We present and evaluate a framework called "ILearner-LLM" to scaffold the task of automated explanation generation.
The framework generates high-quality student-aligned explanations by iteratively feeding the quality rating score from the evaluation model back into the instruction prompt.
Our findings represent a promising path to enrich the learnersourcing experience for students.
- Score: 23.12128710240935
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Large language models exhibit superior capabilities in processing and
understanding language, yet their applications in educational contexts remain
underexplored. Learnersourcing enhances learning by engaging students in
creating their own educational content. When learnersourcing multiple-choice
questions, creating explanations for the solution of a question is a crucial
step; it helps other students understand the solution and promotes a deeper
understanding of related concepts. However, it is often difficult for students
to craft effective solution explanations, due to limited subject understanding.
To help scaffold the task of automated explanation generation, we present and
evaluate a framework called "ILearner-LLM", that iteratively enhances the
generated explanations for the given questions with large language models.
Comprising an explanation generation model and an explanation evaluation model,
the framework generates high-quality student-aligned explanations by
iteratively feeding the quality rating score from the evaluation model back
into the instruction prompt of the explanation generation model. Experimental
results demonstrate the effectiveness of our ILearner-LLM on LLaMA2-13B and
GPT-4 to generate higher quality explanations that are closer to those written
by students on five PeerWise datasets. Our findings represent a promising path
to enrich the learnersourcing experience for students and to enhance the
capabilities of large language models for educational applications.
Related papers
- From Feature Importance to Natural Language Explanations Using LLMs with RAG [4.204990010424084]
We introduce traceable question-answering, leveraging an external knowledge repository to inform responses of Large Language Models (LLMs)
This knowledge repository comprises contextual details regarding the model's output, containing high-level features, feature importance, and alternative probabilities.
We integrate four key characteristics - social, causal, selective, and contrastive - drawn from social science research on human explanations into a single-shot prompt, guiding the response generation process.
arXiv Detail & Related papers (2024-07-30T17:27:20Z) - Evaluating and Optimizing Educational Content with Large Language Model Judgments [52.33701672559594]
We use Language Models (LMs) as educational experts to assess the impact of various instructions on learning outcomes.
We introduce an instruction optimization approach in which one LM generates instructional materials using the judgments of another LM as a reward function.
Human teachers' evaluations of these LM-generated worksheets show a significant alignment between the LM judgments and human teacher preferences.
arXiv Detail & Related papers (2024-03-05T09:09:15Z) - YODA: Teacher-Student Progressive Learning for Language Models [82.0172215948963]
This paper introduces YODA, a teacher-student progressive learning framework.
It emulates the teacher-student education process to improve the efficacy of model fine-tuning.
Experiments show that training LLaMA2 with data from YODA improves SFT with significant performance gain.
arXiv Detail & Related papers (2024-01-28T14:32:15Z) - Advancing Large Multi-modal Models with Explicit Chain-of-Reasoning and Visual Question Generation [34.45251681923171]
This paper presents a novel approach to develop a large Vision-and-Language Models (VLMs)
We introduce a system that can ask a question to acquire necessary knowledge, thereby enhancing the robustness and explicability of the reasoning process.
The dataset covers a range of tasks, from common ones like caption generation to specialized VQA tasks that require expert knowledge.
arXiv Detail & Related papers (2024-01-18T14:21:56Z) - Assertion Enhanced Few-Shot Learning: Instructive Technique for Large
Language Models to Generate Educational Explanations [0.0]
Human educators possess an intrinsic ability to anticipate and seek educational explanations from students.
We aim to imbue Intelligent Tutoring Systems with this ability using few-shot learning capability of Large Language Models.
arXiv Detail & Related papers (2023-12-05T20:41:34Z) - Explanation-aware Soft Ensemble Empowers Large Language Model In-context
Learning [50.00090601424348]
Large language models (LLMs) have shown remarkable capabilities in various natural language understanding tasks.
We propose EASE, an Explanation-Aware Soft Ensemble framework to empower in-context learning with LLMs.
arXiv Detail & Related papers (2023-11-13T06:13:38Z) - Automating question generation from educational text [1.9325905076281444]
The use of question-based activities (QBAs) is wide-spread in education, forming an integral part of the learning and assessment process.
We design and evaluate an automated question generation tool for formative and summative assessment in schools.
arXiv Detail & Related papers (2023-09-26T15:18:44Z) - Complementary Explanations for Effective In-Context Learning [77.83124315634386]
Large language models (LLMs) have exhibited remarkable capabilities in learning from explanations in prompts.
This work aims to better understand the mechanisms by which explanations are used for in-context learning.
arXiv Detail & Related papers (2022-11-25T04:40:47Z) - Explanations from Large Language Models Make Small Reasoners Better [61.991772773700006]
We show that our method can consistently and significantly outperform finetuning baselines across different settings.
As a side benefit, human evaluation shows that our method can generate high-quality explanations to justify its predictions.
arXiv Detail & Related papers (2022-10-13T04:50:02Z) - Enhancing Dialogue Generation via Multi-Level Contrastive Learning [57.005432249952406]
We propose a multi-level contrastive learning paradigm to model the fine-grained quality of the responses with respect to the query.
A Rank-aware (RC) network is designed to construct the multi-level contrastive optimization objectives.
We build a Knowledge Inference (KI) component to capture the keyword knowledge from the reference during training and exploit such information to encourage the generation of informative words.
arXiv Detail & Related papers (2020-09-19T02:41:04Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.