Connecting Feedback to Choice: Understanding Educator Preferences in GenAI vs. Human-Created Lesson Plans in K-12 Education -- A Comparative Analysis

• URL: http://arxiv.org/abs/2504.05449v1
• Date: Mon, 07 Apr 2025 19:28:19 GMT
• Title: Connecting Feedback to Choice: Understanding Educator Preferences in GenAI vs. Human-Created Lesson Plans in K-12 Education -- A Comparative Analysis
• Authors: Shawon Sarkar, Min Sun, Alex Liu, Zewei Tian, Lief Esbenshade, Jian He, Zachary Zhang,
• Abstract summary: generative AI (GenAI) models are increasingly explored for educational applications.<n>This study compares lesson plans authored by human curriculum designers, a fine-tuned LLaMA-2-13b model trained on K-12 content, and a customized GPT-4 model.<n>Using a large-scale preference study with K-12 math educators, we examine how preferences vary across grade levels and instructional components.
• Score: 11.204345070162592
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: As generative AI (GenAI) models are increasingly explored for educational applications, understanding educator preferences for AI-generated lesson plans is critical for their effective integration into K-12 instruction. This exploratory study compares lesson plans authored by human curriculum designers, a fine-tuned LLaMA-2-13b model trained on K-12 content, and a customized GPT-4 model to evaluate their pedagogical quality across multiple instructional measures: warm-up activities, main tasks, cool-down activities, and overall quality. Using a large-scale preference study with K-12 math educators, we examine how preferences vary across grade levels and instructional components. We employ both qualitative and quantitative analyses. The raw preference results indicate that human-authored lesson plans are generally favored, particularly for elementary education, where educators emphasize student engagement, scaffolding, and collaborative learning. However, AI-generated models demonstrate increasing competitiveness in cool-down tasks and structured learning activities, particularly in high school settings. Beyond quantitative results, we conduct thematic analysis using LDA and manual coding to identify key factors influencing educator preferences. Educators value human-authored plans for their nuanced differentiation, real-world contextualization, and student discourse facilitation. Meanwhile, AI-generated lesson plans are often praised for their structure and adaptability for specific instructional tasks. Findings suggest a human-AI collaborative approach to lesson planning, where GenAI can serve as an assistive tool rather than a replacement for educator expertise in lesson planning. This study contributes to the growing discourse on responsible AI integration in education, highlighting both opportunities and challenges in leveraging GenAI for curriculum development.

Related papers

• EducationQ: Evaluating LLMs' Teaching Capabilities Through Multi-Agent Dialogue Framework [9.76455227840645]
  Large language models (LLMs) increasingly serve as educational tools, yet evaluating their teaching capabilities remains challenging. We introduce EducationQ, a multi-agent dialogue framework that efficiently assesses teaching capabilities through simulated dynamic educational scenarios.
  arXiv  Detail & Related papers  (2025-04-21T07:48:20Z)
• LLM-powered Multi-agent Framework for Goal-oriented Learning in Intelligent Tutoring System [54.71619734800526]
  GenMentor is a multi-agent framework designed to deliver goal-oriented, personalized learning within ITS.<n>It maps learners' goals to required skills using a fine-tuned LLM trained on a custom goal-to-skill dataset.<n>GenMentor tailors learning content with an exploration-drafting-integration mechanism to align with individual learner needs.
  arXiv  Detail & Related papers  (2025-01-27T03:29:44Z)
• From Automation to Cognition: Redefining the Roles of Educators and Generative AI in Computing Education [2.0628700367476203]
  Generative Artificial Intelligence (GenAI) offers opportunities to revolutionise teaching and learning in Computing Education (CE)<n>However, educators have expressed concerns that students may over-rely on GenAI and use these tools to generate solutions without engaging in the learning process.<n>This paper describes our experiences with using GenAI in CS-focused educational settings and the changes we have implemented accordingly in our teaching.
  arXiv  Detail & Related papers  (2024-12-16T03:36:25Z)
• Enhancing Instructional Quality: Leveraging Computer-Assisted Textual Analysis to Generate In-Depth Insights from Educational Artifacts [13.617709093240231]
  We examine how artificial intelligence (AI) and machine learning (ML) methods can analyze educational content, teacher discourse, and student responses to foster instructional improvement. We identify key areas where AI/ML integration offers significant advantages, including teacher coaching, student support, and content development. This paper emphasizes the importance of aligning AI/ML technologies with pedagogical goals to realize their full potential in educational settings.
  arXiv  Detail & Related papers  (2024-03-06T18:29:18Z)
• Evaluating and Optimizing Educational Content with Large Language Model Judgments [52.33701672559594]
  We use Language Models (LMs) as educational experts to assess the impact of various instructions on learning outcomes. We introduce an instruction optimization approach in which one LM generates instructional materials using the judgments of another LM as a reward function. Human teachers' evaluations of these LM-generated worksheets show a significant alignment between the LM judgments and human teacher preferences.
  arXiv  Detail & Related papers  (2024-03-05T09:09:15Z)
• Tailoring Education with GenAI: A New Horizon in Lesson Planning [0.21427777919040414]
  This study introduces a GenAI tool, designed as a digital assistant for educators, enabling the creation of customized lesson plans. The tool utilizes an innovative feature termed 'interactive mega-prompt,' a comprehensive query system that allows educators to input detailed classroom specifics. To evaluate the tool's effectiveness, a comprehensive methodology incorporating both quantitative (i.e., % of time savings) and qualitative (i.e., user satisfaction) criteria was implemented.
  arXiv  Detail & Related papers  (2024-02-12T17:30:05Z)
• Designing for human-AI complementarity in K-12 education [2.741266294612776]
  We present the iterative design and evaluation of Lumilo, smart glasses that help teachers help their students in AI-supported classrooms. Results from a field study conducted in K-12 classrooms indicate that students learn more when teachers and AI tutors work together.
  arXiv  Detail & Related papers  (2021-04-02T22:38:50Z)
• Personalized Education in the AI Era: What to Expect Next? [76.37000521334585]
  The objective of personalized learning is to design an effective knowledge acquisition track that matches the learner's strengths and bypasses her weaknesses to meet her desired goal. In recent years, the boost of artificial intelligence (AI) and machine learning (ML) has unfolded novel perspectives to enhance personalized education.
  arXiv  Detail & Related papers  (2021-01-19T12:23:32Z)
• Creation and Evaluation of a Pre-tertiary Artificial Intelligence (AI) Curriculum [58.86139968005518]
  The Chinese University of Hong Kong (CUHK)-Jockey Club AI for the Future Project (AI4Future) co-created an AI curriculum for pre-tertiary education. A team of 14 professors with expertise in engineering and education collaborated with 17 principals and teachers from 6 secondary schools to co-create the curriculum. The co-creation process generated a variety of resources which enhanced the teachers knowledge in AI, as well as fostered teachers autonomy in bringing the subject matter into their classrooms.
  arXiv  Detail & Related papers  (2021-01-19T11:26:19Z)
• Designing AI Learning Experiences for K-12: Emerging Works, Future Opportunities and a Design Framework [18.512494098690144]
  We analyze a collection of K-12 AI and education literature to show how core competencies of AI literacy are applied successfully. We synthesized into a conceptual framework to support researchers, designers, and educators in creating K-12 AI learning experiences.
  arXiv  Detail & Related papers  (2020-09-22T00:08:04Z)
• Dual Policy Distillation [58.43610940026261]
  Policy distillation, which transfers a teacher policy to a student policy, has achieved great success in challenging tasks of deep reinforcement learning. In this work, we introduce dual policy distillation(DPD), a student-student framework in which two learners operate on the same environment to explore different perspectives of the environment. The key challenge in developing this dual learning framework is to identify the beneficial knowledge from the peer learner for contemporary learning-based reinforcement learning algorithms.
  arXiv  Detail & Related papers  (2020-06-07T06:49:47Z)
• Explainable Active Learning (XAL): An Empirical Study of How Local Explanations Impact Annotator Experience [76.9910678786031]
  We propose a novel paradigm of explainable active learning (XAL), by introducing techniques from the recently surging field of explainable AI (XAI) into an Active Learning setting. Our study shows benefits of AI explanation as interfaces for machine teaching--supporting trust calibration and enabling rich forms of teaching feedback, and potential drawbacks--anchoring effect with the model judgment and cognitive workload.
  arXiv  Detail & Related papers  (2020-01-24T22:52:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.