Learning to Use AI for Learning: How Can We Effectively Teach and Measure Prompting Literacy for K-12 Students?

• URL: http://arxiv.org/abs/2508.13962v1
• Date: Tue, 19 Aug 2025 15:54:51 GMT
• Title: Learning to Use AI for Learning: How Can We Effectively Teach and Measure Prompting Literacy for K-12 Students?
• Authors: Ruiwei Xiao, Xinying Hou, Ying-Jui Tseng, Hsuan Nieu, Guanze Liao, John Stamper, Kenneth R. Koedinger,
• Abstract summary: There is a growing need to equip the next generation with the ability to apply, interact with, evaluate, and collaborate with AI systems responsibly.<n>To address this need, we designed an Large-Language Model (LLM)-based module to teach prompting literacy.<n>We conducted two iterations of classroom deployment in 11 authentic secondary education classrooms, and evaluated 1) AI-based auto-grader's capability; 2) students' prompting performance and confidence changes towards using AI for learning; and 3) the quality of learning and assessment materials.
• Score: 1.413488665073795
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As Artificial Intelligence (AI) becomes increasingly integrated into daily life, there is a growing need to equip the next generation with the ability to apply, interact with, evaluate, and collaborate with AI systems responsibly. Prior research highlights the urgent demand from K-12 educators to teach students the ethical and effective use of AI for learning. To address this need, we designed an Large-Language Model (LLM)-based module to teach prompting literacy. This includes scenario-based deliberate practice activities with direct interaction with intelligent LLM agents, aiming to foster secondary school students' responsible engagement with AI chatbots. We conducted two iterations of classroom deployment in 11 authentic secondary education classrooms, and evaluated 1) AI-based auto-grader's capability; 2) students' prompting performance and confidence changes towards using AI for learning; and 3) the quality of learning and assessment materials. Results indicated that the AI-based auto-grader could grade student-written prompts with satisfactory quality. In addition, the instructional materials supported students in improving their prompting skills through practice and led to positive shifts in their perceptions of using AI for learning. Furthermore, data from Study 1 informed assessment revisions in Study 2. Analyses of item difficulty and discrimination in Study 2 showed that True/False and open-ended questions could measure prompting literacy more effectively than multiple-choice questions for our target learners. These promising outcomes highlight the potential for broader deployment and highlight the need for broader studies to assess learning effectiveness and assessment design.

Related papers

• AI Literacy as a Key Driver of User Experience in AI-Powered Assessment: Insights from Socratic Mind [2.0272430076690027]
  This study examines how students' AI literacy and prior exposure to AI technologies shape their perceptions of Socratic Mind.<n>Data from 309 undergraduates in Computer Science and Business courses were collected.
  arXiv  Detail & Related papers  (2025-07-29T10:11:24Z)
• Robots and Children that Learn Together : Improving Knowledge Retention by Teaching Peer-Like Interactive Robots [41.94295877935867]
  This study introduces Interactive Reinforcement Learning as a cognitive model for teachable social robots.<n>Children in the LbT condition achieved significantly higher retention gains compared to those in the self-practice condition.<n>This work makes two contributions: (1) it introduces Interactive RL as a pedagogically effective and scalable model for peer-robot learning, and (2) it demonstrates, for the first time, the feasibility of deploying multiple autonomous robots simultaneously in real classrooms.
  arXiv  Detail & Related papers  (2025-06-23T07:51:04Z)
• Evaluating AI-Powered Learning Assistants in Engineering Higher Education: Student Engagement, Ethical Challenges, and Policy Implications [0.2812395851874055]
  This study evaluates the use of the Educational AI Hub, an AI-powered learning framework, in undergraduate civil and environmental engineering courses at a large R1 public university.<n>Students appreciated the AI assistant for its convenience and comfort, with nearly half reporting greater ease in using the AI tool.<n>While most students viewed AI use as ethically acceptable, many expressed uncertainties about institutional policies and apprehension about potential academic misconduct.
  arXiv  Detail & Related papers  (2025-06-06T03:02:49Z)
• "From Unseen Needs to Classroom Solutions": Exploring AI Literacy Challenges & Opportunities with Project-based Learning Toolkit in K-12 Education [0.3994567502796064]
  There is a growing need to equip K-12 students with AI literacy skills that extend beyond computer science.<n>This paper explores the integration of a Project-Based Learning (PBL) AI toolkit into diverse subject areas, aimed at helping educators teach AI concepts more effectively.
  arXiv  Detail & Related papers  (2024-12-23T03:31:02Z)
• AI in Education: Rationale, Principles, and Instructional Implications [0.0]
  Generative AI, like ChatGPT, can create human-like content, prompting questions about its educational role.<n>The study emphasizes deliberate strategies to ensure AI complements, not replaces, genuine cognitive effort.
  arXiv  Detail & Related papers  (2024-12-02T14:08:07Z)
• Human-Centric eXplainable AI in Education [0.0]
  This paper explores Human-Centric eXplainable AI (HCXAI) in the educational landscape. It emphasizes its role in enhancing learning outcomes, fostering trust among users, and ensuring transparency in AI-driven tools. It outlines comprehensive frameworks for developing HCXAI systems that prioritize user understanding and engagement.
  arXiv  Detail & Related papers  (2024-10-18T14:02:47Z)
• Do great minds think alike? Investigating Human-AI Complementarity in Question Answering with CAIMIRA [43.116608441891096]
  Humans outperform AI systems in knowledge-grounded abductive and conceptual reasoning. State-of-the-art LLMs like GPT-4 and LLaMA show superior performance on targeted information retrieval.
  arXiv  Detail & Related papers  (2024-10-09T03:53:26Z)
• Effects of a Prompt Engineering Intervention on Undergraduate Students' AI Self-Efficacy, AI Knowledge and Prompt Engineering Ability: A Mixed Methods Study [36.48421439947282]
  This study designed and implemented a prompt engineering intervention at a university in Hong Kong. It examined students' AI self-efficacy, AI knowledge, and proficiency in creating effective prompts.
  arXiv  Detail & Related papers  (2024-07-30T15:05:24Z)
• Charting the Future of AI in Project-Based Learning: A Co-Design Exploration with Students [35.05435052195561]
  The increasing use of Artificial Intelligence (AI) by students in learning presents new challenges for assessing their learning outcomes. This paper introduces a co-design study to explore the potential of students' AI usage data as a novel material for assessment.
  arXiv  Detail & Related papers  (2024-01-26T14:49:29Z)
• Learning to Prompt in the Classroom to Understand AI Limits: A pilot study [35.06607166918901]
  Large Language Models (LLM) and the derived chatbots, like ChatGPT, have highly improved the natural language processing capabilities of AI systems. However, excitement has led to negative sentiments, even as AI methods demonstrate remarkable contributions. A pilot educational intervention was performed in a high school with 21 students.
  arXiv  Detail & Related papers  (2023-07-04T07:51:37Z)
• AGI: Artificial General Intelligence for Education [41.45039606933712]
  This position paper reviews artificial general intelligence (AGI)'s key concepts, capabilities, scope, and potential within future education. It highlights that AGI can significantly improve intelligent tutoring systems, educational assessment, and evaluation procedures. The paper emphasizes that AGI's capabilities extend to understanding human emotions and social interactions.
  arXiv  Detail & Related papers  (2023-04-24T22:31:59Z)
• Reinforcement Learning Tutor Better Supported Lower Performers in a Math Task [32.6507926764587]
  Reinforcement learning could be a key tool to reduce the development cost and improve the effectiveness of intelligent tutoring software. We show that deep reinforcement learning can be used to provide adaptive pedagogical support to students learning about the concept of volume.
  arXiv  Detail & Related papers  (2023-04-11T02:11:24Z)
• Personalized Education in the AI Era: What to Expect Next? [76.37000521334585]
  The objective of personalized learning is to design an effective knowledge acquisition track that matches the learner's strengths and bypasses her weaknesses to meet her desired goal. In recent years, the boost of artificial intelligence (AI) and machine learning (ML) has unfolded novel perspectives to enhance personalized education.
  arXiv  Detail & Related papers  (2021-01-19T12:23:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.