CodeTailor: LLM-Powered Personalized Parsons Puzzles for Engaging Support While Learning Programming

• URL: http://arxiv.org/abs/2401.12125v3
• Date: Thu, 30 May 2024 17:46:29 GMT
• Title: CodeTailor: LLM-Powered Personalized Parsons Puzzles for Engaging Support While Learning Programming
• Authors: Xinying Hou, Zihan Wu, Xu Wang, Barbara J. Ericson,
• Abstract summary: Generative AI can create a solution for most intro-level programming problems. Students might use these tools to just generate code for them, resulting in reduced engagement and limited learning. We present CodeTailor, a system that leverages a large language model (LLM) to provide personalized help to students.
• Score: 6.43344619836303
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Learning to program can be challenging, and providing high-quality and timely support at scale is hard. Generative AI and its products, like ChatGPT, can create a solution for most intro-level programming problems. However, students might use these tools to just generate code for them, resulting in reduced engagement and limited learning. In this paper, we present CodeTailor, a system that leverages a large language model (LLM) to provide personalized help to students while still encouraging cognitive engagement. CodeTailor provides a personalized Parsons puzzle to support struggling students. In a Parsons puzzle, students place mixed-up code blocks in the correct order to solve a problem. A technical evaluation with previous incorrect student code snippets demonstrated that CodeTailor could deliver high-quality (correct, personalized, and concise) Parsons puzzles based on their incorrect code. We conducted a within-subjects study with 18 novice programmers. Participants perceived CodeTailor as more engaging than just receiving an LLM-generated solution (the baseline condition). In addition, participants applied more supported elements from the scaffolded practice to the posttest when using CodeTailor than baseline. Overall, most participants preferred using CodeTailor versus just receiving the LLM-generated code for learning. Qualitative observations and interviews also provided evidence for the benefits of CodeTailor, including thinking more about solution construction, fostering continuity in learning, promoting reflection, and boosting confidence. We suggest future design ideas to facilitate active learning opportunities with generative AI techniques.

Related papers

• Automating Personalized Parsons Problems with Customized Contexts and Concepts [2.185263087861945]
  Large language models (LLMs) may offer a solution by allowing students to produce on-demand Parsons problems. In this paper, we introduce PuzzleMakerPy, an educational tool that uses an LLM to generate unlimited contextualized drag-and-drop programming exercises. We evaluated PuzzleMakerPy by deploying it in a large introductory programming course, and found that the ability to personalize the contextual framing was highly engaging for students.
  arXiv  Detail & Related papers  (2024-04-17T02:01:50Z)
• Interactions with Prompt Problems: A New Way to Teach Programming with Large Language Models [4.1599514827277355]
  We propose a new way to teach programming with Prompt Problems. Students receive a problem visually, indicating how input should be transformed to output, and must translate that to a prompt for an LLM to decipher. The problem is considered correct when the code that is generated by the student prompt can pass all test cases.
  arXiv  Detail & Related papers  (2024-01-19T15:32:46Z)
• Code Prompting Elicits Conditional Reasoning Abilities in Text+Code LLMs [65.2379940117181]
  We introduce code prompting, a chain of prompts that transforms a natural language problem into code. We find that code prompting exhibits a high-performance boost for multiple LLMs. Our analysis of GPT 3.5 reveals that the code formatting of the input problem is essential for performance improvement.
  arXiv  Detail & Related papers  (2024-01-18T15:32:24Z)
• Integrating Personalized Parsons Problems with Multi-Level Textual Explanations to Scaffold Code Writing [7.277912553209182]
  Novice programmers need to write basic code as part of the learning process, but they often face difficulties. To assist struggling students, we recently implemented personalized Parsons problems, where students arrange blocks of code to solve them as pop-up scaffolding. Students found them to be more engaging and preferred them for learning, instead of simply receiving the correct answer.
  arXiv  Detail & Related papers  (2024-01-06T07:27:46Z)
• Lessons from Building StackSpot AI: A Contextualized AI Coding Assistant [2.268415020650315]
  A new breed of tools, built atop Large Language Models, is emerging. These tools aim to mitigate drawbacks by employing techniques like fine-tuning or enriching user prompts with contextualized information.
  arXiv  Detail & Related papers  (2023-11-30T10:51:26Z)
• Promptly: Using Prompt Problems to Teach Learners How to Effectively Utilize AI Code Generators [5.458849730200646]
  This paper introduces a novel pedagogical concept known as a Prompt Problem' A Prompt Problem challenges a student to create a natural language prompt that leads an LLM to produce the correct code for a specific problem. We report empirical findings from a field study in which Promptly was deployed in a first-year Python programming course.
  arXiv  Detail & Related papers  (2023-07-31T01:46:42Z)
• Evaluating Language Models for Mathematics through Interactions [116.67206980096513]
  We introduce CheckMate, a prototype platform for humans to interact with and evaluate large language models (LLMs) We conduct a study with CheckMate to evaluate three language models (InstructGPT, ChatGPT, and GPT-4) as assistants in proving undergraduate-level mathematics. We derive a taxonomy of human behaviours and uncover that despite a generally positive correlation, there are notable instances of divergence between correctness and perceived helpfulness.
  arXiv  Detail & Related papers  (2023-06-02T17:12:25Z)
• CodeT5+: Open Code Large Language Models for Code Understanding and Generation [72.1638273937025]
  Large language models (LLMs) pretrained on vast source code have achieved prominent progress in code intelligence. CodeT5+ is a family of encoder-decoder LLMs for code in which component modules can be flexibly combined to suit a wide range of downstream code tasks. We extensively evaluate CodeT5+ on over 20 code-related benchmarks in different settings, including zero-shot, finetuning, and instruction-tuning.
  arXiv  Detail & Related papers  (2023-05-13T14:23:07Z)
• JiuZhang: A Chinese Pre-trained Language Model for Mathematical Problem Understanding [74.12405417718054]
  This paper aims to advance the mathematical intelligence of machines by presenting the first Chinese mathematical pre-trained language model(PLM) Unlike other standard NLP tasks, mathematical texts are difficult to understand, since they involve mathematical terminology, symbols and formulas in the problem statement. We design a novel curriculum pre-training approach for improving the learning of mathematical PLMs, consisting of both basic and advanced courses.
  arXiv  Detail & Related papers  (2022-06-13T17:03:52Z)
• ProtoTransformer: A Meta-Learning Approach to Providing Student Feedback [54.142719510638614]
  In this paper, we frame the problem of providing feedback as few-shot classification. A meta-learner adapts to give feedback to student code on a new programming question from just a few examples by instructors. Our approach was successfully deployed to deliver feedback to 16,000 student exam-solutions in a programming course offered by a tier 1 university.
  arXiv  Detail & Related papers  (2021-07-23T22:41:28Z)
• Measuring Coding Challenge Competence With APPS [54.22600767666257]
  We introduce APPS, a benchmark for code generation. Our benchmark includes 10,000 problems, which range from having simple one-line solutions to being substantial algorithmic challenges. Recent models such as GPT-Neo can pass approximately 15% of the test cases of introductory problems.
  arXiv  Detail & Related papers  (2021-05-20T17:58:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.