Adaptive Scaffolding in Block-Based Programming via Synthesizing New
Tasks as Pop Quizzes
- URL: http://arxiv.org/abs/2303.16359v1
- Date: Tue, 28 Mar 2023 23:52:15 GMT
- Title: Adaptive Scaffolding in Block-Based Programming via Synthesizing New
Tasks as Pop Quizzes
- Authors: Ahana Ghosh, Sebastian Tschiatschek, Sam Devlin, Adish Singla
- Abstract summary: We introduce a scaffolding framework based on pop quizzes presented as multi-choice programming tasks.
To automatically generate these pop quizzes, we propose a novel algorithm, PQuizSyn.
Our algorithm synthesizes new tasks for pop quizzes with the following features: (a) Adaptive (i.e., individualized to the student's current attempt), (b) Comprehensible (i.e., easy to comprehend and solve), and (c) Concealing, do not reveal the solution code.
- Score: 30.127552292093384
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Block-based programming environments are increasingly used to introduce
computing concepts to beginners. However, novice students often struggle in
these environments, given the conceptual and open-ended nature of programming
tasks. To effectively support a student struggling to solve a given task, it is
important to provide adaptive scaffolding that guides the student towards a
solution. We introduce a scaffolding framework based on pop quizzes presented
as multi-choice programming tasks. To automatically generate these pop quizzes,
we propose a novel algorithm, PQuizSyn. More formally, given a reference task
with a solution code and the student's current attempt, PQuizSyn synthesizes
new tasks for pop quizzes with the following features: (a) Adaptive (i.e.,
individualized to the student's current attempt), (b) Comprehensible (i.e.,
easy to comprehend and solve), and (c) Concealing (i.e., do not reveal the
solution code). Our algorithm synthesizes these tasks using techniques based on
symbolic reasoning and graph-based code representations. We show that our
algorithm can generate hundreds of pop quizzes for different student attempts
on reference tasks from Hour of Code: Maze Challenge and Karel. We assess the
quality of these pop quizzes through expert ratings using an evaluation rubric.
Further, we have built an online platform for practicing block-based
programming tasks empowered via pop quiz based feedback, and report results
from an initial user study.
Related papers
- A Knowledge-Component-Based Methodology for Evaluating AI Assistants [9.412070852474313]
We evaluate an automatic hint generator for CS1 programming assignments powered by GPT-4.
This system provides natural language guidance about how students can improve their incorrect solutions to short programming exercises.
arXiv Detail & Related papers (2024-06-09T00:58:39Z) - Promptly: Using Prompt Problems to Teach Learners How to Effectively
Utilize AI Code Generators [5.458849730200646]
This paper introduces a novel pedagogical concept known as a Prompt Problem'
A Prompt Problem challenges a student to create a natural language prompt that leads an LLM to produce the correct code for a specific problem.
We report empirical findings from a field study in which Promptly was deployed in a first-year Python programming course.
arXiv Detail & Related papers (2023-07-31T01:46:42Z) - Synthesizing a Progression of Subtasks for Block-Based Visual
Programming Tasks [21.33708484899808]
We propose a novel synthesis algorithm that generates a progression of subtasks that are high-quality, well-spaced in terms of their complexity.
We show the utility of our synthesis algorithm in improving the efficacy of AI agents for solving tasks in the Karel programming environment.
arXiv Detail & Related papers (2023-05-27T16:24:36Z) - Reinforcement Learning with Success Induced Task Prioritization [68.8204255655161]
We introduce Success Induced Task Prioritization (SITP), a framework for automatic curriculum learning.
The algorithm selects the order of tasks that provide the fastest learning for agents.
We demonstrate that SITP matches or surpasses the results of other curriculum design methods.
arXiv Detail & Related papers (2022-12-30T12:32:43Z) - JiuZhang: A Chinese Pre-trained Language Model for Mathematical Problem
Understanding [74.12405417718054]
This paper aims to advance the mathematical intelligence of machines by presenting the first Chinese mathematical pre-trained language model(PLM)
Unlike other standard NLP tasks, mathematical texts are difficult to understand, since they involve mathematical terminology, symbols and formulas in the problem statement.
We design a novel curriculum pre-training approach for improving the learning of mathematical PLMs, consisting of both basic and advanced courses.
arXiv Detail & Related papers (2022-06-13T17:03:52Z) - From {Solution Synthesis} to {Student Attempt Synthesis} for Block-Based
Visual Programming Tasks [20.64766977405438]
We introduce a novel benchmark, StudentSyn, centered around the following challenge.
For a given student, synthesize the student's attempt on a new target task after observing the student's attempt on a fixed reference task.
This challenge is akin to that of program synthesis; however, instead of a solution (i.e., program an expert would write), the goal here is to synthesize a student attempt (i.e., program that a given student would write)
arXiv Detail & Related papers (2022-05-03T01:32:47Z) - Continuous Examination by Automatic Quiz Assessment Using Spiral Codes
and Image Processing [69.35569554213679]
Paper quizzes are affordable and within reach of campus education in classrooms.
correction of the quiz is a considerable obstacle.
We suggest mitigating the issue by a novel image processing technique.
arXiv Detail & Related papers (2022-01-26T22:58:15Z) - ProtoTransformer: A Meta-Learning Approach to Providing Student Feedback [54.142719510638614]
In this paper, we frame the problem of providing feedback as few-shot classification.
A meta-learner adapts to give feedback to student code on a new programming question from just a few examples by instructors.
Our approach was successfully deployed to deliver feedback to 16,000 student exam-solutions in a programming course offered by a tier 1 university.
arXiv Detail & Related papers (2021-07-23T22:41:28Z) - BUSTLE: Bottom-Up Program Synthesis Through Learning-Guided Exploration [72.88493072196094]
We present a new synthesis approach that leverages learning to guide a bottom-up search over programs.
In particular, we train a model to prioritize compositions of intermediate values during search conditioned on a set of input-output examples.
We show that the combination of learning and bottom-up search is remarkably effective, even with simple supervised learning approaches.
arXiv Detail & Related papers (2020-07-28T17:46:18Z) - Sequential Transfer in Reinforcement Learning with a Generative Model [48.40219742217783]
We show how to reduce the sample complexity for learning new tasks by transferring knowledge from previously-solved ones.
We derive PAC bounds on its sample complexity which clearly demonstrate the benefits of using this kind of prior knowledge.
We empirically verify our theoretical findings in simple simulated domains.
arXiv Detail & Related papers (2020-07-01T19:53:35Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.