From {Solution Synthesis} to {Student Attempt Synthesis} for Block-Based Visual Programming Tasks

• URL: http://arxiv.org/abs/2205.01265v3
• Date: Wed, 29 Mar 2023 00:07:18 GMT
• Title: From {Solution Synthesis} to {Student Attempt Synthesis} for Block-Based Visual Programming Tasks
• Authors: Adish Singla, Nikitas Theodoropoulos
• Abstract summary: We introduce a novel benchmark, StudentSyn, centered around the following challenge. For a given student, synthesize the student's attempt on a new target task after observing the student's attempt on a fixed reference task. This challenge is akin to that of program synthesis; however, instead of a solution (i.e., program an expert would write), the goal here is to synthesize a student attempt (i.e., program that a given student would write)
• Score: 20.64766977405438
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Block-based visual programming environments are increasingly used to introduce computing concepts to beginners. Given that programming tasks are open-ended and conceptual, novice students often struggle when learning in these environments. AI-driven programming tutors hold great promise in automatically assisting struggling students, and need several components to realize this potential. We investigate the crucial component of student modeling, in particular, the ability to automatically infer students' misconceptions for predicting (synthesizing) their behavior. We introduce a novel benchmark, StudentSyn, centered around the following challenge: For a given student, synthesize the student's attempt on a new target task after observing the student's attempt on a fixed reference task. This challenge is akin to that of program synthesis; however, instead of synthesizing a {solution} (i.e., program an expert would write), the goal here is to synthesize a {student attempt} (i.e., program that a given student would write). We first show that human experts (TutorSS) can achieve high performance on the benchmark, whereas simple baselines perform poorly. Then, we develop two neuro/symbolic techniques (NeurSS and SymSS) in a quest to close this gap with TutorSS.

Related papers

• ORES: Open-vocabulary Responsible Visual Synthesis [104.7572323359984]
  We formalize a new task, Open-vocabulary Responsible Visual Synthesis (ORES), where the synthesis model is able to avoid forbidden visual concepts. To address this problem, we present a Two-stage Intervention (TIN) framework. By introducing 1) rewriting with learnable instruction through a large-scale language model (LLM) and 2) synthesizing with prompt intervention on a diffusion model, it can effectively synthesize images avoiding any concepts but following the user's query as much as possible.
  arXiv  Detail & Related papers  (2023-08-26T06:47:34Z)
• Synthesizing a Progression of Subtasks for Block-Based Visual Programming Tasks [21.33708484899808]
  We propose a novel synthesis algorithm that generates a progression of subtasks that are high-quality, well-spaced in terms of their complexity. We show the utility of our synthesis algorithm in improving the efficacy of AI agents for solving tasks in the Karel programming environment.
  arXiv  Detail & Related papers  (2023-05-27T16:24:36Z)
• Neural Task Synthesis for Visual Programming [22.918489170257704]
  We seek to design neural models that can automatically generate programming tasks for a given specification in the context of visual programming domains. We propose a novel neuro-symbolic technique, NeurTaskSyn, that can synthesize programming tasks for a specification given in the form of desired programming concepts.
  arXiv  Detail & Related papers  (2023-05-26T01:08:18Z)
• Adaptive Scaffolding in Block-Based Programming via Synthesizing New Tasks as Pop Quizzes [30.127552292093384]
  We introduce a scaffolding framework based on pop quizzes presented as multi-choice programming tasks. To automatically generate these pop quizzes, we propose a novel algorithm, PQuizSyn. Our algorithm synthesizes new tasks for pop quizzes with the following features: (a) Adaptive (i.e., individualized to the student's current attempt), (b) Comprehensible (i.e., easy to comprehend and solve), and (c) Concealing, do not reveal the solution code.
  arXiv  Detail & Related papers  (2023-03-28T23:52:15Z)
• Learning Neuro-Symbolic Skills for Bilevel Planning [63.388694268198655]
  Decision-making is challenging in robotics environments with continuous object-centric states, continuous actions, long horizons, and sparse feedback. Hierarchical approaches, such as task and motion planning (TAMP), address these challenges by decomposing decision-making into two or more levels of abstraction. Our main contribution is a method for learning parameterized polices in combination with operators and samplers.
  arXiv  Detail & Related papers  (2022-06-21T19:01:19Z)
• Compositional Generalization and Decomposition in Neural Program Synthesis [59.356261137313275]
  In this paper, we focus on measuring the ability of learned program synthesizers to compositionally generalize. We first characterize several different axes along which program synthesis methods would be desired to generalize. We introduce a benchmark suite of tasks to assess these abilities based on two popular existing datasets.
  arXiv  Detail & Related papers  (2022-04-07T22:16:05Z)
• ProtoTransformer: A Meta-Learning Approach to Providing Student Feedback [54.142719510638614]
  In this paper, we frame the problem of providing feedback as few-shot classification. A meta-learner adapts to give feedback to student code on a new programming question from just a few examples by instructors. Our approach was successfully deployed to deliver feedback to 16,000 student exam-solutions in a programming course offered by a tier 1 university.
  arXiv  Detail & Related papers  (2021-07-23T22:41:28Z)
• Program Synthesis Guided Reinforcement Learning [34.342362868490525]
  Key challenge for reinforcement learning is solving long-horizon planning and control problems. Recent work has proposed leveraging programs to help guide the learning algorithm in these settings. We propose an approach that leverages program synthesis to automatically generate the guiding program.
  arXiv  Detail & Related papers  (2021-02-22T16:05:32Z)
• CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning [138.40338621974954]
  CausalWorld is a benchmark for causal structure and transfer learning in a robotic manipulation environment. Tasks consist of constructing 3D shapes from a given set of blocks - inspired by how children learn to build complex structures.
  arXiv  Detail & Related papers  (2020-10-08T23:01:13Z)
• BUSTLE: Bottom-Up Program Synthesis Through Learning-Guided Exploration [72.88493072196094]
  We present a new synthesis approach that leverages learning to guide a bottom-up search over programs. In particular, we train a model to prioritize compositions of intermediate values during search conditioned on a set of input-output examples. We show that the combination of learning and bottom-up search is remarkably effective, even with simple supervised learning approaches.
  arXiv  Detail & Related papers  (2020-07-28T17:46:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.