Evaluating Contextually Personalized Programming Exercises Created with Generative AI

• URL: http://arxiv.org/abs/2407.11994v1
• Date: Tue, 11 Jun 2024 12:59:52 GMT
• Title: Evaluating Contextually Personalized Programming Exercises Created with Generative AI
• Authors: Evanfiya Logacheva, Arto Hellas, James Prather, Sami Sarsa, Juho Leinonen,
• Abstract summary: This article reports on a user study conducted in an elective programming course that included contextually personalized programming exercises created with GPT-4. The results demonstrate that the quality of exercises generated with GPT-4 was generally high. This suggests that AI-generated programming problems can be a worthwhile addition to introductory programming courses.
• Score: 4.046163999707179
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Programming skills are typically developed through completing various hands-on exercises. Such programming problems can be contextualized to students' interests and cultural backgrounds. Prior research in educational psychology has demonstrated that context personalization of exercises stimulates learners' situational interests and positively affects their engagement. However, creating a varied and comprehensive set of programming exercises for students to practice on is a time-consuming and laborious task for computer science educators. Previous studies have shown that large language models can generate conceptually and contextually relevant programming exercises. Thus, they offer a possibility to automatically produce personalized programming problems to fit students' interests and needs. This article reports on a user study conducted in an elective introductory programming course that included contextually personalized programming exercises created with GPT-4. The quality of the exercises was evaluated by both the students and the authors. Additionally, this work investigated student attitudes towards the created exercises and their engagement with the system. The results demonstrate that the quality of exercises generated with GPT-4 was generally high. What is more, the course participants found them engaging and useful. This suggests that AI-generated programming problems can be a worthwhile addition to introductory programming courses, as they provide students with a practically unlimited pool of practice material tailored to their personal interests and educational needs.

Related papers

• Recommending the right academic programs: An interest mining approach using BERTopic [46.133648730062035]
  This paper presents the first information system that provides students with efficient recommendations based on both program content and personal preferences. BERTopic, a powerful topic modeling algorithm, is used that leverages text embedding techniques to generate topic representations. A case study at a post-secondary school shows that the system provides immediate and effective decision support.
  arXiv  Detail & Related papers  (2025-01-11T16:34:10Z)
• Dynamic Skill Adaptation for Large Language Models [78.31322532135272]
  We present Dynamic Skill Adaptation (DSA), an adaptive and dynamic framework to adapt novel and complex skills to Large Language Models (LLMs) For every skill, we utilize LLMs to generate both textbook-like data which contains detailed descriptions of skills for pre-training and exercise-like data which targets at explicitly utilizing the skills to solve problems for instruction-tuning. Experiments on large language models such as LLAMA and Mistral demonstrate the effectiveness of our proposed methods in adapting math reasoning skills and social study skills.
  arXiv  Detail & Related papers  (2024-12-26T22:04:23Z)
• "Give me the code" -- Log Analysis of First-Year CS Students' Interactions With GPT [0.0]
  This paper analyzes the prompts used by 69 freshmen undergraduate students to solve a certain programming problem within a project assignment. Despite using unsophisticated prompting techniques, our findings suggest that the majority of students successfully leveraged GPT. Half of the students demonstrated the ability to exercise judgment in selecting from multiple GPT-generated solutions.
  arXiv  Detail & Related papers  (2024-11-26T20:11:46Z)
• Code Interviews: Design and Evaluation of a More Authentic Assessment for Introductory Programming Assignments [15.295438618760164]
  We describe code interviews: a more authentic assessment method for take-home programming assignments. Code interviews pushed students to discuss their work, motivating more nuanced but sometimes repetitive insights. We conclude by discussing the different decisions about the design of code interviews with implications for student experience, academic integrity, and teaching workload.
  arXiv  Detail & Related papers  (2024-10-01T19:01:41Z)
• Personalization, Cognition, and Gamification-based Programming Language Learning: A State-of-the-Art Systematic Literature Review [0.13053649021965597]
  Programming courses in computing science are important because they are often the first introduction to computer programming for many students. The current teacher-lecturer model of learning commonly employed in university lecture halls often results in a lack of motivation and participation in learning. This paper provides insights into designing and implementing effective personalized gamification interventions in programming courses.
  arXiv  Detail & Related papers  (2023-09-05T05:14:23Z)
• Exploring the Use of ChatGPT as a Tool for Learning and Assessment in Undergraduate Computer Science Curriculum: Opportunities and Challenges [0.3553493344868413]
  This paper addresses the prospects and obstacles associated with utilizing ChatGPT as a tool for learning and assessment in undergraduate Computer Science curriculum. Group B students were given access to ChatGPT and were encouraged to use it to help solve the programming challenges. Results show that students using ChatGPT had an advantage in terms of earned scores, however there were inconsistencies and inaccuracies in the submitted code.
  arXiv  Detail & Related papers  (2023-04-16T21:04:52Z)
• Hierarchical Programmatic Reinforcement Learning via Learning to Compose Programs [58.94569213396991]
  We propose a hierarchical programmatic reinforcement learning framework to produce program policies. By learning to compose programs, our proposed framework can produce program policies that describe out-of-distributionally complex behaviors. The experimental results in the Karel domain show that our proposed framework outperforms baselines.
  arXiv  Detail & Related papers  (2023-01-30T14:50:46Z)
• Giving Feedback on Interactive Student Programs with Meta-Exploration [74.5597783609281]
  Developing interactive software, such as websites or games, is a particularly engaging way to learn computer science. Standard approaches require instructors to manually grade student-implemented interactive programs. Online platforms that serve millions, like Code.org, are unable to provide any feedback on assignments for implementing interactive programs.
  arXiv  Detail & Related papers  (2022-11-16T10:00:23Z)
• Competition-Level Code Generation with AlphaCode [74.87216298566942]
  We introduce AlphaCode, a system for code generation that can create novel solutions to problems that require deeper reasoning. In simulated evaluations on recent programming competitions on the Codeforces platform, AlphaCode achieved on average a ranking of top 54.3%.
  arXiv  Detail & Related papers  (2022-02-08T23:16:31Z)
• ProtoTransformer: A Meta-Learning Approach to Providing Student Feedback [54.142719510638614]
  In this paper, we frame the problem of providing feedback as few-shot classification. A meta-learner adapts to give feedback to student code on a new programming question from just a few examples by instructors. Our approach was successfully deployed to deliver feedback to 16,000 student exam-solutions in a programming course offered by a tier 1 university.
  arXiv  Detail & Related papers  (2021-07-23T22:41:28Z)
• On the Nature of Programming Exercises [0.0]
  It is essential to understand that the nature of a programming exercise is an important factor for the success and consistent learning. This paper explores different approaches on the creation of a programming exercise.
  arXiv  Detail & Related papers  (2020-06-25T15:22:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.