Evaluating Contextually Personalized Programming Exercises Created with Generative AI
- URL: http://arxiv.org/abs/2407.11994v1
- Date: Tue, 11 Jun 2024 12:59:52 GMT
- Title: Evaluating Contextually Personalized Programming Exercises Created with Generative AI
- Authors: Evanfiya Logacheva, Arto Hellas, James Prather, Sami Sarsa, Juho Leinonen,
- Abstract summary: This article reports on a user study conducted in an elective programming course that included contextually personalized programming exercises created with GPT-4.
The results demonstrate that the quality of exercises generated with GPT-4 was generally high.
This suggests that AI-generated programming problems can be a worthwhile addition to introductory programming courses.
- Score: 4.046163999707179
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Programming skills are typically developed through completing various hands-on exercises. Such programming problems can be contextualized to students' interests and cultural backgrounds. Prior research in educational psychology has demonstrated that context personalization of exercises stimulates learners' situational interests and positively affects their engagement. However, creating a varied and comprehensive set of programming exercises for students to practice on is a time-consuming and laborious task for computer science educators. Previous studies have shown that large language models can generate conceptually and contextually relevant programming exercises. Thus, they offer a possibility to automatically produce personalized programming problems to fit students' interests and needs. This article reports on a user study conducted in an elective introductory programming course that included contextually personalized programming exercises created with GPT-4. The quality of the exercises was evaluated by both the students and the authors. Additionally, this work investigated student attitudes towards the created exercises and their engagement with the system. The results demonstrate that the quality of exercises generated with GPT-4 was generally high. What is more, the course participants found them engaging and useful. This suggests that AI-generated programming problems can be a worthwhile addition to introductory programming courses, as they provide students with a practically unlimited pool of practice material tailored to their personal interests and educational needs.
Related papers
- Code Interviews: Design and Evaluation of a More Authentic Assessment for Introductory Programming Assignments [15.295438618760164]
We describe code interviews as a more authentic assessment method for take-home programming assignments.
Code interviews pushed students to discuss their work, motivating more nuanced but sometimes repetitive insights.
We conclude by discussing the different decisions about the design of code interviews with implications for student experience, academic integrity, and teaching workload.
arXiv Detail & Related papers (2024-10-01T19:01:41Z) - Beyond the Hype: A Cautionary Tale of ChatGPT in the Programming Classroom [0.0]
The paper provides insights for academics who teach programming to create more challenging exercises and how to engage responsibly in the use of ChatGPT to promote classroom integrity.
We analyzed the various practical programming examples from past IS exercises and compared those with memos created by tutors and lecturers in a university setting.
arXiv Detail & Related papers (2024-06-16T23:52:37Z) - Personalization, Cognition, and Gamification-based Programming Language
Learning: A State-of-the-Art Systematic Literature Review [0.13053649021965597]
Programming courses in computing science are important because they are often the first introduction to computer programming for many students.
The current teacher-lecturer model of learning commonly employed in university lecture halls often results in a lack of motivation and participation in learning.
This paper provides insights into designing and implementing effective personalized gamification interventions in programming courses.
arXiv Detail & Related papers (2023-09-05T05:14:23Z) - AGI: Artificial General Intelligence for Education [41.45039606933712]
This position paper reviews artificial general intelligence (AGI)'s key concepts, capabilities, scope, and potential within future education.
It highlights that AGI can significantly improve intelligent tutoring systems, educational assessment, and evaluation procedures.
The paper emphasizes that AGI's capabilities extend to understanding human emotions and social interactions.
arXiv Detail & Related papers (2023-04-24T22:31:59Z) - Exploring the Use of ChatGPT as a Tool for Learning and Assessment in
Undergraduate Computer Science Curriculum: Opportunities and Challenges [0.3553493344868413]
This paper addresses the prospects and obstacles associated with utilizing ChatGPT as a tool for learning and assessment in undergraduate Computer Science curriculum.
Group B students were given access to ChatGPT and were encouraged to use it to help solve the programming challenges.
Results show that students using ChatGPT had an advantage in terms of earned scores, however there were inconsistencies and inaccuracies in the submitted code.
arXiv Detail & Related papers (2023-04-16T21:04:52Z) - Hierarchical Programmatic Reinforcement Learning via Learning to Compose
Programs [58.94569213396991]
We propose a hierarchical programmatic reinforcement learning framework to produce program policies.
By learning to compose programs, our proposed framework can produce program policies that describe out-of-distributionally complex behaviors.
The experimental results in the Karel domain show that our proposed framework outperforms baselines.
arXiv Detail & Related papers (2023-01-30T14:50:46Z) - A Survey on Natural Language Processing for Programming [42.850340313115765]
Natural language processing for programming aims to use NLP techniques to assist programming.
Structure-based representation and functionality-oriented algorithm are at the heart of program understanding and generation.
arXiv Detail & Related papers (2022-12-12T08:51:30Z) - Giving Feedback on Interactive Student Programs with Meta-Exploration [74.5597783609281]
Developing interactive software, such as websites or games, is a particularly engaging way to learn computer science.
Standard approaches require instructors to manually grade student-implemented interactive programs.
Online platforms that serve millions, like Code.org, are unable to provide any feedback on assignments for implementing interactive programs.
arXiv Detail & Related papers (2022-11-16T10:00:23Z) - Competition-Level Code Generation with AlphaCode [74.87216298566942]
We introduce AlphaCode, a system for code generation that can create novel solutions to problems that require deeper reasoning.
In simulated evaluations on recent programming competitions on the Codeforces platform, AlphaCode achieved on average a ranking of top 54.3%.
arXiv Detail & Related papers (2022-02-08T23:16:31Z) - ProtoTransformer: A Meta-Learning Approach to Providing Student Feedback [54.142719510638614]
In this paper, we frame the problem of providing feedback as few-shot classification.
A meta-learner adapts to give feedback to student code on a new programming question from just a few examples by instructors.
Our approach was successfully deployed to deliver feedback to 16,000 student exam-solutions in a programming course offered by a tier 1 university.
arXiv Detail & Related papers (2021-07-23T22:41:28Z) - On the Nature of Programming Exercises [0.0]
It is essential to understand that the nature of a programming exercise is an important factor for the success and consistent learning.
This paper explores different approaches on the creation of a programming exercise.
arXiv Detail & Related papers (2020-06-25T15:22:26Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.