How Do Programming Students Use Generative AI?

• URL: http://arxiv.org/abs/2501.10091v1
• Date: Fri, 17 Jan 2025 10:25:41 GMT
• Title: How Do Programming Students Use Generative AI?
• Authors: Christian Rahe, Walid Maalej,
• Abstract summary: We studied how programming students actually use generative AI tools like ChatGPT. We observed two prevalent usage strategies: to seek knowledge about general concepts and to directly generate solutions. Our findings indicate that concerns about potential decrease in programmers' agency and productivity with Generative AI are justified.
• Score: 7.863638253070439
• License:
• Abstract: Programming students have a widespread access to powerful Generative AI tools like ChatGPT. While this can help understand the learning material and assist with exercises, educators are voicing more and more concerns about an over-reliance on generated outputs and lack of critical thinking skills. It is thus important to understand how students actually use generative AI and what impact this could have on their learning behavior. To this end, we conducted a study including an exploratory experiment with 37 programming students, giving them monitored access to ChatGPT while solving a code understanding and improving exercise. While only 23 of the students actually opted to use the chatbot, the majority of those eventually prompted it to simply generate a full solution. We observed two prevalent usage strategies: to seek knowledge about general concepts and to directly generate solutions. Instead of using the bot to comprehend the code and their own mistakes, students often got trapped in a vicious cycle of submitting wrong generated code and then asking the bot for a fix. Those who self-reported using generative AI regularly were more likely to prompt the bot to generate a solution. Our findings indicate that concerns about potential decrease in programmers' agency and productivity with Generative AI are justified. We discuss how researchers and educators can respond to the potential risk of students uncritically over-relying on generative AI. We also discuss potential modifications to our study design for large-scale replications.

Related papers

• Encouraging Responsible Use of Generative AI in Education: A Reward-Based Learning Approach [0.7889270818022226]
  This research introduces an innovative mathematical learning approach that integrates generative AI to cultivate a structured learning rather than quick solution. The goal is to transition students from seeking quick fixes to engaging actively in a comprehensive learning experience.
  arXiv  Detail & Related papers  (2024-06-26T14:27:24Z)
• The Widening Gap: The Benefits and Harms of Generative AI for Novice Programmers [1.995977018536036]
  Novice programmers often struggle through programming problem solving due to a lack of metacognitive awareness and strategies. Many novices are now programming with generative AI (GenAI) Our findings show an unfortunate divide in the use of GenAI tools between students who accelerated and students who struggled.
  arXiv  Detail & Related papers  (2024-05-28T01:48:28Z)
• Genetic Auto-prompt Learning for Pre-trained Code Intelligence Language Models [54.58108387797138]
  We investigate the effectiveness of prompt learning in code intelligence tasks. Existing automatic prompt design methods are very limited to code intelligence tasks. We propose Genetic Auto Prompt (GenAP) which utilizes an elaborate genetic algorithm to automatically design prompts.
  arXiv  Detail & Related papers  (2024-03-20T13:37:00Z)
• Enhancing Programming Error Messages in Real Time with Generative AI [0.0]
  We implement feedback from ChatGPT for all programs submitted to our automated assessment tool, Athene. Our results indicate that adding generative AI to an automated assessment tool does not necessarily make it better.
  arXiv  Detail & Related papers  (2024-02-12T21:32:05Z)
• Identifying and Mitigating the Security Risks of Generative AI [179.2384121957896]
  This paper reports the findings of a workshop held at Google on the dual-use dilemma posed by GenAI. GenAI can be used just as well by attackers to generate new attacks and increase the velocity and efficacy of existing attacks. We discuss short-term and long-term goals for the community on this topic.
  arXiv  Detail & Related papers  (2023-08-28T18:51:09Z)
• Self-Improving Robots: End-to-End Autonomous Visuomotor Reinforcement Learning [54.636562516974884]
  In imitation and reinforcement learning, the cost of human supervision limits the amount of data that robots can be trained on. In this work, we propose MEDAL++, a novel design for self-improving robotic systems. The robot autonomously practices the task by learning to both do and undo the task, simultaneously inferring the reward function from the demonstrations.
  arXiv  Detail & Related papers  (2023-03-02T18:51:38Z)
• How Generative AI models such as ChatGPT can be (Mis)Used in SPC Practice, Education, and Research? An Exploratory Study [2.0841728192954663]
  Generative Artificial Intelligence (AI) models have the potential to revolutionize Statistical Process Control (SPC) practice, learning, and research. These tools are in the early stages of development and can be easily misused or misunderstood. We explore ChatGPT's ability to provide code, explain basic concepts, and create knowledge related to SPC practice, learning, and research.
  arXiv  Detail & Related papers  (2023-02-17T15:48:37Z)
• Smart tutor to provide feedback in programming courses [0.0]
  We present an AI based intelligent tutor that answers students programming questions. The tool has been tested by university students at the URJC along a whole course.
  arXiv  Detail & Related papers  (2023-01-24T11:00:06Z)
• Seamful XAI: Operationalizing Seamful Design in Explainable AI [59.89011292395202]
  Mistakes in AI systems are inevitable, arising from both technical limitations and sociotechnical gaps. We propose that seamful design can foster AI explainability by revealing sociotechnical and infrastructural mismatches. We explore this process with 43 AI practitioners and real end-users.
  arXiv  Detail & Related papers  (2022-11-12T21:54:05Z)
• The MineRL BASALT Competition on Learning from Human Feedback [58.17897225617566]
  The MineRL BASALT competition aims to spur forward research on this important class of techniques. We design a suite of four tasks in Minecraft for which we expect it will be hard to write down hardcoded reward functions. We provide a dataset of human demonstrations on each of the four tasks, as well as an imitation learning baseline.
  arXiv  Detail & Related papers  (2021-07-05T12:18:17Z)
• Explainable Active Learning (XAL): An Empirical Study of How Local Explanations Impact Annotator Experience [76.9910678786031]
  We propose a novel paradigm of explainable active learning (XAL), by introducing techniques from the recently surging field of explainable AI (XAI) into an Active Learning setting. Our study shows benefits of AI explanation as interfaces for machine teaching--supporting trust calibration and enabling rich forms of teaching feedback, and potential drawbacks--anchoring effect with the model judgment and cognitive workload.
  arXiv  Detail & Related papers  (2020-01-24T22:52:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.