Examining the Usage of Generative AI Models in Student Learning Activities for Software Programming

• URL: http://arxiv.org/abs/2511.13271v1
• Date: Mon, 17 Nov 2025 11:42:24 GMT
• Title: Examining the Usage of Generative AI Models in Student Learning Activities for Software Programming
• Authors: Rufeng Chen, Shuaishuai Jiang, Jiyun Shen, AJung Moon, Lili Wei,
• Abstract summary: This study investigates how GenAI assistance compares to conventional online resources in supporting knowledge gains across different proficiency levels.<n>We conducted a controlled user experiment with 24 undergraduate students of two different levels of programming experience.<n>Our findings reveal that generating complete solutions with GenAI significantly improves task performance, especially for beginners, but does not consistently result in knowledge gains.
• Score: 3.2383459424229835
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The rise of Generative AI (GenAI) tools like ChatGPT has created new opportunities and challenges for computing education. Existing research has primarily focused on GenAI's ability to complete educational tasks and its impact on student performance, often overlooking its effects on knowledge gains. In this study, we investigate how GenAI assistance compares to conventional online resources in supporting knowledge gains across different proficiency levels. We conducted a controlled user experiment with 24 undergraduate students of two different levels of programming experience (beginner, intermediate) to examine how students interact with ChatGPT while solving programming tasks. We analyzed task performance, conceptual understanding, and interaction behaviors. Our findings reveal that generating complete solutions with GenAI significantly improves task performance, especially for beginners, but does not consistently result in knowledge gains. Importantly, usage strategies differ by experience: beginners tend to rely heavily on GenAI toward task completion often without knowledge gain in the process, while intermediates adopt more selective approaches. We find that both over-reliance and minimal use result in weaker knowledge gains overall. Based on our results, we call on students and educators to adopt GenAI as a learning rather than a problem solving tool. Our study highlights the urgent need for guidance when integrating GenAI into programming education to foster deeper understanding.

Related papers

• On the Role and Impact of GenAI Tools in Software Engineering Education [2.867517731896504]
  generative AI (GenAI) tools like ChatGPT and GitHub Copilot have transformed how software is learned and written.<n>In software engineering (SE) education, these tools offer new opportunities for support, but also raise concerns about over-reliance, ethical use, and impacts on learning.<n>This study investigates how undergraduate SE students use GenAI tools, focusing on the benefits, challenges, ethical concerns, and instructional expectations that shape their experiences.
  arXiv  Detail & Related papers  (2025-12-03T20:51:16Z)
• Generative AI in Training and Coaching: Redefining the Design Process of Learning Materials [44.99833362998488]
  We explore how AI integrates into the design process of learning materials, assessing its impact on efficiency, pedagogical quality, and the evolving role of human trainers and coaches.<n>Through qualitative interviews with professionals in education and corporate training, we identify the following key topics.<n>We derive how tools based on GenAI can successfully be implemented for trainers and coaches on an individual, organizational, systemic, and strategic level.
  arXiv  Detail & Related papers  (2025-08-06T03:42:43Z)
• Encouraging Students' Responsible Use of GenAI in Software Engineering Education: A Causal Model and Two Institutional Applications [0.4306143768014156]
  As generative AI (GenAI) tools become pervasive in education, concerns are rising about students using them to complete rather than learn.<n>This paper proposes and empirically applies a causal model to help educators scaffold responsible GenAI use in Software Engineering (SE) education.
  arXiv  Detail & Related papers  (2025-05-31T19:27:40Z)
• From Recall to Reasoning: Automated Question Generation for Deeper Math Learning through Large Language Models [44.99833362998488]
  We investigated the first steps for optimizing content creation for advanced math.<n>We looked at the ability of GenAI to produce high-quality practice problems that are relevant to the course content.
  arXiv  Detail & Related papers  (2025-05-17T08:30:10Z)
• Evaluating the AI-Lab Intervention: Impact on Student Perception and Use of Generative AI in Early Undergraduate Computer Science Courses [0.0]
  Generative AI (GenAI) is rapidly entering computer science education.<n>Concerns about overreliance coexist with a gap in research on structured scaffolding to guide tool use in formal courses.<n>This study examines the impact of a dedicated "AI-Lab" intervention on undergraduate students.
  arXiv  Detail & Related papers  (2025-04-30T18:12:42Z)
• Student's Use of Generative AI as a Support Tool in an Advanced Web Development Course [0.5371337604556311]
  We analyze the use of GenAI as a support tool for learning, creativity, and productivity in a web development course for undergraduate students.<n>Students used GenAI on different tasks with a reported increase in learning and productivity.
  arXiv  Detail & Related papers  (2025-03-19T20:34:21Z)
• Model-based Maintenance and Evolution with GenAI: A Look into the Future [47.93555901495955]
  We argue that Generative Artificial Intelligence (GenAI) can be used as a means to address the limitations of Model-Based Engineering (MBM&E) We propose that GenAI can be used in MBM&E for: reducing engineers' learning curve, maximizing efficiency with recommendations, or serving as a reasoning tool to understand domain problems.
  arXiv  Detail & Related papers  (2024-07-09T23:13:26Z)
• The Widening Gap: The Benefits and Harms of Generative AI for Novice Programmers [1.995977018536036]
  Novice programmers often struggle through programming problem solving due to a lack of metacognitive awareness and strategies. Many novices are now programming with generative AI (GenAI) Our findings show an unfortunate divide in the use of GenAI tools between students who accelerated and students who struggled.
  arXiv  Detail & Related papers  (2024-05-28T01:48:28Z)
• Genetic Auto-prompt Learning for Pre-trained Code Intelligence Language Models [54.58108387797138]
  We investigate the effectiveness of prompt learning in code intelligence tasks. Existing automatic prompt design methods are very limited to code intelligence tasks. We propose Genetic Auto Prompt (GenAP) which utilizes an elaborate genetic algorithm to automatically design prompts.
  arXiv  Detail & Related papers  (2024-03-20T13:37:00Z)
• Toward enriched Cognitive Learning with XAI [44.99833362998488]
  We introduce an intelligent system (CL-XAI) for Cognitive Learning which is supported by artificial intelligence (AI) tools. The use of CL-XAI is illustrated with a game-inspired virtual use case where learners tackle problems to enhance problem-solving skills.
  arXiv  Detail & Related papers  (2023-12-19T16:13:47Z)
• Innovating Computer Programming Pedagogy: The AI-Lab Framework for Generative AI Adoption [0.0]
  We introduce "AI-Lab," a framework for guiding students in effectively leveraging GenAI within core programming courses. By identifying and rectifying GenAI's errors, students enrich their learning process. For educators, AI-Lab provides mechanisms to explore students' perceptions of GenAI's role in their learning experience.
  arXiv  Detail & Related papers  (2023-08-23T17:20:37Z)
• Automated Graph Self-supervised Learning via Multi-teacher Knowledge Distillation [43.903582264697974]
  This paper studies the problem of how to automatically, adaptively, and dynamically learn instance-level self-supervised learning strategies for each node. We propose a novel multi-teacher knowledge distillation framework for Automated Graph Self-Supervised Learning (AGSSL) Experiments on eight datasets show that AGSSL can benefit from multiple pretext tasks, outperforming the corresponding individual tasks.
  arXiv  Detail & Related papers  (2022-10-05T08:39:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.