Evaluating the AI-Lab Intervention: Impact on Student Perception and Use of Generative AI in Early Undergraduate Computer Science Courses

• URL: http://arxiv.org/abs/2505.00100v1
• Date: Wed, 30 Apr 2025 18:12:42 GMT
• Title: Evaluating the AI-Lab Intervention: Impact on Student Perception and Use of Generative AI in Early Undergraduate Computer Science Courses
• Authors: Ethan Dickey, Andres Bejarano, Rhianna Kuperus, Bárbara Fagundes,
• Abstract summary: Generative AI (GenAI) is rapidly entering computer science education.<n>Concerns about overreliance coexist with a gap in research on structured scaffolding to guide tool use in formal courses.<n>This study examines the impact of a dedicated "AI-Lab" intervention on undergraduate students.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generative AI (GenAI) is rapidly entering computer science education, yet its effects on student learning, skill development, and perceptions remain underexplored. Concerns about overreliance coexist with a gap in research on structured scaffolding to guide tool use in formal courses. This study examines the impact of a dedicated "AI-Lab" intervention -- emphasizing guided scaffolding and mindful engagement -- on undergraduate students in Data Structures and Algorithms, Competitive Programming, and first-year engineering courses at Purdue University. Over three semesters, we integrated AI-Lab modules into four mandatory and elective courses, yielding 831 matched pre- and post-intervention survey responses, alongside focus group discussions. Employing a mixed-methods approach, we analyzed quantitative shifts in usage patterns and attitudes as well as qualitative narratives of student experiences. While the overall frequency of GenAI usage for homework or programming projects remained largely stable, we observed large effect sizes in comfort and openness across conceptual, debugging, and homework problems. Notably, usage patterns for debugging also shifted statistically significantly, reflecting students' more mindful and deliberate approach. Focus group discussions corroborated these results, suggesting that the intervention "bridged the gap" between naive GenAI usage and more nuanced, reflective integration of AI tools into coursework, ultimately heightening students' awareness of their own skill development. These findings suggest that structured, scaffolded interventions can enable students to harness GenAI's benefits without undermining essential competencies. We offer evidence-based recommendations for educators seeking to integrate GenAI responsibly into computing curricula and identify avenues for future research on GenAI-supported pedagogy.

Related papers

• Student's Use of Generative AI as a Support Tool in an Advanced Web Development Course [0.5371337604556311]
  We analyze the use of GenAI as a support tool for learning, creativity, and productivity in a web development course for undergraduate students.<n>Students used GenAI on different tasks with a reported increase in learning and productivity.
  arXiv  Detail & Related papers  (2025-03-19T20:34:21Z)
• Computational Safety for Generative AI: A Signal Processing Perspective [65.268245109828]
  computational safety is a mathematical framework that enables the quantitative assessment, formulation, and study of safety challenges in GenAI.<n>We show how sensitivity analysis and loss landscape analysis can be used to detect malicious prompts with jailbreak attempts.<n>We discuss key open research challenges, opportunities, and the essential role of signal processing in computational AI safety.
  arXiv  Detail & Related papers  (2025-02-18T02:26:50Z)
• Navigating Ethical Challenges in Generative AI-Enhanced Research: The ETHICAL Framework for Responsible Generative AI Use [0.0]
  The rapid adoption of generative artificial intelligence (GenAI) in research presents both opportunities and ethical challenges.<n>This paper develops the ETHICAL framework, which is a practical guide for responsible GenAI use in research.
  arXiv  Detail & Related papers  (2024-12-11T05:49:11Z)
• Early Adoption of Generative Artificial Intelligence in Computing Education: Emergent Student Use Cases and Perspectives in 2023 [38.83649319653387]
  There is limited prior research on computing students' use and perceptions of GenAI. We surveyed all computer science majors in a small engineering-focused R1 university. We discuss the impact of our findings on the emerging conversation around GenAI and education.
  arXiv  Detail & Related papers  (2024-11-17T20:17:47Z)
• LLMs Integration in Software Engineering Team Projects: Roles, Impact, and a Pedagogical Design Space for AI Tools in Computing Education [7.058964784190549]
  This work takes a pedagogical lens to explore the implications of generative AI (GenAI) models and tools, such as ChatGPT and GitHub Copilot. Our results address a particular gap in understanding the role and implications of GenAI on teamwork, team-efficacy, and team dynamics.
  arXiv  Detail & Related papers  (2024-10-30T14:43:33Z)
• Hey GPT, Can You be More Racist? Analysis from Crowdsourced Attempts to Elicit Biased Content from Generative AI [41.96102438774773]
  This work presents the findings from a university-level competition, which challenged participants to design prompts for eliciting biased outputs from GenAI tools. We quantitatively and qualitatively analyze the competition submissions and identify a diverse set of biases in GenAI and strategies employed by participants to induce bias in GenAI.
  arXiv  Detail & Related papers  (2024-10-20T18:44:45Z)
• Enhancing Feature Selection and Interpretability in AI Regression Tasks Through Feature Attribution [38.53065398127086]
  This study investigates the potential of feature attribution methods to filter out uninformative features in input data for regression problems. We introduce a feature selection pipeline that combines Integrated Gradients with k-means clustering to select an optimal set of variables from the initial data space. To validate the effectiveness of this approach, we apply it to a real-world industrial problem - blade vibration analysis in the development process of turbo machinery.
  arXiv  Detail & Related papers  (2024-09-25T09:50:51Z)
• BoilerTAI: A Platform for Enhancing Instruction Using Generative AI in Educational Forums [0.0]
  This paper describes a practical, scalable platform that seamlessly integrates Generative AI (GenAI) with online educational forums. The platform empowers instructional staff to efficiently manage, refine, and approve responses by facilitating interaction between student posts and a Large Language Model (LLM)
  arXiv  Detail & Related papers  (2024-09-20T04:00:30Z)
• Model-based Maintenance and Evolution with GenAI: A Look into the Future [47.93555901495955]
  We argue that Generative Artificial Intelligence (GenAI) can be used as a means to address the limitations of Model-Based Engineering (MBM&E) We propose that GenAI can be used in MBM&E for: reducing engineers' learning curve, maximizing efficiency with recommendations, or serving as a reasoning tool to understand domain problems.
  arXiv  Detail & Related papers  (2024-07-09T23:13:26Z)
• OpenHEXAI: An Open-Source Framework for Human-Centered Evaluation of Explainable Machine Learning [43.87507227859493]
  This paper presents OpenHEXAI, an open-source framework for human-centered evaluation of XAI methods. OpenHEAXI is the first large-scale infrastructural effort to facilitate human-centered benchmarks of XAI methods.
  arXiv  Detail & Related papers  (2024-02-20T22:17:59Z)
• Toward enriched Cognitive Learning with XAI [44.99833362998488]
  We introduce an intelligent system (CL-XAI) for Cognitive Learning which is supported by artificial intelligence (AI) tools. The use of CL-XAI is illustrated with a game-inspired virtual use case where learners tackle problems to enhance problem-solving skills.
  arXiv  Detail & Related papers  (2023-12-19T16:13:47Z)
• Innovating Computer Programming Pedagogy: The AI-Lab Framework for Generative AI Adoption [0.0]
  We introduce "AI-Lab," a framework for guiding students in effectively leveraging GenAI within core programming courses. By identifying and rectifying GenAI's errors, students enrich their learning process. For educators, AI-Lab provides mechanisms to explore students' perceptions of GenAI's role in their learning experience.
  arXiv  Detail & Related papers  (2023-08-23T17:20:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.