Scaling CS1 Support with Compiler-Integrated Conversational AI

• URL: http://arxiv.org/abs/2408.02378v1
• Date: Mon, 22 Jul 2024 10:53:55 GMT
• Title: Scaling CS1 Support with Compiler-Integrated Conversational AI
• Authors: Jake Renzella, Alexandra Vassar, Lorenzo Lee Solano, Andrew Taylor,
• Abstract summary: DCC Sidekick is a web-based AI tool that enhances an existing LLM-powered C/C++ compiler by generating educational programming error explanations. We analyse usage data from a large Australian CS1 course, where 959 students engaged in 11,222 DCC Sidekick sessions, resulting in 17,982 error explanations over seven weeks.
• Score: 43.77796322595561
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper introduces DCC Sidekick, a web-based conversational AI tool that enhances an existing LLM-powered C/C++ compiler by generating educational programming error explanations. The tool seamlessly combines code display, compile- and run-time error messages, and stack frame read-outs alongside an AI interface, leveraging compiler error context for improved explanations. We analyse usage data from a large Australian CS1 course, where 959 students engaged in 11,222 DCC Sidekick sessions, resulting in 17,982 error explanations over seven weeks. Notably, over 50% of interactions occurred outside business hours, underscoring the tool's value as an always-available resource. Our findings reveal strong adoption of AI-assisted debugging tools, demonstrating their scalability in supporting extensive CS1 courses. We provide implementation insights and recommendations for educators seeking to incorporate AI tools with appropriate pedagogical safeguards.

Related papers

• LLM-aided explanations of EDA synthesis errors [10.665347817363623]
  Large Language Models (LLMs) have demonstrated text comprehension and question-answering capabilities. We generate 936 error message explanations using three OpenAI LLMs over 21 different buggy code samples. These are then graded for relevance and correctness, and we find that in approximately 71% of cases the LLMs give correct & complete explanations suitable for novice learners.
  arXiv  Detail & Related papers  (2024-04-07T07:12:16Z)
• Enhancing Programming Error Messages in Real Time with Generative AI [0.0]
  We implement feedback from ChatGPT for all programs submitted to our automated assessment tool, Athene. Our results indicate that adding generative AI to an automated assessment tool does not necessarily make it better.
  arXiv  Detail & Related papers  (2024-02-12T21:32:05Z)
• Developer Experiences with a Contextualized AI Coding Assistant: Usability, Expectations, and Outcomes [11.520721038793285]
  This study focuses on the initial experiences of 62 participants who used a contextualized coding AI assistant -- named StackSpot AI -- in a controlled setting. Assistants' use resulted in significant time savings, easier access to documentation, and the generation of accurate codes for internal APIs. challenges associated with the knowledge sources necessary to make the coding assistant access more contextual information as well as variable responses and limitations in handling complex codes were observed.
  arXiv  Detail & Related papers  (2023-11-30T10:52:28Z)
• How to Build an AI Tutor that Can Adapt to Any Course and Provide Accurate Answers Using Large Language Model and Retrieval-Augmented Generation [0.0]
  The OpenAI Assistants API allows AI Tutor to easily embed, store, retrieve, and manage files and chat history. The AI Tutor prototype demonstrates its ability to generate relevant, accurate answers with source citations.
  arXiv  Detail & Related papers  (2023-11-29T15:02:46Z)
• Dcc --help: Generating Context-Aware Compiler Error Explanations with Large Language Models [53.04357141450459]
  dcc --help was deployed to our CS1 and CS2 courses, with 2,565 students using the tool over 64,000 times in ten weeks. We found that the LLM-generated explanations were conceptually accurate in 90% of compile-time and 75% of run-time cases, but often disregarded the instruction not to provide solutions in code.
  arXiv  Detail & Related papers  (2023-08-23T02:36:19Z)
• Giving Feedback on Interactive Student Programs with Meta-Exploration [74.5597783609281]
  Developing interactive software, such as websites or games, is a particularly engaging way to learn computer science. Standard approaches require instructors to manually grade student-implemented interactive programs. Online platforms that serve millions, like Code.org, are unable to provide any feedback on assignments for implementing interactive programs.
  arXiv  Detail & Related papers  (2022-11-16T10:00:23Z)
• AI Explainability 360: Impact and Design [120.95633114160688]
  In 2019, we created AI Explainability 360 (Arya et al. 2020), an open source software toolkit featuring ten diverse and state-of-the-art explainability methods. This paper examines the impact of the toolkit with several case studies, statistics, and community feedback. The paper also describes the flexible design of the toolkit, examples of its use, and the significant educational material and documentation available to its users.
  arXiv  Detail & Related papers  (2021-09-24T19:17:09Z)
• ProtoTransformer: A Meta-Learning Approach to Providing Student Feedback [54.142719510638614]
  In this paper, we frame the problem of providing feedback as few-shot classification. A meta-learner adapts to give feedback to student code on a new programming question from just a few examples by instructors. Our approach was successfully deployed to deliver feedback to 16,000 student exam-solutions in a programming course offered by a tier 1 university.
  arXiv  Detail & Related papers  (2021-07-23T22:41:28Z)
• OpenHoldem: An Open Toolkit for Large-Scale Imperfect-Information Game Research [82.09426894653237]
  OpenHoldem is an integrated toolkit for large-scale imperfect-information game research using NLTH. OpenHoldem makes three main contributions to this research direction: 1) a standardized evaluation protocol for thoroughly evaluating different NLTH AIs, 2) three publicly available strong baselines for NLTH AI, and 3) an online testing platform with easy-to-use APIs for public NLTH AI evaluation.
  arXiv  Detail & Related papers  (2020-12-11T07:24:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.