GenAI Voice Mode in Programming Education

• URL: http://arxiv.org/abs/2509.10596v1
• Date: Fri, 12 Sep 2025 15:25:08 GMT
• Title: GenAI Voice Mode in Programming Education
• Abstract summary: Real-time voice interfaces using multimodal Generative AI (GenAI) can potentially address the accessibility needs of novice programmers with disabilities.<n>This paper analyzes audio dialogues from nine 9th-grade students using a voice-enabled tutor in an authentic classroom setting while learning Python.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Real-time voice interfaces using multimodal Generative AI (GenAI) can potentially address the accessibility needs of novice programmers with disabilities (e.g., related to vision). Yet, little is known about how novices interact with GenAI tools and their feedback quality in the form of audio output. This paper analyzes audio dialogues from nine 9th-grade students using a voice-enabled tutor (powered by OpenAI's Realtime API) in an authentic classroom setting while learning Python. We examined the students' voice prompts and AI's responses (1210 messages) by using qualitative coding. We also gathered students' perceptions via the Partner Modeling Questionnaire. The GenAI Voice Tutor primarily offered feedback on mistakes and next steps, but its correctness was limited (71.4% correct out of 416 feedback outputs). Quality issues were observed, particularly when the AI attempted to utter programming code elements. Students used the GenAI voice tutor primarily for debugging. They perceived it as competent, only somewhat human-like, and flexible. The present study is the first to explore the interaction dynamics of real-time voice GenAI tutors and novice programmers, informing future educational tool design and potentially addressing accessibility needs of diverse learners.

Related papers

• Quick Build, Careful Check? Generative AI Use in Hackathons [83.0493885475068]
  We report an exploratory interview study with participants at an AI-themed hackathon in central Europe.<n>We found that participants used GenAI for purposes beyond coding, including learning unfamiliar topics, brainstorming, and preparing documentation.<n>We also found that all participants converged on an unwritten practice of checking GenAI output before using it, yet their ability to actually verify was constrained by time pressure and limited domain knowledge.
  arXiv  Detail & Related papers  (2026-07-31T08:59:18Z)
• CodeVaani: A Multilingual, Voice-Based Code Learning Assistant [13.984753319694763]
  We present CodeVaani, a multilingual speech-driven assistant for understanding code built into Bodhitree.<n>It is a voice-enabled assistant that helps learners explore programming concepts in their native languages.<n>In a study with 28 beginner programmers, CodeVaani achieved 75% response accuracy, with over 80% of participants rating the experience positively.
  arXiv  Detail & Related papers  (2025-10-08T17:15:59Z)
• Exploring Student Choice and the Use of Multimodal Generative AI in Programming Learning [6.678639464153991]
  We selected a commercially available multimodal GenAI platform for interaction.<n>We investigated student modality choices for GenAI tools when completing programming problems.<n>This work aims to spark continued exploration on understanding student interaction with multimodal GenAI in the context of Computer Science education.
  arXiv  Detail & Related papers  (2025-10-06T22:16:44Z)
• Students' Feedback Requests and Interactions with the SCRIPT Chatbot: Do They Get What They Ask For? [0.0]
  Building on prior research on Generative AI (GenAI) and related tools for programming education, we developed SCRIPT to support novice learners.<n>SCRIPT allows for open-ended interactions and structured guidance through predefined prompts.<n>We analyzed how students interacted with SCRIPT while solving programming tasks with a focus on their feedback preferences.
  arXiv  Detail & Related papers  (2025-07-23T06:56:26Z)
• Using Generative AI in Software Design Education: An Experience Report [0.6827423171182154]
  Students were required to use GenAI to help complete a team-based assignment.<n>Students identified numerous ways ChatGPT helped them in their design process.<n>We identified several key lessons for educators in how to deploy GenAI in a software design class effectively.
  arXiv  Detail & Related papers  (2025-06-26T18:40:16Z)
• That's Not the Feedback I Need! -- Student Engagement with GenAI Feedback in the Tutor Kai [0.0]
  We build a custom web application providing students with Python programming tasks, a code editor, GenAI feedback, and compiler feedback.<n>We investigate how much attention the generated feedback received from learners and to what extent the generated feedback is helpful (or not)<n>The findings indicate that GenAI feedback generally receives a lot of visual attention, with inexperienced students spending twice as much fixation time.
  arXiv  Detail & Related papers  (2025-06-25T13:43:23Z)
• FunAudioLLM: Voice Understanding and Generation Foundation Models for Natural Interaction Between Humans and LLMs [63.8261207950923]
  FunAudioLLM is a model family designed to enhance natural voice interactions between humans and large language models (LLMs) At its core are two innovative models: SenseVoice, which handles multilingual speech recognition, emotion recognition, and audio event detection; and CosyVoice, which facilitates natural speech generation with control over multiple languages, timbre, speaking style, and speaker identity. The models related to SenseVoice and CosyVoice have been open-sourced on Modelscope and Huggingface, along with the corresponding training, inference, and fine-tuning codes released on GitHub.
  arXiv  Detail & Related papers  (2024-07-04T16:49:02Z)
• Large Language Models Meet User Interfaces: The Case of Provisioning Feedback [6.626949691937476]
  We present a framework for incorporating GenAI into educational tools and demonstrate its application in our tool, Feedback Copilot. This work charts a course for the future of GenAI in education.
  arXiv  Detail & Related papers  (2024-04-17T05:05:05Z)
• Can Language Models Learn to Listen? [96.01685069483025]
  We present a framework for generating appropriate facial responses from a listener in dyadic social interactions based on the speaker's words. Our approach autoregressively predicts a response of a listener: a sequence of listener facial gestures, quantized using a VQ-VAE. We show that our generated listener motion is fluent and reflective of language semantics through quantitative metrics and a qualitative user study.
  arXiv  Detail & Related papers  (2023-08-21T17:59:02Z)
• Giving Feedback on Interactive Student Programs with Meta-Exploration [74.5597783609281]
  Developing interactive software, such as websites or games, is a particularly engaging way to learn computer science. Standard approaches require instructors to manually grade student-implemented interactive programs. Online platforms that serve millions, like Code.org, are unable to provide any feedback on assignments for implementing interactive programs.
  arXiv  Detail & Related papers  (2022-11-16T10:00:23Z)
• GenNI: Human-AI Collaboration for Data-Backed Text Generation [102.08127062293111]
  Table2Text systems generate textual output based on structured data utilizing machine learning. GenNI (Generation Negotiation Interface) is an interactive visual system for high-level human-AI collaboration in producing descriptive text.
  arXiv  Detail & Related papers  (2021-10-19T18:07:07Z)
• Explainable Active Learning (XAL): An Empirical Study of How Local Explanations Impact Annotator Experience [76.9910678786031]
  We propose a novel paradigm of explainable active learning (XAL), by introducing techniques from the recently surging field of explainable AI (XAI) into an Active Learning setting. Our study shows benefits of AI explanation as interfaces for machine teaching--supporting trust calibration and enabling rich forms of teaching feedback, and potential drawbacks--anchoring effect with the model judgment and cognitive workload.
  arXiv  Detail & Related papers  (2020-01-24T22:52:18Z)
• VoiceCoach: Interactive Evidence-based Training for Voice Modulation Skills in Public Speaking [55.366941476863644]
  The modulation of voice properties, such as pitch, volume, and speed, is crucial for delivering a successful public speech. We present VoiceCoach, an interactive evidence-based approach to facilitate the effective training of voice modulation skills.
  arXiv  Detail & Related papers  (2020-01-22T04:52:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.