Exploring Student Choice and the Use of Multimodal Generative AI in Programming Learning

• URL: http://arxiv.org/abs/2510.05417v1
• Date: Mon, 06 Oct 2025 22:16:44 GMT
• Title: Exploring Student Choice and the Use of Multimodal Generative AI in Programming Learning
• Authors: Xinying Hou, Ruiwei Xiao, Runlong Ye, Michael Liut, John Stamper,
• Abstract summary: We selected a commercially available multimodal GenAI platform for interaction.<n>We investigated student modality choices for GenAI tools when completing programming problems.<n>This work aims to spark continued exploration on understanding student interaction with multimodal GenAI in the context of Computer Science education.
• Score: 6.678639464153991
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The broad adoption of Generative AI (GenAI) is impacting Computer Science education, and recent studies found its benefits and potential concerns when students use it for programming learning. However, most existing explorations focus on GenAI tools that primarily support text-to-text interaction. With recent developments, GenAI applications have begun supporting multiple modes of communication, known as multimodality. In this work, we explored how undergraduate programming novices choose and work with multimodal GenAI tools, and their criteria for choices. We selected a commercially available multimodal GenAI platform for interaction, as it supports multiple input and output modalities, including text, audio, image upload, and real-time screen-sharing. Through 16 think-aloud sessions that combined participant observation with follow-up semi-structured interviews, we investigated student modality choices for GenAI tools when completing programming problems and the underlying criteria for modality selections. With multimodal communication emerging as the future of AI in education, this work aims to spark continued exploration on understanding student interaction with multimodal GenAI in the context of CS education.

Related papers

• Examining the Usage of Generative AI Models in Student Learning Activities for Software Programming [3.2383459424229835]
  This study investigates how GenAI assistance compares to conventional online resources in supporting knowledge gains across different proficiency levels.<n>We conducted a controlled user experiment with 24 undergraduate students of two different levels of programming experience.<n>Our findings reveal that generating complete solutions with GenAI significantly improves task performance, especially for beginners, but does not consistently result in knowledge gains.
  arXiv  Detail & Related papers  (2025-11-17T11:42:24Z)
• From Recall to Reasoning: Automated Question Generation for Deeper Math Learning through Large Language Models [44.99833362998488]
  We investigated the first steps for optimizing content creation for advanced math.<n>We looked at the ability of GenAI to produce high-quality practice problems that are relevant to the course content.
  arXiv  Detail & Related papers  (2025-05-17T08:30:10Z)
• Evaluating the AI-Lab Intervention: Impact on Student Perception and Use of Generative AI in Early Undergraduate Computer Science Courses [0.0]
  Generative AI (GenAI) is rapidly entering computer science education.<n>Concerns about overreliance coexist with a gap in research on structured scaffolding to guide tool use in formal courses.<n>This study examines the impact of a dedicated "AI-Lab" intervention on undergraduate students.
  arXiv  Detail & Related papers  (2025-04-30T18:12:42Z)
• Student's Use of Generative AI as a Support Tool in an Advanced Web Development Course [0.5371337604556311]
  We analyze the use of GenAI as a support tool for learning, creativity, and productivity in a web development course for undergraduate students.<n>Students used GenAI on different tasks with a reported increase in learning and productivity.
  arXiv  Detail & Related papers  (2025-03-19T20:34:21Z)
• Beyond the Hype: A Comprehensive Review of Current Trends in Generative AI Research, Teaching Practices, and Tools [4.352985782794601]
  In 2024, new research started to emerge on the effects of GenAI usage in the computing classroom.<n>New data involve the use of GenAI to support classroom instruction at scale and to teach students how to code with GenAI.<n>New class of tools is emerging that can provide personalized feedback to students on their programming assignments or teach both programming and prompting skills at the same time.
  arXiv  Detail & Related papers  (2024-12-19T11:01:11Z)
• Students' Perceptions and Use of Generative AI Tools for Programming Across Different Computing Courses [1.7811951520198]
  Investigation of students' perceptions and opinions on the use of generative artificial intelligence (GenAI) in education is a topic gaining much interest. How students perceive and use GenAI tools can potentially depend on many factors, including their background knowledge. We conducted three surveys among students of all computing programs of a large European research university.
  arXiv  Detail & Related papers  (2024-10-09T13:24:06Z)
• The Roles of Generative Artificial Intelligence in Internet of Electric Vehicles [65.14115295214636]
  We specifically consider Internet of electric vehicles (IoEV) and we categorize GenAI for IoEV into four different layers. We introduce various GenAI techniques used in each layer of IoEV applications. Public datasets available for training the GenAI models are summarized.
  arXiv  Detail & Related papers  (2024-09-24T05:12:10Z)
• LLMs Meet Multimodal Generation and Editing: A Survey [89.76691959033323]
  This survey elaborates on multimodal generation and editing across various domains, comprising image, video, 3D, and audio. We summarize the notable advancements with milestone works in these fields and categorize these studies into LLM-based and CLIP/T5-based methods. We dig into tool-augmented multimodal agents that can leverage existing generative models for human-computer interaction.
  arXiv  Detail & Related papers  (2024-05-29T17:59:20Z)
• An Interactive Agent Foundation Model [49.77861810045509]
  We propose an Interactive Agent Foundation Model that uses a novel multi-task agent training paradigm for training AI agents. Our training paradigm unifies diverse pre-training strategies, including visual masked auto-encoders, language modeling, and next-action prediction. We demonstrate the performance of our framework across three separate domains -- Robotics, Gaming AI, and Healthcare.
  arXiv  Detail & Related papers  (2024-02-08T18:58:02Z)
• LLaVA-Interactive: An All-in-One Demo for Image Chat, Segmentation, Generation and Editing [99.80742991922992]
  The system can have multi-turn dialogues with human users by taking multimodal user inputs and generating multimodal responses. LLaVA-Interactive goes beyond language prompt, where visual prompt is enabled to align human intents in the interaction.
  arXiv  Detail & Related papers  (2023-11-01T15:13:43Z)
• A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT [63.58711128819828]
  ChatGPT and other Generative AI (GAI) techniques belong to the category of Artificial Intelligence Generated Content (AIGC) The goal of AIGC is to make the content creation process more efficient and accessible, allowing for the production of high-quality content at a faster pace.
  arXiv  Detail & Related papers  (2023-03-07T20:36:13Z)
• GenNI: Human-AI Collaboration for Data-Backed Text Generation [102.08127062293111]
  Table2Text systems generate textual output based on structured data utilizing machine learning. GenNI (Generation Negotiation Interface) is an interactive visual system for high-level human-AI collaboration in producing descriptive text.
  arXiv  Detail & Related papers  (2021-10-19T18:07:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.