Evolution of Programmers' Trust in Generative AI Programming Assistants

• URL: http://arxiv.org/abs/2509.13253v1
• Date: Tue, 16 Sep 2025 17:06:47 GMT
• Title: Evolution of Programmers' Trust in Generative AI Programming Assistants
• Authors: Anshul Shah, Thomas Rexin, Elena Tomson, Leo Porter, William G. Griswold, Adalbert Gerald Soosai Raj,
• Abstract summary: This study aims to understand programmers' evolution of trust after immediate (one hour) and extended (10 days) use of GitHub Copilot.<n>After completing a project with Copilot, students felt that Copilot requires a competent programmer to complete some tasks manually.<n>Students mentioned that seeing Copilot's correctness, understanding how Copilot uses context from the code base, and learning some basics of natural language processing contributed to their elevated trust.
• Score: 2.006647079907382
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Motivation. Trust in generative AI programming assistants is a vital attitude that impacts how programmers use those programming assistants. Programmers that are over-trusting may be too reliant on their tools, leading to incorrect or vulnerable code; programmers that are under-trusting may avoid using tools that can improve their productivity and well-being. Methods. Since trust is a dynamic attitude that may change over time, this study aims to understand programmers' evolution of trust after immediate (one hour) and extended (10 days) use of GitHub Copilot. We collected survey data from 71 upper-division computer science students working on a legacy code base, representing a population that is about to enter the workforce. In this study, we quantitatively measure student trust levels and qualitatively uncover why student trust changes. Findings. Student trust, on average, increased over time. After completing a project with Copilot, however, students felt that Copilot requires a competent programmer to complete some tasks manually. Students mentioned that seeing Copilot's correctness, understanding how Copilot uses context from the code base, and learning some basics of natural language processing contributed to their elevated trust. Implications. Our study helps instructors and industry managers understand the factors that influence how students calibrate their trust with AI assistants. We make four pedagogical recommendations, which are that CS educators should 1) provide opportunities for students to work with Copilot on challenging software engineering tasks to calibrate their trust, 2) teach traditional skills of comprehending, debugging, and testing so students can verify output, 3) teach students about the basics of natural language processing, and 4) explicitly introduce and demonstrate the range of features available in Copilot.

Related papers

• More code, less validation: Risk factors for over-reliance on AI coding tools among scientists [3.5398689122254763]
  Generative AI tools capable of code generation may support scientific programmers, but user studies indicate risks of over-reliance.<n>We surveyed 868 scientists who program, examining adoption patterns, tool preferences, and factors associated with perceived productivity.
  arXiv  Detail & Related papers  (2025-12-22T18:17:54Z)
• Code with Me or for Me? How Increasing AI Automation Transforms Developer Workflows [60.04362496037186]
  We present the first controlled study of developer interactions with coding agents.<n>We evaluate two leading copilot and agentic coding assistants.<n>Our results show agents can assist developers in ways that surpass copilots.
  arXiv  Detail & Related papers  (2025-07-10T20:12:54Z)
• The Effects of GitHub Copilot on Computing Students' Programming Effectiveness, Efficiency, and Processes in Brownfield Programming Tasks [0.6282171844772422]
  GitHub Copilot is a generative artificial intelligence (GenAI) coding assistant.<n>This paper investigates how GitHub Copilot influences undergraduate students' programming performance, behaviors, and understanding.
  arXiv  Detail & Related papers  (2025-06-11T16:18:53Z)
• From Developer Pairs to AI Copilots: A Comparative Study on Knowledge Transfer [8.567835367628787]
  With the rise of AI coding assistants, developers now not only work with human partners but also, as some claim, with AI pair programmers.<n>To analyze knowledge transfer in both human-human and human-AI settings, we conducted an empirical study.<n>We found a similar frequency of successful knowledge transfer episodes and overlapping topical categories across both settings.
  arXiv  Detail & Related papers  (2025-06-05T09:13:30Z)
• Don't lie to your friends: Learning what you know from collaborative self-play [63.0121884175594]
  We propose a radically new approach to teaching AI agents what they know.<n>We construct multi-agent collaborations in which the group is rewarded for collectively arriving at correct answers.<n>The desired meta-knowledge emerges from the incentives built into the structure of the interaction.
  arXiv  Detail & Related papers  (2025-03-18T17:53:20Z)
• "It's Weird That it Knows What I Want": Usability and Interactions with Copilot for Novice Programmers [5.317693153442043]
  We present the first study that observes students at the introductory level using one such code auto-generating tool, Github Copilot, on a typical programming assignment. We explore student perceptions of the benefits and pitfalls of this technology for learning, present new observed interaction patterns, and discuss cognitive and metacognitive difficulties faced by students.
  arXiv  Detail & Related papers  (2023-04-05T15:07:50Z)
• Self-Improving Robots: End-to-End Autonomous Visuomotor Reinforcement Learning [54.636562516974884]
  In imitation and reinforcement learning, the cost of human supervision limits the amount of data that robots can be trained on. In this work, we propose MEDAL++, a novel design for self-improving robotic systems. The robot autonomously practices the task by learning to both do and undo the task, simultaneously inferring the reward function from the demonstrations.
  arXiv  Detail & Related papers  (2023-03-02T18:51:38Z)
• Generation Probabilities Are Not Enough: Uncertainty Highlighting in AI Code Completions [54.55334589363247]
  We study whether conveying information about uncertainty enables programmers to more quickly and accurately produce code. We find that highlighting tokens with the highest predicted likelihood of being edited leads to faster task completion and more targeted edits.
  arXiv  Detail & Related papers  (2023-02-14T18:43:34Z)
• Giving Feedback on Interactive Student Programs with Meta-Exploration [74.5597783609281]
  Developing interactive software, such as websites or games, is a particularly engaging way to learn computer science. Standard approaches require instructors to manually grade student-implemented interactive programs. Online platforms that serve millions, like Code.org, are unable to provide any feedback on assignments for implementing interactive programs.
  arXiv  Detail & Related papers  (2022-11-16T10:00:23Z)
• Explainable Active Learning (XAL): An Empirical Study of How Local Explanations Impact Annotator Experience [76.9910678786031]
  We propose a novel paradigm of explainable active learning (XAL), by introducing techniques from the recently surging field of explainable AI (XAI) into an Active Learning setting. Our study shows benefits of AI explanation as interfaces for machine teaching--supporting trust calibration and enabling rich forms of teaching feedback, and potential drawbacks--anchoring effect with the model judgment and cognitive workload.
  arXiv  Detail & Related papers  (2020-01-24T22:52:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.