"It's Weird That it Knows What I Want": Usability and Interactions with Copilot for Novice Programmers

• URL: http://arxiv.org/abs/2304.02491v1
• Date: Wed, 5 Apr 2023 15:07:50 GMT
• Title: "It's Weird That it Knows What I Want": Usability and Interactions with Copilot for Novice Programmers
• Authors: James Prather, Brent N. Reeves, Paul Denny, Brett A. Becker, Juho Leinonen, Andrew Luxton-Reilly, Garrett Powell, James Finnie-Ansley, Eddie Antonio Santos
• Abstract summary: We present the first study that observes students at the introductory level using one such code auto-generating tool, Github Copilot, on a typical programming assignment. We explore student perceptions of the benefits and pitfalls of this technology for learning, present new observed interaction patterns, and discuss cognitive and metacognitive difficulties faced by students.
• Score: 5.317693153442043
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent developments in deep learning have resulted in code-generation models that produce source code from natural language and code-based prompts with high accuracy. This is likely to have profound effects in the classroom, where novices learning to code can now use free tools to automatically suggest solutions to programming exercises and assignments. However, little is currently known about how novices interact with these tools in practice. We present the first study that observes students at the introductory level using one such code auto-generating tool, Github Copilot, on a typical introductory programming (CS1) assignment. Through observations and interviews we explore student perceptions of the benefits and pitfalls of this technology for learning, present new observed interaction patterns, and discuss cognitive and metacognitive difficulties faced by students. We consider design implications of these findings, specifically in terms of how tools like Copilot can better support and scaffold the novice programming experience.

Related papers

• Investigating Developers' Preferences for Learning and Issue Resolution Resources in the ChatGPT Era [1.3124513975412255]
  The landscape of software developer learning resources has continuously evolved, with recent trends favoring engaging formats like video tutorials. The emergence of Large Language Models (LLMs) like ChatGPT presents a new learning paradigm. We conducted a survey targeting software developers and computer science students, gathering 341 responses, of which 268 were completed and analyzed.
  arXiv  Detail & Related papers  (2024-10-10T22:57:29Z)
• Code Compass: A Study on the Challenges of Navigating Unfamiliar Codebases [2.808331566391181]
  We propose a novel tool, Code, to address these issues. Our study highlights a significant gap in current tools and methodologies. Our formative study demonstrates how effectively the tool reduces the time developers spend navigating documentation.
  arXiv  Detail & Related papers  (2024-05-10T06:58:31Z)
• Application-Driven Innovation in Machine Learning [56.85396167616353]
  We describe the paradigm of application-driven research in machine learning. We show how this approach can productively synergize with methods-driven work. Despite these benefits, we find that reviewing, hiring, and teaching practices in machine learning often hold back application-driven innovation.
  arXiv  Detail & Related papers  (2024-03-26T04:59:27Z)
• Charting a Path to Efficient Onboarding: The Role of Software Visualization [49.1574468325115]
  The present study aims to explore the familiarity of managers, leaders, and developers with software visualization tools. This approach incorporated quantitative and qualitative analyses of data collected from practitioners using questionnaires and semi-structured interviews.
  arXiv  Detail & Related papers  (2024-01-17T21:30:45Z)
• How to Do Things with Deep Learning Code [0.0]
  We draw attention to the means by which ordinary users might interact with, and even direct, the behavior of deep learning systems. What is at stake is the possibility of achieving an informed sociotechnical consensus about the responsible applications of large language models.
  arXiv  Detail & Related papers  (2023-04-19T03:46:12Z)
• Learning Robotic Navigation from Experience: Principles, Methods, and Recent Results [94.60414567852536]
  Real-world navigation presents a complex set of physical challenges that defies simple geometric abstractions. Machine learning offers a promising way to go beyond geometry and conventional planning. We present a toolkit for experiential learning of robotic navigation skills that unifies several recent approaches.
  arXiv  Detail & Related papers  (2022-12-13T17:41:58Z)
• Conversing with Copilot: Exploring Prompt Engineering for Solving CS1 Problems Using Natural Language [3.155277175705079]
  GitHub Copilot is an artificial intelligence model for automatically generating source code from natural language problem descriptions. Since June 2022, Copilot has officially been available for free to all students as a plug-in to development environments like Visual Studio Code.
  arXiv  Detail & Related papers  (2022-10-27T03:48:24Z)
• Reading Between the Lines: Modeling User Behavior and Costs in AI-Assisted Programming [28.254978977288868]
  We studied GitHub Copilot, a code-recommendation system used by millions of programmers daily. We developed CUPS, a taxonomy of common programmer activities when interacting with Copilot. Our insights reveal how programmers interact with Copilot and motivate new interface designs and metrics.
  arXiv  Detail & Related papers  (2022-10-25T20:01:15Z)
• Improving Compositionality of Neural Networks by Decoding Representations to Inputs [83.97012077202882]
  We bridge the benefits of traditional and deep learning programs by jointly training a generative model to constrain neural network activations to "decode" back to inputs. We demonstrate applications of decodable representations to out-of-distribution detection, adversarial examples, calibration, and fairness.
  arXiv  Detail & Related papers  (2021-06-01T20:07:16Z)
• Ten Quick Tips for Deep Learning in Biology [116.78436313026478]
  Machine learning is concerned with the development and applications of algorithms that can recognize patterns in data and use them for predictive modeling. Deep learning has become its own subfield of machine learning. In the context of biological research, deep learning has been increasingly used to derive novel insights from high-dimensional biological data.
  arXiv  Detail & Related papers  (2021-05-29T21:02:44Z)
• Hierarchical Affordance Discovery using Intrinsic Motivation [69.9674326582747]
  We propose an algorithm using intrinsic motivation to guide the learning of affordances for a mobile robot. This algorithm is capable to autonomously discover, learn and adapt interrelated affordances without pre-programmed actions. Once learned, these affordances may be used by the algorithm to plan sequences of actions in order to perform tasks of various difficulties.
  arXiv  Detail & Related papers  (2020-09-23T07:18:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.