How AI Impacts Skill Formation

• URL: http://arxiv.org/abs/2601.20245v2
• Date: Sun, 01 Feb 2026 05:05:10 GMT
• Title: How AI Impacts Skill Formation
• Authors: Judy Hanwen Shen, Alex Tamkin,
• Abstract summary: We study how developers gained mastery of a new asynchronous programming library with and without the assistance of AI.<n>We find that AI use impairs conceptual understanding, code reading, and debug abilities, without delivering significant efficiency gains on average.<n>We identify six distinct AI interaction patterns, three of which involve cognitive engagement and preserve learning outcomes even when participants receive AI assistance.
• Score: 12.295096074858932
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: AI assistance produces significant productivity gains across professional domains, particularly for novice workers. Yet how this assistance affects the development of skills required to effectively supervise AI remains unclear. Novice workers who rely heavily on AI to complete unfamiliar tasks may compromise their own skill acquisition in the process. We conduct randomized experiments to study how developers gained mastery of a new asynchronous programming library with and without the assistance of AI. We find that AI use impairs conceptual understanding, code reading, and debugging abilities, without delivering significant efficiency gains on average. Participants who fully delegated coding tasks showed some productivity improvements, but at the cost of learning the library. We identify six distinct AI interaction patterns, three of which involve cognitive engagement and preserve learning outcomes even when participants receive AI assistance. Our findings suggest that AI-enhanced productivity is not a shortcut to competence and AI assistance should be carefully adopted into workflows to preserve skill formation -- particularly in safety-critical domains.

Related papers

• AI as Cognitive Amplifier: Rethinking Human Judgment in the Age of Generative AI [0.65268245109828]
  I propose a three-level model of AI engagement.<n>I argue that the transition between levels requires not technical training but development of domain expertise and metacognitive skills.
  arXiv  Detail & Related papers  (2025-10-30T11:55:34Z)
• Efficiency Without Cognitive Change: Evidence from Human Interaction with Narrow AI Systems [0.0]
  Short-term exposure to narrow AI tools enhances core cognitive abilities.<n>No significant pre-post differences emerged in standardized measures of problem solving or verbal comprehension.
  arXiv  Detail & Related papers  (2025-10-28T18:55:44Z)
• The SPACE of AI: Real-World Lessons on AI's Impact on Developers [0.807084206814932]
  We study how developers perceive AI's influence across the dimensions of the SPACE framework: Satisfaction, Performance, Activity, Collaboration and Efficiency.<n>We find that AI is broadly adopted and widely seen as enhancing productivity, particularly for routine tasks.<n>Developers report increased efficiency and satisfaction, with less evidence of impact on collaboration.
  arXiv  Detail & Related papers  (2025-07-31T21:45:54Z)
• From Teacher to Colleague: How Coding Experience Shapes Developer Perceptions of AI Tools [0.0]
  AI-assisted development tools promise productivity gains and improved code quality, yet their adoption among developers remains inconsistent.<n>We analyze survey data from 3380 developers to examine how coding experience relates to AI awareness, adoption, and the roles developers assign to AI in their workflow.
  arXiv  Detail & Related papers  (2025-04-08T08:58:06Z)
• Human-AI Collaborative Game Testing with Vision Language Models [0.0]
  This study investigates how AI can improve game testing by developing and experimenting with an AI-assisted workflow.<n>We evaluate the effectiveness of AI assistance under four conditions: with or without AI support, and with or without detailed knowledge of defects and design documentation.<n>Results indicate that AI assistance significantly improves defect identification performance, particularly when paired with detailed knowledge.
  arXiv  Detail & Related papers  (2025-01-20T23:14:23Z)
• Imagining and building wise machines: The centrality of AI metacognition [78.76893632793497]
  We examine what is known about human wisdom and sketch a vision of its AI counterpart.<n>We argue that AI systems particularly struggle with metacognition.<n>We discuss how wise AI might be benchmarked, trained, and implemented.
  arXiv  Detail & Related papers  (2024-11-04T18:10:10Z)
• Generation Probabilities Are Not Enough: Uncertainty Highlighting in AI Code Completions [54.55334589363247]
  We study whether conveying information about uncertainty enables programmers to more quickly and accurately produce code. We find that highlighting tokens with the highest predicted likelihood of being edited leads to faster task completion and more targeted edits.
  arXiv  Detail & Related papers  (2023-02-14T18:43:34Z)
• Seamful XAI: Operationalizing Seamful Design in Explainable AI [59.89011292395202]
  Mistakes in AI systems are inevitable, arising from both technical limitations and sociotechnical gaps. We propose that seamful design can foster AI explainability by revealing sociotechnical and infrastructural mismatches. We explore this process with 43 AI practitioners and real end-users.
  arXiv  Detail & Related papers  (2022-11-12T21:54:05Z)
• Human Decision Makings on Curriculum Reinforcement Learning with Difficulty Adjustment [52.07473934146584]
  We guide the curriculum reinforcement learning results towards a preferred performance level that is neither too hard nor too easy via learning from the human decision process. Our system is highly parallelizable, making it possible for a human to train large-scale reinforcement learning applications. It shows reinforcement learning performance can successfully adjust in sync with the human desired difficulty level.
  arXiv  Detail & Related papers  (2022-08-04T23:53:51Z)
• Cybertrust: From Explainable to Actionable and Interpretable AI (AI2) [58.981120701284816]
  Actionable and Interpretable AI (AI2) will incorporate explicit quantifications and visualizations of user confidence in AI recommendations. It will allow examining and testing of AI system predictions to establish a basis for trust in the systems' decision making.
  arXiv  Detail & Related papers  (2022-01-26T18:53:09Z)
• Building Bridges: Generative Artworks to Explore AI Ethics [56.058588908294446]
  In recent years, there has been an increased emphasis on understanding and mitigating adverse impacts of artificial intelligence (AI) technologies on society. A significant challenge in the design of ethical AI systems is that there are multiple stakeholders in the AI pipeline, each with their own set of constraints and interests. This position paper outlines some potential ways in which generative artworks can play this role by serving as accessible and powerful educational tools.
  arXiv  Detail & Related papers  (2021-06-25T22:31:55Z)
• Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making [53.62514158534574]
  We study whether features that reveal case-specific model information can calibrate trust and improve the joint performance of the human and AI. We show that confidence score can help calibrate people's trust in an AI model, but trust calibration alone is not sufficient to improve AI-assisted decision making.
  arXiv  Detail & Related papers  (2020-01-07T15:33:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.