Related papers: Pre-AI Baseline: Developer IDE Satisfaction and Tool Autonomy in 2022

Pre-AI Baseline: Developer IDE Satisfaction and Tool Autonomy in 2022

URL: http://arxiv.org/abs/2603.06050v1
Date: Fri, 06 Mar 2026 09:01:53 GMT
Title: Pre-AI Baseline: Developer IDE Satisfaction and Tool Autonomy in 2022
Abstract summary: This study analyzes satisfaction data from 1,155 software developers collected in July 2022, immediately preceding the mainstream adoption of generative AI tools.<n>We report a high-satisfaction ecosystem, dominated by Visual Studio Code (79% usage)<n>We identify an "experimenter" segment (29.9%) characterized by high tool churn but no significant satisfaction difference.
Score: 0.0
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
Abstract: To quantify the impact of AI on software development, the community requires a robust pre-AI baseline. This study analyzes valid satisfaction data from 1,155 software developers collected in July 2022, immediately preceding the mainstream adoption of generative AI tools. We report a high-satisfaction ecosystem (Mean = 8.14 [95% CI 8.01-8.25]), dominated by Visual Studio Code (79% usage). Multivariable regression confirms that autonomy in tool choice is the strongest predictor of IDE satisfaction (beta = 0.51), significantly outweighing demographic or role-based factors. Conversely, cloud IDE adoption was negligible (4.3% regular usage), with 40.1% citing network dependency as the primary barrier, a constraint that remains relevant for modern cloud-reliant AI agents. Additionally, we identify an "experimenter" segment (29.9%) characterized by high tool churn but no significant satisfaction difference (t = 0.43, p = 0.67), and demonstrate significant variation in IDE retention rates (VS Code: 68.5%, traditional IDEs: 3.9-25%), suggesting underlying dissatisfaction despite high overall satisfaction. By providing a quantitative snapshot of developer sentiment and workflows on the eve of the AI revolution, this study establishes a verifiable baseline for longitudinal research into the productivity-satisfaction misalignment observed in the post-AI era.

Related papers

MatrAIx: Simulating the World with 8.3 Billion Persona Agents [175.06699039914625]
MatrAIx is a population-scale simulated-user evaluation infrastructure for testing AI systems and digital products with heterogeneous users.<n>MatrAIx has three core components: First, Persona 8B contains 8.3 billion persona records represented by 1,290 categorical dimensions.<n>Second, the MatrAIx Playground provides four environments in which users evaluate and interact with digital products.<n>Third, MatrAIx provides 1,010 application tasks spanning more than 25 domains, including Commerce, Software, Finance, and Healthcare.
arXiv Detail & Related papers (2026-08-04T20:04:49Z)
Predicting Developer Acceptance of AI-Generated Code Suggestions [12.254014187965831]
This paper conducts an empirical study using 66,329 industrial developer-AI interactions from a large technology company.<n>We analyze features that are significantly different between accepted code suggestions and rejected ones.<n>We introduce CSAP (Code Suggestion Acceptance Prediction) to predict whether a developer will accept the code suggestion before it is displayed.
arXiv Detail & Related papers (2026-01-29T08:14:24Z)
AI IDEs or Autonomous Agents? Measuring the Impact of Coding Agents on Software Development [12.50615284537175]
Large language model (LLM) based coding agents increasingly act as autonomous contributors that generate and merge pull requests.<n>We present a longitudinal causal study of agent adoption in open-source repositories using staggered difference-in-differences with matched controls.
arXiv Detail & Related papers (2026-01-20T04:51:56Z)
WhatsCode: Large-Scale GenAI Deployment for Developer Efficiency at WhatsApp [0.8197659035200293]
Report on the industrial deployment of WhatsCode, a domain-specific AI development system that supports WhatsApp.<n>WhatsCode evolved from targeted privacy automation to autonomous agentic integrated with end-to-end feature development and DevOps processes.<n>System committed 692 automated/fix changes, 711 framework adoptions, 141 feature development assists and maintained precision in bug triage.
arXiv Detail & Related papers (2025-12-04T23:25:06Z)
Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoning [84.70211451226835]
Large Language Model (LLM) Agents are constrained by a dependency on human-curated data.<n>We introduce Agent0, a fully autonomous framework that evolves high-performing agents without external data.<n>Agent0 substantially boosts reasoning capabilities, improving the Qwen3-8B-Base model by 18% on mathematical reasoning and 24% on general reasoning benchmarks.
arXiv Detail & Related papers (2025-11-20T05:01:57Z)
Intuition to Evidence: Measuring AI's True Impact on Developer Productivity [30.02516976149379]
We present a comprehensive real-world evaluation of AI-assisted software development tools deployed at enterprise scale.<n>Over one year, 300 engineers across multiple teams integrated an in-house AI platform (DeputyDev) that combines code generation and automated review capabilities.
arXiv Detail & Related papers (2025-09-24T02:34:11Z)
LIMI: Less is More for Agency [49.63355240818081]
LIMI (Less Is More for Intelligent Agency) demonstrates that agency follows radically different development principles.<n>We show that sophisticated agentic intelligence can emerge from minimal but strategically curated demonstrations of autonomous behavior.<n>Our findings establish the Agency Efficiency Principle: machine autonomy emerges not from data abundance but from strategic curation of high-quality agentic demonstrations.
arXiv Detail & Related papers (2025-09-22T10:59:32Z)
UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learning [155.51875080423883]
The development of autonomous agents for graphical user interfaces presents major challenges in artificial intelligence.<n>We present UI-TARS-2, a native GUI-centered agent model that addresses these challenges through a systematic training methodology.<n> Empirical evaluation demonstrates that UI-TARS-2 achieves significant improvements over its predecessor UI-TARS-1.5.
arXiv Detail & Related papers (2025-09-02T17:44:45Z)
OmniEAR: Benchmarking Agent Reasoning in Embodied Tasks [52.87238755666243]
We present OmniEAR, a framework for evaluating how language models reason about physical interactions, tool usage, and multi-agent coordination in embodied tasks.<n>We model continuous physical properties and complex spatial relationships across 1,500 scenarios spanning household and industrial domains.<n>Our systematic evaluation reveals severe performance degradation when models must reason from constraints.
arXiv Detail & Related papers (2025-08-07T17:54:15Z)
Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity [0.5789840336223054]
Despite widespread adoption, the impact of AI tools on software development in the wild remains understudied.<n>We conduct a randomized controlled trial (RCT) to understand how AI tools at the February-June 2025 frontier affect the productivity of experienced open-source developers.<n>16 developers with moderate AI experience complete 246 tasks in mature projects on which they have an average of 5 years of prior experience.
arXiv Detail & Related papers (2025-07-12T00:16:33Z)
Tool-Augmented Reward Modeling [58.381678612409]
We propose a tool-augmented preference modeling approach, named Themis, to address limitations by empowering RMs with access to external environments. Our study delves into the integration of external tools into RMs, enabling them to interact with diverse external sources. In human evaluations, RLHF trained with Themis attains an average win rate of 32% when compared to baselines.
arXiv Detail & Related papers (2023-10-02T09:47:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.