Bridging Developer Needs and Feasible Features for AI Assistants in IDEs

• URL: http://arxiv.org/abs/2410.08676v2
• Date: Mon, 04 Aug 2025 09:11:06 GMT
• Title: Bridging Developer Needs and Feasible Features for AI Assistants in IDEs
• Authors: Agnia Sergeyuk, Ekaterina Koshchenko, Ilya Zakharov, Timofey Bryksin, Maliheh Izadi,
• Abstract summary: We interviewed 35 professional developers to uncover unmet needs and expectations.<n>Our analysis revealed five key areas: Technology Improvement, Interaction, and Alignment, as well as Simplifying Skill Building, and Programming Tasks.<n>The results demonstrate a strong alignment between the developers' needs and the practitioners' judgment for features focused on implementation and context awareness.
• Score: 6.05260196829912
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite the increasing presence of AI assistants in Integrated Development Environments, it remains unclear what developers actually need from these tools and which features are likely to be implemented in practice. To investigate this gap, we conducted a two-phase study. First, we interviewed 35 professional developers from three user groups (Adopters, Churners, and Non-Users) to uncover unmet needs and expectations. Our analysis revealed five key areas: Technology Improvement, Interaction, and Alignment, as well as Simplifying Skill Building, and Programming Tasks. We then examined the feasibility of addressing selected needs through an internal prediction market involving 102 practitioners. The results demonstrate a strong alignment between the developers' needs and the practitioners' judgment for features focused on implementation and context awareness. However, features related to proactivity and maintenance remain both underestimated and technically unaddressed. Our findings reveal gaps in current AI support and provide practical directions for developing more effective and sustainable in-IDE AI systems.

Related papers

• The SPACE of AI: Real-World Lessons on AI's Impact on Developers [0.807084206814932]
  We study how developers perceive AI's influence across the dimensions of the SPACE framework: Satisfaction, Performance, Activity, Collaboration and Efficiency.<n>We find that AI is broadly adopted and widely seen as enhancing productivity, particularly for routine tasks.<n>Developers report increased efficiency and satisfaction, with less evidence of impact on collaboration.
  arXiv  Detail & Related papers  (2025-07-31T21:45:54Z)
• Bridging the Gap: Integrating Ethics and Environmental Sustainability in AI Research and Practice [57.94036023167952]
  We argue that the efforts aiming to study AI's ethical ramifications should be made in tandem with those evaluating its impacts on the environment.<n>We propose best practices to better integrate AI ethics and sustainability in AI research and practice.
  arXiv  Detail & Related papers  (2025-04-01T13:53:11Z)
• The Impact of Generative AI Coding Assistants on Developers Who Are Visually Impaired [3.2895414694900684]
  We conducted a study where developers who are visually impaired completed a series of programming tasks using a generative AI coding assistant.<n>While participants found the AI assistant beneficial and reported significant advantages, they also highlighted accessibility challenges.<n>Our findings emphasize the need to apply activity-centered design principles to generative AI assistants.
  arXiv  Detail & Related papers  (2025-03-10T22:06:43Z)
• Human-AI Experience in Integrated Development Environments: A Systematic Literature Review [2.1749194587826026]
  In-IDE HAX explores the evolving dynamics of Human-Computer Interaction in AI-assisted coding environments. Our findings reveal that AI-assisted coding enhances developer productivity but also introduces challenges, such as verification overhead, automation bias, and over-reliance. Concerns about code correctness, security, and maintainability highlight the urgent need for explainability, verification mechanisms, and adaptive user control.
  arXiv  Detail & Related papers  (2025-03-08T12:40:18Z)
• AISafetyLab: A Comprehensive Framework for AI Safety Evaluation and Improvement [73.0700818105842]
  We introduce AISafetyLab, a unified framework and toolkit that integrates representative attack, defense, and evaluation methodologies for AI safety. AISafetyLab features an intuitive interface that enables developers to seamlessly apply various techniques. We conduct empirical studies on Vicuna, analyzing different attack and defense strategies to provide valuable insights into their comparative effectiveness.
  arXiv  Detail & Related papers  (2025-02-24T02:11:52Z)
• Interactive Agents to Overcome Ambiguity in Software Engineering [61.40183840499932]
  AI agents are increasingly being deployed to automate tasks, often based on ambiguous and underspecified user instructions.<n>Making unwarranted assumptions and failing to ask clarifying questions can lead to suboptimal outcomes.<n>We study the ability of LLM agents to handle ambiguous instructions in interactive code generation settings by evaluating proprietary and open-weight models on their performance.
  arXiv  Detail & Related papers  (2025-02-18T17:12:26Z)
• Understanding User Mental Models in AI-Driven Code Completion Tools: Insights from an Elicitation Study [5.534104886050636]
  We conduct an elicitation study with 56 developers using focus groups to elicit their mental models when interacting with AI-powered code completion tools. The study findings provide actionable insights for designing human-centered CCTs that align with user expectations, enhance satisfaction and productivity, and foster trust in AI-powered development tools. We also develop ATHENA, a proof-of-concept CCT that dynamically adapts to developers' coding preferences and environments, ensuring seamless integration into diverse environments.
  arXiv  Detail & Related papers  (2025-02-04T10:20:49Z)
• How Developers Interact with AI: A Taxonomy of Human-AI Collaboration in Software Engineering [8.65285948382426]
  We propose a taxonomy of interaction types between developers and AI tools, identifying eleven distinct interaction types. Building on this taxonomy, we outline a research agenda focused on optimizing AI interactions, improving developer control, and addressing trust and usability challenges in AI-assisted development.
  arXiv  Detail & Related papers  (2025-01-15T12:53:49Z)
• TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks [52.46737975742287]
  We build a self-contained environment with data that mimics a small software company environment. We find that with the most competitive agent, 24% of the tasks can be completed autonomously. This paints a nuanced picture on task automation with LM agents.
  arXiv  Detail & Related papers  (2024-12-18T18:55:40Z)
• "I Don't Use AI for Everything": Exploring Utility, Attitude, and Responsibility of AI-empowered Tools in Software Development [19.851794567529286]
  This study investigates the adoption, impact, and security considerations of AI-empowered tools in the software development process. Our findings reveal widespread adoption of AI tools across various stages of software development.
  arXiv  Detail & Related papers  (2024-09-20T09:17:10Z)
• Constraining Participation: Affordances of Feedback Features in Interfaces to Large Language Models [49.74265453289855]
  Large language models (LLMs) are now accessible to anyone with a computer, a web browser, and an internet connection via browser-based interfaces. This paper examines the affordances of interactive feedback features in ChatGPT's interface, analysing how they shape user input and participation in iteration.
  arXiv  Detail & Related papers  (2024-08-27T13:50:37Z)
• OpenHands: An Open Platform for AI Software Developers as Generalist Agents [109.8507367518992]
  We introduce OpenHands, a platform for the development of AI agents that interact with the world in similar ways to a human developer. We describe how the platform allows for the implementation of new agents, safe interaction with sandboxed environments for code execution, and incorporation of evaluation benchmarks.
  arXiv  Detail & Related papers  (2024-07-23T17:50:43Z)
• The Ethics of Advanced AI Assistants [53.89899371095332]
  This paper focuses on the opportunities and the ethical and societal risks posed by advanced AI assistants. We define advanced AI assistants as artificial agents with natural language interfaces, whose function is to plan and execute sequences of actions on behalf of a user. We consider the deployment of advanced assistants at a societal scale, focusing on cooperation, equity and access, misinformation, economic impact, the environment and how best to evaluate advanced AI assistants.
  arXiv  Detail & Related papers  (2024-04-24T23:18:46Z)
• Bridging Gaps, Building Futures: Advancing Software Developer Diversity and Inclusion Through Future-Oriented Research [50.545824691484796]
  We present insights from SE researchers and practitioners on challenges and solutions regarding diversity and inclusion in SE. We share potential utopian and dystopian visions of the future and provide future research directions and implications for academia and industry.
  arXiv  Detail & Related papers  (2024-04-10T16:18:11Z)
• Open-Source AI-based SE Tools: Opportunities and Challenges of Collaborative Software Learning [23.395624804517034]
  Large Language Models (LLMs) have become instrumental in advancing software engineering (SE) tasks. The collaboration of these AI-based SE models hinges on maximising the sources of high-quality data. Data especially of high quality, often holds commercial or sensitive value, making it less accessible for open-source AI-based SE projects.
  arXiv  Detail & Related papers  (2024-04-09T10:47:02Z)
• In-IDE Human-AI Experience in the Era of Large Language Models; A Literature Review [2.6703221234079946]
  The study of in-IDE Human-AI Experience is critical in understanding how these AI tools are transforming the software development process. We conducted a literature review to study the current state of in-IDE Human-AI Experience research.
  arXiv  Detail & Related papers  (2024-01-19T14:55:51Z)
• Exploring the intersection of Generative AI and Software Development [0.0]
  The synergy between generative AI and Software Engineering emerges as a transformative frontier. This whitepaper delves into the unexplored realm, elucidating how generative AI techniques can revolutionize software development. It serves as a guide for stakeholders, urging discussions and experiments in the application of generative AI in Software Engineering.
  arXiv  Detail & Related papers  (2023-12-21T19:23:23Z)
• Federated Learning-Empowered AI-Generated Content in Wireless Networks [58.48381827268331]
  Federated learning (FL) can be leveraged to improve learning efficiency and achieve privacy protection for AIGC. We present FL-based techniques for empowering AIGC, and aim to enable users to generate diverse, personalized, and high-quality content.
  arXiv  Detail & Related papers  (2023-07-14T04:13:11Z)
• Empowered and Embedded: Ethics and Agile Processes [60.63670249088117]
  We argue that ethical considerations need to be embedded into the (agile) software development process. We put emphasis on the possibility to implement ethical deliberations in already existing and well established agile software development processes.
  arXiv  Detail & Related papers  (2021-07-15T11:14:03Z)
• Artificial Intelligence for IT Operations (AIOPS) Workshop White Paper [50.25428141435537]
  Artificial Intelligence for IT Operations (AIOps) is an emerging interdisciplinary field arising in the intersection between machine learning, big data, streaming analytics, and the management of IT operations. Main aim of the AIOPS workshop is to bring together researchers from both academia and industry to present their experiences, results, and work in progress in this field.
  arXiv  Detail & Related papers  (2021-01-15T10:43:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.