Enhancing Software Development with Context-Aware Conversational Agents: A User Study on Developer Interactions with Chatbots

• URL: http://arxiv.org/abs/2505.08648v1
• Date: Tue, 13 May 2025 15:08:55 GMT
• Title: Enhancing Software Development with Context-Aware Conversational Agents: A User Study on Developer Interactions with Chatbots
• Authors: Glaucia Melo, Paulo Alencar, Donald Cowan,
• Abstract summary: We conducted a user study with 29 developers using a prototype text-based chatbots to investigate preferred functionalities.<n>Our findings reveal strong interest in task automation, version control support, and contextual adaptability.<n>We highlight the importance of deep contextual understanding, historical interaction awareness, and personalized support in CA design.
• Score: 3.6321891270689055
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Software development is a cognitively intensive process requiring multitasking, adherence to evolving workflows, and continuous learning. With the rise of large language model (LLM)-based tools, such as conversational agents (CAs), there is growing interest in supporting developers through natural language interaction. However, little is known about the specific features developers seek in these systems. We conducted a user study with 29 developers using a prototype text-based chatbot to investigate preferred functionalities. Our findings reveal strong interest in task automation, version control support, and contextual adaptability, especially the need to tailor assistance for both novice and experienced users. We highlight the importance of deep contextual understanding, historical interaction awareness, and personalized support in CA design. This study contributes to the development of context-aware chatbots that enhance productivity and satisfaction, and it outlines opportunities for future research on human-AI collaboration in software engineering.

Related papers

• Past, Present and Future: Exploring Adaptive AI in Software Development Bots [3.2228025627337864]
  This paper examines the role of adaptive AI-powered conversational agents in software development.<n>We look at how these tools have evolved from simple query-based systems to advanced AI-driven solutions like GitHub Copilot and Microsoft Teams bots.<n>The study aims to assess the benefits and limitations of these systems, address concerns like data privacy and ethical issues, and offer insights into their future use in the field.
  arXiv  Detail & Related papers  (2025-07-14T21:40:03Z)
• Code with Me or for Me? How Increasing AI Automation Transforms Developer Workflows [66.1850490474361]
  We conduct the first academic study to explore developer interactions with coding agents.<n>We evaluate two leading copilot and agentic coding assistants, GitHub Copilot and OpenHands.<n>Our results show agents have the potential to assist developers in ways that surpass copilots.
  arXiv  Detail & Related papers  (2025-07-10T20:12:54Z)
• A Desideratum for Conversational Agents: Capabilities, Challenges, and Future Directions [51.96890647837277]
  Large Language Models (LLMs) have propelled conversational AI from traditional dialogue systems into sophisticated agents capable of autonomous actions, contextual awareness, and multi-turn interactions with users.<n>This survey paper presents a desideratum for next-generation Conversational Agents - what has been achieved, what challenges persist, and what must be done for more scalable systems that approach human-level intelligence.
  arXiv  Detail & Related papers  (2025-04-07T21:01:25Z)
• How Developers Interact with AI: A Taxonomy of Human-AI Collaboration in Software Engineering [8.65285948382426]
  We propose a taxonomy of interaction types between developers and AI tools, identifying eleven distinct interaction types.<n>Building on this taxonomy, we outline a research agenda focused on optimizing AI interactions, improving developer control, and addressing trust and usability challenges in AI-assisted development.
  arXiv  Detail & Related papers  (2025-01-15T12:53:49Z)
• Developers' Perceptions on the Impact of ChatGPT in Software Development: A Survey [13.257222195239375]
  We conducted a survey with 207 software developers to understand the impact of ChatGPT on software quality, productivity, and job satisfaction. The study delves into developers' expectations regarding future adaptations of ChatGPT, concerns about potential job displacement, and perspectives on regulatory interventions.
  arXiv  Detail & Related papers  (2024-05-20T17:31:16Z)
• Rocks Coding, Not Development--A Human-Centric, Experimental Evaluation of LLM-Supported SE Tasks [9.455579863269714]
  We examined whether and to what degree working with ChatGPT was helpful in the coding task and typical software development task. We found that while ChatGPT performed well in solving simple coding problems, its performance in supporting typical software development tasks was not that good. Our study thus provides first-hand insights into using ChatGPT to fulfill software engineering tasks with real-world developers.
  arXiv  Detail & Related papers  (2024-02-08T13:07:31Z)
• In-IDE Human-AI Experience in the Era of Large Language Models; A Literature Review [2.6703221234079946]
  The study of in-IDE Human-AI Experience is critical in understanding how these AI tools are transforming the software development process. We conducted a literature review to study the current state of in-IDE Human-AI Experience research.
  arXiv  Detail & Related papers  (2024-01-19T14:55:51Z)
• ChatDev: Communicative Agents for Software Development [84.90400377131962]
  ChatDev is a chat-powered software development framework in which specialized agents are guided in what to communicate. These agents actively contribute to the design, coding, and testing phases through unified language-based communication.
  arXiv  Detail & Related papers  (2023-07-16T02:11:34Z)
• The GitHub Development Workflow Automation Ecosystems [47.818229204130596]
  Large-scale software development has become a highly collaborative endeavour. This chapter explores the ecosystems of development bots and GitHub Actions. It provides an extensive survey of the state-of-the-art in this domain.
  arXiv  Detail & Related papers  (2023-05-08T15:24:23Z)
• CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society [58.04479313658851]
  This paper explores the potential of building scalable techniques to facilitate autonomous cooperation among communicative agents. We propose a novel communicative agent framework named role-playing. Our contributions include introducing a novel communicative agent framework, offering a scalable approach for studying the cooperative behaviors and capabilities of multi-agent systems.
  arXiv  Detail & Related papers  (2023-03-31T01:09:00Z)
• Can You be More Social? Injecting Politeness and Positivity into Task-Oriented Conversational Agents [60.27066549589362]
  Social language used by human agents is associated with greater users' responsiveness and task completion. The model uses a sequence-to-sequence deep learning architecture, extended with a social language understanding element. Evaluation in terms of content preservation and social language level using both human judgment and automatic linguistic measures shows that the model can generate responses that enable agents to address users' issues in a more socially appropriate way.
  arXiv  Detail & Related papers  (2020-12-29T08:22:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.