No Two Developers Think Alike: How Problem-Solving Styles and Experience Shape Needs in Conversational Interaction with Copilot
- URL: http://arxiv.org/abs/2606.19216v1
- Date: Wed, 17 Jun 2026 15:52:39 GMT
- Title: No Two Developers Think Alike: How Problem-Solving Styles and Experience Shape Needs in Conversational Interaction with Copilot
- Abstract summary: This study explores the role of cognitive diversity in shaping interactions with GitHub Copilot chat.<n>We characterize 5 distinct interaction modes'' and 10 underlying needs in developers' interactions, forming a conceptual model.
- Score: 1.1145952934885128
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Conversational LLM-based ``programming assistants'' provide a range of benefits to developers. However, recent studies demonstrate the variety in individual developers' needs regarding programming assistants, and challenges encountered by only specific groups of developers. In this study, we explore the role of cognitive diversity in shaping interactions with GitHub Copilot chat. Through a mixed-methods think aloud study with 27 professional developers and students, we characterize 5 distinct ``interaction modes'' and 10 underlying needs in developers' interactions, forming a conceptual model. We characterize links between these modes, needs, and developers' problem-solving styles and experience profiles, showing how cognitive diversity may shape developers' interactions. We provide insights and recommendations for researchers and practitioners on how to design, research, and employ programming assistants to better account for diverse developer needs.
Related papers
- Intentmaking and Sensemaking: Human Interaction with AI-Guided Mathematical Discovery [72.49593499512092]
We identify and characterize a distinct workflow we term intentmaking.<n>We frame this as a natural extension to sensemaking.<n>Our documentation of these themes suggests an approach to designing AI tools for scientific discovery.
arXiv Detail & Related papers (2026-05-07T09:30:25Z) - Personalizing LLM-Based Conversational Programming Assistants [0.0]
Large Language Models (LLMs) have shown much promise in powering a variety of software engineering (SE) tools.<n>This paper discusses our current and future work towards characterizing how diversity in cognition and organizational context impacts developers' needs.
arXiv Detail & Related papers (2026-04-14T17:31:28Z) - Collaborative Multi-Agent Scripts Generation for Enhancing Imperfect-Information Reasoning in Murder Mystery Games [57.64247186502836]
Vision-language models (VLMs) have shown impressive capabilities in perceptual tasks, yet they degrade in complex multi-hop reasoning under multiplayer game settings.<n>In this paper, we study a representative multiplayer task, Murder Mystery Games, which require inferring hidden truths based on partial clues provided by roles with different intentions.<n>We propose a collaborative multi-agent framework for evaluating and synthesizing high-quality, role-driven multiplayer game scripts.
arXiv Detail & Related papers (2026-04-13T17:16:23Z) - FronTalk: Benchmarking Front-End Development as Conversational Code Generation with Multi-Modal Feedback [92.67587639164908]
We present FronTalk, a benchmark for front-end code generation with multi-modal feedback.<n>We focus on the front-end development task and curate FronTalk, a collection of 100 multi-turn dialogues.<n> Evaluation of 20 models reveals two key challenges that are under-explored systematically in the literature.
arXiv Detail & Related papers (2025-12-05T23:28:09Z) - (R)evolution of Programming: Vibe Coding as a Post-Coding Paradigm [3.058685580689604]
We investigate the emerging Vibe Coding (VC) paradigm that emphasizes intuitive, affect-driven, and improvisational interactions between developers and AI systems.<n>Our analysis conceptualizes VC within the metaphor of co-drifting, contrasting it with the prevalent co-piloting perspective of AI-assisted development.
arXiv Detail & Related papers (2025-10-14T10:25:56Z) - Code with Me or for Me? How Increasing AI Automation Transforms Developer Workflows [60.04362496037186]
We present the first controlled study of developer interactions with coding agents.<n>We evaluate two leading copilot and agentic coding assistants.<n>Our results show agents can assist developers in ways that surpass copilots.
arXiv Detail & Related papers (2025-07-10T20:12:54Z) - Understanding User Mental Models in AI-Driven Code Completion Tools: Insights from an Elicitation Study [4.96900793617394]
We conducted an elicitation study with 56 developers using co-design workshops to elicit their mental models when interacting with CCTs.<n>Developers expressed diverse preferences on when and how code suggestions should be triggered.<n>It also emerged that developers need to be supported by customization of activation timing, display modality, suggestion granularity, and explanation content.<n>To demonstrate the feasibility of these and the other guidelines that emerged during the study, we developed ATHENA, a proof-of-concept CCT.
arXiv Detail & Related papers (2025-02-04T10:20:49Z) - What Could Possibly Go Wrong: Undesirable Patterns in Collective Development [4.2330023661329355]
Various studies have attempted to capture the social dynamics within software engineering.
Certain teamwork issues remain unstudied.
This paper introduces the concept of undesirable patterns in collective development.
arXiv Detail & Related papers (2024-09-02T15:13:18Z) - MockLLM: A Multi-Agent Behavior Collaboration Framework for Online Job Seeking and Recruiting [29.676163697160945]
We propose textbfMockLLM, a novel framework to generate and evaluate mock interview interactions.<n>By simulating both interviewer and candidate roles, MockLLM enables consistent and collaborative interactions for real-time and two-sided matching.<n>We evaluate MockLLM on real-world data Boss Zhipin, a major Chinese recruitment platform.
arXiv Detail & Related papers (2024-05-28T12:23:16Z) - Diversity's Double-Edged Sword: Analyzing Race's Effect on Remote Pair Programming Interactions [0.5999777817331317]
Mixed-race pairs excelled in task distribution, shared decision-making, and role-exchange but encountered communication challenges, discomfort, and anxiety.
Our study emphasizes race's impact on remote pair programming and underscores the need for diverse tools and methods to address racial disparities for collaboration.
arXiv Detail & Related papers (2024-04-11T01:58:38Z) - Making Software Development More Diverse and Inclusive: Key Themes, Challenges, and Future Directions [50.545824691484796]
We identify six themes around the theme challenges and opportunities to improve Software Developer Diversity and Inclusion (SDDI)<n>We identify benefits, harms, and future research directions for the four main themes.<n>We discuss the remaining two themes, Artificial Intelligence & SDDI and AI & Computer Science education, which have a cross-cutting effect on the other themes.
arXiv Detail & Related papers (2024-04-10T16:18:11Z) - ChatDev: Communicative Agents for Software Development [84.90400377131962]
ChatDev is a chat-powered software development framework in which specialized agents are guided in what to communicate.
These agents actively contribute to the design, coding, and testing phases through unified language-based communication.
arXiv Detail & Related papers (2023-07-16T02:11:34Z) - A Case Study in Engineering a Conversational Programming Assistant's
Persona [72.47187215119664]
Conversational capability was achieved by using an existing code-fluent Large Language Model.
A discussion of the evolution of the prompt provides a case study in how to coax an existing foundation model to behave in a desirable manner for a particular application.
arXiv Detail & Related papers (2023-01-13T14:48:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.