ChatCollab: Exploring Collaboration Between Humans and AI Agents in Software Teams

• URL: http://arxiv.org/abs/2412.01992v1
• Date: Mon, 02 Dec 2024 21:56:46 GMT
• Title: ChatCollab: Exploring Collaboration Between Humans and AI Agents in Software Teams
• Authors: Benjamin Klieger, Charis Charitsis, Miroslav Suzara, Sierra Wang, Nick Haber, John C. Mitchell,
• Abstract summary: ChatCollab's novel architecture allows agents - human or AI - to join collaborations in any role. Using software engineering as a case study, we find that our AI agents successfully identify their roles and responsibilities. In relation to three prior multi-agent AI systems for software development, we find ChatCollab AI agents produce comparable or better software in an interactive game development task.
• Score: 1.3967206132709542
• License:
• Abstract: We explore the potential for productive team-based collaboration between humans and Artificial Intelligence (AI) by presenting and conducting initial tests with a general framework that enables multiple human and AI agents to work together as peers. ChatCollab's novel architecture allows agents - human or AI - to join collaborations in any role, autonomously engage in tasks and communication within Slack, and remain agnostic to whether their collaborators are human or AI. Using software engineering as a case study, we find that our AI agents successfully identify their roles and responsibilities, coordinate with other agents, and await requested inputs or deliverables before proceeding. In relation to three prior multi-agent AI systems for software development, we find ChatCollab AI agents produce comparable or better software in an interactive game development task. We also propose an automated method for analyzing collaboration dynamics that effectively identifies behavioral characteristics of agents with distinct roles, allowing us to quantitatively compare collaboration dynamics in a range of experimental conditions. For example, in comparing ChatCollab AI agents, we find that an AI CEO agent generally provides suggestions 2-4 times more often than an AI product manager or AI developer, suggesting agents within ChatCollab can meaningfully adopt differentiated collaborative roles. Our code and data can be found at: https://github.com/ChatCollab.

Related papers

• CowPilot: A Framework for Autonomous and Human-Agent Collaborative Web Navigation [70.3224918173672]
  CowPilot is a framework supporting autonomous as well as human-agent collaborative web navigation. It reduces the number of steps humans need to perform by allowing agents to propose next steps, while users are able to pause, reject, or take alternative actions. CowPilot can serve as a useful tool for data collection and agent evaluation across websites.
  arXiv  Detail & Related papers  (2025-01-28T00:56:53Z)
• YETI (YET to Intervene) Proactive Interventions by Multimodal AI Agents in Augmented Reality Tasks [16.443149180969776]
  Augmented Reality (AR) head worn devices can uniquely improve the user experience of solving procedural day-to-day tasks. Such AR capabilities can help AI Agents see and listen to actions that users take which can relate to multimodal capabilities of human users. Proactivity of AI Agents on the other hand can help the human user detect and correct any mistakes in agent observed tasks.
  arXiv  Detail & Related papers  (2025-01-16T08:06:02Z)
• Collaborative Gym: A Framework for Enabling and Evaluating Human-Agent Collaboration [51.452664740963066]
  Collaborative Gym is a framework enabling asynchronous, tripartite interaction among agents, humans, and task environments. We instantiate Co-Gym with three representative tasks in both simulated and real-world conditions. Our findings reveal that collaborative agents consistently outperform their fully autonomous counterparts in task performance.
  arXiv  Detail & Related papers  (2024-12-20T09:21:15Z)
• TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks [52.46737975742287]
  We build a self-contained environment with data that mimics a small software company environment. We find that with the most competitive agent, 24% of the tasks can be completed autonomously. This paints a nuanced picture on task automation with LM agents.
  arXiv  Detail & Related papers  (2024-12-18T18:55:40Z)
• Two Heads Are Better Than One: Collaborative LLM Embodied Agents for Human-Robot Interaction [1.6574413179773757]
  Large language models (LLMs) should be able to leverage their large breadth of understanding to interpret natural language commands. However, these models suffer from hallucinations, which may cause safety issues or deviations from the task. In this research, multiple collaborative AI systems were tested against a single independent AI agent to determine whether the success in other domains would translate into improved human-robot interaction performance.
  arXiv  Detail & Related papers  (2024-11-23T02:47:12Z)
• The AI Collaborator: Bridging Human-AI Interaction in Educational and Professional Settings [3.506120162002989]
  AI Collaborator, powered by OpenAI's GPT-4, is a groundbreaking tool designed for human-AI collaboration research. Its standout feature is the ability for researchers to create customized AI personas for diverse experimental setups. This functionality is essential for simulating various interpersonal dynamics in team settings.
  arXiv  Detail & Related papers  (2024-05-16T22:14:54Z)
• CACA Agent: Capability Collaboration based AI Agent [18.84686313298908]
  We propose CACA Agent (Capability Collaboration based AI Agent) using an open architecture inspired by service computing. CACA Agent integrates a set of collaborative capabilities to implement AI Agents, not only reducing the dependence on a single LLM. We present a demo to illustrate the operation and the application scenario extension of CACA Agent.
  arXiv  Detail & Related papers  (2024-03-22T11:42:47Z)
• The Rise and Potential of Large Language Model Based Agents: A Survey [91.71061158000953]
  Large language models (LLMs) are regarded as potential sparks for Artificial General Intelligence (AGI) We start by tracing the concept of agents from its philosophical origins to its development in AI, and explain why LLMs are suitable foundations for agents. We explore the extensive applications of LLM-based agents in three aspects: single-agent scenarios, multi-agent scenarios, and human-agent cooperation.
  arXiv  Detail & Related papers  (2023-09-14T17:12:03Z)
• ProAgent: Building Proactive Cooperative Agents with Large Language Models [89.53040828210945]
  ProAgent is a novel framework that harnesses large language models to create proactive agents. ProAgent can analyze the present state, and infer the intentions of teammates from observations. ProAgent exhibits a high degree of modularity and interpretability, making it easily integrated into various coordination scenarios.
  arXiv  Detail & Related papers  (2023-08-22T10:36:56Z)
• Improving Grounded Language Understanding in a Collaborative Environment by Interacting with Agents Through Help Feedback [42.19685958922537]
  We argue that human-AI collaboration should be interactive, with humans monitoring the work of AI agents and providing feedback that the agent can understand and utilize. In this work, we explore these directions using the challenging task defined by the IGLU competition, an interactive grounded language understanding task in a MineCraft-like world.
  arXiv  Detail & Related papers  (2023-04-21T05:37:59Z)
• CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society [58.04479313658851]
  This paper explores the potential of building scalable techniques to facilitate autonomous cooperation among communicative agents. We propose a novel communicative agent framework named role-playing. Our contributions include introducing a novel communicative agent framework, offering a scalable approach for studying the cooperative behaviors and capabilities of multi-agent systems.
  arXiv  Detail & Related papers  (2023-03-31T01:09:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.