Building Your Own Product Copilot: Challenges, Opportunities, and Needs

• URL: http://arxiv.org/abs/2312.14231v1
• Date: Thu, 21 Dec 2023 18:37:43 GMT
• Title: Building Your Own Product Copilot: Challenges, Opportunities, and Needs
• Authors: Chris Parnin, Gustavo Soares, Rahul Pandita, Sumit Gulwani, Jessica Rich, Austin Z. Henley
• Abstract summary: We interviewed 26 professional software engineers responsible for building product copilots at various companies. We found pain points at every step of the engineering process and the challenges that strained existing development practices.
• Score: 16.710056957807353
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A race is underway to embed advanced AI capabilities into products. These product copilots enable users to ask questions in natural language and receive relevant responses that are specific to the user's context. In fact, virtually every large technology company is looking to add these capabilities to their software products. However, for most software engineers, this is often their first encounter with integrating AI-powered technology. Furthermore, software engineering processes and tools have not caught up with the challenges and scale involved with building AI-powered applications. In this work, we present the findings of an interview study with 26 professional software engineers responsible for building product copilots at various companies. From our interviews, we found pain points at every step of the engineering process and the challenges that strained existing development practices. We then conducted group brainstorming sessions to collaborative on opportunities and tool designs for the broader software engineering community.

Related papers

• A Human Centric Requirements Engineering Framework for Assessing Github Copilot Output [0.0]
  GitHub Copilot introduces new challenges in how these software tools address human needs.<n>I analyzed GitHub Copilot's interaction with users through its chat interface.<n>I established a human-centered requirements framework with clear metrics to evaluate these qualities.
  arXiv  Detail & Related papers  (2025-08-05T21:33:23Z)
• A roadmap for AI in robotics [55.87087746398059]
  We are witnessing growing excitement in robotics at the prospect of leveraging the potential of AI to tackle some of the outstanding barriers to the full deployment of robots in our daily lives.<n>This article offers an assessment of what AI for robotics has achieved since the 1990s and proposes a short- and medium-term research roadmap listing challenges and promises.
  arXiv  Detail & Related papers  (2025-07-26T15:18:28Z)
• Code with Me or for Me? How Increasing AI Automation Transforms Developer Workflows [66.1850490474361]
  We conduct the first academic study to explore developer interactions with coding agents.<n>We evaluate two leading copilot and agentic coding assistants, GitHub Copilot and OpenHands.<n>Our results show agents have the potential to assist developers in ways that surpass copilots.
  arXiv  Detail & Related papers  (2025-07-10T20:12:54Z)
• Automated Testing of the GUI of a Real-Life Engineering Software using Large Language Models [45.498315114762484]
  Tests aim to determine unintuitive behavior of the software as it is presented to the end-user.<n>They provide valuable feedback for the development of the software, but are time-intensive to conduct.<n>We present GERALLT, a system that uses Large Language Models (LLMs) to perform exploratory tests of the Graphical User Interface (GUI) of a real-life engineering software.
  arXiv  Detail & Related papers  (2025-05-23T12:53:28Z)
• Challenges and Paths Towards AI for Software Engineering [55.95365538122656]
  We discuss progress in AI for software engineering in threefold manner. First, we provide a structured taxonomy of concrete tasks in AI for software engineering. Second, we outline several key bottlenecks that limit current approaches.
  arXiv  Detail & Related papers  (2025-03-28T17:17:57Z)
• Fundamental Risks in the Current Deployment of General-Purpose AI Models: What Have We (Not) Learnt From Cybersecurity? [60.629883024152576]
  Large Language Models (LLMs) have seen rapid deployment in a wide range of use cases. OpenAIs Altera are just a few examples of increased autonomy, data access, and execution capabilities. These methods come with a range of cybersecurity challenges.
  arXiv  Detail & Related papers  (2024-12-19T14:44:41Z)
• Overview of Current Challenges in Multi-Architecture Software Engineering and a Vision for the Future [0.0]
  The presented system architecture is based on the concept of dynamic, knowledge graph-based WebAssembly Twins. The resulting systems are to possess advanced autonomous capabilities, with full transparency and controllability by the end user.
  arXiv  Detail & Related papers  (2024-10-28T13:03:09Z)
• OpenHands: An Open Platform for AI Software Developers as Generalist Agents [109.8507367518992]
  We introduce OpenHands, a platform for the development of AI agents that interact with the world in similar ways to a human developer. We describe how the platform allows for the implementation of new agents, safe interaction with sandboxed environments for code execution, and incorporation of evaluation benchmarks.
  arXiv  Detail & Related papers  (2024-07-23T17:50:43Z)
• Bridging Gaps, Building Futures: Advancing Software Developer Diversity and Inclusion Through Future-Oriented Research [50.545824691484796]
  We present insights from SE researchers and practitioners on challenges and solutions regarding diversity and inclusion in SE. We share potential utopian and dystopian visions of the future and provide future research directions and implications for academia and industry.
  arXiv  Detail & Related papers  (2024-04-10T16:18:11Z)
• Embedded Software Development with Digital Twins: Specific Requirements for Small and Medium-Sized Enterprises [55.57032418885258]
  Digital twins have the potential for cost-effective software development and maintenance strategies. We interviewed SMEs about their current development processes. First results show that real-time requirements prevent, to date, a Software-in-the-Loop development approach.
  arXiv  Detail & Related papers  (2023-09-17T08:56:36Z)
• AI in Software Engineering: A Survey on Project Management Applications [3.156791351998142]
  Machine Learning (ML) employs algorithms that undergo training on data sets, enabling them to carry out specific tasks autonomously. AI holds immense potential in the field of software engineering, particularly in project management and planning.
  arXiv  Detail & Related papers  (2023-07-27T23:02:24Z)
• Artificial Intelligence Impact On The Labour Force -- Searching For The Analytical Skills Of The Future Software Engineers [0.0]
  This systematic literature review aims to investigate the impact of artificial intelligence on the labour force in software engineering. It focuses on the skills needed for future software engineers, the impact of AI on the demand for software engineering skills, and the future of work for software engineers.
  arXiv  Detail & Related papers  (2023-02-26T03:49:53Z)
• Enabling Automated Machine Learning for Model-Driven AI Engineering [60.09869520679979]
  We propose a novel approach to enable Model-Driven Software Engineering and Model-Driven AI Engineering. In particular, we support Automated ML, thus assisting software engineers without deep AI knowledge in developing AI-intensive systems.
  arXiv  Detail & Related papers  (2022-03-06T10:12:56Z)
• Explainable AI for Software Engineering [12.552048647904591]
  We first highlight the need for explainable AI in software engineering. Then, we summarize three successful case studies on how explainable AI techniques can be used to address the aforementioned challenges.
  arXiv  Detail & Related papers  (2020-12-03T00:42:29Z)
• A Reference Software Architecture for Social Robots [64.86618385090416]
  We propose a series of principles that social robots may benefit from. These principles lay also the foundations for the design of a reference software architecture for Social Robots.
  arXiv  Detail & Related papers  (2020-07-09T17:03:21Z)
• Machine Learning for Software Engineering: A Systematic Mapping [73.30245214374027]
  The software development industry is rapidly adopting machine learning for transitioning modern day software systems towards highly intelligent and self-learning systems. No comprehensive study exists that explores the current state-of-the-art on the adoption of machine learning across software engineering life cycle stages. This study introduces a machine learning for software engineering (MLSE) taxonomy classifying the state-of-the-art machine learning techniques according to their applicability to various software engineering life cycle stages.
  arXiv  Detail & Related papers  (2020-05-27T11:56:56Z)
• Understanding What Software Engineers Are Working on -- The Work-Item Prediction Challenge [0.0]
  Understanding what a software engineer (a developer, an incident responder, a production engineer, etc.) is working on is a challenging problem. In this paper, we explain the corresponding 'work-item prediction challenge' on the grounds of representative scenarios.
  arXiv  Detail & Related papers  (2020-04-13T19:59:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.