What Slows Down FMware Development? An Empirical Study of Developer Challenges and Resolution Times

• URL: http://arxiv.org/abs/2510.11138v1
• Date: Mon, 13 Oct 2025 08:26:48 GMT
• Title: What Slows Down FMware Development? An Empirical Study of Developer Challenges and Resolution Times
• Authors: Zitao Wang, Zhimin Zhao, Michael W. Godfrey,
• Abstract summary: This study is the first large-scale analysis of FMware development across cloud-based platforms and open-source repositories.<n>We investigate the FMware ecosystem through three focus areas: (1) the most common application domains of FMware, (2) the key challenges developers encounter, and (3) the types of issues that demand the greatest effort to resolve.<n>Our findings reveal a strong focus on education, content creation, and business strategy, alongside persistent technical challenges in memory management, dependency handling, and tokenizer configuration.
• Score: 2.451770418371169
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Foundation Models (FMs), such as OpenAI's GPT, are fundamentally transforming the practice of software engineering by enabling the development of \emph{FMware} -- applications and infrastructures built around these models. FMware systems now support tasks such as code generation, natural-language interaction, knowledge integration, and multi-modal content creation, underscoring their disruptive impact on current software engineering workflows. However, the design, implementation, and evolution of FMware present significant new challenges, particularly across cloud-based and on-premise platforms where goals, processes, and tools often diverge from those of traditional software development. To our knowledge, this is the first large-scale analysis of FMware development across both cloud-based platforms and open-source repositories. We empirically investigate the FMware ecosystem through three focus areas: (1) the most common application domains of FMware, (2) the key challenges developers encounter, and (3) the types of issues that demand the greatest effort to resolve. Our analysis draws on data from GitHub repositories and from leading FMware platforms, including HuggingFace, GPTStore, Ora, and Poe. Our findings reveal a strong focus on education, content creation, and business strategy, alongside persistent technical challenges in memory management, dependency handling, and tokenizer configuration. On GitHub, bug reports and core functionality issues are the most frequently reported problems, while code review, similarity search, and prompt template design are the most time-consuming to resolve. By uncovering developer practices and pain points, this study points to opportunities to improve FMware tools, workflows, and community support, and provides actionable insights to help guide the future of FMware development.

Related papers

• Advances and Frontiers of LLM-based Issue Resolution in Software Engineering: A Comprehensive Survey [59.3507264893654]
  Issue resolution is a complex Software Engineering task integral to real-world development.<n> benchmarks like SWE-bench revealed this task as profoundly difficult for large language models.<n>This paper presents a systematic survey of this emerging domain.
  arXiv  Detail & Related papers  (2026-01-15T18:55:03Z)
• Hierarchical Federated Foundation Models over Wireless Networks for Multi-Modal Multi-Task Intelligence: Integration of Edge Learning with D2D/P2P-Enabled Fog Learning Architectures [58.72593025539547]
  In this paper, we unveil an unexplored variation of M3T FFMs by proposing hierarchical federated foundation models (HF-FMs)<n>HF-FMs strategically align the modular structure of M3T FMs, comprising modality encoders, prompts, mixture-of-experts (MoEs), adapters, and task heads.<n>To demonstrate their potential, we prototype HF-FMs in a wireless network setting and release the open-source code for the development of HF-FMs.
  arXiv  Detail & Related papers  (2025-09-03T20:23:19Z)
• A Survey on Code Generation with LLM-based Agents [61.474191493322415]
  Code generation agents powered by large language models (LLMs) are revolutionizing the software development paradigm.<n>LLMs are characterized by three core features.<n>This paper presents a systematic survey of the field of LLM-based code generation agents.
  arXiv  Detail & Related papers  (2025-07-31T18:17:36Z)
• The Hitchhikers Guide to Production-ready Trustworthy Foundation Model powered Software (FMware) [10.438253230778844]
  Foundation Models (FMs) are reshaping the software industry by enabling FMware, systems that integrate these FMs as core components.<n>In this KDD 2025 tutorial, we present a comprehensive exploration of FMware that combines a curated catalogue of challenges with real-world production concerns.
  arXiv  Detail & Related papers  (2025-05-15T18:22:45Z)
• Foundational Models for 3D Point Clouds: A Survey and Outlook [50.61473863985571]
  3D point cloud representation plays a crucial role in preserving the geometric fidelity of the physical world.<n>To bridge this gap, it becomes essential to incorporate multiple modalities.<n>Foundation models (FMs) can seamlessly integrate and reason across these modalities.
  arXiv  Detail & Related papers  (2025-01-30T18:59:43Z)
• Software Performance Engineering for Foundation Model-Powered Software (FMware) [6.283211168007636]
  Foundation Models (FMs) like Large Language Models (LLMs) are revolutionizing software development. This paper highlights the significance of Software Performance Engineering (SPE) in FMware. We identify four key challenges: cognitive architecture design, communication protocols, tuning and optimization, and deployment.
  arXiv  Detail & Related papers  (2024-11-14T16:42:19Z)
• From Cool Demos to Production-Ready FMware: Core Challenges and a Technology Roadmap [12.313710667597897]
  We conduct a semi-structured thematic synthesis to identify the key challenges in productionizing FMware across diverse data sources.<n>We identify critical issues in FM selection, data and model alignment, prompt engineering, agent orchestration, system testing, and deployment.<n>We discuss needed technologies and strategies to address these challenges and offer guidance on how to enable the transition from demonstration systems to scalable, production-ready FMware solutions.
  arXiv  Detail & Related papers  (2024-10-28T07:16:00Z)
• Foundation Model Engineering: Engineering Foundation Models Just as Engineering Software [8.14005646330662]
  Foundation Models (FMs) become a new type of software by treating data and models as the source code. We outline our vision of introducing Foundation Model (FM) engineering, a strategic response to the anticipated FM crisis.
  arXiv  Detail & Related papers  (2024-07-11T04:40:02Z)
• Rethinking Software Engineering in the Foundation Model Era: A Curated Catalogue of Challenges in the Development of Trustworthy FMware [13.21876203209586]
  We identify 10 key SE4FMware challenges that have caused enterprise FMware development to be unproductive, costly, and risky. We present FMArts, which is our long-term effort towards creating a cradle-to-grave platform for the engineering of trustworthy FMware.
  arXiv  Detail & Related papers  (2024-02-25T00:53:16Z)
• Embedded Software Development with Digital Twins: Specific Requirements for Small and Medium-Sized Enterprises [55.57032418885258]
  Digital twins have the potential for cost-effective software development and maintenance strategies. We interviewed SMEs about their current development processes. First results show that real-time requirements prevent, to date, a Software-in-the-Loop development approach.
  arXiv  Detail & Related papers  (2023-09-17T08:56:36Z)
• ChatDev: Communicative Agents for Software Development [84.90400377131962]
  ChatDev is a chat-powered software development framework in which specialized agents are guided in what to communicate. These agents actively contribute to the design, coding, and testing phases through unified language-based communication.
  arXiv  Detail & Related papers  (2023-07-16T02:11:34Z)
• The GitHub Development Workflow Automation Ecosystems [47.818229204130596]
  Large-scale software development has become a highly collaborative endeavour. This chapter explores the ecosystems of development bots and GitHub Actions. It provides an extensive survey of the state-of-the-art in this domain.
  arXiv  Detail & Related papers  (2023-05-08T15:24:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.