Software Performance Engineering for Foundation Model-Powered Software (FMware)

• URL: http://arxiv.org/abs/2411.09580v1
• Date: Thu, 14 Nov 2024 16:42:19 GMT
• Title: Software Performance Engineering for Foundation Model-Powered Software (FMware)
• Authors: Haoxiang Zhang, Shi Chang, Arthur Leung, Kishanthan Thangarajah, Boyuan Chen, Hanan Lutfiyya, Ahmed E. Hassan,
• Abstract summary: Foundation Models (FMs) like Large Language Models (LLMs) are revolutionizing software development. This paper highlights the significance of Software Performance Engineering (SPE) in FMware. We identify four key challenges: cognitive architecture design, communication protocols, tuning and optimization, and deployment.
• Score: 6.283211168007636
• License:
• Abstract: The rise of Foundation Models (FMs) like Large Language Models (LLMs) is revolutionizing software development. Despite the impressive prototypes, transforming FMware into production-ready products demands complex engineering across various domains. A critical but overlooked aspect is performance engineering, which aims at ensuring FMware meets performance goals such as throughput and latency to avoid user dissatisfaction and financial loss. Often, performance considerations are an afterthought, leading to costly optimization efforts post-deployment. FMware's high computational resource demands highlight the need for efficient hardware use. Continuous performance engineering is essential to prevent degradation. This paper highlights the significance of Software Performance Engineering (SPE) in FMware, identifying four key challenges: cognitive architecture design, communication protocols, tuning and optimization, and deployment. These challenges are based on literature surveys and experiences from developing an in-house FMware system. We discuss problems, current practices, and innovative paths for the software engineering community.

Related papers

• Lingma SWE-GPT: An Open Development-Process-Centric Language Model for Automated Software Improvement [62.94719119451089]
  Lingma SWE-GPT series learns from and simulating real-world code submission activities. Lingma SWE-GPT 72B resolves 30.20% of GitHub issues, marking a significant improvement in automatic issue resolution.
  arXiv  Detail & Related papers  (2024-11-01T14:27:16Z)
• From Cool Demos to Production-Ready FMware: Core Challenges and a Technology Roadmap [12.313710667597897]
  The rapid expansion of foundation models (FMs) has given rise to FMware--software systems that integrate FMs as core components. transitioning to production-ready systems presents numerous challenges, including reliability, high implementation costs, scalability, and compliance with privacy regulations. We identify critical issues in FM selection, data and model alignment, prompt engineering, agent orchestration, system testing, and deployment, alongside cross-cutting concerns such as memory management, observability, and feedback integration.
  arXiv  Detail & Related papers  (2024-10-28T07:16:00Z)
• Foundation Model Engineering: Engineering Foundation Models Just as Engineering Software [8.14005646330662]
  Foundation Models (FMs) become a new type of software by treating data and models as the source code. We outline our vision of introducing Foundation Model (FM) engineering, a strategic response to the anticipated FM crisis.
  arXiv  Detail & Related papers  (2024-07-11T04:40:02Z)
• Agent-Driven Automatic Software Improvement [55.2480439325792]
  This research proposal aims to explore innovative solutions by focusing on the deployment of agents powered by Large Language Models (LLMs) The iterative nature of agents, which allows for continuous learning and adaptation, can help surpass common challenges in code generation. We aim to use the iterative feedback in these systems to further fine-tune the LLMs underlying the agents, becoming better aligned to the task of automated software improvement.
  arXiv  Detail & Related papers  (2024-06-24T15:45:22Z)
• Prioritizing Software Requirements Using Large Language Models [3.9422957660677476]
  This article focuses on requirements engineering, typically seen as the initial phase of software development. The challenge of identifying requirements and satisfying all stakeholders within time and budget constraints remains significant. This study introduces a web-based software tool utilizing AI agents and prompt engineering to automate task prioritization.
  arXiv  Detail & Related papers  (2024-04-05T15:20:56Z)
• Rethinking Software Engineering in the Foundation Model Era: A Curated Catalogue of Challenges in the Development of Trustworthy FMware [13.21876203209586]
  We identify 10 key SE4FMware challenges that have caused enterprise FMware development to be unproductive, costly, and risky. We present FMArts, which is our long-term effort towards creating a cradle-to-grave platform for the engineering of trustworthy FMware.
  arXiv  Detail & Related papers  (2024-02-25T00:53:16Z)
• Large Language Models for Software Engineering: Survey and Open Problems [35.29302720251483]
  This paper provides a survey of the emerging area of Large Language Models (LLMs) for Software Engineering (SE) Our survey reveals the pivotal role that hybrid techniques (traditional SE plus LLMs) have to play in the development and deployment of reliable, efficient and effective LLM-based SE.
  arXiv  Detail & Related papers  (2023-10-05T13:33:26Z)
• Embedded Software Development with Digital Twins: Specific Requirements for Small and Medium-Sized Enterprises [55.57032418885258]
  Digital twins have the potential for cost-effective software development and maintenance strategies. We interviewed SMEs about their current development processes. First results show that real-time requirements prevent, to date, a Software-in-the-Loop development approach.
  arXiv  Detail & Related papers  (2023-09-17T08:56:36Z)
• Empowered and Embedded: Ethics and Agile Processes [60.63670249088117]
  We argue that ethical considerations need to be embedded into the (agile) software development process. We put emphasis on the possibility to implement ethical deliberations in already existing and well established agile software development processes.
  arXiv  Detail & Related papers  (2021-07-15T11:14:03Z)
• Technology Readiness Levels for Machine Learning Systems [107.56979560568232]
  Development and deployment of machine learning systems can be executed easily with modern tools, but the process is typically rushed and means-to-an-end. We have developed a proven systems engineering approach for machine learning development and deployment. Our "Machine Learning Technology Readiness Levels" framework defines a principled process to ensure robust, reliable, and responsible systems.
  arXiv  Detail & Related papers  (2021-01-11T15:54:48Z)
• Technology Readiness Levels for AI & ML [79.22051549519989]
  Development of machine learning systems can be executed easily with modern tools, but the process is typically rushed and means-to-an-end. Engineering systems follow well-defined processes and testing standards to streamline development for high-quality, reliable results. We propose a proven systems engineering approach for machine learning development and deployment.
  arXiv  Detail & Related papers  (2020-06-21T17:14:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.