A Taxonomy of Foundation Model based Systems through the Lens of
Software Architecture
- URL: http://arxiv.org/abs/2305.05352v6
- Date: Mon, 22 Jan 2024 04:15:13 GMT
- Title: A Taxonomy of Foundation Model based Systems through the Lens of
Software Architecture
- Authors: Qinghua Lu, Liming Zhu, Xiwei Xu, Yue Liu, Zhenchang Xing, Jon Whittle
- Abstract summary: We propose a taxonomy of foundation model based systems, which classifies and compares the characteristics of foundation models and design options.
Our taxonomy comprises three categories: the pretraining and adaptation of foundation models, the architecture design of foundation model based systems, and responsible-AI-by-design.
- Score: 35.20191493188642
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The recent release of large language model (LLM) based chatbots, such as
ChatGPT, has attracted huge interest in foundation models. It is widely
believed that foundation models will serve as the fundamental building blocks
for future AI systems. As foundation models are in their early stages, the
design of foundation model based systems has not yet been systematically
explored. There is limited understanding about the impact of introducing
foundation models in software architecture. Therefore, in this paper, we
propose a taxonomy of foundation model based systems, which classifies and
compares the characteristics of foundation models and design options of
foundation model based systems. Our taxonomy comprises three categories: the
pretraining and adaptation of foundation models, the architecture design of
foundation model based systems, and responsible-AI-by-design. This taxonomy can
serve as concrete guidance for making major architectural design decisions when
designing foundation model based systems and highlights trade-offs arising from
design decisions.
Related papers
- Exploring the design space of deep-learning-based weather forecasting systems [56.129148006412855]
This paper systematically analyzes the impact of different design choices on deep-learning-based weather forecasting systems.
We study fixed-grid architectures such as UNet, fully convolutional architectures, and transformer-based models.
We propose a hybrid system that combines the strong performance of fixed-grid models with the flexibility of grid-invariant architectures.
arXiv Detail & Related papers (2024-10-09T22:25:50Z) - High-Performance Few-Shot Segmentation with Foundation Models: An Empirical Study [64.06777376676513]
We develop a few-shot segmentation (FSS) framework based on foundation models.
To be specific, we propose a simple approach to extract implicit knowledge from foundation models to construct coarse correspondence.
Experiments on two widely used datasets demonstrate the effectiveness of our approach.
arXiv Detail & Related papers (2024-09-10T08:04:11Z) - An overview of domain-specific foundation model: key technologies, applications and challenges [25.146374189455784]
This article provides a timely and thorough overview of the methodology for customizing domain-specific foundation models.
It introduces basic concepts, outlines the general architecture, and surveys key methods for constructing domain-specific models.
arXiv Detail & Related papers (2024-09-06T13:24:22Z) - A Taxonomy of Architecture Options for Foundation Model-based Agents: Analysis and Decision Model [25.78239568393706]
This paper introduces a taxonomy focused on the architectures of foundation-model-based agents.
By unifying and detailing these classifications, our taxonomy aims to improve the design of foundation-model-based agents.
arXiv Detail & Related papers (2024-08-06T03:10:52Z) - Agent Design Pattern Catalogue: A Collection of Architectural Patterns for Foundation Model based Agents [22.94671478021277]
Foundation model-enabled generative artificial intelligence facilitates the development and implementation of agents.
There is a lack of systematic knowledge to guide practitioners in designing the agents.
We present a pattern catalogue consisting of 18 architectural patterns with analyses of the context, forces, and trade-offs.
arXiv Detail & Related papers (2024-05-16T23:24:48Z) - Towards Responsible Generative AI: A Reference Architecture for Designing Foundation Model based Agents [28.406492378232695]
Foundation model based agents derive their autonomy from the capabilities of foundation models.
This paper presents a pattern-oriented reference architecture that serves as guidance when designing foundation model based agents.
arXiv Detail & Related papers (2023-11-22T04:21:47Z) - ZhiJian: A Unifying and Rapidly Deployable Toolbox for Pre-trained Model
Reuse [59.500060790983994]
This paper introduces ZhiJian, a comprehensive and user-friendly toolbox for model reuse, utilizing the PyTorch backend.
ZhiJian presents a novel paradigm that unifies diverse perspectives on model reuse, encompassing target architecture construction with PTM, tuning target model with PTM, and PTM-based inference.
arXiv Detail & Related papers (2023-08-17T19:12:13Z) - A Reference Architecture for Designing Foundation Model based Systems [28.826700360670515]
There is a broad consensus that foundations models will be the fundamental building blocks for future AI systems.
incorporating foundations models into AI systems raises significant concerns about responsible and safe AI.
The paper proposes a pattern-oriented reference architecture for designing responsible foundation-model-based systems.
arXiv Detail & Related papers (2023-04-13T05:01:03Z) - Towards Efficient Task-Driven Model Reprogramming with Foundation Models [52.411508216448716]
Vision foundation models exhibit impressive power, benefiting from the extremely large model capacity and broad training data.
However, in practice, downstream scenarios may only support a small model due to the limited computational resources or efficiency considerations.
This brings a critical challenge for the real-world application of foundation models: one has to transfer the knowledge of a foundation model to the downstream task.
arXiv Detail & Related papers (2023-04-05T07:28:33Z) - On the Opportunities and Risks of Foundation Models [256.61956234436553]
We call these models foundation models to underscore their critically central yet incomplete character.
This report provides a thorough account of the opportunities and risks of foundation models.
To tackle these questions, we believe much of the critical research on foundation models will require deep interdisciplinary collaboration.
arXiv Detail & Related papers (2021-08-16T17:50:08Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.