A Taxonomy of Foundation Model based Systems through the Lens of
Software Architecture
- URL: http://arxiv.org/abs/2305.05352v6
- Date: Mon, 22 Jan 2024 04:15:13 GMT
- Title: A Taxonomy of Foundation Model based Systems through the Lens of
Software Architecture
- Authors: Qinghua Lu, Liming Zhu, Xiwei Xu, Yue Liu, Zhenchang Xing, Jon Whittle
- Abstract summary: We propose a taxonomy of foundation model based systems, which classifies and compares the characteristics of foundation models and design options.
Our taxonomy comprises three categories: the pretraining and adaptation of foundation models, the architecture design of foundation model based systems, and responsible-AI-by-design.
- Score: 35.20191493188642
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The recent release of large language model (LLM) based chatbots, such as
ChatGPT, has attracted huge interest in foundation models. It is widely
believed that foundation models will serve as the fundamental building blocks
for future AI systems. As foundation models are in their early stages, the
design of foundation model based systems has not yet been systematically
explored. There is limited understanding about the impact of introducing
foundation models in software architecture. Therefore, in this paper, we
propose a taxonomy of foundation model based systems, which classifies and
compares the characteristics of foundation models and design options of
foundation model based systems. Our taxonomy comprises three categories: the
pretraining and adaptation of foundation models, the architecture design of
foundation model based systems, and responsible-AI-by-design. This taxonomy can
serve as concrete guidance for making major architectural design decisions when
designing foundation model based systems and highlights trade-offs arising from
design decisions.
Related papers
- Understanding the differences in Foundation Models: Attention, State Space Models, and Recurrent Neural Networks [50.29356570858905]
We introduce the Dynamical Systems Framework (DSF), which allows a principled investigation of all these architectures in a common representation.
We provide principled comparisons between softmax attention and other model classes, discussing the theoretical conditions under which softmax attention can be approximated.
This shows the DSF's potential to guide the systematic development of future more efficient and scalable foundation models.
arXiv Detail & Related papers (2024-05-24T17:19:57Z) - Agent Design Pattern Catalogue: A Collection of Architectural Patterns for Foundation Model based Agents [22.94671478021277]
Foundation model-enabled generative artificial intelligence facilitates the development and implementation of agents.
We present a pattern catalogue consisting of 17 architectural patterns with analyses of the context, forces, and trade-offs.
arXiv Detail & Related papers (2024-05-16T23:24:48Z) - On a Foundation Model for Operating Systems [36.17124600729315]
This paper lays down the research agenda for a domain-specific foundation model for operating systems (OSes)
Our case for a foundation model revolves around the observations that several OS components such as CPU, memory, and network subsystems are interrelated.
OS traces offer the ideal dataset for a foundation model to grasp the intricacies of diverse OS components.
arXiv Detail & Related papers (2023-12-13T00:23:22Z) - Towards Responsible Generative AI: A Reference Architecture for Designing Foundation Model based Agents [28.406492378232695]
Foundation model based agents derive their autonomy from the capabilities of foundation models.
This paper presents a pattern-oriented reference architecture that serves as guidance when designing foundation model based agents.
arXiv Detail & Related papers (2023-11-22T04:21:47Z) - Compositional Foundation Models for Hierarchical Planning [52.18904315515153]
We propose a foundation model which leverages expert foundation model trained on language, vision and action data individually together to solve long-horizon tasks.
We use a large language model to construct symbolic plans that are grounded in the environment through a large video diffusion model.
Generated video plans are then grounded to visual-motor control, through an inverse dynamics model that infers actions from generated videos.
arXiv Detail & Related papers (2023-09-15T17:44:05Z) - ZhiJian: A Unifying and Rapidly Deployable Toolbox for Pre-trained Model
Reuse [59.500060790983994]
This paper introduces ZhiJian, a comprehensive and user-friendly toolbox for model reuse, utilizing the PyTorch backend.
ZhiJian presents a novel paradigm that unifies diverse perspectives on model reuse, encompassing target architecture construction with PTM, tuning target model with PTM, and PTM-based inference.
arXiv Detail & Related papers (2023-08-17T19:12:13Z) - A Reference Architecture for Designing Foundation Model based Systems [28.826700360670515]
There is a broad consensus that foundations models will be the fundamental building blocks for future AI systems.
incorporating foundations models into AI systems raises significant concerns about responsible and safe AI.
The paper proposes a pattern-oriented reference architecture for designing responsible foundation-model-based systems.
arXiv Detail & Related papers (2023-04-13T05:01:03Z) - Towards Efficient Task-Driven Model Reprogramming with Foundation Models [52.411508216448716]
Vision foundation models exhibit impressive power, benefiting from the extremely large model capacity and broad training data.
However, in practice, downstream scenarios may only support a small model due to the limited computational resources or efficiency considerations.
This brings a critical challenge for the real-world application of foundation models: one has to transfer the knowledge of a foundation model to the downstream task.
arXiv Detail & Related papers (2023-04-05T07:28:33Z) - Decision Models for Selecting Federated Learning Architecture Patterns [7.468413169676602]
We present a set of decision models for the selection of patterns for federated machine learning architecture design.
Each decision model maps functional and non-functional requirements of federated machine learning systems to a set of patterns.
arXiv Detail & Related papers (2022-04-28T05:07:26Z) - On the Opportunities and Risks of Foundation Models [256.61956234436553]
We call these models foundation models to underscore their critically central yet incomplete character.
This report provides a thorough account of the opportunities and risks of foundation models.
To tackle these questions, we believe much of the critical research on foundation models will require deep interdisciplinary collaboration.
arXiv Detail & Related papers (2021-08-16T17:50:08Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.