Design Patterns for AI-based Systems: A Multivocal Literature Review and Pattern Repository

• URL: http://arxiv.org/abs/2303.13173v1
• Date: Thu, 23 Mar 2023 10:57:11 GMT
• Title: Design Patterns for AI-based Systems: A Multivocal Literature Review and Pattern Repository
• Abstract summary: This study provides an overview of design patterns for AI-based systems, both new and adapted ones. New patterns for AI-based systems are emerging, but existing patterns have also been adapted to this new context. Our results can be used by researchers as a foundation for follow-up studies and by practitioners to discover relevant patterns for informing the design of AI-based systems.
• Score: 6.184279908456794
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Systems with artificial intelligence components, so-called AI-based systems, have gained considerable attention recently. However, many organizations have issues with achieving production readiness with such systems. As a means to improve certain software quality attributes and to address frequently occurring problems, design patterns represent proven solution blueprints. While new patterns for AI-based systems are emerging, existing patterns have also been adapted to this new context. The goal of this study is to provide an overview of design patterns for AI-based systems, both new and adapted ones. We want to collect and categorize patterns, and make them accessible for researchers and practitioners. To this end, we first performed a multivocal literature review (MLR) to collect design patterns used with AI-based systems. We then integrated the created pattern collection into a web-based pattern repository to make the patterns browsable and easy to find. As a result, we selected 51 resources (35 white and 16 gray ones), from which we extracted 70 unique patterns used for AI-based systems. Among these are 34 new patterns and 36 traditional ones that have been adapted to this context. Popular pattern categories include "architecture" (25 patterns), "deployment" (16), "implementation" (9), or "security & safety" (9). While some patterns with four or more mentions already seem established, the majority of patterns have only been mentioned once or twice (51 patterns). Our results in this emerging field can be used by researchers as a foundation for follow-up studies and by practitioners to discover relevant patterns for informing the design of AI-based systems.

Related papers

• A Methodology for Investigating AI Patterns Prevalence in Software Repositories [1.3077849036833011]
  We identify 14 AI pattern classes by mining 44 published AI pattern-related sources.<n>We then use an active learning approach to determine the prevalence of the most common pattern class across 100 GitHub open AI repositories.
  arXiv  Detail & Related papers  (2026-07-01T07:46:06Z)
• Barbarians at the Gate: How AI is Upending Systems Research [58.95406995634148]
  We argue that systems research, long focused on designing and evaluating new performance-oriented algorithms, is particularly well-suited for AI-driven solution discovery.<n>We term this approach as AI-Driven Research for Systems ( ADRS), which iteratively generates, evaluates, and refines solutions.<n>Our results highlight both the disruptive potential and the urgent need to adapt systems research practices in the age of AI.
  arXiv  Detail & Related papers  (2025-10-07T17:49:24Z)
• Continual Learning for Generative AI: From LLMs to MLLMs and Beyond [56.29231194002407]
  We present a comprehensive survey of continual learning methods for mainstream generative AI models.<n>We categorize these approaches into three paradigms: architecture-based, regularization-based, and replay-based.<n>We analyze continual learning setups for different generative models, including training objectives, benchmarks, and core backbones.
  arXiv  Detail & Related papers  (2025-06-16T02:27:25Z)
• Applying Cognitive Design Patterns to General LLM Agents [4.055489363682198]
  This paper outlines a few recurring cognitive design patterns that have appeared in various pre-transformer AI architectures.<n>We then explore how these patterns are evident in systems using large language models (LLMs)<n>Examining and applying these recurring patterns enables predictions of gaps or deficiencies in today's Agentic LLM Systems.
  arXiv  Detail & Related papers  (2025-05-11T18:29:54Z)
• The Method for Storing Patterns in Neural Networks-Memorization and Recall of QR code Patterns- [0.0]
  We propose a mechanism for storing complex patterns within a neural network and subsequently recalling them. The advantage of storing patterns in a neural network lies in its ability to recall the original pattern even when an incomplete version is presented.
  arXiv  Detail & Related papers  (2025-04-09T07:09:40Z)
• A Survey of Circuit Foundation Model: Foundation AI Models for VLSI Circuit Design and EDA [13.026106814794696]
  Foundation AI models for circuits have emerged as a new technology trend. New AI models are developed through two stages: 1) self-supervised pre-training on a large amount of unlabeled data to learn intrinsic circuit properties; and 2) efficient fine-tuning for specific downstream applications.
  arXiv  Detail & Related papers  (2025-03-28T07:27:27Z)
• Unlocking the Potential of Past Research: Using Generative AI to Reconstruct Healthcare Simulation Models [0.0]
  This study explores the feasibility of using generative artificial intelligence (AI) to recreate published models using Free and Open Source Software (FOSS) We successfully generated, tested and internally reproduced two DES models, including user interfaces. The reported results were replicated for one model, but not the other, likely due to missing information on distributions.
  arXiv  Detail & Related papers  (2025-03-27T16:10:02Z)
• A Survey of Model Architectures in Information Retrieval [59.61734783818073]
  The period from 2019 to the present has represented one of the biggest paradigm shifts in information retrieval (IR) and natural language processing (NLP)<n>We trace the development from traditional term-based methods to modern neural approaches, particularly highlighting the impact of transformer-based models and subsequent large language models (LLMs)<n>We conclude with a forward-looking discussion of emerging challenges and future directions.
  arXiv  Detail & Related papers  (2025-02-20T18:42:58Z)
• AI-Aided Kalman Filters [65.35350122917914]
  The Kalman filter (KF) and its variants are among the most celebrated algorithms in signal processing. Recent developments illustrate the possibility of fusing deep neural networks (DNNs) with classic Kalman-type filtering. This article provides a tutorial-style overview of design approaches for incorporating AI in aiding KF-type algorithms.
  arXiv  Detail & Related papers  (2024-10-16T06:47:53Z)
• Computer Vision Model Compression Techniques for Embedded Systems: A Survey [75.38606213726906]
  This paper covers the main model compression techniques applied for computer vision tasks. We present the characteristics of compression subareas, compare different approaches, and discuss how to choose the best technique. We also share codes to assist researchers and new practitioners in overcoming initial implementation challenges.
  arXiv  Detail & Related papers  (2024-08-15T16:41:55Z)
• Generalization Properties of Retrieval-based Models [50.35325326050263]
  Retrieval-based machine learning methods have enjoyed success on a wide range of problems. Despite growing literature showcasing the promise of these models, the theoretical underpinning for such models remains underexplored. We present a formal treatment of retrieval-based models to characterize their generalization ability.
  arXiv  Detail & Related papers  (2022-10-06T00:33:01Z)
• Artefact Retrieval: Overview of NLP Models with Knowledge Base Access [18.098224374478598]
  This paper systematically describes the typology of artefacts (items retrieved from a knowledge base), retrieval mechanisms and the way these artefacts are fused into the model. Most of the focus is given to language models, though we also show how question answering, fact-checking and dialogue models fit into this system as well.
  arXiv  Detail & Related papers  (2022-01-24T13:15:33Z)
• Redefining Neural Architecture Search of Heterogeneous Multi-Network Models by Characterizing Variation Operators and Model Components [71.03032589756434]
  We investigate the effect of different variation operators in a complex domain, that of multi-network heterogeneous neural models. We characterize both the variation operators, according to their effect on the complexity and performance of the model; and the models, relying on diverse metrics which estimate the quality of the different parts composing it.
  arXiv  Detail & Related papers  (2021-06-16T17:12:26Z)
• Characterizing Technical Debt and Antipatterns in AI-Based Systems: A Systematic Mapping Study [14.437695080681259]
  The goal of our study is to provide a clear overview and characterization of the types of Technical Debt (TD) that appear in AI-based systems. Our results show that (i) established TD types, variations of them, and four new TD types (data, model, configuration, and ethics debt) are present in AI-based systems. Our results can support AI professionals with reasoning about and communicating aspects of TD present in their systems.
  arXiv  Detail & Related papers  (2021-03-17T17:11:43Z)
• Modular Design Patterns for Hybrid Learning and Reasoning Systems: a taxonomy, patterns and use cases [0.3849857432787594]
  unification of statistical (data-driven) and symbolic (knowledge-driven) methods is one of the key challenges of modern AI. Recent years have seen large number of publications on such hybrid neuro-symbolic AI systems. We propose a set of modular design patterns for such hybrid, neuro-symbolic systems.
  arXiv  Detail & Related papers  (2021-02-23T22:16:05Z)
• Architectural Patterns for the Design of Federated Learning Systems [12.330671239159102]
  Federated learning has received fast-growing interests from academia and industry to tackle the challenges of data hungriness and privacy in machine learning. This paper presents a collection of architectural patterns to deal with the design challenges of federated learning systems.
  arXiv  Detail & Related papers  (2021-01-07T05:11:09Z)
• Few-Shot Named Entity Recognition: A Comprehensive Study [92.40991050806544]
  We investigate three schemes to improve the model generalization ability for few-shot settings. We perform empirical comparisons on 10 public NER datasets with various proportions of labeled data. We create new state-of-the-art results on both few-shot and training-free settings.
  arXiv  Detail & Related papers  (2020-12-29T23:43:16Z)
• Hierarchically Organized Latent Modules for Exploratory Search in Morphogenetic Systems [21.23182328329019]
  We introduce a novel dynamic and modular architecture that enables unsupervised learning of a hierarchy of diverse representations. We show that this system forms a discovery assistant that can efficiently adapt its diversity search towards preferences of a user.
  arXiv  Detail & Related papers  (2020-07-02T15:28:27Z)
• Neural Entity Linking: A Survey of Models Based on Deep Learning [82.43751915717225]
  This survey presents a comprehensive description of recent neural entity linking (EL) systems developed since 2015. Its goal is to systemize design features of neural entity linking systems and compare their performance to the remarkable classic methods on common benchmarks. The survey touches on applications of entity linking, focusing on the recently emerged use-case of enhancing deep pre-trained masked language models.
  arXiv  Detail & Related papers  (2020-05-31T18:02:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.