A quantitative framework for evaluating architectural patterns in ML systems

• URL: http://arxiv.org/abs/2501.11543v1
• Date: Mon, 20 Jan 2025 15:30:09 GMT
• Title: A quantitative framework for evaluating architectural patterns in ML systems
• Authors: Simeon Emanuilov, Aleksandar Dimov,
• Abstract summary: This study proposes a framework for quantitative assessment of architectural patterns in ML systems. We focus on scalability and performance metrics for cost-effective CPU-based inference.
• Score: 49.1574468325115
• License:
• Abstract: Contemporary intelligent systems incorporate software components, including machine learning components. As they grow in complexity and data volume such machine learning systems face unique quality challenges like scalability and performance. To overcome them, engineers may often use specific architectural patterns, however their impact on ML systems is difficult to quantify. The effect of software architecture on traditional systems is well studied, however more work is needed in the area of machine learning systems. This study proposes a framework for quantitative assessment of architectural patterns in ML systems, focusing on scalability and performance metrics for cost-effective CPU-based inference. We integrate these metrics into a systematic evaluation process for selection of architectural patterns and demonstrate its application through a case study. The approach shown in the paper should enable software architects to objectively analyze and select optimal patterns, addressing key challenges in ML system design.

Related papers

• A Functional Software Reference Architecture for LLM-Integrated Systems [8.68898878009242]
  Integration of large language models into software systems is transforming capabilities such as natural language understanding, decision-making, and autonomous task execution. The absence of a commonly accepted software reference architecture hinders systematic reasoning about their design and quality attributes. We describe our textitemerging results for a preliminary functional reference architecture as a conceptual framework to address these challenges.
  arXiv  Detail & Related papers  (2025-01-22T14:30:40Z)
• A Survey on Inference Optimization Techniques for Mixture of Experts Models [50.40325411764262]
  Large-scale Mixture of Experts (MoE) models offer enhanced model capacity and computational efficiency through conditional computation. deploying and running inference on these models presents significant challenges in computational resources, latency, and energy efficiency. This survey analyzes optimization techniques for MoE models across the entire system stack.
  arXiv  Detail & Related papers  (2024-12-18T14:11:15Z)
• Enhanced FIWARE-Based Architecture for Cyberphysical Systems With Tiny Machine Learning and Machine Learning Operations: A Case Study on Urban Mobility Systems [0.0]
  Mobility computing presents specific barriers due to its real-time requirements, decentralization, and connectivity through wireless networks. New research on edge computing and tiny machine learning (tinyML) explores the execution of AI models on low-performance devices to address these issues. This article extends a previous architecture based on FIWARE software components to implement the machine learning operations flow.
  arXiv  Detail & Related papers  (2024-11-16T13:14:29Z)
• Architectural Patterns for Designing Quantum Artificial Intelligence Systems [25.42535682546052]
  Utilising quantum computing technology to enhance artificial intelligence systems is expected to improve training and inference times, increase robustness against noise and adversarial attacks, and reduce the number of parameters without compromising accuracy. However, moving beyond proof-of-concept or simulations to develop practical applications of these systems faces significant challenges due to the limitations of quantum hardware and the underdeveloped knowledge base in software engineering for such systems. We have conducted a systematic mapping study to identify the challenges and solutions associated with the software architecture of quantum-enhanced artificial intelligence systems.
  arXiv  Detail & Related papers  (2024-11-14T05:09:07Z)
• Enhancing Architecture Frameworks by Including Modern Stakeholders and their Views/Viewpoints [48.87872564630711]
  The stakeholders with data science and Machine Learning related concerns, such as data scientists and data engineers, are yet to be included in existing architecture frameworks. We surveyed 61 subject matter experts from over 25 organizations in 10 countries.
  arXiv  Detail & Related papers  (2023-08-09T21:54:34Z)
• Real-world Machine Learning Systems: A survey from a Data-Oriented Architecture Perspective [7.574538335342942]
  Data-oriented Architecture (DOA) is an emerging concept that equips systems better for integrating ML models. DOA extends current architectures to create data-driven, loosely coupled, decentralised, open systems. This paper answers these questions by surveying real-world deployments of ML-based systems.
  arXiv  Detail & Related papers  (2023-02-09T17:57:02Z)
• An Extensible Benchmark Suite for Learning to Simulate Physical Systems [60.249111272844374]
  We introduce a set of benchmark problems to take a step towards unified benchmarks and evaluation protocols. We propose four representative physical systems, as well as a collection of both widely used classical time-based and representative data-driven methods.
  arXiv  Detail & Related papers  (2021-08-09T17:39:09Z)
• A Survey of Machine Learning for Computer Architecture and Systems [18.620218353713476]
  It has been a long time that computer architecture and systems are optimized to enable efficient execution of machine learning (ML) algorithms or models. Now, it is time to reconsider the relationship between ML and systems, and let ML transform the way that computer architecture and systems are designed.
  arXiv  Detail & Related papers  (2021-02-16T04:09:57Z)
• Technology Readiness Levels for Machine Learning Systems [107.56979560568232]
  Development and deployment of machine learning systems can be executed easily with modern tools, but the process is typically rushed and means-to-an-end. We have developed a proven systems engineering approach for machine learning development and deployment. Our "Machine Learning Technology Readiness Levels" framework defines a principled process to ensure robust, reliable, and responsible systems.
  arXiv  Detail & Related papers  (2021-01-11T15:54:48Z)
• Technology Readiness Levels for AI & ML [79.22051549519989]
  Development of machine learning systems can be executed easily with modern tools, but the process is typically rushed and means-to-an-end. Engineering systems follow well-defined processes and testing standards to streamline development for high-quality, reliable results. We propose a proven systems engineering approach for machine learning development and deployment.
  arXiv  Detail & Related papers  (2020-06-21T17:14:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.