Characterizing Technical Debt and Antipatterns in AI-Based Systems: A
Systematic Mapping Study
- URL: http://arxiv.org/abs/2103.09783v1
- Date: Wed, 17 Mar 2021 17:11:43 GMT
- Title: Characterizing Technical Debt and Antipatterns in AI-Based Systems: A
Systematic Mapping Study
- Authors: Justus Bogner, Roberto Verdecchia, Ilias Gerostathopoulos
- Abstract summary: The goal of our study is to provide a clear overview and characterization of the types of Technical Debt (TD) that appear in AI-based systems.
Our results show that (i) established TD types, variations of them, and four new TD types (data, model, configuration, and ethics debt) are present in AI-based systems.
Our results can support AI professionals with reasoning about and communicating aspects of TD present in their systems.
- Score: 14.437695080681259
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Background: With the rising popularity of Artificial Intelligence (AI), there
is a growing need to build large and complex AI-based systems in a
cost-effective and manageable way. Like with traditional software, Technical
Debt (TD) will emerge naturally over time in these systems, therefore leading
to challenges and risks if not managed appropriately. The influence of data
science and the stochastic nature of AI-based systems may also lead to new
types of TD or antipatterns, which are not yet fully understood by researchers
and practitioners. Objective: The goal of our study is to provide a clear
overview and characterization of the types of TD (both established and new
ones) that appear in AI-based systems, as well as the antipatterns and related
solutions that have been proposed. Method: Following the process of a
systematic mapping study, 21 primary studies are identified and analyzed.
Results: Our results show that (i) established TD types, variations of them,
and four new TD types (data, model, configuration, and ethics debt) are present
in AI-based systems, (ii) 72 antipatterns are discussed in the literature, the
majority related to data and model deficiencies, and (iii) 46 solutions have
been proposed, either to address specific TD types, antipatterns, or TD in
general. Conclusions: Our results can support AI professionals with reasoning
about and communicating aspects of TD present in their systems. Additionally,
they can serve as a foundation for future research to further our understanding
of TD in AI-based systems.
Related papers
- A Survey on Failure Analysis and Fault Injection in AI Systems [28.30817443151044]
The complexity of AI systems has exposed their vulnerabilities, necessitating robust methods for failure analysis (FA) and fault injection (FI) to ensure resilience and reliability.
This study fills this gap by presenting a detailed survey of existing FA and FI approaches across six layers of AI systems.
Our findings reveal a taxonomy of AI system failures, assess the capabilities of existing FI tools, and highlight discrepancies between real-world and simulated failures.
arXiv Detail & Related papers (2024-06-28T00:32:03Z) - Testing autonomous vehicles and AI: perspectives and challenges from cybersecurity, transparency, robustness and fairness [53.91018508439669]
The study explores the complexities of integrating Artificial Intelligence into Autonomous Vehicles (AVs)
It examines the challenges introduced by AI components and the impact on testing procedures.
The paper identifies significant challenges and suggests future directions for research and development of AI in AV technology.
arXiv Detail & Related papers (2024-02-21T08:29:42Z) - Towards Cognitive AI Systems: a Survey and Prospective on Neuro-Symbolic
AI [33.0761784111292]
Neuro-symbolic AI emerges as a promising paradigm to enhance interpretability, robustness, and trustworthiness.
Recent NSAI systems have demonstrated great potential in collaborative human-AI scenarios with reasoning and cognitive capabilities.
arXiv Detail & Related papers (2024-01-02T05:00:54Z) - Predictable Artificial Intelligence [67.79118050651908]
We argue that achieving predictability is crucial for fostering trust, liability, control, alignment and safety of AI ecosystems.
This paper aims to elucidate the questions, hypotheses and challenges relevant to Predictable AI.
arXiv Detail & Related papers (2023-10-09T21:36:21Z) - Unmasking Biases and Navigating Pitfalls in the Ophthalmic Artificial
Intelligence Lifecycle: A Review [3.1929071422400446]
This review article breaks down the AI lifecycle into seven steps.
Data collection; defining the model task; data pre-processing and labeling; model development; model evaluation and validation; deployment.
Finally, post-deployment evaluation, monitoring, and system recalibration and delves into the risks for harm at each step and strategies for mitigating them.
arXiv Detail & Related papers (2023-10-08T03:49:42Z) - Bridging MDE and AI: A Systematic Review of Domain-Specific Languages and Model-Driven Practices in AI Software Systems Engineering [1.4853133497896698]
This study aims to investigate the existing model-driven approaches relying on DSL in support of the engineering of AI software systems.
The use of MDE for AI is still in its early stages, and there is no single tool or method that is widely used.
arXiv Detail & Related papers (2023-07-10T14:38:38Z) - Human-Centric Multimodal Machine Learning: Recent Advances and Testbed
on AI-based Recruitment [66.91538273487379]
There is a certain consensus about the need to develop AI applications with a Human-Centric approach.
Human-Centric Machine Learning needs to be developed based on four main requirements: (i) utility and social good; (ii) privacy and data ownership; (iii) transparency and accountability; and (iv) fairness in AI-driven decision-making processes.
We study how current multimodal algorithms based on heterogeneous sources of information are affected by sensitive elements and inner biases in the data.
arXiv Detail & Related papers (2023-02-13T16:44:44Z) - The Principles of Data-Centric AI (DCAI) [9.211953610948862]
Data-centric AI (DCAI) as an emerging concept brings data, its quality and its dynamism to the forefront.
This article brings together data-centric perspectives and concepts to outline the foundations of DCAI.
arXiv Detail & Related papers (2022-11-26T16:43:40Z) - Human-Centered AI for Data Science: A Systematic Approach [48.71756559152512]
Human-Centered AI (HCAI) refers to the research effort that aims to design and implement AI techniques to support various human tasks.
We illustrate how we approach HCAI using a series of research projects around Data Science (DS) works as a case study.
arXiv Detail & Related papers (2021-10-03T21:47:13Z) - Model-Based Deep Learning [155.063817656602]
Signal processing, communications, and control have traditionally relied on classical statistical modeling techniques.
Deep neural networks (DNNs) use generic architectures which learn to operate from data, and demonstrate excellent performance.
We are interested in hybrid techniques that combine principled mathematical models with data-driven systems to benefit from the advantages of both approaches.
arXiv Detail & Related papers (2020-12-15T16:29:49Z) - Distributed and Democratized Learning: Philosophy and Research
Challenges [80.39805582015133]
We propose a novel design philosophy called democratized learning (Dem-AI)
Inspired by the societal groups of humans, the specialized groups of learning agents in the proposed Dem-AI system are self-organized in a hierarchical structure to collectively perform learning tasks more efficiently.
We present a reference design as a guideline to realize future Dem-AI systems, inspired by various interdisciplinary fields.
arXiv Detail & Related papers (2020-03-18T08:45:10Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.