A Case Study on AI Engineering Practices: Developing an Autonomous Stock
Trading System
- URL: http://arxiv.org/abs/2303.13216v1
- Date: Thu, 23 Mar 2023 12:27:27 GMT
- Title: A Case Study on AI Engineering Practices: Developing an Autonomous Stock
Trading System
- Authors: Marcel Grote, Justus Bogner
- Abstract summary: Solid AI engineering practices are required to ensure the quality of a production-ready AI-based system.
While several practices have already been proposed for the development of AI-based systems, detailed practical experiences of applying these practices are rare.
We selected 10 AI engineering practices from the literature and systematically applied them during development.
- Score: 8.211107836178083
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Today, many systems use artificial intelligence (AI) to solve complex
problems. While this often increases system effectiveness, developing a
production-ready AI-based system is a difficult task. Thus, solid AI
engineering practices are required to ensure the quality of the resulting
system and to improve the development process. While several practices have
already been proposed for the development of AI-based systems, detailed
practical experiences of applying these practices are rare.
In this paper, we aim to address this gap by collecting such experiences
during a case study, namely the development of an autonomous stock trading
system that uses machine learning functionality to invest in stocks. We
selected 10 AI engineering practices from the literature and systematically
applied them during development, with the goal to collect evidence about their
applicability and effectiveness. Using structured field notes, we documented
our experiences. Furthermore, we also used field notes to document challenges
that occurred during the development, and the solutions we applied to overcome
them. Afterwards, we analyzed the collected field notes, and evaluated how each
practice improved the development. Lastly, we compared our evidence with
existing literature.
Most applied practices improved our system, albeit to varying extent, and we
were able to overcome all major challenges. The qualitative results provide
detailed accounts about 10 AI engineering practices, as well as challenges and
solutions associated with such a project. Our experiences therefore enrich the
emerging body of evidence in this field, which may be especially helpful for
practitioner teams new to AI engineering.
Related papers
- Collaborative AI in Sentiment Analysis: System Architecture, Data Prediction and Deployment Strategies [3.3374611485861116]
Large language model (LLM) based artificial intelligence technologies have been a game-changer, particularly in sentiment analysis.
However, integrating diverse AI models for processing complex multimodal data and the associated high costs of feature extraction presents significant challenges.
This study introduces a collaborative AI framework designed to efficiently distribute and resolve tasks across various AI systems.
arXiv Detail & Related papers (2024-10-17T06:14:34Z) - AI for IT Operations (AIOps) on Cloud Platforms: Reviews, Opportunities
and Challenges [60.56413461109281]
Artificial Intelligence for IT operations (AIOps) aims to combine the power of AI with the big data generated by IT Operations processes.
We discuss in depth the key types of data emitted by IT Operations activities, the scale and challenges in analyzing them, and where they can be helpful.
We categorize the key AIOps tasks as - incident detection, failure prediction, root cause analysis and automated actions.
arXiv Detail & Related papers (2023-04-10T15:38:12Z) - Lessons from Formally Verified Deployed Software Systems (Extended version) [65.69802414600832]
This article examines a range of projects, in various application areas, that have produced formally verified systems and deployed them for actual use.
It considers the technologies used, the form of verification applied, the results obtained, and the lessons that the software industry should draw regarding its ability to benefit from formal verification techniques and tools.
arXiv Detail & Related papers (2023-01-05T18:18:46Z) - Selected Trends in Artificial Intelligence for Space Applications [69.3474006357492]
This chapter focuses on differentiable intelligence and on-board machine learning.
We discuss a few selected projects originating from the European Space Agency's (ESA) Advanced Concepts Team (ACT)
arXiv Detail & Related papers (2022-12-10T07:49:50Z) - Enabling Automated Machine Learning for Model-Driven AI Engineering [60.09869520679979]
We propose a novel approach to enable Model-Driven Software Engineering and Model-Driven AI Engineering.
In particular, we support Automated ML, thus assisting software engineers without deep AI knowledge in developing AI-intensive systems.
arXiv Detail & Related papers (2022-03-06T10:12:56Z) - Machine Learning Application Development: Practitioners' Insights [18.114724750441724]
We report about a survey that aimed to understand the challenges and best practices of ML application development.
We synthesize the results obtained from 80 practitioners into 17 findings; outlining challenges and best practices for ML application development.
We hope that the reported challenges will inform the research community about topics that need to be investigated to improve the engineering process and the quality of ML-based applications.
arXiv Detail & Related papers (2021-12-31T03:38:37Z) - Machine Learning Model Development from a Software Engineering
Perspective: A Systematic Literature Review [0.0]
Data scientists often develop machine learning models to solve a variety of problems in the industry and academy.
This paper is an effort to investigate the challenges and practices that emerge during the development of ML models from the software engineering perspective.
arXiv Detail & Related papers (2021-02-15T14:25:13Z) - Technology Readiness Levels for Machine Learning Systems [107.56979560568232]
Development and deployment of machine learning systems can be executed easily with modern tools, but the process is typically rushed and means-to-an-end.
We have developed a proven systems engineering approach for machine learning development and deployment.
Our "Machine Learning Technology Readiness Levels" framework defines a principled process to ensure robust, reliable, and responsible systems.
arXiv Detail & Related papers (2021-01-11T15:54:48Z) - Software engineering for artificial intelligence and machine learning
software: A systematic literature review [6.681725960709127]
This study aims to investigate how software engineering has been applied in the development of AI/ML systems.
Main challenges faced by professionals are in areas of testing, AI software quality, and data management.
arXiv Detail & Related papers (2020-11-07T11:06:28Z) - Machine Learning for Software Engineering: A Systematic Mapping [73.30245214374027]
The software development industry is rapidly adopting machine learning for transitioning modern day software systems towards highly intelligent and self-learning systems.
No comprehensive study exists that explores the current state-of-the-art on the adoption of machine learning across software engineering life cycle stages.
This study introduces a machine learning for software engineering (MLSE) taxonomy classifying the state-of-the-art machine learning techniques according to their applicability to various software engineering life cycle stages.
arXiv Detail & Related papers (2020-05-27T11:56:56Z) - ECCOLA -- a Method for Implementing Ethically Aligned AI Systems [11.31664099885664]
We present a method for implementing AI ethics into practice.
The method, ECCOLA, has been iteratively developed using a cyclical action design research approach.
arXiv Detail & Related papers (2020-04-17T17:57:07Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.