ML-On-Rails: Safeguarding Machine Learning Models in Software Systems A
Case Study
- URL: http://arxiv.org/abs/2401.06513v1
- Date: Fri, 12 Jan 2024 11:27:15 GMT
- Title: ML-On-Rails: Safeguarding Machine Learning Models in Software Systems A
Case Study
- Authors: Hala Abdelkader, Mohamed Abdelrazek, Scott Barnett, Jean-Guy
Schneider, Priya Rani, Rajesh Vasa
- Abstract summary: We introduce ML-On-Rails, a protocol designed to safeguard machine learning models.
ML-On-Rails establishes a well-defined endpoint interface for different ML tasks, and clear communication between ML providers and ML consumers.
We evaluate the protocol through a real-world case study of the MoveReminder application.
- Score: 4.087995998278127
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Machine learning (ML), especially with the emergence of large language models
(LLMs), has significantly transformed various industries. However, the
transition from ML model prototyping to production use within software systems
presents several challenges. These challenges primarily revolve around ensuring
safety, security, and transparency, subsequently influencing the overall
robustness and trustworthiness of ML models. In this paper, we introduce
ML-On-Rails, a protocol designed to safeguard ML models, establish a
well-defined endpoint interface for different ML tasks, and clear communication
between ML providers and ML consumers (software engineers). ML-On-Rails
enhances the robustness of ML models via incorporating detection capabilities
to identify unique challenges specific to production ML. We evaluated the
ML-On-Rails protocol through a real-world case study of the MoveReminder
application. Through this evaluation, we emphasize the importance of
safeguarding ML models in production.
Related papers
- Verbalized Machine Learning: Revisiting Machine Learning with Language Models [63.10391314749408]
We introduce the framework of verbalized machine learning (VML)
VML constrains the parameter space to be human-interpretable natural language.
We conduct several studies to empirically evaluate the effectiveness of VML.
arXiv Detail & Related papers (2024-06-06T17:59:56Z) - When Code Smells Meet ML: On the Lifecycle of ML-specific Code Smells in
ML-enabled Systems [13.718420553401662]
We aim to investigate the emergence and evolution of specific types of quality-related concerns known as ML-specific code smells.
More specifically, we present a plan to study ML-specific code smells by empirically analyzing their prevalence in real ML-enabled systems.
We will conduct an exploratory study, mining a large dataset of ML-enabled systems and analyzing over 400k commits about 337 projects.
arXiv Detail & Related papers (2024-03-13T07:43:45Z) - Simultaneous Machine Translation with Large Language Models [51.470478122113356]
We investigate the possibility of applying Large Language Models to SimulMT tasks.
We conducted experiments using the textttLlama2-7b-chat model on nine different languages from the MUST-C dataset.
The results show that LLM outperforms dedicated MT models in terms of BLEU and LAAL metrics.
arXiv Detail & Related papers (2023-09-13T04:06:47Z) - MLGuard: Defend Your Machine Learning Model! [3.4069804433026314]
We propose MLGuard, a new approach to specify contracts for Machine Learning applications.
Our work is intended to provide the overarching framework required for building ML applications and monitoring their safety.
arXiv Detail & Related papers (2023-09-04T06:08:11Z) - Vulnerability of Machine Learning Approaches Applied in IoT-based Smart Grid: A Review [51.31851488650698]
Machine learning (ML) sees an increasing prevalence of being used in the internet-of-things (IoT)-based smart grid.
adversarial distortion injected into the power signal will greatly affect the system's normal control and operation.
It is imperative to conduct vulnerability assessment for MLsgAPPs applied in the context of safety-critical power systems.
arXiv Detail & Related papers (2023-08-30T03:29:26Z) - Understanding the Complexity and Its Impact on Testing in ML-Enabled
Systems [8.630445165405606]
We study Rasa 3.0, an industrial dialogue system that has been widely adopted by various companies around the world.
Our goal is to characterize the complexity of such a largescale ML-enabled system and to understand the impact of the complexity on testing.
Our study reveals practical implications for software engineering for ML-enabled systems.
arXiv Detail & Related papers (2023-01-10T08:13:24Z) - Scaling up Trustless DNN Inference with Zero-Knowledge Proofs [47.42532753464726]
We present the first practical ImageNet-scale method to verify ML model inference non-interactively, i.e., after the inference has been done.
We provide the first ZKSNARK proof of valid inference for a full resolution ImageNet model, achieving 79% top-5 accuracy.
arXiv Detail & Related papers (2022-10-17T00:35:38Z) - Practical Machine Learning Safety: A Survey and Primer [81.73857913779534]
Open-world deployment of Machine Learning algorithms in safety-critical applications such as autonomous vehicles needs to address a variety of ML vulnerabilities.
New models and training techniques to reduce generalization error, achieve domain adaptation, and detect outlier examples and adversarial attacks.
Our organization maps state-of-the-art ML techniques to safety strategies in order to enhance the dependability of the ML algorithm from different aspects.
arXiv Detail & Related papers (2021-06-09T05:56:42Z) - Learning by Design: Structuring and Documenting the Human Choices in
Machine Learning Development [6.903929927172917]
We present a method consisting of eight design questions that outline the deliberation and normative choices going into creating a machine learning model.
Our method affords several benefits, such as supporting critical assessment through methodological transparency.
We believe that our method can help ML practitioners structure and justify their choices and assumptions when developing ML models.
arXiv Detail & Related papers (2021-05-03T08:47:45Z) - MLDemon: Deployment Monitoring for Machine Learning Systems [10.074466859579571]
We propose a novel approach, MLDemon, for ML DEployment MONitoring.
MLDemon integrates both unlabeled features and a small amount of on-demand labeled examples over time to produce a real-time estimate.
On temporal datasets with diverse distribution drifts and models, MLDemon substantially outperforms existing monitoring approaches.
arXiv Detail & Related papers (2021-04-28T07:59:10Z) - Transfer Learning without Knowing: Reprogramming Black-box Machine
Learning Models with Scarce Data and Limited Resources [78.72922528736011]
We propose a novel approach, black-box adversarial reprogramming (BAR), that repurposes a well-trained black-box machine learning model.
Using zeroth order optimization and multi-label mapping techniques, BAR can reprogram a black-box ML model solely based on its input-output responses.
BAR outperforms state-of-the-art methods and yields comparable performance to the vanilla adversarial reprogramming method.
arXiv Detail & Related papers (2020-07-17T01:52:34Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.