MLGuard: Defend Your Machine Learning Model!
- URL: http://arxiv.org/abs/2309.01379v1
- Date: Mon, 4 Sep 2023 06:08:11 GMT
- Title: MLGuard: Defend Your Machine Learning Model!
- Authors: Sheng Wong, Scott Barnett, Jessica Rivera-Villicana, Anj Simmons, Hala
Abdelkader, Jean-Guy Schneider, Rajesh Vasa
- Abstract summary: We propose MLGuard, a new approach to specify contracts for Machine Learning applications.
Our work is intended to provide the overarching framework required for building ML applications and monitoring their safety.
- Score: 3.4069804433026314
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Machine Learning (ML) is used in critical highly regulated and high-stakes
fields such as finance, medicine, and transportation. The correctness of these
ML applications is important for human safety and economic benefit. Progress
has been made on improving ML testing and monitoring of ML. However, these
approaches do not provide i) pre/post conditions to handle uncertainty, ii)
defining corrective actions based on probabilistic outcomes, or iii) continual
verification during system operation. In this paper, we propose MLGuard, a new
approach to specify contracts for ML applications. Our approach consists of a)
an ML contract specification defining pre/post conditions, invariants, and
altering behaviours, b) generated validation models to determine the
probability of contract violation, and c) an ML wrapper generator to enforce
the contract and respond to violations. Our work is intended to provide the
overarching framework required for building ML applications and monitoring
their safety.
Related papers
- Uncertainty is Fragile: Manipulating Uncertainty in Large Language Models [79.76293901420146]
Large Language Models (LLMs) are employed across various high-stakes domains, where the reliability of their outputs is crucial.
Our research investigates the fragility of uncertainty estimation and explores potential attacks.
We demonstrate that an attacker can embed a backdoor in LLMs, which, when activated by a specific trigger in the input, manipulates the model's uncertainty without affecting the final output.
arXiv Detail & Related papers (2024-07-15T23:41:11Z) - Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph [85.51252685938564]
Uncertainty quantification (UQ) is becoming increasingly recognized as a critical component of applications that rely on machine learning (ML)
As with other ML models, large language models (LLMs) are prone to make incorrect predictions, hallucinate'' by fabricating claims, or simply generate low-quality output for a given input.
We introduce a novel benchmark that implements a collection of state-of-the-art UQ baselines, and provides an environment for controllable and consistent evaluation of novel techniques.
arXiv Detail & Related papers (2024-06-21T20:06:31Z) - Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformation [98.02846901473697]
We propose ECSO (Eyes Closed, Safety On), a training-free protecting approach that exploits the inherent safety awareness of MLLMs.
ECSO generates safer responses via adaptively transforming unsafe images into texts to activate the intrinsic safety mechanism of pre-aligned LLMs.
arXiv Detail & Related papers (2024-03-14T17:03:04Z) - Evaluation of Predictive Reliability to Foster Trust in Artificial
Intelligence. A case study in Multiple Sclerosis [0.34473740271026115]
Spotting Machine Learning failures is of paramount importance when ML predictions are used to drive clinical decisions.
We propose a simple approach that can be used in the deployment phase of any ML model to suggest whether to trust predictions or not.
Our method holds the promise to provide effective support to clinicians by spotting potential ML failures during deployment.
arXiv Detail & Related papers (2024-02-27T14:48:07Z) - ML-On-Rails: Safeguarding Machine Learning Models in Software Systems A
Case Study [4.087995998278127]
We introduce ML-On-Rails, a protocol designed to safeguard machine learning models.
ML-On-Rails establishes a well-defined endpoint interface for different ML tasks, and clear communication between ML providers and ML consumers.
We evaluate the protocol through a real-world case study of the MoveReminder application.
arXiv Detail & Related papers (2024-01-12T11:27:15Z) - Vulnerability of Machine Learning Approaches Applied in IoT-based Smart Grid: A Review [51.31851488650698]
Machine learning (ML) sees an increasing prevalence of being used in the internet-of-things (IoT)-based smart grid.
adversarial distortion injected into the power signal will greatly affect the system's normal control and operation.
It is imperative to conduct vulnerability assessment for MLsgAPPs applied in the context of safety-critical power systems.
arXiv Detail & Related papers (2023-08-30T03:29:26Z) - Scaling up Trustless DNN Inference with Zero-Knowledge Proofs [47.42532753464726]
We present the first practical ImageNet-scale method to verify ML model inference non-interactively, i.e., after the inference has been done.
We provide the first ZKSNARK proof of valid inference for a full resolution ImageNet model, achieving 79% top-5 accuracy.
arXiv Detail & Related papers (2022-10-17T00:35:38Z) - Practical Machine Learning Safety: A Survey and Primer [81.73857913779534]
Open-world deployment of Machine Learning algorithms in safety-critical applications such as autonomous vehicles needs to address a variety of ML vulnerabilities.
New models and training techniques to reduce generalization error, achieve domain adaptation, and detect outlier examples and adversarial attacks.
Our organization maps state-of-the-art ML techniques to safety strategies in order to enhance the dependability of the ML algorithm from different aspects.
arXiv Detail & Related papers (2021-06-09T05:56:42Z) - MLDemon: Deployment Monitoring for Machine Learning Systems [10.074466859579571]
We propose a novel approach, MLDemon, for ML DEployment MONitoring.
MLDemon integrates both unlabeled features and a small amount of on-demand labeled examples over time to produce a real-time estimate.
On temporal datasets with diverse distribution drifts and models, MLDemon substantially outperforms existing monitoring approaches.
arXiv Detail & Related papers (2021-04-28T07:59:10Z) - Insights into Performance Fitness and Error Metrics for Machine Learning [1.827510863075184]
Machine learning (ML) is the field of training machines to achieve high level of cognition and perform human-like analysis.
This paper examines a number of the most commonly-used performance fitness and error metrics for regression and classification algorithms.
arXiv Detail & Related papers (2020-05-17T22:59:04Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.