Verifying Machine Learning Interpretability Requirements through Provenance

• URL: http://arxiv.org/abs/2604.21599v1
• Date: Thu, 23 Apr 2026 12:22:10 GMT
• Title: Verifying Machine Learning Interpretability Requirements through Provenance
• Abstract summary: This paper shows how ML provenance can be used to verify ML interpretability requirements.<n>It provides an approach for how ML engineers can save various types of model and data provenance to make the model's behavior transparent and interpretable.
• Score: 0.7466390172678973
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine Learning (ML) Engineering is a growing field that necessitates an increase in the rigor of ML development. It draws many ideas from software engineering and more specifically, from requirements engineering. Existing literature on ML Engineering defines quality models and Non-Functional Requirements (NFRs) specific to ML, in particular interpretability being one such NFR. However, a major challenge occurs in verifying ML NFRs, including interpretability. Although existing literature defines interpretability in terms of ML, it remains an immeasurable requirement, making it impossible to definitively confirm whether a model meets its interpretability requirement. This paper shows how ML provenance can be used to verify ML interpretability requirements. This work provides an approach for how ML engineers can save various types of model and data provenance to make the model's behavior transparent and interpretable. Saving this data forms the basis of quantifiable Functional Requirements (FRs) whose verification in turn verifies the interpretability NFR. Ultimately, this paper contributes a method to verify interpretability NFRs for ML models.

Related papers

• LLM-enabled Behavior Driven Development Workflow for Formally Verified Hardware Designs [6.993866597392993]
  Controlled Natural Language (CNL) specifications can serve as a middle-ground, reducing ambiguity while retaining interpretability.<n>We introduce and define Formal Verification Gherkin Scenarios (FV Gherkin Scenarios), unlocking CNL specifications as the foundation for formally verified hardware designs.
  arXiv  Detail & Related papers  (2026-09-14T10:08:48Z)
• From Machine Learning Documentation to Requirements: Bridging Processes with Requirements Languages [2.2012529546432633]
  This study first investigates the amount and nature of RE-relevant information in 20 publicly available ModelCards and DataSheets.<n>Next, we evaluate how effectively three established RE representations (EARS, Rupp's template, and Volere) can structure this knowledge into requirements.
  arXiv  Detail & Related papers  (2025-11-19T10:59:28Z)
• Uncertainty Quantification for Data-Driven Machine Learning Models in Nuclear Engineering Applications: Where We Are and What Do We Need? [2.026805178426999]
  Machine learning (ML) has been leveraged to tackle a diverse range of tasks in almost all branches of nuclear engineering.<n>An important but under-rated area is uncertainty quantification (UQ) of ML.<n>We will elucidate the differences in the basic concepts of UQ of physics-based models and data-driven ML models.
  arXiv  Detail & Related papers  (2025-03-16T19:54:55Z)
• Verbalized Machine Learning: Revisiting Machine Learning with Language Models [63.10391314749408]
  We introduce the framework of verbalized machine learning (VML)<n>VML constrains the parameter space to be human-interpretable natural language.<n>We empirically verify the effectiveness of VML, and hope that VML can serve as a stepping stone to stronger interpretability.
  arXiv  Detail & Related papers  (2024-06-06T17:59:56Z)
• LLMs for XAI: Future Directions for Explaining Explanations [50.87311607612179]
  We focus on refining explanations computed using existing XAI algorithms. Initial experiments and user study suggest that LLMs offer a promising way to enhance the interpretability and usability of XAI.
  arXiv  Detail & Related papers  (2024-05-09T19:17:47Z)
• ML-On-Rails: Safeguarding Machine Learning Models in Software Systems A Case Study [4.087995998278127]
  We introduce ML-On-Rails, a protocol designed to safeguard machine learning models. ML-On-Rails establishes a well-defined endpoint interface for different ML tasks, and clear communication between ML providers and ML consumers. We evaluate the protocol through a real-world case study of the MoveReminder application.
  arXiv  Detail & Related papers  (2024-01-12T11:27:15Z)
• Simultaneous Machine Translation with Large Language Models [51.470478122113356]
  We investigate the possibility of applying Large Language Models to SimulMT tasks. We conducted experiments using the textttLlama2-7b-chat model on nine different languages from the MUST-C dataset. The results show that LLM outperforms dedicated MT models in terms of BLEU and LAAL metrics.
  arXiv  Detail & Related papers  (2023-09-13T04:06:47Z)
• MLGuard: Defend Your Machine Learning Model! [3.4069804433026314]
  We propose MLGuard, a new approach to specify contracts for Machine Learning applications. Our work is intended to provide the overarching framework required for building ML applications and monitoring their safety.
  arXiv  Detail & Related papers  (2023-09-04T06:08:11Z)
• Scaling up Trustless DNN Inference with Zero-Knowledge Proofs [47.42532753464726]
  We present the first practical ImageNet-scale method to verify ML model inference non-interactively, i.e., after the inference has been done. We provide the first ZKSNARK proof of valid inference for a full resolution ImageNet model, achieving 79% top-5 accuracy.
  arXiv  Detail & Related papers  (2022-10-17T00:35:38Z)
• The Need for Interpretable Features: Motivation and Taxonomy [69.07189753428553]
  We claim that the term "interpretable feature" is not specific nor detailed enough to capture the full extent to which features impact the usefulness of machine learning explanations. In this paper, we motivate and discuss three key lessons: 1) more attention should be given to what we refer to as the interpretable feature space, or the state of features that are useful to domain experts taking real-world actions.
  arXiv  Detail & Related papers  (2022-02-23T19:19:14Z)
• Transfer Learning without Knowing: Reprogramming Black-box Machine Learning Models with Scarce Data and Limited Resources [78.72922528736011]
  We propose a novel approach, black-box adversarial reprogramming (BAR), that repurposes a well-trained black-box machine learning model. Using zeroth order optimization and multi-label mapping techniques, BAR can reprogram a black-box ML model solely based on its input-output responses. BAR outperforms state-of-the-art methods and yields comparable performance to the vanilla adversarial reprogramming method.
  arXiv  Detail & Related papers  (2020-07-17T01:52:34Z)
• Insights into Performance Fitness and Error Metrics for Machine Learning [1.827510863075184]
  Machine learning (ML) is the field of training machines to achieve high level of cognition and perform human-like analysis. This paper examines a number of the most commonly-used performance fitness and error metrics for regression and classification algorithms.
  arXiv  Detail & Related papers  (2020-05-17T22:59:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.