Engineering Trustworthy Machine-Learning Operations with Zero-Knowledge Proofs

• URL: http://arxiv.org/abs/2505.20136v1
• Date: Mon, 26 May 2025 15:39:11 GMT
• Title: Engineering Trustworthy Machine-Learning Operations with Zero-Knowledge Proofs
• Authors: Filippo Scaramuzza, Giovanni Quattrocchi, Damian A. Tamburri,
• Abstract summary: Zero-Knowledge Proofs (ZKPs) offer a cryptographic solution that enables provers to demonstrate, through verified computations, adherence to set requirements without revealing sensitive model details or data.<n>We identify five key properties (non-interactivity, transparent setup, standard representations, succinctness, and post-quantum security) critical for their application in AI validation and verification pipelines.
• Score: 1.7723990552388873
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As Artificial Intelligence (AI) systems, particularly those based on machine learning (ML), become integral to high-stakes applications, their probabilistic and opaque nature poses significant challenges to traditional verification and validation methods. These challenges are exacerbated in regulated sectors requiring tamper-proof, auditable evidence, as highlighted by apposite legal frameworks, e.g., the EU AI Act. Conversely, Zero-Knowledge Proofs (ZKPs) offer a cryptographic solution that enables provers to demonstrate, through verified computations, adherence to set requirements without revealing sensitive model details or data. Through a systematic survey of ZKP protocols, we identify five key properties (non-interactivity, transparent setup, standard representations, succinctness, and post-quantum security) critical for their application in AI validation and verification pipelines. Subsequently, we perform a follow-up systematic survey analyzing ZKP-enhanced ML applications across an adaptation of the Team Data Science Process (TDSP) model (Data & Preprocessing, Training & Offline Metrics, Inference, and Online Metrics), detailing verification objectives, ML models, and adopted protocols. Our findings indicate that current research on ZKP-Enhanced ML primarily focuses on inference verification, while the data preprocessing and training stages remain underexplored. Most notably, our analysis identifies a significant convergence within the research domain toward the development of a unified Zero-Knowledge Machine Learning Operations (ZKMLOps) framework. This emerging framework leverages ZKPs to provide robust cryptographic guarantees of correctness, integrity, and privacy, thereby promoting enhanced accountability, transparency, and compliance with Trustworthy AI principles.

Related papers

• ZKPROV: A Zero-Knowledge Approach to Dataset Provenance for Large Language Models [46.71493672772134]
  We introduce ZKPROV, a novel cryptographic framework that enables zero-knowledge proofs of LLM provenance.<n>It allows users to verify that a model is trained on a reliable dataset without revealing sensitive information about it or its parameters.<n>Our method cryptographically binds a trained model to its authorized training dataset(s) through zero-knowledge proofs while avoiding proof of every training step.
  arXiv  Detail & Related papers  (2025-06-26T00:49:02Z)
• Does Machine Unlearning Truly Remove Model Knowledge? A Framework for Auditing Unlearning in LLMs [58.24692529185971]
  We introduce a comprehensive auditing framework for unlearning evaluation comprising three benchmark datasets, six unlearning algorithms, and five prompt-based auditing methods.<n>We evaluate the effectiveness and robustness of different unlearning strategies.
  arXiv  Detail & Related papers  (2025-05-29T09:19:07Z)
• Enhancing LLM Reliability via Explicit Knowledge Boundary Modeling [48.15636223774418]
  Large language models (LLMs) are prone to hallucination stemming from misaligned self-awareness.<n>We propose the Explicit Knowledge Boundary Modeling framework to integrate fast and slow reasoning systems to harmonize reliability and usability.
  arXiv  Detail & Related papers  (2025-03-04T03:16:02Z)
• A Survey of Zero-Knowledge Proof Based Verifiable Machine Learning [11.935644882980233]
  Zero-knowledge proof (ZKP) technology enables effective validation of model performance and authenticity in both training and inference processes without disclosing sensitive data.<n>ZKP ensures the verifiability and security of machine learning models, making it a valuable tool for privacy-preserving AI.<n>This survey paper aims to bridge the gap by reviewing and analyzing all the existing Zero-Knowledge Machine Learning (ZKML) research from June 2017 to December 2024.
  arXiv  Detail & Related papers  (2025-02-25T05:04:27Z)
• Do We Need to Verify Step by Step? Rethinking Process Supervision from a Theoretical Perspective [59.61868506896214]
  We show that under standard data coverage assumptions, reinforcement learning is no more statistically difficult than through process supervision.<n>We prove that any policy's advantage function can serve as an optimal process reward model.
  arXiv  Detail & Related papers  (2025-02-14T22:21:56Z)
• Automated Phishing Detection Using URLs and Webpages [35.66275851732625]
  This project addresses the constraints of traditional reference-based phishing detection by developing an LLM agent framework. This agent harnesses Large Language Models to actively fetch and utilize online information. Our approach has achieved with accuracy of 0.945, significantly outperforms the existing solution(DynaPhish) by 0.445.
  arXiv  Detail & Related papers  (2024-08-03T05:08:27Z)
• TRACE: TRansformer-based Attribution using Contrastive Embeddings in LLMs [50.259001311894295]
  We propose a novel TRansformer-based Attribution framework using Contrastive Embeddings called TRACE. We show that TRACE significantly improves the ability to attribute sources accurately, making it a valuable tool for enhancing the reliability and trustworthiness of large language models.
  arXiv  Detail & Related papers  (2024-07-06T07:19:30Z)
• Trust the Process: Zero-Knowledge Machine Learning to Enhance Trust in Generative AI Interactions [1.3688201404977818]
  It explores using cryptographic techniques, particularly Zero-Knowledge Proofs (ZKPs), to address concerns regarding performance fairness and accuracy. Applying ZKPs to Machine Learning models, known as ZKML (Zero-Knowledge Machine Learning), enables independent validation of AI-generated content. We introduce snarkGPT, a practical ZKML implementation for transformers, to empower users to verify output accuracy and quality while preserving model privacy.
  arXiv  Detail & Related papers  (2024-02-09T14:00:16Z)
• Data Poisoning for In-context Learning [49.77204165250528]
  In-context learning (ICL) has been recognized for its innovative ability to adapt to new tasks. This paper delves into the critical issue of ICL's susceptibility to data poisoning attacks. We introduce ICLPoison, a specialized attacking framework conceived to exploit the learning mechanisms of ICL.
  arXiv  Detail & Related papers  (2024-02-03T14:20:20Z)
• Zero-Knowledge Proof-based Verifiable Decentralized Machine Learning in Communication Network: A Comprehensive Survey [31.111210313340454]
  Decentralized approaches to machine learning introduce challenges related to trust and verifiability.<n>We present a comprehensive review of Zero-Knowledge Proof-based Verifiable Machine Learning (ZKP-VML)
  arXiv  Detail & Related papers  (2023-10-23T12:15:23Z)
• Trustworthy AI [75.99046162669997]
  Brittleness to minor adversarial changes in the input data, ability to explain the decisions, address the bias in their training data, are some of the most prominent limitations. We propose the tutorial on Trustworthy AI to address six critical issues in enhancing user and public trust in AI systems.
  arXiv  Detail & Related papers  (2020-11-02T20:04:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.