Under manipulations, are some AI models harder to audit?

• URL: http://arxiv.org/abs/2402.09043v1
• Date: Wed, 14 Feb 2024 09:38:09 GMT
• Title: Under manipulations, are some AI models harder to audit?
• Authors: Augustin Godinot, Gilles Tredan, Erwan Le Merrer, Camilla Penzo, Francois Ta\"iani
• Abstract summary: We study the feasibility of robust audits in realistic settings, in which models exhibit large capacities. We first prove a constraining result: if a web platform uses models that may fit any data, no audit strategy can outperform random sampling. We then relate the manipulability of audits to the capacity of the targeted models, using the Rademacher complexity.
• Score: 2.699900017799093
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Auditors need robust methods to assess the compliance of web platforms with the law. However, since they hardly ever have access to the algorithm, implementation, or training data used by a platform, the problem is harder than a simple metric estimation. Within the recent framework of manipulation-proof auditing, we study in this paper the feasibility of robust audits in realistic settings, in which models exhibit large capacities. We first prove a constraining result: if a web platform uses models that may fit any data, no audit strategy -- whether active or not -- can outperform random sampling when estimating properties such as demographic parity. To better understand the conditions under which state-of-the-art auditing techniques may remain competitive, we then relate the manipulability of audits to the capacity of the targeted models, using the Rademacher complexity. We empirically validate these results on popular models of increasing capacities, thus confirming experimentally that large-capacity models, which are commonly used in practice, are particularly hard to audit robustly. These results refine the limits of the auditing problem, and open up enticing questions on the connection between model capacity and the ability of platforms to manipulate audit attempts.

Related papers

• Automatically Adaptive Conformal Risk Control [49.95190019041905]
  We propose a methodology for achieving approximate conditional control of statistical risks by adapting to the difficulty of test samples. Our framework goes beyond traditional conditional risk control based on user-provided conditioning events to the algorithmic, data-driven determination of appropriate function classes for conditioning.
  arXiv  Detail & Related papers  (2024-06-25T08:29:32Z)
• Large Language Models Must Be Taught to Know What They Don't Know [97.90008709512921]
  We show that fine-tuning on a small dataset of correct and incorrect answers can create an uncertainty estimate with good generalization and small computational overhead. We also investigate the mechanisms that enable reliable uncertainty estimation, finding that many models can be used as general-purpose uncertainty estimators.
  arXiv  Detail & Related papers  (2024-06-12T16:41:31Z)
• Trustless Audits without Revealing Data or Models [49.23322187919369]
  We show that it is possible to allow model providers to keep their model weights (but not architecture) and data secret while allowing other parties to trustlessly audit model and data properties. We do this by designing a protocol called ZkAudit in which model providers publish cryptographic commitments of datasets and model weights.
  arXiv  Detail & Related papers  (2024-04-06T04:43:06Z)
• Verifiable evaluations of machine learning models using zkSNARKs [40.538081946945596]
  This work presents a method of verifiable model evaluation using model inference through zkSNARKs. The resulting zero-knowledge computational proofs of model outputs over datasets can be packaged into verifiable evaluation attestations. For the first time, we demonstrate this across a sample of real-world models and highlight key challenges and design solutions.
  arXiv  Detail & Related papers  (2024-02-05T02:21:11Z)
• TrustFed: A Reliable Federated Learning Framework with Malicious-Attack Resistance [8.924352407824566]
  Federated learning (FL) enables collaborative learning among multiple clients while ensuring individual data privacy. In this paper, we propose a hierarchical audit-based FL (HiAudit-FL) framework to enhance the reliability and security of the learning process. Our simulation results demonstrate that HiAudit-FL can effectively identify and handle potential malicious users accurately, with small system overhead.
  arXiv  Detail & Related papers  (2023-12-06T13:56:45Z)
• Tight Auditing of Differentially Private Machine Learning [77.38590306275877]
  For private machine learning, existing auditing mechanisms are tight. They only give tight estimates under implausible worst-case assumptions. We design an improved auditing scheme that yields tight privacy estimates for natural (not adversarially crafted) datasets.
  arXiv  Detail & Related papers  (2023-02-15T21:40:33Z)
• Federated and Privacy-Preserving Learning of Accounting Data in Financial Statement Audits [1.4986031916712106]
  We propose a Federated Learning framework to train DL models on auditing relevant accounting data of multiple clients. We evaluate our approach to detect accounting anomalies in three real-world datasets of city payments.
  arXiv  Detail & Related papers  (2022-08-26T15:09:18Z)
• Soundness of Data-Aware Processes with Arithmetic Conditions [8.914271888521652]
  Data Petri nets (DPNs) have gained increasing popularity thanks to their ability to balance simplicity with expressiveness. The interplay of data and control-flow makes checking the correctness of such models, specifically the well-known property of soundness, crucial and challenging. We provide a framework for assessing soundness of DPNs enriched with arithmetic data conditions.
  arXiv  Detail & Related papers  (2022-03-28T14:46:10Z)
• Sample-Efficient Reinforcement Learning via Conservative Model-Based Actor-Critic [67.00475077281212]
  Model-based reinforcement learning algorithms are more sample efficient than their model-free counterparts. We propose a novel approach that achieves high sample efficiency without the strong reliance on accurate learned models. We show that CMBAC significantly outperforms state-of-the-art approaches in terms of sample efficiency on several challenging tasks.
  arXiv  Detail & Related papers  (2021-12-16T15:33:11Z)
• How Training Data Impacts Performance in Learning-based Control [67.7875109298865]
  This paper derives an analytical relationship between the density of the training data and the control performance. We formulate a quality measure for the data set, which we refer to as $rho$-gap. We show how the $rho$-gap can be applied to a feedback linearizing control law.
  arXiv  Detail & Related papers  (2020-05-25T12:13:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.