Data-Free Model Extraction

• URL: http://arxiv.org/abs/2011.14779v2
• Date: Wed, 31 Mar 2021 16:12:34 GMT
• Title: Data-Free Model Extraction
• Authors: Jean-Baptiste Truong, Pratyush Maini, Robert J. Walls, Nicolas Papernot
• Abstract summary: Current model extraction attacks assume that the adversary has access to a surrogate dataset with characteristics similar to the proprietary data used to train the victim model. We propose data-free model extraction methods that do not require a surrogate dataset. We find that the proposed data-free model extraction approach achieves high-accuracy with reasonable query complexity.
• Score: 16.007030173299984
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Current model extraction attacks assume that the adversary has access to a surrogate dataset with characteristics similar to the proprietary data used to train the victim model. This requirement precludes the use of existing model extraction techniques on valuable models, such as those trained on rare or hard to acquire datasets. In contrast, we propose data-free model extraction methods that do not require a surrogate dataset. Our approach adapts techniques from the area of data-free knowledge transfer for model extraction. As part of our study, we identify that the choice of loss is critical to ensuring that the extracted model is an accurate replica of the victim model. Furthermore, we address difficulties arising from the adversary's limited access to the victim model in a black-box setting. For example, we recover the model's logits from its probability predictions to approximate gradients. We find that the proposed data-free model extraction approach achieves high-accuracy with reasonable query complexity -- 0.99x and 0.92x the victim model accuracy on SVHN and CIFAR-10 datasets given 2M and 20M queries respectively.

Related papers

• Towards a Theoretical Understanding of Memorization in Diffusion Models [76.85077961718875]
  Diffusion probabilistic models (DPMs) are being employed as mainstream models for Generative Artificial Intelligence (GenAI) We provide a theoretical understanding of memorization in both conditional and unconditional DPMs under the assumption of model convergence. We propose a novel data extraction method named textbfSurrogate condItional Data Extraction (SIDE) that leverages a time-dependent classifier trained on the generated data as a surrogate condition to extract training data from unconditional DPMs.
  arXiv  Detail & Related papers  (2024-10-03T13:17:06Z)
• CaBaGe: Data-Free Model Extraction using ClAss BAlanced Generator Ensemble [4.029642441688877]
  We propose a data-free model extraction approach, CaBaGe, to achieve higher model extraction accuracy with a small number of queries. Our evaluation shows that CaBaGe outperforms existing techniques on seven datasets.
  arXiv  Detail & Related papers  (2024-09-16T18:19:19Z)
• MisGUIDE : Defense Against Data-Free Deep Learning Model Extraction [0.8437187555622164]
  "MisGUIDE" is a two-step defense framework for Deep Learning models that disrupts the adversarial sample generation process. The aim of the proposed defense method is to reduce the accuracy of the cloned model while maintaining accuracy on authentic queries.
  arXiv  Detail & Related papers  (2024-03-27T13:59:21Z)
• SCME: A Self-Contrastive Method for Data-free and Query-Limited Model Extraction Attack [18.998300969035885]
  Model extraction attacks fool the target model by generating adversarial examples on a substitute model. We propose a novel data-free model extraction method named SCME, which considers both the inter- and intra-class diversity in synthesizing fake data.
  arXiv  Detail & Related papers  (2023-10-15T10:41:45Z)
• Self-Supervised Dataset Distillation for Transfer Learning [77.4714995131992]
  We propose a novel problem of distilling an unlabeled dataset into a set of small synthetic samples for efficient self-supervised learning (SSL) We first prove that a gradient of synthetic samples with respect to a SSL objective in naive bilevel optimization is textitbiased due to randomness originating from data augmentations or masking. We empirically validate the effectiveness of our method on various applications involving transfer learning.
  arXiv  Detail & Related papers  (2023-10-10T10:48:52Z)
• Beyond Labeling Oracles: What does it mean to steal ML models? [52.63413852460003]
  Model extraction attacks are designed to steal trained models with only query access. We investigate factors influencing the success of model extraction attacks. Our findings urge the community to redefine the adversarial goals of ME attacks.
  arXiv  Detail & Related papers  (2023-10-03T11:10:21Z)
• Data-Free Model Extraction Attacks in the Context of Object Detection [0.6719751155411076]
  A significant number of machine learning models are vulnerable to model extraction attacks. We propose an adversary black box attack extending to a regression problem for predicting bounding box coordinates in object detection. We find that the proposed model extraction method achieves significant results by using reasonable queries.
  arXiv  Detail & Related papers  (2023-08-09T06:23:54Z)
• ADT-SSL: Adaptive Dual-Threshold for Semi-Supervised Learning [68.53717108812297]
  Semi-Supervised Learning (SSL) has advanced classification tasks by inputting both labeled and unlabeled data to train a model jointly. This paper proposes an Adaptive Dual-Threshold method for Semi-Supervised Learning (ADT-SSL) Experimental results show that the proposed ADT-SSL achieves state-of-the-art classification accuracy.
  arXiv  Detail & Related papers  (2022-05-21T11:52:08Z)
• MEGEX: Data-Free Model Extraction Attack against Gradient-Based Explainable AI [1.693045612956149]
  Deep neural networks deployed in Machine Learning as a Service (ML) face the threat of model extraction attacks. A model extraction attack is an attack to violate intellectual property and privacy in which an adversary steals trained models in a cloud using only their predictions. In this paper, we propose MEGEX, a data-free model extraction attack against a gradient-based explainable AI.
  arXiv  Detail & Related papers  (2021-07-19T14:25:06Z)
• Contrastive Model Inversion for Data-Free Knowledge Distillation [60.08025054715192]
  We propose Contrastive Model Inversion, where the data diversity is explicitly modeled as an optimizable objective. Our main observation is that, under the constraint of the same amount of data, higher data diversity usually indicates stronger instance discrimination. Experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet demonstrate that CMI achieves significantly superior performance when the generated data are used for knowledge distillation.
  arXiv  Detail & Related papers  (2021-05-18T15:13:00Z)
• Data from Model: Extracting Data from Non-robust and Robust Models [83.60161052867534]
  This work explores the reverse process of generating data from a model, attempting to reveal the relationship between the data and the model. We repeat the process of Data to Model (DtM) and Data from Model (DfM) in sequence and explore the loss of feature mapping information. Our results show that the accuracy drop is limited even after multiple sequences of DtM and DfM, especially for robust models.
  arXiv  Detail & Related papers  (2020-07-13T05:27:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.