A Scalable and Efficient Iterative Method for Copying Machine Learning Classifiers

• URL: http://arxiv.org/abs/2302.02667v2
• Date: Tue, 7 Feb 2023 09:33:37 GMT
• Title: A Scalable and Efficient Iterative Method for Copying Machine Learning Classifiers
• Authors: Nahuel Statuto, Irene Unceta, Jordi Nin and Oriol Pujol
• Abstract summary: This paper introduces a novel sequential approach that significantly reduces the amount of computational resources needed to train or maintain a copy of a machine learning model. The effectiveness of the sequential approach is demonstrated through experiments with synthetic and real-world datasets, showing significant reductions in time and resources, while maintaining or improving accuracy.
• Score: 0.802904964931021
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Differential replication through copying refers to the process of replicating the decision behavior of a machine learning model using another model that possesses enhanced features and attributes. This process is relevant when external constraints limit the performance of an industrial predictive system. Under such circumstances, copying enables the retention of original prediction capabilities while adapting to new demands. Previous research has focused on the single-pass implementation for copying. This paper introduces a novel sequential approach that significantly reduces the amount of computational resources needed to train or maintain a copy, leading to reduced maintenance costs for companies using machine learning models in production. The effectiveness of the sequential approach is demonstrated through experiments with synthetic and real-world datasets, showing significant reductions in time and resources, while maintaining or improving accuracy.

Related papers

• Attribute-to-Delete: Machine Unlearning via Datamodel Matching [65.13151619119782]
  Machine unlearning -- efficiently removing a small "forget set" training data on a pre-divertrained machine learning model -- has recently attracted interest. Recent research shows that machine unlearning techniques do not hold up in such a challenging setting.
  arXiv  Detail & Related papers  (2024-10-30T17:20:10Z)
• Adjusting Pretrained Backbones for Performativity [34.390793811659556]
  We propose a novel technique to adjust pretrained backbones for performativity in a modular way. We show how it leads to smaller loss along the retraining trajectory and enables us to effectively select among candidate models to anticipate performance degradations.
  arXiv  Detail & Related papers  (2024-10-06T14:41:13Z)
• Dataset Condensation Driven Machine Unlearning [0.0]
  Current trend in data regulation requirements and privacy-preserving machine learning has emphasized the importance of machine unlearning. We propose new dataset condensation techniques and an innovative unlearning scheme that strikes a balance between machine unlearning privacy, utility, and efficiency. We present a novel and effective approach to instrumenting machine unlearning and propose its application in defending against membership inference and model inversion attacks.
  arXiv  Detail & Related papers  (2024-01-31T21:48:25Z)
• Pseudo Replay-based Class Continual Learning for Online New Category Anomaly Detection in Advanced Manufacturing [5.012204041812572]
  This paper develops a novel pseudo replay-based continual learning framework. It integrates class incremental learning and oversampling-based data generation. The effectiveness of the proposed framework is validated in three cases studies.
  arXiv  Detail & Related papers  (2023-12-05T04:43:23Z)
• Continual Learning of Diffusion Models with Generative Distillation [34.52513912701778]
  Diffusion models are powerful generative models that achieve state-of-the-art performance in image synthesis. In this paper, we propose generative distillation, an approach that distils the entire reverse process of a diffusion model.
  arXiv  Detail & Related papers  (2023-11-23T14:33:03Z)
• Uncovering the Hidden Cost of Model Compression [43.62624133952414]
  Visual Prompting has emerged as a pivotal method for transfer learning in computer vision. Model compression detrimentally impacts the performance of visual prompting-based transfer. However, negative effects on calibration are not present when models are compressed via quantization.
  arXiv  Detail & Related papers  (2023-08-29T01:47:49Z)
• PROM: A Phrase-level Copying Mechanism with Pre-training for Abstractive Summarization [139.242907155883]
  This work proposes PROM, a new PhRase-level cOpying Mechanism that enhances attention on n-grams. PROM adds an indicator layer to explicitly pick up tokens in n-gram that can be copied from the source, and calculates an auxiliary loss for the copying prediction. In zero-shot setting, PROM is utilized in the self-supervised pre-training on raw corpora and provides new general baselines on a wide range of summarization datasets.
  arXiv  Detail & Related papers  (2023-05-11T08:29:05Z)
• VCNet: A self-explaining model for realistic counterfactual generation [52.77024349608834]
  Counterfactual explanation is a class of methods to make local explanations of machine learning decisions. We present VCNet-Variational Counter Net, a model architecture that combines a predictor and a counterfactual generator. We show that VCNet is able to both generate predictions, and to generate counterfactual explanations without having to solve another minimisation problem.
  arXiv  Detail & Related papers  (2022-12-21T08:45:32Z)
• On the Copying Behaviors of Pre-Training for Neural Machine Translation [63.914940899327966]
  Previous studies have shown that initializing neural machine translation (NMT) models with the pre-trained language models (LM) can speed up the model training and boost the model performance. In this work, we identify a critical side-effect of pre-training for NMT, which is due to the discrepancy between the training objectives of LM-based pre-training and NMT. We propose a simple and effective method named copying penalty to control the copying behaviors in decoding.
  arXiv  Detail & Related papers  (2021-07-17T10:02:30Z)
• On Learning Text Style Transfer with Direct Rewards [101.97136885111037]
  Lack of parallel corpora makes it impossible to directly train supervised models for the text style transfer task. We leverage semantic similarity metrics originally used for fine-tuning neural machine translation models. Our model provides significant gains in both automatic and human evaluation over strong baselines.
  arXiv  Detail & Related papers  (2020-10-24T04:30:02Z)
• Automatic Recall Machines: Internal Replay, Continual Learning and the Brain [104.38824285741248]
  Replay in neural networks involves training on sequential data with memorized samples, which counteracts forgetting of previous behavior caused by non-stationarity. We present a method where these auxiliary samples are generated on the fly, given only the model that is being trained for the assessed objective. Instead the implicit memory of learned samples within the assessed model itself is exploited.
  arXiv  Detail & Related papers  (2020-06-22T15:07:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.