Localized Shortcut Removal

• URL: http://arxiv.org/abs/2211.15510v2
• Date: Tue, 23 May 2023 08:27:43 GMT
• Title: Localized Shortcut Removal
• Authors: Nicolas M. M\"uller, Jochen Jacobs, Jennifer Williams, Konstantin B\"ottinger
• Abstract summary: High performance on held-out test data does not necessarily indicate that a model generalizes or learns anything meaningful. This is often due to the existence of machine learning shortcuts - features in the data that are predictive but unrelated to the problem at hand. We use an adversarially trained lens to detect and eliminate highly predictive but semantically unconnected clues in images.
• Score: 4.511561231517167
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Machine learning is a data-driven field, and the quality of the underlying datasets plays a crucial role in learning success. However, high performance on held-out test data does not necessarily indicate that a model generalizes or learns anything meaningful. This is often due to the existence of machine learning shortcuts - features in the data that are predictive but unrelated to the problem at hand. To address this issue for datasets where the shortcuts are smaller and more localized than true features, we propose a novel approach to detect and remove them. We use an adversarially trained lens to detect and eliminate highly predictive but semantically unconnected clues in images. In our experiments on both synthetic and real-world data, we show that our proposed approach reliably identifies and neutralizes such shortcuts without causing degradation of model performance on clean data. We believe that our approach can lead to more meaningful and generalizable machine learning models, especially in scenarios where the quality of the underlying datasets is crucial.

Related papers

• The Frontier of Data Erasure: Machine Unlearning for Large Language Models [56.26002631481726]
  Large Language Models (LLMs) are foundational to AI advancements. LLMs pose risks by potentially memorizing and disseminating sensitive, biased, or copyrighted information. Machine unlearning emerges as a cutting-edge solution to mitigate these concerns.
  arXiv  Detail & Related papers  (2024-03-23T09:26:15Z)
• An Information Theoretic Approach to Machine Unlearning [45.600917449314444]
  Key challenge in unlearning is forgetting the necessary data in a timely manner, while preserving model performance. In this work, we address the zero-shot unlearning scenario, whereby an unlearning algorithm must be able to remove data given only a trained model and the data to be forgotten. We derive a simple but principled zero-shot unlearning method based on the geometry of the model.
  arXiv  Detail & Related papers  (2024-02-02T13:33:30Z)
• Zero-shot Retrieval: Augmenting Pre-trained Models with Search Engines [83.65380507372483]
  Large pre-trained models can dramatically reduce the amount of task-specific data required to solve a problem, but they often fail to capture domain-specific nuances out of the box. This paper shows how to leverage recent advances in NLP and multi-modal learning to augment a pre-trained model with search engine retrieval.
  arXiv  Detail & Related papers  (2023-11-29T05:33:28Z)
• Learning Defect Prediction from Unrealistic Data [57.53586547895278]
  Pretrained models of code have become popular choices for code understanding and generation tasks. Such models tend to be large and require commensurate volumes of training data. It has become popular to train models with far larger but less realistic datasets, such as functions with artificially injected bugs. Models trained on such data tend to only perform well on similar data, while underperforming on real world programs.
  arXiv  Detail & Related papers  (2023-11-02T01:51:43Z)
• On Inductive Biases for Machine Learning in Data Constrained Settings [0.0]
  This thesis explores a different answer to the problem of learning expressive models in data constrained settings. Instead of relying on big datasets to learn neural networks, we will replace some modules by known functions reflecting the structure of the data. Our approach falls under the hood of "inductive biases", which can be defined as hypothesis on the data at hand restricting the space of models to explore.
  arXiv  Detail & Related papers  (2023-02-21T14:22:01Z)
• PROMISSING: Pruning Missing Values in Neural Networks [0.0]
  We propose a simple and intuitive yet effective method for pruning missing values (PROMISSING) during learning and inference steps in neural networks. Our experiments show that PROMISSING results in similar prediction performance compared to various imputation techniques.
  arXiv  Detail & Related papers  (2022-06-03T15:37:27Z)
• Vertical Machine Unlearning: Selectively Removing Sensitive Information From Latent Feature Space [21.8933559159369]
  We investigate a vertical unlearning mode, aiming at removing only sensitive information from latent feature space. We introduce intuitive and formal definitions for this unlearning and show its relationship with existing horizontal unlearning. We propose an approximation with an upper bound to estimate it, with rigorous theoretical analysis.
  arXiv  Detail & Related papers  (2022-02-27T05:25:15Z)
• Machine Unlearning of Features and Labels [72.81914952849334]
  We propose first scenarios for unlearning and labels in machine learning models. Our approach builds on the concept of influence functions and realizes unlearning through closed-form updates of model parameters.
  arXiv  Detail & Related papers  (2021-08-26T04:42:24Z)
• Hidden Biases in Unreliable News Detection Datasets [60.71991809782698]
  We show that selection bias during data collection leads to undesired artifacts in the datasets. We observed a significant drop (>10%) in accuracy for all models tested in a clean split with no train/test source overlap. We suggest future dataset creation include a simple model as a difficulty/bias probe and future model development use a clean non-overlapping site and date split.
  arXiv  Detail & Related papers  (2021-04-20T17:16:41Z)
• Diverse Complexity Measures for Dataset Curation in Self-driving [80.55417232642124]
  We propose a new data selection method that exploits a diverse set of criteria that quantize interestingness of traffic scenes. Our experiments show that the proposed curation pipeline is able to select datasets that lead to better generalization and higher performance.
  arXiv  Detail & Related papers  (2021-01-16T23:45:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.