Shortcut Detection with Variational Autoencoders

• URL: http://arxiv.org/abs/2302.04246v2
• Date: Fri, 21 Jul 2023 09:15:42 GMT
• Title: Shortcut Detection with Variational Autoencoders
• Authors: Nicolas M. M\"uller, Simon Roschmann, Shahbaz Khan, Philip Sperl, Konstantin B\"ottinger
• Abstract summary: We present a novel approach to detect shortcuts in image and audio datasets by leveraging variational autoencoders (VAEs) The disentanglement of features in the latent space of VAEs allows us to discover feature-target correlations in datasets and semi-automatically evaluate them for ML shortcuts. We demonstrate the applicability of our method on several real-world datasets and identify shortcuts that have not been discovered before.
• Score: 1.3174512123890016
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: For real-world applications of machine learning (ML), it is essential that models make predictions based on well-generalizing features rather than spurious correlations in the data. The identification of such spurious correlations, also known as shortcuts, is a challenging problem and has so far been scarcely addressed. In this work, we present a novel approach to detect shortcuts in image and audio datasets by leveraging variational autoencoders (VAEs). The disentanglement of features in the latent space of VAEs allows us to discover feature-target correlations in datasets and semi-automatically evaluate them for ML shortcuts. We demonstrate the applicability of our method on several real-world datasets and identify shortcuts that have not been discovered before.

Related papers

• Protecting Publicly Available Data With Machine Learning Shortcuts [3.8709855706783105]
  We show that even simple shortcuts are difficult to detect by explainable AI methods. We then exploit this fact and design an approach to defend online databases against crawlers. We show that a deterrent can be created by deliberately adding ML shortcuts.
  arXiv  Detail & Related papers  (2023-10-30T09:38:03Z)
• Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective [106.92016199403042]
  We empirically investigate knowledge transfer from larger to smaller models through a parametric perspective. We employ sensitivity-based techniques to extract and align knowledge-specific parameters between different large language models. Our findings highlight the critical factors contributing to the process of parametric knowledge transfer.
  arXiv  Detail & Related papers  (2023-10-17T17:58:34Z)
• A Discrepancy Aware Framework for Robust Anomaly Detection [51.710249807397695]
  We present a Discrepancy Aware Framework (DAF), which demonstrates robust performance consistently with simple and cheap strategies. Our method leverages an appearance-agnostic cue to guide the decoder in identifying defects, thereby alleviating its reliance on synthetic appearance. Under the simple synthesis strategies, it outperforms existing methods by a large margin. Furthermore, it also achieves the state-of-the-art localization performance.
  arXiv  Detail & Related papers  (2023-10-11T15:21:40Z)
• Localized Shortcut Removal [4.511561231517167]
  High performance on held-out test data does not necessarily indicate that a model generalizes or learns anything meaningful. This is often due to the existence of machine learning shortcuts - features in the data that are predictive but unrelated to the problem at hand. We use an adversarially trained lens to detect and eliminate highly predictive but semantically unconnected clues in images.
  arXiv  Detail & Related papers  (2022-11-24T13:05:33Z)
• Finding Dataset Shortcuts with Grammar Induction [85.47127659108637]
  We propose to use probabilistic grammars to characterize and discover shortcuts in NLP datasets. Specifically, we use a context-free grammar to model patterns in sentence classification datasets and use a synchronous context-free grammar to model datasets involving sentence pairs. The resulting grammars reveal interesting shortcut features in a number of datasets, including both simple and high-level features.
  arXiv  Detail & Related papers  (2022-10-20T19:54:11Z)
• Monitoring Shortcut Learning using Mutual Information [16.17600110257266]
  Shortcut learning is evaluated on real-world data that does not contain spurious correlations. Experiments demonstrate that MI can be used as a metric network shortcut network.
  arXiv  Detail & Related papers  (2022-06-27T03:55:23Z)
• Continual Object Detection via Prototypical Task Correlation Guided Gating Mechanism [120.1998866178014]
  We present a flexible framework for continual object detection via pRotOtypical taSk corrElaTion guided gaTingAnism (ROSETTA) Concretely, a unified framework is shared by all tasks while task-aware gates are introduced to automatically select sub-models for specific tasks. Experiments on COCO-VOC, KITTI-Kitchen, class-incremental detection on VOC and sequential learning of four tasks show that ROSETTA yields state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-05-06T07:31:28Z)
• STC-IDS: Spatial-Temporal Correlation Feature Analyzing based Intrusion Detection System for Intelligent Connected Vehicles [7.301018758489822]
  We present a novel model for automotive intrusion detection by spatial-temporal correlation features of in-vehicle communication traffic (STC-IDS) Specifically, the proposed model exploits an encoding-detection architecture. In the encoder part, spatial and temporal relations are encoded simultaneously. The encoded information is then passed to the detector for generating forceful spatial-temporal attention features and enabling anomaly classification.
  arXiv  Detail & Related papers  (2022-04-23T04:22:58Z)
• ITSA: An Information-Theoretic Approach to Automatic Shortcut Avoidance and Domain Generalization in Stereo Matching Networks [14.306250516592305]
  We show that learning of feature representations in stereo matching networks is heavily influenced by synthetic data artefacts. We propose an Information-Theoretic Shortcut Avoidance(ITSA) approach to automatically restrict shortcut-related information from being encoded into the feature representations. We show that using this method, state-of-the-art stereo matching networks that are trained purely on synthetic data can effectively generalize to challenging and previously unseen real data scenarios.
  arXiv  Detail & Related papers  (2022-01-06T22:03:50Z)
• Combining Feature and Instance Attribution to Detect Artifacts [62.63504976810927]
  We propose methods to facilitate identification of training data artifacts. We show that this proposed training-feature attribution approach can be used to uncover artifacts in training data. We execute a small user study to evaluate whether these methods are useful to NLP researchers in practice.
  arXiv  Detail & Related papers  (2021-07-01T09:26:13Z)
• Local Metrics for Multi-Object Tracking [102.16141039252652]
  This paper introduces temporally local metrics for Multi-Object Tracking. The horizon parameter offers a novel, meaningful mechanism by which to define the relative importance of detection and association. It is shown that the historical Average Tracking Accuracy (ATA) metric exhibits superior sensitivity to association.
  arXiv  Detail & Related papers  (2021-04-06T16:07:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.