MAGDiff: Covariate Data Set Shift Detection via Activation Graphs of Deep Neural Networks

• URL: http://arxiv.org/abs/2305.13271v2
• Date: Sun, 12 May 2024 18:19:11 GMT
• Title: MAGDiff: Covariate Data Set Shift Detection via Activation Graphs of Deep Neural Networks
• Authors: Charles Arnal, Felix Hensel, Mathieu Carrière, Théo Lacombe, Hiroaki Kurihara, Yuichi Ike, Frédéric Chazal,
• Abstract summary: We propose a new family of representations, called MAGDiff, that we extract from any given neural network classifier. These representations are computed by comparing the activation graphs of the neural network for samples belonging to the training distribution and to the target distribution. We show that our novel representations induce significant improvements over a state-of-the-art baseline relying on the network output.
• Score: 8.887179103071388
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite their successful application to a variety of tasks, neural networks remain limited, like other machine learning methods, by their sensitivity to shifts in the data: their performance can be severely impacted by differences in distribution between the data on which they were trained and that on which they are deployed. In this article, we propose a new family of representations, called MAGDiff, that we extract from any given neural network classifier and that allows for efficient covariate data shift detection without the need to train a new model dedicated to this task. These representations are computed by comparing the activation graphs of the neural network for samples belonging to the training distribution and to the target distribution, and yield powerful data- and task-adapted statistics for the two-sample tests commonly used for data set shift detection. We demonstrate this empirically by measuring the statistical powers of two-sample Kolmogorov-Smirnov (KS) tests on several different data sets and shift types, and showing that our novel representations induce significant improvements over a state-of-the-art baseline relying on the network output.

Related papers

• NIDS Neural Networks Using Sliding Time Window Data Processing with Trainable Activations and its Generalization Capability [0.0]
  This paper presents neural networks for network intrusion detection systems (NIDS) that operate on flow data preprocessed with a time window. It requires only eleven features which do not rely on deep packet inspection and can be found in most NIDS datasets and easily obtained from conventional flow collectors. The reported training accuracy exceeds 99% for the proposed method with as little as twenty neural network input features.
  arXiv  Detail & Related papers  (2024-10-24T11:36:19Z)
• A Novel Explainable Out-of-Distribution Detection Approach for Spiking Neural Networks [6.100274095771616]
  This work presents a novel OoD detector that can identify whether test examples input to a Spiking Neural Network belong to the distribution of the data over which it was trained. We characterize the internal activations of the hidden layers of the network in the form of spike count patterns. A local explanation method is devised to produce attribution maps revealing which parts of the input instance push most towards the detection of an example as an OoD sample.
  arXiv  Detail & Related papers  (2022-09-30T11:16:35Z)
• Decomposing neural networks as mappings of correlation functions [57.52754806616669]
  We study the mapping between probability distributions implemented by a deep feed-forward network. We identify essential statistics in the data, as well as different information representations that can be used by neural networks.
  arXiv  Detail & Related papers  (2022-02-10T09:30:31Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• CRNNTL: convolutional recurrent neural network and transfer learning for QSAR modelling [4.090810719630087]
  We propose the convolutional recurrent neural network and transfer learning (CRNNTL) for QSAR modelling. Our strategy takes advantages of both convolutional and recurrent neural networks for feature extraction, as well as the data augmentation method.
  arXiv  Detail & Related papers  (2021-09-07T20:04:55Z)
• Anomaly Detection on Attributed Networks via Contrastive Self-Supervised Learning [50.24174211654775]
  We present a novel contrastive self-supervised learning framework for anomaly detection on attributed networks. Our framework fully exploits the local information from network data by sampling a novel type of contrastive instance pair. A graph neural network-based contrastive learning model is proposed to learn informative embedding from high-dimensional attributes and local structure.
  arXiv  Detail & Related papers  (2021-02-27T03:17:20Z)
• Surgical Mask Detection with Convolutional Neural Networks and Data Augmentations on Spectrograms [8.747840760772268]
  We show the impact of data augmentation on the binary classification task of surgical mask detection in samples of human voice. Results show that most of the baselines given by ComParE are outperformed.
  arXiv  Detail & Related papers  (2020-08-11T09:02:47Z)
• On Robustness and Transferability of Convolutional Neural Networks [147.71743081671508]
  Modern deep convolutional networks (CNNs) are often criticized for not generalizing under distributional shifts. We study the interplay between out-of-distribution and transfer performance of modern image classification CNNs for the first time. We find that increasing both the training set and model sizes significantly improve the distributional shift robustness.
  arXiv  Detail & Related papers  (2020-07-16T18:39:04Z)
• Beyond Data Samples: Aligning Differential Networks Estimation with Scientific Knowledge [18.980524563441975]
  The proposed estimator is scalable to a large number of variables and achieves a sharp convergence rate. Our results highlight significant benefits of integrating group, spatial and anatomic knowledge during differential genetic network identification and brain connectome change discovery.
  arXiv  Detail & Related papers  (2020-04-24T00:01:15Z)
• Diversity inducing Information Bottleneck in Model Ensembles [73.80615604822435]
  In this paper, we target the problem of generating effective ensembles of neural networks by encouraging diversity in prediction. We explicitly optimize a diversity inducing adversarial loss for learning latent variables and thereby obtain diversity in the output predictions necessary for modeling multi-modal data. Compared to the most competitive baselines, we show significant improvements in classification accuracy, under a shift in the data distribution.
  arXiv  Detail & Related papers  (2020-03-10T03:10:41Z)
• Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Data [87.61504710345528]
  We propose two strategies for freeing a neural network from tuning with OoD data, while improving its OoD detection performance. We specifically propose to decompose confidence scoring as well as a modified input pre-processing method. Our further analysis on a larger scale image dataset shows that the two types of distribution shifts, specifically semantic shift and non-semantic shift, present a significant difference.
  arXiv  Detail & Related papers  (2020-02-26T04:18:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.