Neural Representations Reveal Distinct Modes of Class Fitting in Residual Convolutional Networks

• URL: http://arxiv.org/abs/2212.00771v1
• Date: Thu, 1 Dec 2022 18:55:58 GMT
• Title: Neural Representations Reveal Distinct Modes of Class Fitting in Residual Convolutional Networks
• Authors: Micha{\l} Jamro\.z and Marcin Kurdziel
• Abstract summary: We leverage probabilistic models of neural representations to investigate how residual networks fit classes. We find that classes in the investigated models are not fitted in an uniform way. We show that the uncovered structure in neural representations correlate with robustness of training examples and adversarial memorization.
• Score: 5.1271832547387115
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We leverage probabilistic models of neural representations to investigate how residual networks fit classes. To this end, we estimate class-conditional density models for representations learned by deep ResNets. We then use these models to characterize distributions of representations across learned classes. Surprisingly, we find that classes in the investigated models are not fitted in an uniform way. On the contrary: we uncover two groups of classes that are fitted with markedly different distributions of representations. These distinct modes of class-fitting are evident only in the deeper layers of the investigated models, indicating that they are not related to low-level image features. We show that the uncovered structure in neural representations correlate with memorization of training examples and adversarial robustness. Finally, we compare class-conditional distributions of neural representations between memorized and typical examples. This allows us to uncover where in the network structure class labels arise for memorized and standard inputs.

Related papers

• Context-Specific Refinements of Bayesian Network Classifiers [1.9136291802656262]
  We study the relationship between our novel classes of classifiers and Bayesian networks. We introduce and implement data-driven learning routines for our models. The study demonstrates that models embedding asymmetric information can enhance classification accuracy.
  arXiv  Detail & Related papers  (2024-05-28T15:50:50Z)
• Probing clustering in neural network representations [30.640266399583613]
  We study how the many design choices involved in neural network training affect the clusters formed in the hidden representations. We isolate the training dataset and architecture as important factors affecting clusterability. We find that normalization strategies affect which layers yield the best clustering performance.
  arXiv  Detail & Related papers  (2023-11-14T02:33:54Z)
• Diffusion Models Beat GANs on Image Classification [37.70821298392606]
  Diffusion models have risen to prominence as a state-of-the-art method for image generation, denoising, inpainting, super-resolution, manipulation, etc. We present our findings that these embeddings are useful beyond the noise prediction task, as they contain discriminative information and can also be leveraged for classification. We find that with careful feature selection and pooling, diffusion models outperform comparable generative-discriminative methods for classification tasks.
  arXiv  Detail & Related papers  (2023-07-17T17:59:40Z)
• Linear Connectivity Reveals Generalization Strategies [54.947772002394736]
  Some pairs of finetuned models have large barriers of increasing loss on the linear paths between them. We find distinct clusters of models which are linearly connected on the test loss surface, but are disconnected from models outside the cluster. Our work demonstrates how the geometry of the loss surface can guide models towards different functions.
  arXiv  Detail & Related papers  (2022-05-24T23:43:02Z)
• On the Role of Neural Collapse in Transfer Learning [29.972063833424215]
  Recent results show that representations learned by a single classifier over many classes are competitive on few-shot learning problems. We show that neural collapse generalizes to new samples from the training classes, and -- more importantly -- to new classes as well.
  arXiv  Detail & Related papers  (2021-12-30T16:36:26Z)
• Learning Debiased and Disentangled Representations for Semantic Segmentation [52.35766945827972]
  We propose a model-agnostic and training scheme for semantic segmentation. By randomly eliminating certain class information in each training iteration, we effectively reduce feature dependencies among classes. Models trained with our approach demonstrate strong results on multiple semantic segmentation benchmarks.
  arXiv  Detail & Related papers  (2021-10-31T16:15:09Z)
• Dynamic Inference with Neural Interpreters [72.90231306252007]
  We present Neural Interpreters, an architecture that factorizes inference in a self-attention network as a system of modules. inputs to the model are routed through a sequence of functions in a way that is end-to-end learned. We show that Neural Interpreters perform on par with the vision transformer using fewer parameters, while being transferrable to a new task in a sample efficient manner.
  arXiv  Detail & Related papers  (2021-10-12T23:22:45Z)
• Learning and Evaluating Representations for Deep One-class Classification [59.095144932794646]
  We present a two-stage framework for deep one-class classification. We first learn self-supervised representations from one-class data, and then build one-class classifiers on learned representations. In experiments, we demonstrate state-of-the-art performance on visual domain one-class classification benchmarks.
  arXiv  Detail & Related papers  (2020-11-04T23:33:41Z)
• Understanding Classifier Mistakes with Generative Models [88.20470690631372]
  Deep neural networks are effective on supervised learning tasks, but have been shown to be brittle. In this paper, we leverage generative models to identify and characterize instances where classifiers fail to generalize. Our approach is agnostic to class labels from the training set which makes it applicable to models trained in a semi-supervised way.
  arXiv  Detail & Related papers  (2020-10-05T22:13:21Z)
• ReMarNet: Conjoint Relation and Margin Learning for Small-Sample Image Classification [49.87503122462432]
  We introduce a novel neural network termed Relation-and-Margin learning Network (ReMarNet) Our method assembles two networks of different backbones so as to learn the features that can perform excellently in both of the aforementioned two classification mechanisms. Experiments on four image datasets demonstrate that our approach is effective in learning discriminative features from a small set of labeled samples.
  arXiv  Detail & Related papers  (2020-06-27T13:50:20Z)
• Classify and Generate: Using Classification Latent Space Representations for Image Generations [17.184760662429834]
  We propose a discriminative modeling framework that employs manipulated supervised latent representations to reconstruct and generate new samples belonging to a given class. ReGene has higher classification accuracy than existing conditional generative models while being competitive in terms of FID.
  arXiv  Detail & Related papers  (2020-04-16T09:13:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.