Do Deep Networks Transfer Invariances Across Classes?
- URL: http://arxiv.org/abs/2203.09739v1
- Date: Fri, 18 Mar 2022 04:38:18 GMT
- Title: Do Deep Networks Transfer Invariances Across Classes?
- Authors: Allan Zhou, Fahim Tajwar, Alexander Robey, Tom Knowles, George J.
Pappas, Hamed Hassani, Chelsea Finn
- Abstract summary: We show how a generative approach for learning the nuisance transformations can help transfer invariances across classes.
Our results provide one explanation for why classifiers generalize poorly on unbalanced and longtailed distributions.
- Score: 123.84237389985236
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: To generalize well, classifiers must learn to be invariant to nuisance
transformations that do not alter an input's class. Many problems have
"class-agnostic" nuisance transformations that apply similarly to all classes,
such as lighting and background changes for image classification. Neural
networks can learn these invariances given sufficient data, but many real-world
datasets are heavily class imbalanced and contain only a few examples for most
of the classes. We therefore pose the question: how well do neural networks
transfer class-agnostic invariances learned from the large classes to the small
ones? Through careful experimentation, we observe that invariance to
class-agnostic transformations is still heavily dependent on class size, with
the networks being much less invariant on smaller classes. This result holds
even when using data balancing techniques, and suggests poor invariance
transfer across classes. Our results provide one explanation for why
classifiers generalize poorly on unbalanced and long-tailed distributions.
Based on this analysis, we show how a generative approach for learning the
nuisance transformations can help transfer invariances across classes and
improve performance on a set of imbalanced image classification benchmarks.
Source code for our experiments is available at
https://github.com/AllanYangZhou/generative-invariance-transfer.
Related papers
- Understanding the Role of Invariance in Transfer Learning [9.220104991339104]
Transfer learning is a powerful technique for knowledge-sharing between different tasks.
Recent work has found that the representations of models with certain invariances, such as to adversarial input perturbations, achieve higher performance on downstream tasks.
arXiv Detail & Related papers (2024-07-05T07:53:52Z) - Revisiting Data Augmentation for Rotational Invariance in Convolutional
Neural Networks [0.29127054707887967]
We investigate how best to include rotational invariance in a CNN for image classification.
Our experiments show that networks trained with data augmentation alone can classify rotated images nearly as well as in the normal unrotated case.
arXiv Detail & Related papers (2023-10-12T15:53:24Z) - Learning to Transform for Generalizable Instance-wise Invariance [48.647925994707855]
Given any image, we use a normalizing flow to predict a distribution over transformations and average the predictions over them.
This normalizing flow is trained end-to-end and can learn a much larger range of transformations than Augerino and InstaAug.
When used as data augmentation, our method shows accuracy and robustness gains on CIFAR 10, CIFAR10-LT, and TinyImageNet.
arXiv Detail & Related papers (2023-09-28T17:59:58Z) - Adjusting Logit in Gaussian Form for Long-Tailed Visual Recognition [37.62659619941791]
We study the problem of long-tailed visual recognition from the perspective of feature level.
Two novel logit adjustment methods are proposed to improve model performance at a modest computational overhead.
Experiments conducted on benchmark datasets demonstrate the superior performance of the proposed method over the state-of-the-art ones.
arXiv Detail & Related papers (2023-05-18T02:06:06Z) - GistNet: a Geometric Structure Transfer Network for Long-Tailed
Recognition [95.93760490301395]
Long-tailed recognition is a problem where the number of examples per class is highly unbalanced.
GistNet is proposed to support this goal, using constellations of classifier parameters to encode the class geometry.
A new learning algorithm is then proposed for GeometrIc Structure Transfer (GIST), with resort to a combination of loss functions that combine class-balanced and random sampling to guarantee that, while overfitting to the popular classes is restricted to geometric parameters, it is leveraged to transfer class geometry from popular to few-shot classes.
arXiv Detail & Related papers (2021-05-01T00:37:42Z) - Learning Invariances in Neural Networks [51.20867785006147]
We show how to parameterize a distribution over augmentations and optimize the training loss simultaneously with respect to the network parameters and augmentation parameters.
We can recover the correct set and extent of invariances on image classification, regression, segmentation, and molecular property prediction from a large space of augmentations.
arXiv Detail & Related papers (2020-10-22T17:18:48Z) - Variational Feature Disentangling for Fine-Grained Few-Shot
Classification [30.350307891161865]
Fine-grained few-shot recognition often suffers from the problem of training data scarcity for novel categories.
In this paper, we focus one enlarging the intra-class variance of the unseen class to improve few-shot classification performance.
arXiv Detail & Related papers (2020-10-07T08:13:42Z) - Feature Space Augmentation for Long-Tailed Data [74.65615132238291]
Real-world data often follow a long-tailed distribution as the frequency of each class is typically different.
Class-balanced loss and advanced methods on data re-sampling and augmentation are among the best practices to alleviate the data imbalance problem.
We present a novel approach to address the long-tailed problem by augmenting the under-represented classes in the feature space with the features learned from the classes with ample samples.
arXiv Detail & Related papers (2020-08-09T06:38:00Z) - M2m: Imbalanced Classification via Major-to-minor Translation [79.09018382489506]
In most real-world scenarios, labeled training datasets are highly class-imbalanced, where deep neural networks suffer from generalizing to a balanced testing criterion.
In this paper, we explore a novel yet simple way to alleviate this issue by augmenting less-frequent classes via translating samples from more-frequent classes.
Our experimental results on a variety of class-imbalanced datasets show that the proposed method improves the generalization on minority classes significantly compared to other existing re-sampling or re-weighting methods.
arXiv Detail & Related papers (2020-04-01T13:21:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.