PatchMix Augmentation to Identify Causal Features in Few-shot Learning

• URL: http://arxiv.org/abs/2211.16019v1
• Date: Tue, 29 Nov 2022 08:41:29 GMT
• Title: PatchMix Augmentation to Identify Causal Features in Few-shot Learning
• Authors: Chengming Xu, Chen Liu, Xinwei Sun, Siqian Yang, Yabiao Wang, Chengjie Wang, Yanwei Fu
• Abstract summary: Few-shot learning aims to transfer knowledge learned from base with sufficient categories labelled data to novel categories with scarce known information. We propose a novel data augmentation strategy dubbed as PatchMix that can break this spurious dependency. We show that such an augmentation mechanism, different from existing ones, is able to identify the causal features.
• Score: 55.64873998196191
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The task of Few-shot learning (FSL) aims to transfer the knowledge learned from base categories with sufficient labelled data to novel categories with scarce known information. It is currently an important research question and has great practical values in the real-world applications. Despite extensive previous efforts are made on few-shot learning tasks, we emphasize that most existing methods did not take into account the distributional shift caused by sample selection bias in the FSL scenario. Such a selection bias can induce spurious correlation between the semantic causal features, that are causally and semantically related to the class label, and the other non-causal features. Critically, the former ones should be invariant across changes in distributions, highly related to the classes of interest, and thus well generalizable to novel classes, while the latter ones are not stable to changes in the distribution. To resolve this problem, we propose a novel data augmentation strategy dubbed as PatchMix that can break this spurious dependency by replacing the patch-level information and supervision of the query images with random gallery images from different classes from the query ones. We theoretically show that such an augmentation mechanism, different from existing ones, is able to identify the causal features. To further make these features to be discriminative enough for classification, we propose Correlation-guided Reconstruction (CGR) and Hardness-Aware module for instance discrimination and easier discrimination between similar classes. Moreover, such a framework can be adapted to the unsupervised FSL scenario.

Related papers

• CLOSER: Towards Better Representation Learning for Few-Shot Class-Incremental Learning [52.63674911541416]
  Few-shot class-incremental learning (FSCIL) faces several challenges, such as overfitting and forgetting. Our primary focus is representation learning on base classes to tackle the unique challenge of FSCIL. We find that trying to secure the spread of features within a more confined feature space enables the learned representation to strike a better balance between transferability and discriminability.
  arXiv  Detail & Related papers  (2024-10-08T02:23:16Z)
• Semantic Enhanced Few-shot Object Detection [37.715912401900745]
  We propose a fine-tuning based FSOD framework that utilizes semantic embeddings for better detection. Our method allows each novel class to construct a compact feature space without being confused with similar base classes.
  arXiv  Detail & Related papers  (2024-06-19T12:40:55Z)
• Enlarging Instance-specific and Class-specific Information for Open-set Action Recognition [47.69171542776917]
  We find that features with richer semantic diversity can significantly improve the open-set performance under the same uncertainty scores. A novel Prototypical Similarity Learning (PSL) framework is proposed to keep the instance variance within the same class to retain more IS information.
  arXiv  Detail & Related papers  (2023-03-25T04:07:36Z)
• Few-Shot Object Detection via Variational Feature Aggregation [32.34871873486389]
  We propose a meta-learning framework with two novel feature aggregation schemes. We first present a Class-Agnostic Aggregation (CAA) method, where the query and support features can be aggregated regardless of their categories. We then propose a Variational Feature Aggregation (VFA) method, which encodes support examples into class-level support features.
  arXiv  Detail & Related papers  (2023-01-31T04:58:21Z)
• Generalization Bounds for Few-Shot Transfer Learning with Pretrained Classifiers [26.844410679685424]
  We study the ability of foundation models to learn representations for classification that are transferable to new, unseen classes. We show that the few-shot error of the learned feature map on new classes is small in case of class-feature-variability collapse.
  arXiv  Detail & Related papers  (2022-12-23T18:46:05Z)
• Bi-directional Feature Reconstruction Network for Fine-Grained Few-Shot Image Classification [61.411869453639845]
  We introduce a bi-reconstruction mechanism that can simultaneously accommodate for inter-class and intra-class variations. This design effectively helps the model to explore more subtle and discriminative features. Experimental results on three widely used fine-grained image classification datasets consistently show considerable improvements.
  arXiv  Detail & Related papers  (2022-11-30T16:55:14Z)
• Exploring Category-correlated Feature for Few-shot Image Classification [27.13708881431794]
  We present a simple yet effective feature rectification method by exploring the category correlation between novel and base classes as the prior knowledge. The proposed approach consistently obtains considerable performance gains on three widely used benchmarks.
  arXiv  Detail & Related papers  (2021-12-14T08:25:24Z)
• Learning Debiased and Disentangled Representations for Semantic Segmentation [52.35766945827972]
  We propose a model-agnostic and training scheme for semantic segmentation. By randomly eliminating certain class information in each training iteration, we effectively reduce feature dependencies among classes. Models trained with our approach demonstrate strong results on multiple semantic segmentation benchmarks.
  arXiv  Detail & Related papers  (2021-10-31T16:15:09Z)
• Revisiting Deep Local Descriptor for Improved Few-Shot Classification [56.74552164206737]
  We show how one can improve the quality of embeddings by leveraging textbfDense textbfClassification and textbfAttentive textbfPooling. We suggest to pool feature maps by applying attentive pooling instead of the widely used global average pooling (GAP) to prepare embeddings for few-shot classification.
  arXiv  Detail & Related papers  (2021-03-30T00:48:28Z)
• Rethinking Generative Zero-Shot Learning: An Ensemble Learning Perspective for Recognising Visual Patches [52.67723703088284]
  We propose a novel framework called multi-patch generative adversarial nets (MPGAN) MPGAN synthesises local patch features and labels unseen classes with a novel weighted voting strategy. MPGAN has significantly greater accuracy than state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-07-27T05:49:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.