Informed Mixing -- Improving Open Set Recognition via Attribution-based Augmentation

• URL: http://arxiv.org/abs/2505.12803v1
• Date: Mon, 19 May 2025 07:32:06 GMT
• Title: Informed Mixing -- Improving Open Set Recognition via Attribution-based Augmentation
• Authors: Jiawen Xu, Odej Kao, Margret Keuper,
• Abstract summary: Open set recognition (OSR) is devised to address the problem of detecting novel classes during model inference.<n>We propose GradMix, a data augmentation method that dynamically leverages gradient-based attribution maps of the model during training to mask out already learned concepts.<n>Experiments on open set recognition, close set classification, and out-of-distribution detection reveal that our method can often outperform the state-of-the-art.
• Score: 17.110010140066134
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Open set recognition (OSR) is devised to address the problem of detecting novel classes during model inference. Even in recent vision models, this remains an open issue which is receiving increasing attention. Thereby, a crucial challenge is to learn features that are relevant for unseen categories from given data, for which these features might not be discriminative. To facilitate this process and "optimize to learn" more diverse features, we propose GradMix, a data augmentation method that dynamically leverages gradient-based attribution maps of the model during training to mask out already learned concepts. Thus GradMix encourages the model to learn a more complete set of representative features from the same data source. Extensive experiments on open set recognition, close set classification, and out-of-distribution detection reveal that our method can often outperform the state-of-the-art. GradMix can further increase model robustness to corruptions as well as downstream classification performance for self-supervised learning, indicating its benefit for model generalization.

Related papers

• SphOR: A Representation Learning Perspective on Open-set Recognition for Identifying Unknown Classes in Deep Learning Models [1.2172320168050468]
  We introduce SphOR, a representation learning method that models the feature space as a mixture of von Mises-Fisher distributions.<n>This approach enables the use of semantically ambiguous samples during training, to improve the detection of samples from unknown classes.
  arXiv  Detail & Related papers  (2025-03-11T05:06:11Z)
• Learning from Neighbors: Category Extrapolation for Long-Tail Learning [62.30734737735273]
  We offer a novel perspective on long-tail learning, inspired by an observation: datasets with finer granularity tend to be less affected by data imbalance.<n>We introduce open-set auxiliary classes that are visually similar to existing ones, aiming to enhance representation learning for both head and tail classes.<n>To prevent the overwhelming presence of auxiliary classes from disrupting training, we introduce a neighbor-silencing loss.
  arXiv  Detail & Related papers  (2024-10-21T13:06:21Z)
• Enhancing Fine-Grained Visual Recognition in the Low-Data Regime Through Feature Magnitude Regularization [23.78498670529746]
  We introduce a regularization technique to ensure that the magnitudes of the extracted features are evenly distributed. Despite its apparent simplicity, our approach has demonstrated significant performance improvements across various fine-grained visual recognition datasets.
  arXiv  Detail & Related papers  (2024-09-03T07:32:46Z)
• Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapters [65.15700861265432]
  We present a parameter-efficient continual learning framework to alleviate long-term forgetting in incremental learning with vision-language models. Our approach involves the dynamic expansion of a pre-trained CLIP model, through the integration of Mixture-of-Experts (MoE) adapters. To preserve the zero-shot recognition capability of vision-language models, we introduce a Distribution Discriminative Auto-Selector.
  arXiv  Detail & Related papers  (2024-03-18T08:00:23Z)
• Leveraging vision-language models for fair facial attribute classification [19.93324644519412]
  General-purpose vision-language model (VLM) is a rich knowledge source for common sensitive attributes. We analyze the correspondence between VLM predicted and human defined sensitive attribute distribution. Experiments on multiple benchmark facial attribute classification datasets show fairness gains of the model over existing unsupervised baselines.
  arXiv  Detail & Related papers  (2024-03-15T18:37:15Z)
• Exploring Diverse Representations for Open Set Recognition [51.39557024591446]
  Open set recognition (OSR) requires the model to classify samples that belong to closed sets while rejecting unknown samples during test. Currently, generative models often perform better than discriminative models in OSR. We propose a new model, namely Multi-Expert Diverse Attention Fusion (MEDAF), that learns diverse representations in a discriminative way.
  arXiv  Detail & Related papers  (2024-01-12T11:40:22Z)
• Self-Evolution Learning for Mixup: Enhance Data Augmentation on Few-Shot Text Classification Tasks [75.42002070547267]
  We propose a self evolution learning (SE) based mixup approach for data augmentation in text classification. We introduce a novel instance specific label smoothing approach, which linearly interpolates the model's output and one hot labels of the original samples to generate new soft for label mixing up.
  arXiv  Detail & Related papers  (2023-05-22T23:43:23Z)
• Open-Set Recognition with Gradient-Based Representations [16.80077149399317]
  We propose to utilize gradient-based representations to train an unknown detector with instances of known classes only. We show that our gradient-based approach outperforms state-of-the-art methods by up to 11.6% in open-set classification.
  arXiv  Detail & Related papers  (2022-06-16T14:54:12Z)
• Active Learning for Open-set Annotation [38.739845944840454]
  We propose a new active learning framework called LfOSA, which boosts the classification performance with an effective sampling strategy to precisely detect examples from known classes for annotation. The experimental results show that the proposed method can significantly improve the selection quality of known classes, and achieve higher classification accuracy with lower annotation cost than state-of-the-art active learning methods.
  arXiv  Detail & Related papers  (2022-01-18T06:11:51Z)
• No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Data [78.69828864672978]
  A central challenge in training classification models in the real-world federated system is learning with non-IID data. We propose a novel and simple algorithm called Virtual Representations (CCVR), which adjusts the classifier using virtual representations sampled from an approximated ssian mixture model. Experimental results demonstrate that CCVR state-of-the-art performance on popular federated learning benchmarks including CIFAR-10, CIFAR-100, and CINIC-10.
  arXiv  Detail & Related papers  (2021-06-09T12:02:29Z)
• Incremental Learning from Low-labelled Stream Data in Open-Set Video Face Recognition [0.0]
  We propose a novel incremental learning approach which combines a deep features encoder with an Open-Set Dynamic Ensembles of SVM. Our method can use unsupervised operational data to enhance recognition. Results show a benefit of up to 15% F1-score increase respect to non-adaptive state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-12-17T13:28:13Z)
• Open Set Recognition with Conditional Probabilistic Generative Models [51.40872765917125]
  We propose Conditional Probabilistic Generative Models (CPGM) for open set recognition. CPGM can detect unknown samples but also classify known classes by forcing different latent features to approximate conditional Gaussian distributions. Experiment results on multiple benchmark datasets reveal that the proposed method significantly outperforms the baselines.
  arXiv  Detail & Related papers  (2020-08-12T06:23:49Z)
• Conditional Gaussian Distribution Learning for Open Set Recognition [10.90687687505665]
  We propose Conditional Gaussian Distribution Learning (CGDL) for open set recognition. In addition to detecting unknown samples, this method can also classify known samples by forcing different latent features to approximate different Gaussian models. Experiments on several standard image reveal that the proposed method significantly outperforms the baseline method and achieves new state-of-the-art results.
  arXiv  Detail & Related papers  (2020-03-19T14:32:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.