SMCL: Saliency Masked Contrastive Learning for Long-tailed Recognition

• URL: http://arxiv.org/abs/2406.02223v1
• Date: Tue, 4 Jun 2024 11:33:40 GMT
• Title: SMCL: Saliency Masked Contrastive Learning for Long-tailed Recognition
• Authors: Sanglee Park, Seung-won Hwang, Jungmin So,
• Abstract summary: We propose saliency masked contrastive learning to mitigate the problem of biased predictions. Our key idea is to mask the important part of an image using saliency detection and use contrastive learning to move the masked image towards minor classes in the feature space. Experiment results show that our method achieves state-of-the-art level performance on benchmark long-tailed datasets.
• Score: 19.192861880360347
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Real-world data often follow a long-tailed distribution with a high imbalance in the number of samples between classes. The problem with training from imbalanced data is that some background features, common to all classes, can be unobserved in classes with scarce samples. As a result, this background correlates to biased predictions into ``major" classes. In this paper, we propose saliency masked contrastive learning, a new method that uses saliency masking and contrastive learning to mitigate the problem and improve the generalizability of a model. Our key idea is to mask the important part of an image using saliency detection and use contrastive learning to move the masked image towards minor classes in the feature space, so that background features present in the masked image are no longer correlated with the original class. Experiment results show that our method achieves state-of-the-art level performance on benchmark long-tailed datasets.

Related papers

• Improving Masked Autoencoders by Learning Where to Mask [65.89510231743692]
  Masked image modeling is a promising self-supervised learning method for visual data. We present AutoMAE, a framework that uses Gumbel-Softmax to interlink an adversarially-trained mask generator and a mask-guided image modeling process. In our experiments, AutoMAE is shown to provide effective pretraining models on standard self-supervised benchmarks and downstream tasks.
  arXiv  Detail & Related papers  (2023-03-12T05:28:55Z)
• Supervised Contrastive Learning on Blended Images for Long-tailed Recognition [32.876647081080655]
  Real-world data often have a long-tailed distribution, where the number of samples per class is not equal over training classes. In this paper, we propose a novel long-tailed recognition method to balance the latent feature space.
  arXiv  Detail & Related papers  (2022-11-22T01:19:00Z)
• Constructing Balance from Imbalance for Long-tailed Image Recognition [50.6210415377178]
  The imbalance between majority (head) classes and minority (tail) classes severely skews the data-driven deep neural networks. Previous methods tackle with data imbalance from the viewpoints of data distribution, feature space, and model design. We propose a concise paradigm by progressively adjusting label space and dividing the head classes and tail classes. Our proposed model also provides a feature evaluation method and paves the way for long-tailed feature learning.
  arXiv  Detail & Related papers  (2022-08-04T10:22:24Z)
• Constrained Deep One-Class Feature Learning For Classifying Imbalanced Medical Images [4.211466076086617]
  One-class classification has attracted increasing attention to address the data imbalance problem. We propose a novel deep learning-based method to learn compact features. Our method can learn more relevant features associated with the given class, making the majority and minority samples more distinguishable.
  arXiv  Detail & Related papers  (2021-11-20T15:25:24Z)
• Improving Tail-Class Representation with Centroid Contrastive Learning [145.73991900239017]
  We propose interpolative centroid contrastive learning (ICCL) to improve long-tailed representation learning. ICCL interpolates two images from a class-agnostic sampler and a class-aware sampler, and trains the model such that the representation of the ICCL can be used to retrieve the centroids for both source classes. Our result shows a significant accuracy gain of 2.8% on the iNaturalist 2018 dataset with a real-world long-tailed distribution.
  arXiv  Detail & Related papers  (2021-10-19T15:24:48Z)
• STAR: Noisy Semi-Supervised Transfer Learning for Visual Classification [0.8662293148437356]
  Semi-supervised learning (SSL) has proven to be effective at leveraging large-scale unlabeled data. Recent SSL methods rely on unlabeled image data at a scale of billions to work well. We propose noisy semi-supervised transfer learning, which integrates transfer learning and self-training with noisy student.
  arXiv  Detail & Related papers  (2021-08-18T19:35:05Z)
• Rectifying the Shortcut Learning of Background: Shared Object Concentration for Few-Shot Image Recognition [101.59989523028264]
  Few-Shot image classification aims to utilize pretrained knowledge learned from a large-scale dataset to tackle a series of downstream classification tasks. We propose COSOC, a novel Few-Shot Learning framework, to automatically figure out foreground objects at both pretraining and evaluation stage.
  arXiv  Detail & Related papers  (2021-07-16T07:46:41Z)
• Background Splitting: Finding Rare Classes in a Sea of Background [55.03789745276442]
  We focus on the real-world problem of training accurate deep models for image classification of a small number of rare categories. In these scenarios, almost all images belong to the background category in the dataset (>95% of the dataset is background) We demonstrate that both standard fine-tuning approaches and state-of-the-art approaches for training on imbalanced datasets do not produce accurate deep models in the presence of this extreme imbalance.
  arXiv  Detail & Related papers  (2020-08-28T23:05:15Z)
• Distilling Localization for Self-Supervised Representation Learning [82.79808902674282]
  Contrastive learning has revolutionized unsupervised representation learning. Current contrastive models are ineffective at localizing the foreground object. We propose a data-driven approach for learning in variance to backgrounds.
  arXiv  Detail & Related papers  (2020-04-14T16:29:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.