Soft Neighbors are Positive Supporters in Contrastive Visual Representation Learning

• URL: http://arxiv.org/abs/2303.17142v1
• Date: Thu, 30 Mar 2023 04:22:07 GMT
• Title: Soft Neighbors are Positive Supporters in Contrastive Visual Representation Learning
• Authors: Chongjian Ge, Jiangliu Wang, Zhan Tong, Shoufa Chen, Yibing Song, and Ping Luo
• Abstract summary: Contrastive learning methods train visual encoders by comparing views from one instance to others. This binary instance discrimination is studied extensively to improve feature representations in self-supervised learning. In this paper, we rethink the instance discrimination framework and find the binary instance labeling insufficient to measure correlations between different samples.
• Score: 35.53729744330751
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Contrastive learning methods train visual encoders by comparing views from one instance to others. Typically, the views created from one instance are set as positive, while views from other instances are negative. This binary instance discrimination is studied extensively to improve feature representations in self-supervised learning. In this paper, we rethink the instance discrimination framework and find the binary instance labeling insufficient to measure correlations between different samples. For an intuitive example, given a random image instance, there may exist other images in a mini-batch whose content meanings are the same (i.e., belonging to the same category) or partially related (i.e., belonging to a similar category). How to treat the images that correlate similarly to the current image instance leaves an unexplored problem. We thus propose to support the current image by exploring other correlated instances (i.e., soft neighbors). We first carefully cultivate a candidate neighbor set, which will be further utilized to explore the highly-correlated instances. A cross-attention module is then introduced to predict the correlation score (denoted as positiveness) of other correlated instances with respect to the current one. The positiveness score quantitatively measures the positive support from each correlated instance, and is encoded into the objective for pretext training. To this end, our proposed method benefits in discriminating uncorrelated instances while absorbing correlated instances for SSL. We evaluate our soft neighbor contrastive learning method (SNCLR) on standard visual recognition benchmarks, including image classification, object detection, and instance segmentation. The state-of-the-art recognition performance shows that SNCLR is effective in improving feature representations from both ViT and CNN encoders.

Related papers

• Semantic Positive Pairs for Enhancing Visual Representation Learning of Instance Discrimination methods [4.680881326162484]
  Self-supervised learning algorithms (SSL) based on instance discrimination have shown promising results. We propose an approach to identify those images with similar semantic content and treat them as positive instances. We run experiments on three benchmark datasets: ImageNet, STL-10 and CIFAR-10 with different instance discrimination SSL approaches.
  arXiv  Detail & Related papers  (2023-06-28T11:47:08Z)
• Similarity Contrastive Estimation for Image and Video Soft Contrastive Self-Supervised Learning [0.22940141855172028]
  We propose a novel formulation of contrastive learning using semantic similarity between instances. Our training objective is a soft contrastive one that brings the positives closer and estimates a continuous distribution to push or pull negative instances. We show that SCE reaches state-of-the-art results for pretraining video representation and that the learned representation can generalize to video downstream tasks.
  arXiv  Detail & Related papers  (2022-12-21T16:56:55Z)
• Beyond Instance Discrimination: Relation-aware Contrastive Self-supervised Learning [75.46664770669949]
  We present relation-aware contrastive self-supervised learning (ReCo) to integrate instance relations. Our ReCo consistently gains remarkable performance improvements.
  arXiv  Detail & Related papers  (2022-11-02T03:25:28Z)
• Not All Instances Contribute Equally: Instance-adaptive Class Representation Learning for Few-Shot Visual Recognition [94.04041301504567]
  Few-shot visual recognition refers to recognize novel visual concepts from a few labeled instances. We propose a novel metric-based meta-learning framework termed instance-adaptive class representation learning network (ICRL-Net) for few-shot visual recognition.
  arXiv  Detail & Related papers  (2022-09-07T10:00:18Z)
• Similarity Contrastive Estimation for Self-Supervised Soft Contrastive Learning [0.41998444721319206]
  We argue that a good data representation contains the relations, or semantic similarity, between the instances. We propose a novel formulation of contrastive learning using semantic similarity between instances called Similarity Contrastive Estimation (SCE) Our training objective can be considered as soft contrastive learning.
  arXiv  Detail & Related papers  (2021-11-29T15:19:15Z)
• Weakly Supervised Contrastive Learning [68.47096022526927]
  We introduce a weakly supervised contrastive learning framework (WCL) to tackle this issue. WCL achieves 65% and 72% ImageNet Top-1 Accuracy using ResNet50, which is even higher than SimCLRv2 with ResNet101.
  arXiv  Detail & Related papers  (2021-10-10T12:03:52Z)
• Beyond Single Instance Multi-view Unsupervised Representation Learning [21.449132256091662]
  We impose more accurate instance discrimination capability by measuring the joint similarity between two randomly sampled instances. We believe that learning joint similarity helps to improve the performance when encoded features are distributed more evenly in the latent space.
  arXiv  Detail & Related papers  (2020-11-26T15:43:27Z)
• Unsupervised Feature Learning by Cross-Level Instance-Group Discrimination [68.83098015578874]
  We integrate between-instance similarity into contrastive learning, not directly by instance grouping, but by cross-level discrimination. CLD effectively brings unsupervised learning closer to natural data and real-world applications. New state-of-the-art on self-supervision, semi-supervision, and transfer learning benchmarks, and beats MoCo v2 and SimCLR on every reported performance.
  arXiv  Detail & Related papers  (2020-08-09T21:13:13Z)
• Demystifying Contrastive Self-Supervised Learning: Invariances, Augmentations and Dataset Biases [34.02639091680309]
  Recent gains in performance come from training instance classification models, treating each image and it's augmented versions as samples of a single class. We demonstrate that approaches like MOCO and PIRL learn occlusion-invariant representations. Second, we demonstrate that these approaches obtain further gains from access to a clean object-centric training dataset like Imagenet.
  arXiv  Detail & Related papers  (2020-07-28T00:11:31Z)
• Whitening for Self-Supervised Representation Learning [129.57407186848917]
  We propose a new loss function for self-supervised representation learning (SSL) based on the whitening of latent-space features. Our solution does not require asymmetric networks and it is conceptually simple.
  arXiv  Detail & Related papers  (2020-07-13T12:33:25Z)
• Memory-Augmented Relation Network for Few-Shot Learning [114.47866281436829]
  In this work, we investigate a new metric-learning method, Memory-Augmented Relation Network (MRN) In MRN, we choose the samples that are visually similar from the working context, and perform weighted information propagation to attentively aggregate helpful information from chosen ones to enhance its representation. We empirically demonstrate that MRN yields significant improvement over its ancestor and achieves competitive or even better performance when compared with other few-shot learning approaches.
  arXiv  Detail & Related papers  (2020-05-09T10:09:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.