Seeing the Whole in the Parts in Self-Supervised Representation Learning

• URL: http://arxiv.org/abs/2501.02860v1
• Date: Mon, 06 Jan 2025 09:08:59 GMT
• Title: Seeing the Whole in the Parts in Self-Supervised Representation Learning
• Authors: Arthur Aubret, Céline Teulière, Jochen Triesch,
• Abstract summary: We propose a new way to model spatial co-occurrences by aligning local representations with a global image representation. We present CO-SSL, a family of instance discrimination methods, and show that it outperforms previous methods on several datasets.
• Score: 7.324459578044213
• License:
• Abstract: Recent successes in self-supervised learning (SSL) model spatial co-occurrences of visual features either by masking portions of an image or by aggressively cropping it. Here, we propose a new way to model spatial co-occurrences by aligning local representations (before pooling) with a global image representation. We present CO-SSL, a family of instance discrimination methods and show that it outperforms previous methods on several datasets, including ImageNet-1K where it achieves 71.5% of Top-1 accuracy with 100 pre-training epochs. CO-SSL is also more robust to noise corruption, internal corruption, small adversarial attacks, and large training crop sizes. Our analysis further indicates that CO-SSL learns highly redundant local representations, which offers an explanation for its robustness. Overall, our work suggests that aligning local and global representations may be a powerful principle of unsupervised category learning.

Related papers

• On Higher Adversarial Susceptibility of Contrastive Self-Supervised Learning [104.00264962878956]
  Contrastive self-supervised learning (CSL) has managed to match or surpass the performance of supervised learning in image and video classification. It is still largely unknown if the nature of the representation induced by the two learning paradigms is similar. We identify the uniform distribution of data representation over a unit hypersphere in the CSL representation space as the key contributor to this phenomenon. We devise strategies that are simple, yet effective in improving model robustness with CSL training.
  arXiv  Detail & Related papers  (2022-07-22T03:49:50Z)
• Bag of Image Patch Embedding Behind the Success of Self-Supervised Learning [12.480529556920974]
  This work shows that joint-embedding SSL approaches learn a representation of image patches, which reflects their co-occurrence. We empirically show that learning a representation for fixed-scale patches and aggregating local patch representations as the image representation achieves similar or even better results than the baseline methods.
  arXiv  Detail & Related papers  (2022-06-17T18:11:23Z)
• ReCLIP: A Strong Zero-Shot Baseline for Referring Expression Comprehension [114.85628613911713]
  Large-scale pre-trained models are useful for image classification across domains. We present ReCLIP, a simple but strong zero-shot baseline that repurposes CLIP, a state-of-the-art large-scale model, for ReC.
  arXiv  Detail & Related papers  (2022-04-12T17:55:38Z)
• Learning Where to Learn in Cross-View Self-Supervised Learning [54.14989750044489]
  Self-supervised learning (SSL) has made enormous progress and largely narrowed the gap with supervised ones. Current methods simply adopt uniform aggregation of pixels for embedding. We present a new approach, Learning Where to Learn (LEWEL), to adaptively aggregate spatial information of features.
  arXiv  Detail & Related papers  (2022-03-28T17:02:42Z)
• UniVIP: A Unified Framework for Self-Supervised Visual Pre-training [50.87603616476038]
  We propose a novel self-supervised framework to learn versatile visual representations on either single-centric-object or non-iconic dataset. Massive experiments show that UniVIP pre-trained on non-iconic COCO achieves state-of-the-art transfer performance. Our method can also exploit single-centric-object dataset such as ImageNet and outperforms BYOL by 2.5% with the same pre-training epochs in linear probing.
  arXiv  Detail & Related papers  (2022-03-14T10:04:04Z)
• CoSSL: Co-Learning of Representation and Classifier for Imbalanced Semi-Supervised Learning [98.89092930354273]
  We propose a novel co-learning framework (CoSSL) with decoupled representation learning and classifier learning for imbalanced SSL. To handle the data imbalance, we devise Tail-class Feature Enhancement (TFE) for classifier learning. In experiments, we show that our approach outperforms other methods over a large range of shifted distributions.
  arXiv  Detail & Related papers  (2021-12-08T20:13:13Z)
• Spatially Consistent Representation Learning [12.120041613482558]
  We propose a spatially consistent representation learning algorithm (SCRL) for multi-object and location-specific tasks. We devise a novel self-supervised objective that tries to produce coherent spatial representations of a randomly cropped local region. On various downstream localization tasks with benchmark datasets, the proposed SCRL shows significant performance improvements.
  arXiv  Detail & Related papers  (2021-03-10T15:23:45Z)
• PGL: Prior-Guided Local Self-supervised Learning for 3D Medical Image Segmentation [87.50205728818601]
  We propose a PriorGuided Local (PGL) self-supervised model that learns the region-wise local consistency in the latent feature space. Our PGL model learns the distinctive representations of local regions, and hence is able to retain structural information.
  arXiv  Detail & Related papers  (2020-11-25T11:03:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.