SASSL: Enhancing Self-Supervised Learning via Neural Style Transfer
- URL: http://arxiv.org/abs/2312.01187v3
- Date: Sat, 3 Feb 2024 23:10:16 GMT
- Title: SASSL: Enhancing Self-Supervised Learning via Neural Style Transfer
- Authors: Renan A. Rojas-Gomez, Karan Singhal, Ali Etemad, Alex Bijamov, Warren
R. Morningstar, Philip Andrew Mansfield
- Abstract summary: SASSL: Style Augmentations for Self Supervised Learning is a novel augmentation technique based on Neural Style Transfer.
Our technique boosts top-1 classification accuracy on ImageNet by up to 2$%$ compared to established self-supervised methods.
- Score: 22.007237455091836
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Existing data augmentation in self-supervised learning, while diverse, fails
to preserve the inherent structure of natural images. This results in distorted
augmented samples with compromised semantic information, ultimately impacting
downstream performance. To overcome this, we propose SASSL: Style Augmentations
for Self Supervised Learning, a novel augmentation technique based on Neural
Style Transfer. SASSL decouples semantic and stylistic attributes in images and
applies transformations exclusively to the style while preserving content,
generating diverse samples that better retain semantics. Our technique boosts
top-1 classification accuracy on ImageNet by up to 2$\%$ compared to
established self-supervised methods like MoCo, SimCLR, and BYOL, while
achieving superior transfer learning performance across various datasets.
Related papers
- Learning at a Glance: Towards Interpretable Data-limited Continual Semantic Segmentation via Semantic-Invariance Modelling [21.114359437665364]
Continual semantic segmentation (CSS) based on incremental learning (IL) is a great endeavour in developing human-like segmentation models.
Current CSS approaches encounter challenges in the trade-off between preserving old knowledge and learning new ones.
We present Learning at a Glance (LAG), an efficient, robust, human-like and interpretable approach for CSS.
arXiv Detail & Related papers (2024-07-22T07:17:52Z) - Can Generative Models Improve Self-Supervised Representation Learning? [0.7999703756441756]
We introduce a novel framework that enriches the self-supervised learning paradigm by utilizing generative models to produce semantically consistent image augmentations.
Our results show that our framework significantly enhances the quality of learned visual representations by up to 10% Top-1 accuracy in downstream tasks.
arXiv Detail & Related papers (2024-03-09T17:17:07Z) - CLAP: Isolating Content from Style through Contrastive Learning with Augmented Prompts [11.752632557524969]
We propose contrastive learning with data augmentation to disentangle content features from the original representations.
Our experiments across diverse datasets demonstrate significant improvements in zero-shot and few-shot classification tasks.
arXiv Detail & Related papers (2023-11-28T03:00:59Z) - Augmentation-aware Self-supervised Learning with Conditioned Projector [15.285739610439093]
Self-supervised learning (SSL) is a powerful technique for learning robust representations from unlabeled data.
We propose to foster sensitivity to characteristics in the representation space by modifying the projector network.
Our approach, coined Conditional Augmentation-aware Self-supervised Learning (CASSLE), is directly applicable to typical joint-embedding SSL methods.
arXiv Detail & Related papers (2023-05-31T12:24:06Z) - ALADIN-NST: Self-supervised disentangled representation learning of
artistic style through Neural Style Transfer [60.6863849241972]
We learn a representation of visual artistic style more strongly disentangled from the semantic content depicted in an image.
We show that strongly addressing the disentanglement of style and content leads to large gains in style-specific metrics.
arXiv Detail & Related papers (2023-04-12T10:33:18Z) - A Unified Arbitrary Style Transfer Framework via Adaptive Contrastive
Learning [84.8813842101747]
Unified Contrastive Arbitrary Style Transfer (UCAST) is a novel style representation learning and transfer framework.
We present an adaptive contrastive learning scheme for style transfer by introducing an input-dependent temperature.
Our framework consists of three key components, i.e., a parallel contrastive learning scheme for style representation and style transfer, a domain enhancement module for effective learning of style distribution, and a generative network for style transfer.
arXiv Detail & Related papers (2023-03-09T04:35:00Z) - Virtual embeddings and self-consistency for self-supervised learning [43.086696088061416]
TriMix is a novel concept for self-supervised learning that generates virtual embeddings through linear data.
We validate TriMix on eight benchmark datasets with an improvement of 2.71% and 0.41% better than the second-best models for both data types.
arXiv Detail & Related papers (2022-06-13T10:20:28Z) - Imposing Consistency for Optical Flow Estimation [73.53204596544472]
Imposing consistency through proxy tasks has been shown to enhance data-driven learning.
This paper introduces novel and effective consistency strategies for optical flow estimation.
arXiv Detail & Related papers (2022-04-14T22:58:30Z) - In-N-Out Generative Learning for Dense Unsupervised Video Segmentation [89.21483504654282]
In this paper, we focus on the unsupervised Video Object (VOS) task which learns visual correspondence from unlabeled videos.
We propose the In-aNd-Out (INO) generative learning from a purely generative perspective, which captures both high-level and fine-grained semantics.
Our INO outperforms previous state-of-the-art methods by significant margins.
arXiv Detail & Related papers (2022-03-29T07:56:21Z) - Weak Augmentation Guided Relational Self-Supervised Learning [80.0680103295137]
We introduce a novel relational self-supervised learning (ReSSL) framework that learns representations by modeling the relationship between different instances.
Our proposed method employs sharpened distribution of pairwise similarities among different instances as textitrelation metric.
Experimental results show that our proposed ReSSL substantially outperforms the state-of-the-art methods across different network architectures.
arXiv Detail & Related papers (2022-03-16T16:14:19Z) - ReSSL: Relational Self-Supervised Learning with Weak Augmentation [68.47096022526927]
Self-supervised learning has achieved great success in learning visual representations without data annotations.
We introduce a novel relational SSL paradigm that learns representations by modeling the relationship between different instances.
Our proposed ReSSL significantly outperforms the previous state-of-the-art algorithms in terms of both performance and training efficiency.
arXiv Detail & Related papers (2021-07-20T06:53:07Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.