Self-Supervised Learning from Structural Invariance

• URL: http://arxiv.org/abs/2602.02381v1
• Date: Mon, 02 Feb 2026 17:44:44 GMT
• Title: Self-Supervised Learning from Structural Invariance
• Authors: Yipeng Zhang, Hafez Ghaemi, Jungyoon Lee, Shahab Bakhtiari, Eilif B. Muller, Laurent Charlin,
• Abstract summary: We study the one-to-many mapping problem in joint-embedding self-supervised learning (SSL)<n>We show that existing methods struggle to flexibly capture this conditional uncertainty.<n>We empirically show its versatility in causal representation learning, fine-grained image understanding, and world modeling on videos.
• Score: 6.07374214141791
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Joint-embedding self-supervised learning (SSL), the key paradigm for unsupervised representation learning from visual data, learns from invariances between semantically-related data pairs. We study the one-to-many mapping problem in SSL, where each datum may be mapped to multiple valid targets. This arises when data pairs come from naturally occurring generative processes, e.g., successive video frames. We show that existing methods struggle to flexibly capture this conditional uncertainty. As a remedy, we introduce a latent variable to account for this uncertainty and derive a variational lower bound on the mutual information between paired embeddings. Our derivation yields a simple regularization term for standard SSL objectives. The resulting method, which we call AdaSSL, applies to both contrastive and distillation-based SSL objectives, and we empirically show its versatility in causal representation learning, fine-grained image understanding, and world modeling on videos.

Related papers

• Revisiting semi-supervised learning in the era of foundation models [35.44676829010657]
  Semi-supervised learning (SSL) leverages abundant unlabeled data alongside limited labeled data to enhance learning.<n>We develop new SSL benchmark datasets where frozen vision foundation models (VFMs) underperform and systematically evaluate representative SSL methods.<n>We make a surprising observation: parameter-efficient fine-tuning (PEFT) using only labeled data often matches SSL performance, even without leveraging unlabeled data.<n>To overcome the notorious issue of noisy pseudo-labels, we propose ensembling multiple PEFT approaches and VFM backbones to produce more robust pseudo-labels.
  arXiv  Detail & Related papers  (2025-03-12T18:01:10Z)
• Where Did Your Model Learn That? Label-free Influence for Self-supervised Learning [0.48933451909251774]
  Self-supervised learning has revolutionized learning from large-scale unlabeled datasets.<n>Introductory relationship between pretraining data and learned representations remains poorly understood.<n>We introduce Influence-SSL, a novel and label-free approach for defining influence functions tailored to SSL.
  arXiv  Detail & Related papers  (2024-12-22T21:43:56Z)
• A Closer Look at Benchmarking Self-Supervised Pre-training with Image Classification [51.35500308126506]
  Self-supervised learning (SSL) is a machine learning approach where the data itself provides supervision, eliminating the need for external labels. We study how classification-based evaluation protocols for SSL correlate and how well they predict downstream performance on different dataset types.
  arXiv  Detail & Related papers  (2024-07-16T23:17:36Z)
• A Probabilistic Model Behind Self-Supervised Learning [53.64989127914936]
  In self-supervised learning (SSL), representations are learned via an auxiliary task without annotated labels. We present a generative latent variable model for self-supervised learning. We show that several families of discriminative SSL, including contrastive methods, induce a comparable distribution over representations.
  arXiv  Detail & Related papers  (2024-02-02T13:31:17Z)
• Active Self-Supervised Learning: A Few Low-Cost Relationships Are All You Need [34.013568381942775]
  Self-Supervised Learning (SSL) has emerged as the solution of choice to learn transferable representations from unlabeled data. In this work, we formalize and generalize this principle through Positive Active Learning (PAL) where an oracle queries semantic relationships between samples. First, it unveils a theoretically grounded learning framework beyond SSL, based on similarity graphs, that can be extended to tackle supervised and semi-supervised learning depending on the employed oracle. Second, it provides a consistent algorithm to embed a priori knowledge, e.g. some observed labels, into any SSL losses without any change in the training pipeline.
  arXiv  Detail & Related papers  (2023-03-27T14:44:39Z)
• ReSSL: Relational Self-Supervised Learning with Weak Augmentation [68.47096022526927]
  Self-supervised learning has achieved great success in learning visual representations without data annotations. We introduce a novel relational SSL paradigm that learns representations by modeling the relationship between different instances. Our proposed ReSSL significantly outperforms the previous state-of-the-art algorithms in terms of both performance and training efficiency.
  arXiv  Detail & Related papers  (2021-07-20T06:53:07Z)
• Self-Supervised Learning with Kernel Dependence Maximization [23.618292038419654]
  We propose Self-Supervised Learning with the Hilbert-Schmidt Independence Criterion (SSL-HSIC) SSL-HSIC maximizes dependence between representations of transformed versions of an image and the image identity. This self-supervised learning framework yields a new understanding of InfoNCE, a variational lower bound on the mutual information (MI) between different transformations.
  arXiv  Detail & Related papers  (2021-06-15T17:51:16Z)
• Self-Supervised Learning of Graph Neural Networks: A Unified Review [50.71341657322391]
  Self-supervised learning is emerging as a new paradigm for making use of large amounts of unlabeled samples. We provide a unified review of different ways of training graph neural networks (GNNs) using SSL. Our treatment of SSL methods for GNNs sheds light on the similarities and differences of various methods, setting the stage for developing new methods and algorithms.
  arXiv  Detail & Related papers  (2021-02-22T03:43:45Z)
• On Data-Augmentation and Consistency-Based Semi-Supervised Learning [77.57285768500225]
  Recently proposed consistency-based Semi-Supervised Learning (SSL) methods have advanced the state of the art in several SSL tasks. Despite these advances, the understanding of these methods is still relatively limited.
  arXiv  Detail & Related papers  (2021-01-18T10:12:31Z)
• Information Bottleneck Constrained Latent Bidirectional Embedding for Zero-Shot Learning [59.58381904522967]
  We propose a novel embedding based generative model with a tight visual-semantic coupling constraint. We learn a unified latent space that calibrates the embedded parametric distributions of both visual and semantic spaces. Our method can be easily extended to transductive ZSL setting by generating labels for unseen images.
  arXiv  Detail & Related papers  (2020-09-16T03:54:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.