Self-Supervised Learning as Discrete Communication

• URL: http://arxiv.org/abs/2602.09764v1
• Date: Tue, 10 Feb 2026 13:24:06 GMT
• Title: Self-Supervised Learning as Discrete Communication
• Authors: Kawtar Zaher, Ilyass Moummad, Olivier Buisson, Alexis Joly,
• Abstract summary: Most self-supervised learning (SSL) methods learn continuous visual representations by aligning different views of the same input.<n>We frame visual self-supervised learning as a discrete communication process between a teacher and a student network.<n>We show that periodically reinitializing the projection head strengthens this effect by encouraging embeddings that remain predictive.
• Score: 3.33876524834826
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Most self-supervised learning (SSL) methods learn continuous visual representations by aligning different views of the same input, offering limited control over how information is structured across representation dimensions. In this work, we frame visual self-supervised learning as a discrete communication process between a teacher and a student network, where semantic information is transmitted through a fixed-capacity binary channel. Rather than aligning continuous features, the student predicts multi-label binary messages produced by the teacher. Discrete agreement is enforced through an element-wise binary cross-entropy objective, while a coding-rate regularization term encourages effective utilization of the constrained channel, promoting structured representations. We further show that periodically reinitializing the projection head strengthens this effect by encouraging embeddings that remain predictive across multiple discrete encodings. Extensive experiments demonstrate consistent improvements over continuous agreement baselines on image classification, retrieval, and dense visual prediction tasks, as well as under domain shift through self-supervised adaptation. Beyond backbone representations, we analyze the learned binary codes and show that they form a compact and informative discrete language, capturing semantic factors reusable across classes.

Related papers

• TeachPro: Multi-Label Qualitative Teaching Evaluation via Cross-View Graph Synergy and Semantic Anchored Evidence Encoding [12.647867305450289]
  We propose TeachPro, a multi-label learning framework that systematically assesses five key teaching dimensions.<n>We first propose a Dimension-Anchored Evidence, which integrates three core components.<n>We then propose a Cross-View Graph Synergy Network to represent student comments.
  arXiv  Detail & Related papers  (2026-01-14T07:27:57Z)
• Patch-Level Kernel Alignment for Dense Self-Supervised Learning [7.5866326278176075]
  We introduce Patch-level Kernel Alignment (PaKA), a non-parametric, kernel-based approach that improves the dense representations of pretrained vision encoders with a post-(pre)training.<n>Our framework improves dense representations by conducting a lightweight post-training stage on top of a pretrained model.<n>With only 14 hours of additional training on a single GPU, our method achieves state-of-the-art performance across a range of dense vision benchmarks.
  arXiv  Detail & Related papers  (2025-09-06T05:42:32Z)
• Implicit Counterfactual Learning for Audio-Visual Segmentation [50.69377287012591]
  We propose the implicit counterfactual framework (ICF) to achieve unbiased cross-modal understanding.<n>Due to the lack of semantics, heterogeneous representations may lead to erroneous matches.<n>We introduce the multi-granularity implicit text (MIT) involving video-, segment- and frame-level as the bridge to establish the modality-shared space.
  arXiv  Detail & Related papers  (2025-07-28T11:46:35Z)
• Dude: Dual Distribution-Aware Context Prompt Learning For Large Vision-Language Model [27.56988000960972]
  We introduce a new framework based on a dual context of both domain-shared and class-specific contexts. Such dual prompt methods enhance the model's feature representation by joining implicit and explicit factors encoded in Large Language Models. We also formulate the Unbalanced Optimal Transport (UOT) theory to quantify the relationships between constructed prompts and visual tokens.
  arXiv  Detail & Related papers  (2024-07-05T13:15:29Z)
• Harmony: A Joint Self-Supervised and Weakly-Supervised Framework for Learning General Purpose Visual Representations [6.990891188823598]
  We present Harmony, a framework that combines vision-language training with discriminative and generative self-supervision.<n>Our framework is specifically designed to work on web-scraped data by not relying on negative examples in the self-supervised learning path.<n>We evaluate Harmony across various vision downstream tasks and find that it significantly outperforms the baseline CLIP.
  arXiv  Detail & Related papers  (2024-05-23T07:18:08Z)
• Continual Vision-Language Representation Learning with Off-Diagonal Information [112.39419069447902]
  Multi-modal contrastive learning frameworks like CLIP typically require a large amount of image-text samples for training. This paper discusses the feasibility of continual CLIP training using streaming data.
  arXiv  Detail & Related papers  (2023-05-11T08:04:46Z)
• Cross-modal Representation Learning for Zero-shot Action Recognition [67.57406812235767]
  We present a cross-modal Transformer-based framework, which jointly encodes video data and text labels for zero-shot action recognition (ZSAR) Our model employs a conceptually new pipeline by which visual representations are learned in conjunction with visual-semantic associations in an end-to-end manner. Experiment results show our model considerably improves upon the state of the arts in ZSAR, reaching encouraging top-1 accuracy on UCF101, HMDB51, and ActivityNet benchmark datasets.
  arXiv  Detail & Related papers  (2022-05-03T17:39:27Z)
• Dense Contrastive Visual-Linguistic Pretraining [53.61233531733243]
  Several multimodal representation learning approaches have been proposed that jointly represent image and text. These approaches achieve superior performance by capturing high-level semantic information from large-scale multimodal pretraining. We propose unbiased Dense Contrastive Visual-Linguistic Pretraining to replace the region regression and classification with cross-modality region contrastive learning.
  arXiv  Detail & Related papers  (2021-09-24T07:20:13Z)
• Co$^2$L: Contrastive Continual Learning [69.46643497220586]
  Recent breakthroughs in self-supervised learning show that such algorithms learn visual representations that can be transferred better to unseen tasks. We propose a rehearsal-based continual learning algorithm that focuses on continually learning and maintaining transferable representations.
  arXiv  Detail & Related papers  (2021-06-28T06:14:38Z)
• Cross-Modal Discrete Representation Learning [73.68393416984618]
  We present a self-supervised learning framework that learns a representation that captures finer levels of granularity across different modalities. Our framework relies on a discretized embedding space created via vector quantization that is shared across different modalities.
  arXiv  Detail & Related papers  (2021-06-10T00:23:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.