Self-supervised Learning from a Multi-view Perspective
- URL: http://arxiv.org/abs/2006.05576v4
- Date: Mon, 22 Mar 2021 20:40:22 GMT
- Title: Self-supervised Learning from a Multi-view Perspective
- Authors: Yao-Hung Hubert Tsai, Yue Wu, Ruslan Salakhutdinov, Louis-Philippe
Morency
- Abstract summary: We show that self-supervised representations can extract task-relevant information and discard task-irrelevant information.
Our theoretical framework paves the way to a larger space of self-supervised learning objective design.
- Score: 121.63655399591681
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: As a subset of unsupervised representation learning, self-supervised
representation learning adopts self-defined signals as supervision and uses the
learned representation for downstream tasks, such as object detection and image
captioning. Many proposed approaches for self-supervised learning follow
naturally a multi-view perspective, where the input (e.g., original images) and
the self-supervised signals (e.g., augmented images) can be seen as two
redundant views of the data. Building from this multi-view perspective, this
paper provides an information-theoretical framework to better understand the
properties that encourage successful self-supervised learning. Specifically, we
demonstrate that self-supervised learned representations can extract
task-relevant information and discard task-irrelevant information. Our
theoretical framework paves the way to a larger space of self-supervised
learning objective design. In particular, we propose a composite objective that
bridges the gap between prior contrastive and predictive learning objectives,
and introduce an additional objective term to discard task-irrelevant
information. To verify our analysis, we conduct controlled experiments to
evaluate the impact of the composite objectives. We also explore our
framework's empirical generalization beyond the multi-view perspective, where
the cross-view redundancy may not be clearly observed.
Related papers
- A Probabilistic Model Behind Self-Supervised Learning [53.64989127914936]
In self-supervised learning (SSL), representations are learned via an auxiliary task without annotated labels.
We present a generative latent variable model for self-supervised learning.
We show that several families of discriminative SSL, including contrastive methods, induce a comparable distribution over representations.
arXiv Detail & Related papers (2024-02-02T13:31:17Z) - Revisiting Self-supervised Learning of Speech Representation from a
Mutual Information Perspective [68.20531518525273]
We take a closer look into existing self-supervised methods of speech from an information-theoretic perspective.
We use linear probes to estimate the mutual information between the target information and learned representations.
We explore the potential of evaluating representations in a self-supervised fashion, where we estimate the mutual information between different parts of the data without using any labels.
arXiv Detail & Related papers (2024-01-16T21:13:22Z) - Towards Unsupervised Representation Learning: Learning, Evaluating and
Transferring Visual Representations [1.8130068086063336]
We contribute to the field of unsupervised (visual) representation learning from three perspectives.
We design unsupervised, backpropagation-free Convolutional Self-Organizing Neural Networks (CSNNs)
We build upon the widely used (non-)linear evaluation protocol to define pretext- and target-objective-independent metrics.
We contribute CARLANE, the first 3-way sim-to-real domain adaptation benchmark for 2D lane detection, and a method based on self-supervised learning.
arXiv Detail & Related papers (2023-11-30T15:57:55Z) - Semi-supervised learning made simple with self-supervised clustering [65.98152950607707]
Self-supervised learning models have been shown to learn rich visual representations without requiring human annotations.
We propose a conceptually simple yet empirically powerful approach to turn clustering-based self-supervised methods into semi-supervised learners.
arXiv Detail & Related papers (2023-06-13T01:09:18Z) - Matching Multiple Perspectives for Efficient Representation Learning [0.0]
We present an approach that combines self-supervised learning with a multi-perspective matching technique.
We show that the availability of multiple views of the same object combined with a variety of self-supervised pretraining algorithms can lead to improved object classification performance.
arXiv Detail & Related papers (2022-08-16T10:33:13Z) - Self-Supervised Visual Representation Learning with Semantic Grouping [50.14703605659837]
We tackle the problem of learning visual representations from unlabeled scene-centric data.
We propose contrastive learning from data-driven semantic slots, namely SlotCon, for joint semantic grouping and representation learning.
arXiv Detail & Related papers (2022-05-30T17:50:59Z) - Visual Probing: Cognitive Framework for Explaining Self-Supervised Image
Representations [12.485001250777248]
Recently introduced self-supervised methods for image representation learning provide on par or superior results to their fully supervised competitors.
Motivated by this observation, we introduce a novel visual probing framework for explaining the self-supervised models.
We show the effectiveness and applicability of those analogs in the context of explaining self-supervised representations.
arXiv Detail & Related papers (2021-06-21T12:40:31Z) - Which Mutual-Information Representation Learning Objectives are
Sufficient for Control? [80.2534918595143]
Mutual information provides an appealing formalism for learning representations of data.
This paper formalizes the sufficiency of a state representation for learning and representing the optimal policy.
Surprisingly, we find that two of these objectives can yield insufficient representations given mild and common assumptions on the structure of the MDP.
arXiv Detail & Related papers (2021-06-14T10:12:34Z) - Contrastive learning, multi-view redundancy, and linear models [38.80336134485453]
A popular self-supervised approach to representation learning is contrastive learning.
This work provides a theoretical analysis of contrastive learning in the multi-view setting.
arXiv Detail & Related papers (2020-08-24T01:31:47Z) - Self-Supervised Learning Across Domains [33.86614301708017]
We propose to apply a similar approach to the problem of object recognition across domains.
Our model learns the semantic labels in a supervised fashion, and broadens its understanding of the data by learning from self-supervised signals on the same images.
This secondary task helps the network to focus on object shapes, learning concepts like spatial orientation and part correlation, while acting as a regularizer for the classification task.
arXiv Detail & Related papers (2020-07-24T06:19:53Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.