Semantically Consistent Multi-view Representation Learning

• URL: http://arxiv.org/abs/2303.04366v1
• Date: Wed, 8 Mar 2023 04:27:46 GMT
• Title: Semantically Consistent Multi-view Representation Learning
• Authors: Yiyang Zhou, Qinghai Zheng, Shunshun Bai, Jihua Zhu
• Abstract summary: We propose a novel Semantically Consistent Multi-view Representation Learning (SCMRL) SCMRL excavates underlying multi-view semantic consensus information and utilize the information to guide the unified feature representation learning. Compared with several state-of-the-art algorithms, extensive experiments demonstrate its superiority.
• Score: 11.145085584637744
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this work, we devote ourselves to the challenging task of Unsupervised Multi-view Representation Learning (UMRL), which requires learning a unified feature representation from multiple views in an unsupervised manner. Existing UMRL methods mainly concentrate on the learning process in the feature space while ignoring the valuable semantic information hidden in different views. To address this issue, we propose a novel Semantically Consistent Multi-view Representation Learning (SCMRL), which makes efforts to excavate underlying multi-view semantic consensus information and utilize the information to guide the unified feature representation learning. Specifically, SCMRL consists of a within-view reconstruction module and a unified feature representation learning module, which are elegantly integrated by the contrastive learning strategy to simultaneously align semantic labels of both view-specific feature representations and the learned unified feature representation. In this way, the consensus information in the semantic space can be effectively exploited to constrain the learning process of unified feature representation. Compared with several state-of-the-art algorithms, extensive experiments demonstrate its superiority.

Related papers

• An Information Compensation Framework for Zero-Shot Skeleton-based Action Recognition [49.45660055499103]
  Zero-shot human skeleton-based action recognition aims to construct a model that can recognize actions outside the categories seen during training. Previous research has focused on aligning sequences' visual and semantic spatial distributions. We introduce a new loss function sampling method to obtain a tight and robust representation.
  arXiv  Detail & Related papers  (2024-06-02T06:53:01Z)
• Rethinking Multi-view Representation Learning via Distilled Disentangling [34.14711778177439]
  Multi-view representation learning aims to derive robust representations that are both view-consistent and view-specific from diverse data sources. This paper presents an in-depth analysis of existing approaches in this domain, highlighting the redundancy between view-consistent and view-specific representations. We propose an innovative framework for multi-view representation learning, which incorporates a technique we term 'distilled disentangling'
  arXiv  Detail & Related papers  (2024-03-16T11:21:24Z)
• Self-Supervised Representation Learning with Meta Comprehensive Regularization [11.387994024747842]
  We introduce a module called CompMod with Meta Comprehensive Regularization (MCR), embedded into existing self-supervised frameworks. We update our proposed model through a bi-level optimization mechanism, enabling it to capture comprehensive features. We provide theoretical support for our proposed method from information theory and causal counterfactual perspective.
  arXiv  Detail & Related papers  (2024-03-03T15:53:48Z)
• Constrained Multiview Representation for Self-supervised Contrastive Learning [4.817827522417457]
  We introduce a novel approach predicated on representation distance-based mutual information (MI) for measuring the significance of different views. We harness multi-view representations extracted from the frequency domain, re-evaluating their significance based on mutual information.
  arXiv  Detail & Related papers  (2024-02-05T19:09:33Z)
• A Probabilistic Model Behind Self-Supervised Learning [53.64989127914936]
  In self-supervised learning (SSL), representations are learned via an auxiliary task without annotated labels. We present a generative latent variable model for self-supervised learning. We show that several families of discriminative SSL, including contrastive methods, induce a comparable distribution over representations.
  arXiv  Detail & Related papers  (2024-02-02T13:31:17Z)
• Multi-Task Reinforcement Learning with Mixture of Orthogonal Experts [20.926613438442782]
  Multi-Task Reinforcement Learning (MTRL) tackles the problem of endowing agents with skills that generalize across a variety of problems. To this end, sharing representations plays a fundamental role in capturing both unique and common characteristics of the tasks. We introduce a novel approach for representation learning in MTRL that encapsulates common structures among the tasks using representations to promote diversity.
  arXiv  Detail & Related papers  (2023-11-19T18:09:25Z)
• Multi-View Class Incremental Learning [57.14644913531313]
  Multi-view learning (MVL) has gained great success in integrating information from multiple perspectives of a dataset to improve downstream task performance. This paper investigates a novel paradigm called multi-view class incremental learning (MVCIL), where a single model incrementally classifies new classes from a continual stream of views.
  arXiv  Detail & Related papers  (2023-06-16T08:13:41Z)
• An Empirical Investigation of Representation Learning for Imitation [76.48784376425911]
  Recent work in vision, reinforcement learning, and NLP has shown that auxiliary representation learning objectives can reduce the need for large amounts of expensive, task-specific data. We propose a modular framework for constructing representation learning algorithms, then use our framework to evaluate the utility of representation learning for imitation.
  arXiv  Detail & Related papers  (2022-05-16T11:23:42Z)
• Multi-View representation learning in Multi-Task Scene [4.509968166110557]
  We propose a novel semi-supervised algorithm, termed as Multi-Task Multi-View learning based on Common and Special Features (MTMVCSF) An anti-noise multi-task multi-view algorithm called AN-MTMVCSF is proposed, which has a strong adaptability to noise labels. The effectiveness of these algorithms is proved by a series of well-designed experiments on both real world and synthetic data.
  arXiv  Detail & Related papers  (2022-01-15T11:26:28Z)
• Deep Partial Multi-View Learning [94.39367390062831]
  We propose a novel framework termed Cross Partial Multi-View Networks (CPM-Nets) We fifirst provide a formal defifinition of completeness and versatility for multi-view representation. We then theoretically prove the versatility of the learned latent representations.
  arXiv  Detail & Related papers  (2020-11-12T02:29:29Z)
• Provable Representation Learning for Imitation Learning via Bi-level Optimization [60.059520774789654]
  A common strategy in modern learning systems is to learn a representation that is useful for many tasks. We study this strategy in the imitation learning setting for Markov decision processes (MDPs) where multiple experts' trajectories are available. We instantiate this framework for the imitation learning settings of behavior cloning and observation-alone.
  arXiv  Detail & Related papers  (2020-02-24T21:03:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.