Rethinking Multi-view Representation Learning via Distilled Disentangling

• URL: http://arxiv.org/abs/2403.10897v2
• Date: Fri, 29 Mar 2024 14:49:11 GMT
• Title: Rethinking Multi-view Representation Learning via Distilled Disentangling
• Authors: Guanzhou Ke, Bo Wang, Xiaoli Wang, Shengfeng He,
• Abstract summary: Multi-view representation learning aims to derive robust representations that are both view-consistent and view-specific from diverse data sources. This paper presents an in-depth analysis of existing approaches in this domain, highlighting the redundancy between view-consistent and view-specific representations. We propose an innovative framework for multi-view representation learning, which incorporates a technique we term 'distilled disentangling'
• Score: 34.14711778177439
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Multi-view representation learning aims to derive robust representations that are both view-consistent and view-specific from diverse data sources. This paper presents an in-depth analysis of existing approaches in this domain, highlighting a commonly overlooked aspect: the redundancy between view-consistent and view-specific representations. To this end, we propose an innovative framework for multi-view representation learning, which incorporates a technique we term 'distilled disentangling'. Our method introduces the concept of masked cross-view prediction, enabling the extraction of compact, high-quality view-consistent representations from various sources without incurring extra computational overhead. Additionally, we develop a distilled disentangling module that efficiently filters out consistency-related information from multi-view representations, resulting in purer view-specific representations. This approach significantly reduces redundancy between view-consistent and view-specific representations, enhancing the overall efficiency of the learning process. Our empirical evaluations reveal that higher mask ratios substantially improve the quality of view-consistent representations. Moreover, we find that reducing the dimensionality of view-consistent representations relative to that of view-specific representations further refines the quality of the combined representations. Our code is accessible at: https://github.com/Guanzhou-Ke/MRDD.

Related papers

• Multi-view Aggregation Network for Dichotomous Image Segmentation [76.75904424539543]
  Dichotomous Image (DIS) has recently emerged towards high-precision object segmentation from high-resolution natural images. Existing methods rely on tedious multiple encoder-decoder streams and stages to gradually complete the global localization and local refinement. Inspired by it, we model DIS as a multi-view object perception problem and provide a parsimonious multi-view aggregation network (MVANet) Experiments on the popular DIS-5K dataset show that our MVANet significantly outperforms state-of-the-art methods in both accuracy and speed.
  arXiv  Detail & Related papers  (2024-04-11T03:00:00Z)
• MVEB: Self-Supervised Learning with Multi-View Entropy Bottleneck [53.44358636312935]
  Self-supervised approaches regard two views of an image as both the input and the self-supervised signals. Recent studies show that discarding superfluous information not shared between the views can improve generalization. We propose an objective multi-view entropy bottleneck (MVEB) to learn minimal sufficient representation effectively.
  arXiv  Detail & Related papers  (2024-03-28T00:50:02Z)
• Neural Clustering based Visual Representation Learning [61.72646814537163]
  Clustering is one of the most classic approaches in machine learning and data analysis. We propose feature extraction with clustering (FEC), which views feature extraction as a process of selecting representatives from data. FEC alternates between grouping pixels into individual clusters to abstract representatives and updating the deep features of pixels with current representatives.
  arXiv  Detail & Related papers  (2024-03-26T06:04:50Z)
• Constrained Multiview Representation for Self-supervised Contrastive Learning [4.817827522417457]
  We introduce a novel approach predicated on representation distance-based mutual information (MI) for measuring the significance of different views. We harness multi-view representations extracted from the frequency domain, re-evaluating their significance based on mutual information.
  arXiv  Detail & Related papers  (2024-02-05T19:09:33Z)
• Disentangling Multi-view Representations Beyond Inductive Bias [32.15900989696017]
  We propose a novel multi-view representation disentangling method that ensures both interpretability and generalizability of the resulting representations. Our experiments on four multi-view datasets demonstrate that our proposed method outperforms 12 comparison methods in terms of clustering and classification performance.
  arXiv  Detail & Related papers  (2023-08-03T09:09:28Z)
• A Clustering-guided Contrastive Fusion for Multi-view Representation Learning [7.630965478083513]
  We propose a deep fusion network to fuse view-specific representations into the view-common representation. We also design an asymmetrical contrastive strategy that aligns the view-common representation and each view-specific representation. In the incomplete view scenario, our proposed method resists noise interference better than those of our competitors.
  arXiv  Detail & Related papers  (2022-12-28T07:21:05Z)
• Cross-view Graph Contrastive Representation Learning on Partially Aligned Multi-view Data [52.491074276133325]
  Multi-view representation learning has developed rapidly over the past decades and has been applied in many fields. We propose a new cross-view graph contrastive learning framework, which integrates multi-view information to align data and learn latent representations. Experiments conducted on several real datasets demonstrate the effectiveness of the proposed method on the clustering and classification tasks.
  arXiv  Detail & Related papers  (2022-11-08T09:19:32Z)
• MORI-RAN: Multi-view Robust Representation Learning via Hybrid Contrastive Fusion [4.36488705757229]
  Multi-view representation learning is essential for many multi-view tasks, such as clustering and classification. We propose a hybrid contrastive fusion algorithm to extract robust view-common representation from unlabeled data. Experimental results demonstrated that the proposed method outperforms 12 competitive multi-view methods on four real-world datasets.
  arXiv  Detail & Related papers  (2022-08-26T09:58:37Z)
• Deep Partial Multi-View Learning [94.39367390062831]
  We propose a novel framework termed Cross Partial Multi-View Networks (CPM-Nets) We fifirst provide a formal defifinition of completeness and versatility for multi-view representation. We then theoretically prove the versatility of the learned latent representations.
  arXiv  Detail & Related papers  (2020-11-12T02:29:29Z)
• Generative Partial Multi-View Clustering [133.36721417531734]
  We propose a generative partial multi-view clustering model, named as GP-MVC, to address the incomplete multi-view problem. First, multi-view encoder networks are trained to learn common low-dimensional representations, followed by a clustering layer to capture the consistent cluster structure across multiple views. Second, view-specific generative adversarial networks are developed to generate the missing data of one view conditioning on the shared representation given by other views.
  arXiv  Detail & Related papers  (2020-03-29T17:48:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.