Towards the Generalization of Multi-view Learning: An Information-theoretical Analysis

• URL: http://arxiv.org/abs/2501.16768v1
• Date: Tue, 28 Jan 2025 07:47:19 GMT
• Title: Towards the Generalization of Multi-view Learning: An Information-theoretical Analysis
• Authors: Wen Wen, Tieliang Gong, Yuxin Dong, Shujian Yu, Weizhan Zhang,
• Abstract summary: We develop information-theoretic generalization bounds for multi-view learning. We derive novel data-dependent bounds under both leave-one-out and supersample settings. In the interpolating regime, we further establish the fast-rate bound for multi-view learning.
• Score: 28.009990407017618
• License:
• Abstract: Multiview learning has drawn widespread attention for its efficacy in leveraging cross-view consensus and complementarity information to achieve a comprehensive representation of data. While multi-view learning has undergone vigorous development and achieved remarkable success, the theoretical understanding of its generalization behavior remains elusive. This paper aims to bridge this gap by developing information-theoretic generalization bounds for multi-view learning, with a particular focus on multi-view reconstruction and classification tasks. Our bounds underscore the importance of capturing both consensus and complementary information from multiple different views to achieve maximally disentangled representations. These results also indicate that applying the multi-view information bottleneck regularizer is beneficial for satisfactory generalization performance. Additionally, we derive novel data-dependent bounds under both leave-one-out and supersample settings, yielding computational tractable and tighter bounds. In the interpolating regime, we further establish the fast-rate bound for multi-view learning, exhibiting a faster convergence rate compared to conventional square-root bounds. Numerical results indicate a strong correlation between the true generalization gap and the derived bounds across various learning scenarios.

Related papers

• Discovering Common Information in Multi-view Data [35.37807004353416]
  We introduce an innovative and mathematically rigorous definition for computing common information from multi-view data. We develop a novel supervised multi-view learning framework to capture both common and unique information.
  arXiv  Detail & Related papers  (2024-06-21T10:47:06Z)
• TCGF: A unified tensorized consensus graph framework for multi-view representation learning [27.23929515170454]
  This paper proposes a universal multi-view representation learning framework named Consensus Graph Framework (TCGF) It first provides a unified framework for existing multi-view works to exploit the representations for individual view. Then, stacks them into a tensor under alignment basics as a high-order representation, allowing for the smooth propagation of consistency.
  arXiv  Detail & Related papers  (2023-09-14T19:29:14Z)
• Cross-view Graph Contrastive Representation Learning on Partially Aligned Multi-view Data [52.491074276133325]
  Multi-view representation learning has developed rapidly over the past decades and has been applied in many fields. We propose a new cross-view graph contrastive learning framework, which integrates multi-view information to align data and learn latent representations. Experiments conducted on several real datasets demonstrate the effectiveness of the proposed method on the clustering and classification tasks.
  arXiv  Detail & Related papers  (2022-11-08T09:19:32Z)
• Modeling Multiple Views via Implicitly Preserving Global Consistency and Local Complementarity [61.05259660910437]
  We propose a global consistency and complementarity network (CoCoNet) to learn representations from multiple views. On the global stage, we reckon that the crucial knowledge is implicitly shared among views, and enhancing the encoder to capture such knowledge can improve the discriminability of the learned representations. Lastly on the local stage, we propose a complementarity-factor, which joints cross-view discriminative knowledge, and it guides the encoders to learn not only view-wise discriminability but also cross-view complementary information.
  arXiv  Detail & Related papers  (2022-09-16T09:24:00Z)
• Latent Heterogeneous Graph Network for Incomplete Multi-View Learning [57.49776938934186]
  We propose a novel Latent Heterogeneous Graph Network (LHGN) for incomplete multi-view learning. By learning a unified latent representation, a trade-off between consistency and complementarity among different views is implicitly realized. To avoid any inconsistencies between training and test phase, a transductive learning technique is applied based on graph learning for classification tasks.
  arXiv  Detail & Related papers  (2022-08-29T15:14:21Z)
• Variational Distillation for Multi-View Learning [104.17551354374821]
  We design several variational information bottlenecks to exploit two key characteristics for multi-view representation learning. Under rigorously theoretical guarantee, our approach enables IB to grasp the intrinsic correlation between observations and semantic labels.
  arXiv  Detail & Related papers  (2022-06-20T03:09:46Z)
• A Variational Information Bottleneck Approach to Multi-Omics Data Integration [98.6475134630792]
  We propose a deep variational information bottleneck (IB) approach for incomplete multi-view observations. Our method applies the IB framework on marginal and joint representations of the observed views to focus on intra-view and inter-view interactions that are relevant for the target. Experiments on real-world datasets show that our method consistently achieves gain from data integration and outperforms state-of-the-art benchmarks.
  arXiv  Detail & Related papers  (2021-02-05T06:05:39Z)
• Deep Partial Multi-View Learning [94.39367390062831]
  We propose a novel framework termed Cross Partial Multi-View Networks (CPM-Nets) We fifirst provide a formal defifinition of completeness and versatility for multi-view representation. We then theoretically prove the versatility of the learned latent representations.
  arXiv  Detail & Related papers  (2020-11-12T02:29:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.