Discovering Common Information in Multi-view Data

• URL: http://arxiv.org/abs/2406.15043v1
• Date: Fri, 21 Jun 2024 10:47:06 GMT
• Title: Discovering Common Information in Multi-view Data
• Authors: Qi Zhang, Mingfei Lu, Shujian Yu, Jingmin Xin, Badong Chen,
• Abstract summary: We introduce an innovative and mathematically rigorous definition for computing common information from multi-view data. We develop a novel supervised multi-view learning framework to capture both common and unique information.
• Score: 35.37807004353416
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We introduce an innovative and mathematically rigorous definition for computing common information from multi-view data, drawing inspiration from G\'acs-K\"orner common information in information theory. Leveraging this definition, we develop a novel supervised multi-view learning framework to capture both common and unique information. By explicitly minimizing a total correlation term, the extracted common information and the unique information from each view are forced to be independent of each other, which, in turn, theoretically guarantees the effectiveness of our framework. To estimate information-theoretic quantities, our framework employs matrix-based R{\'e}nyi's $\alpha$-order entropy functional, which forgoes the need for variational approximation and distributional estimation in high-dimensional space. Theoretical proof is provided that our framework can faithfully discover both common and unique information from multi-view data. Experiments on synthetic and seven benchmark real-world datasets demonstrate the superior performance of our proposed framework over state-of-the-art approaches.

Related papers

• Neuro-Inspired Information-Theoretic Hierarchical Perception for Multimodal Learning [16.8379583872582]
  We develop the Information-Theoretic Hierarchical Perception (ITHP) model, which utilizes the concept of information bottleneck. We show that ITHP consistently distills crucial information in multimodal learning scenarios, outperforming state-of-the-art benchmarks.
  arXiv  Detail & Related papers  (2024-04-15T01:34:44Z)
• NativE: Multi-modal Knowledge Graph Completion in the Wild [51.80447197290866]
  We propose a comprehensive framework NativE to achieve MMKGC in the wild. NativE proposes a relation-guided dual adaptive fusion module that enables adaptive fusion for any modalities. We construct a new benchmark called WildKGC with five datasets to evaluate our method.
  arXiv  Detail & Related papers  (2024-03-28T03:04:00Z)
• Towards an Information Theoretic Framework of Context-Based Offline Meta-Reinforcement Learning [50.976910714839065]
  Context-based OMRL (COMRL) as a popular paradigm, aims to learn a universal policy conditioned on effective task representations. We show that COMRL algorithms are essentially optimizing the same mutual information objective between the task variable $boldsymbolM$ and its latent representation $boldsymbolZ$ by implementing various approximate bounds. Based on the theoretical insight and the information bottleneck principle, we arrive at a novel algorithm dubbed UNICORN, which exhibits remarkable generalization across a broad spectrum of RL benchmarks.
  arXiv  Detail & Related papers  (2024-02-04T09:58:42Z)
• TCGF: A unified tensorized consensus graph framework for multi-view representation learning [27.23929515170454]
  This paper proposes a universal multi-view representation learning framework named Consensus Graph Framework (TCGF) It first provides a unified framework for existing multi-view works to exploit the representations for individual view. Then, stacks them into a tensor under alignment basics as a high-order representation, allowing for the smooth propagation of consistency.
  arXiv  Detail & Related papers  (2023-09-14T19:29:14Z)
• Information Screening whilst Exploiting! Multimodal Relation Extraction with Feature Denoising and Multimodal Topic Modeling [96.75821232222201]
  Existing research on multimodal relation extraction (MRE) faces two co-existing challenges, internal-information over-utilization and external-information under-exploitation. We propose a novel framework that simultaneously implements the idea of internal-information screening and external-information exploiting.
  arXiv  Detail & Related papers  (2023-05-19T14:56:57Z)
• Modeling Multiple Views via Implicitly Preserving Global Consistency and Local Complementarity [61.05259660910437]
  We propose a global consistency and complementarity network (CoCoNet) to learn representations from multiple views. On the global stage, we reckon that the crucial knowledge is implicitly shared among views, and enhancing the encoder to capture such knowledge can improve the discriminability of the learned representations. Lastly on the local stage, we propose a complementarity-factor, which joints cross-view discriminative knowledge, and it guides the encoders to learn not only view-wise discriminability but also cross-view complementary information.
  arXiv  Detail & Related papers  (2022-09-16T09:24:00Z)
• Dual Representation Learning for One-Step Clustering of Multi-View Data [30.131568561100817]
  We propose a novel one-step multi-view clustering method by exploiting the dual representation of both the common and specific information of different views. With this framework, the representation learning and clustering partition mutually benefit each other, which effectively improve the clustering performance.
  arXiv  Detail & Related papers  (2022-08-30T14:20:26Z)
• Variational Distillation for Multi-View Learning [104.17551354374821]
  We design several variational information bottlenecks to exploit two key characteristics for multi-view representation learning. Under rigorously theoretical guarantee, our approach enables IB to grasp the intrinsic correlation between observations and semantic labels.
  arXiv  Detail & Related papers  (2022-06-20T03:09:46Z)
• Gacs-Korner Common Information Variational Autoencoder [102.89011295243334]
  We propose a notion of common information that allows one to quantify and separate the information that is shared between two random variables. We demonstrate that our formulation allows us to learn semantically meaningful common and unique factors of variation even on high-dimensional data such as images and videos.
  arXiv  Detail & Related papers  (2022-05-24T17:47:26Z)
• Learning Robust Representations via Multi-View Information Bottleneck [41.65544605954621]
  Original formulation requires labeled data to identify superfluous information. We extend this ability to the multi-view unsupervised setting, where two views of the same underlying entity are provided but the label is unknown. A theoretical analysis leads to the definition of a new multi-view model that produces state-of-the-art results on the Sketchy dataset and label-limited versions of the MIR-Flickr dataset.
  arXiv  Detail & Related papers  (2020-02-17T16:01:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.