Hierarchical Optimal Transport for Robust Multi-View Learning
- URL: http://arxiv.org/abs/2006.03160v2
- Date: Mon, 8 Jun 2020 14:54:32 GMT
- Title: Hierarchical Optimal Transport for Robust Multi-View Learning
- Authors: Dixin Luo, Hongteng Xu, Lawrence Carin
- Abstract summary: Two assumptions may be questionable in practice, which limits the application of multi-view learning.
We propose a hierarchical optimal transport (HOT) method to mitigate the dependency on these two assumptions.
The HOT method is applicable to both unsupervised and semi-supervised learning, and experimental results show that it performs robustly on both synthetic and real-world tasks.
- Score: 97.21355697826345
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Traditional multi-view learning methods often rely on two assumptions: ($i$)
the samples in different views are well-aligned, and ($ii$) their
representations in latent space obey the same distribution. Unfortunately,
these two assumptions may be questionable in practice, which limits the
application of multi-view learning. In this work, we propose a hierarchical
optimal transport (HOT) method to mitigate the dependency on these two
assumptions. Given unaligned multi-view data, the HOT method penalizes the
sliced Wasserstein distance between the distributions of different views. These
sliced Wasserstein distances are used as the ground distance to calculate the
entropic optimal transport across different views, which explicitly indicates
the clustering structure of the views. The HOT method is applicable to both
unsupervised and semi-supervised learning, and experimental results show that
it performs robustly on both synthetic and real-world tasks.
Related papers
- Propensity Score Alignment of Unpaired Multimodal Data [3.8373578956681555]
Multimodal representation learning techniques typically rely on paired samples to learn common representations.
This paper presents an approach to address the challenge of aligning unpaired samples across disparate modalities in multimodal representation learning.
arXiv Detail & Related papers (2024-04-02T02:36:21Z) - 360 Layout Estimation via Orthogonal Planes Disentanglement and Multi-view Geometric Consistency Perception [56.84921040837699]
Existing panoramic layout estimation solutions tend to recover room boundaries from a vertically compressed sequence, yielding imprecise results.
We propose an orthogonal plane disentanglement network (termed DOPNet) to distinguish ambiguous semantics.
We also present an unsupervised adaptation technique tailored for horizon-depth and ratio representations.
Our solution outperforms other SoTA models on both monocular layout estimation and multi-view layout estimation tasks.
arXiv Detail & Related papers (2023-12-26T12:16:03Z) - Hierarchical Mutual Information Analysis: Towards Multi-view Clustering
in The Wild [9.380271109354474]
This work proposes a deep MVC framework where data recovery and alignment are fused in a hierarchically consistent way to maximize the mutual information among different views.
To the best of our knowledge, this could be the first successful attempt to handle the missing and unaligned data problem separately with different learning paradigms.
arXiv Detail & Related papers (2023-10-28T06:43:57Z) - ProbVLM: Probabilistic Adapter for Frozen Vision-Language Models [69.50316788263433]
We propose ProbVLM, a probabilistic adapter that estimates probability distributions for the embeddings of pre-trained vision-language models.
We quantify the calibration of embedding uncertainties in retrieval tasks and show that ProbVLM outperforms other methods.
We present a novel technique for visualizing the embedding distributions using a large-scale pre-trained latent diffusion model.
arXiv Detail & Related papers (2023-07-01T18:16:06Z) - Linking data separation, visual separation, and classifier performance
using pseudo-labeling by contrastive learning [125.99533416395765]
We argue that the performance of the final classifier depends on the data separation present in the latent space and visual separation present in the projection.
We demonstrate our results by the classification of five real-world challenging image datasets of human intestinal parasites with only 1% supervised samples.
arXiv Detail & Related papers (2023-02-06T10:01:38Z) - MORI-RAN: Multi-view Robust Representation Learning via Hybrid
Contrastive Fusion [4.36488705757229]
Multi-view representation learning is essential for many multi-view tasks, such as clustering and classification.
We propose a hybrid contrastive fusion algorithm to extract robust view-common representation from unlabeled data.
Experimental results demonstrated that the proposed method outperforms 12 competitive multi-view methods on four real-world datasets.
arXiv Detail & Related papers (2022-08-26T09:58:37Z) - Variational Distillation for Multi-View Learning [104.17551354374821]
We design several variational information bottlenecks to exploit two key characteristics for multi-view representation learning.
Under rigorously theoretical guarantee, our approach enables IB to grasp the intrinsic correlation between observations and semantic labels.
arXiv Detail & Related papers (2022-06-20T03:09:46Z) - Consistency Regularization for Deep Face Anti-Spoofing [69.70647782777051]
Face anti-spoofing (FAS) plays a crucial role in securing face recognition systems.
Motivated by this exciting observation, we conjecture that encouraging feature consistency of different views may be a promising way to boost FAS models.
We enhance both Embedding-level and Prediction-level Consistency Regularization (EPCR) in FAS.
arXiv Detail & Related papers (2021-11-24T08:03:48Z) - Orthogonal Multi-view Analysis by Successive Approximations via
Eigenvectors [7.870955752916424]
The framework integrates the correlations within multiple views, supervised discriminant capacity, and distance preservation.
It not only includes several existing models as special cases, but also inspires new novel models.
Experiments are conducted on various real-world datasets for multi-view discriminant analysis and multi-view multi-label classification.
arXiv Detail & Related papers (2020-10-04T17:16:15Z) - CO-Optimal Transport [19.267807479856575]
Optimal transport (OT) is a powerful tool for finding correspondences and measuring similarity between two distributions.
We propose a novel OT problem, named COOT for CO- Optimal Transport, that simultaneously optimize two transport maps between both samples and features.
We demonstrate its versatility with two machine learning applications in heterogeneous domain adaptation and co-clustering/data summarization.
arXiv Detail & Related papers (2020-02-10T13:33:15Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.