Learning to Match Unpaired Data with Minimum Entropy Coupling

• URL: http://arxiv.org/abs/2503.08501v1
• Date: Tue, 11 Mar 2025 14:54:14 GMT
• Title: Learning to Match Unpaired Data with Minimum Entropy Coupling
• Authors: Mustapha Bounoua, Giulio Franzese, Pietro Michiardi,
• Abstract summary: Minimum Entropy Coupling seeks to minimize the joint Entropy, while satisfying constraints on the marginals.<n>We propose a novel method to solve the continuous MEC problem, using well-known generative diffusion models.<n>We empirically demonstrate that our method, DDMEC, is general and can be easily used to address challenging tasks.
• Score: 7.399561232927219
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodal data is a precious asset enabling a variety of downstream tasks in machine learning. However, real-world data collected across different modalities is often not paired, which is a significant challenge to learn a joint distribution. A prominent approach to address the modality coupling problem is Minimum Entropy Coupling (MEC), which seeks to minimize the joint Entropy, while satisfying constraints on the marginals. Existing approaches to the MEC problem focus on finite, discrete distributions, limiting their application for cases involving continuous data. In this work, we propose a novel method to solve the continuous MEC problem, using well-known generative diffusion models that learn to approximate and minimize the joint Entropy through a cooperative scheme, while satisfying a relaxed version of the marginal constraints. We empirically demonstrate that our method, DDMEC, is general and can be easily used to address challenging tasks, including unsupervised single-cell multi-omics data alignment and unpaired image translation, outperforming specialized methods.

Related papers

• Attention Bootstrapping for Multi-Modal Test-Time Adaptation [10.524759195371828]
  Test-time adaptation aims to adapt a well-trained model to potential distribution shifts at test time using only unlabeled test data. This paper tackles the problem of multi-modal test-time adaptation by proposing a novel method named Attention Bootstrapping with Principal Entropy Minimization (ABPEM) To mitigate this attention gap and encourage better modality fusion, we propose attention bootstrapping that promotes cross-attention with the guidance of self-attention.
  arXiv  Detail & Related papers  (2025-03-04T02:53:53Z)
• FedMAC: Tackling Partial-Modality Missing in Federated Learning with Cross-Modal Aggregation and Contrastive Regularization [11.954904313477176]
  Federated Learning (FL) is a method for training machine learning models using distributed data sources. This study proposes a novel framework named FedMAC, designed to address multi-modality missing under conditions of partial-modality missing in FL.
  arXiv  Detail & Related papers  (2024-10-04T01:24:02Z)
• Dataless Quadratic Neural Networks for the Maximum Independent Set Problem [23.643727259409744]
  We show that pCQO-MIS scales with only the number of nodes in the graph not the number of edges.<n>Our method avoids out-of-distribution, sampling, and data-centric approaches.
  arXiv  Detail & Related papers  (2024-06-27T21:12:48Z)
• Set-CLIP: Exploring Aligned Semantic From Low-Alignment Multimodal Data Through A Distribution View [35.389116270077324]
  Multimodal fusion breaks through the boundaries between diverse modalities and has already achieved notable performances. In many specialized fields, it is struggling to obtain sufficient alignment data for training. We propose a new methodology based on CLIP, termed Set-CLIP.
  arXiv  Detail & Related papers  (2024-06-09T12:41:14Z)
• Multimodal Representation Learning by Alternating Unimodal Adaptation [73.15829571740866]
  We propose MLA (Multimodal Learning with Alternating Unimodal Adaptation) to overcome challenges where some modalities appear more dominant than others during multimodal learning. MLA reframes the conventional joint multimodal learning process by transforming it into an alternating unimodal learning process. It captures cross-modal interactions through a shared head, which undergoes continuous optimization across different modalities. Experiments are conducted on five diverse datasets, encompassing scenarios with complete modalities and scenarios with missing modalities.
  arXiv  Detail & Related papers  (2023-11-17T18:57:40Z)
• Ensemble Modeling for Multimodal Visual Action Recognition [50.38638300332429]
  We propose an ensemble modeling approach for multimodal action recognition. We independently train individual modality models using a variant of focal loss tailored to handle the long-tailed distribution of the MECCANO [21] dataset.
  arXiv  Detail & Related papers  (2023-08-10T08:43:20Z)
• Learning Multimodal VAEs through Mutual Supervision [72.77685889312889]
  MEME combines information between modalities implicitly through mutual supervision. We demonstrate that MEME outperforms baselines on standard metrics across both partial and complete observation schemes.
  arXiv  Detail & Related papers  (2021-06-23T17:54:35Z)
• Implicit MLE: Backpropagating Through Discrete Exponential Family Distributions [24.389388509299543]
  Implicit Maximum Likelihood Estimation is a framework for end-to-end learning of models combining discrete exponential family distributions and differentiable neural components. We show that I-MLE is competitive with and often outperforms existing approaches which rely on problem-specific relaxations.
  arXiv  Detail & Related papers  (2021-06-03T12:42:21Z)
• Differentiable Causal Discovery from Interventional Data [141.41931444927184]
  We propose a theoretically-grounded method based on neural networks that can leverage interventional data. We show that our approach compares favorably to the state of the art in a variety of settings.
  arXiv  Detail & Related papers  (2020-07-03T15:19:17Z)
• CLUB: A Contrastive Log-ratio Upper Bound of Mutual Information [105.73798100327667]
  We propose a novel Contrastive Log-ratio Upper Bound (CLUB) of mutual information. We provide a theoretical analysis of the properties of CLUB and its variational approximation. Based on this upper bound, we introduce a MI minimization training scheme and further accelerate it with a negative sampling strategy.
  arXiv  Detail & Related papers  (2020-06-22T05:36:16Z)
• MMCGAN: Generative Adversarial Network with Explicit Manifold Prior [78.58159882218378]
  We propose to employ explicit manifold learning as prior to alleviate mode collapse and stabilize training of GAN. Our experiments on both the toy data and real datasets show the effectiveness of MMCGAN in alleviating mode collapse, stabilizing training, and improving the quality of generated samples.
  arXiv  Detail & Related papers  (2020-06-18T07:38:54Z)
• Joint Wasserstein Distribution Matching [89.86721884036021]
  Joint distribution matching (JDM) problem, which aims to learn bidirectional mappings to match joint distributions of two domains, occurs in many machine learning and computer vision applications. We propose to address JDM problem by minimizing the Wasserstein distance of the joint distributions in two domains. We then propose an important theorem to reduce the intractable problem into a simple optimization problem, and develop a novel method to solve it.
  arXiv  Detail & Related papers  (2020-03-01T03:39:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.