FMMI: Flow Matching Mutual Information Estimation

• URL: http://arxiv.org/abs/2511.08552v1
• Date: Wed, 12 Nov 2025 02:04:32 GMT
• Title: FMMI: Flow Matching Mutual Information Estimation
• Authors: Ivan Butakov, Alexander Semenenko, Alexey Frolov, Ivan Oseledets,
• Abstract summary: We introduce a novel Mutual Information (MI) estimator that fundamentally reframes the discriminative approach.<n>Instead of training a classifier to discriminate between joint and marginal distributions, we learn a normalizing flow that transforms one into the other.
• Score: 43.51440237740181
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce a novel Mutual Information (MI) estimator that fundamentally reframes the discriminative approach. Instead of training a classifier to discriminate between joint and marginal distributions, we learn a normalizing flow that transforms one into the other. This technique produces a computationally efficient and precise MI estimate that scales well to high dimensions and across a wide range of ground-truth MI values.

Related papers

• MIST: Mutual Information Via Supervised Training [41.02529625643583]
  We propose a fully data-driven approach to designing mutual information (MI) estimators.<n>Since any MI estimator is a function of the observed sample from two random variables, we parameterize this function with a neural network (MIST)<n>Training is performed on a large meta-dataset of 625,000 synthetic joint distributions with known ground-truth MI.
  arXiv  Detail & Related papers  (2025-11-24T09:55:28Z)
• Contrastive Predictive Coding Done Right for Mutual Information Estimation [21.046609494716865]
  We show why InfoNCE should not be regarded as a valid MI estimator.<n>We introduce a simple modification, which we refer to as InfoNCE-anchor, for accurate MI estimation.<n>We generalize our framework using proper scoring rules, which recover InfoNCE-anchor as a special case when the log score is employed.
  arXiv  Detail & Related papers  (2025-10-29T21:33:59Z)
• MMG: Mutual Information Estimation via the MMSE Gap in Diffusion [25.691925207007795]
  Mutual information (MI) is one of the most general ways to measure relationships between random variables.<n>Denoising diffusion models have recently set a new bar for density estimation.<n>We show the diffusion models can be used in a straightforward way to estimate MI.
  arXiv  Detail & Related papers  (2025-09-24T23:04:48Z)
• Mutual Information Multinomial Estimation [53.58005108981247]
  Estimating mutual information (MI) is a fundamental yet challenging task in data science and machine learning. Our main discovery is that a preliminary estimate of the data distribution can dramatically help estimate. Experiments on diverse tasks including non-Gaussian synthetic problems with known ground-truth and real-world applications demonstrate the advantages of our method.
  arXiv  Detail & Related papers  (2024-08-18T06:27:30Z)
• Mutual Information Estimation via Normalizing Flows [39.58317527488534]
  We propose a novel approach to the problem of mutual information estimation. The estimator maps original data to the target distribution, for which MI is easier to estimate. We additionally explore the target distributions with known closed-form expressions for MI.
  arXiv  Detail & Related papers  (2024-03-04T16:28:04Z)
• Improving Mutual Information Estimation with Annealed and Energy-Based Bounds [20.940022170594816]
  Mutual information (MI) is a fundamental quantity in information theory and machine learning. We present a unifying view of existing MI bounds from the perspective of importance sampling. We propose three novel bounds based on this approach.
  arXiv  Detail & Related papers  (2023-03-13T10:47:24Z)
• Tight Mutual Information Estimation With Contrastive Fenchel-Legendre Optimization [69.07420650261649]
  We introduce a novel, simple, and powerful contrastive MI estimator named as FLO. Empirically, our FLO estimator overcomes the limitations of its predecessors and learns more efficiently. The utility of FLO is verified using an extensive set of benchmarks, which also reveals the trade-offs in practical MI estimation.
  arXiv  Detail & Related papers  (2021-07-02T15:20:41Z)
• Towards Accurate Knowledge Transfer via Target-awareness Representation Disentanglement [56.40587594647692]
  We propose a novel transfer learning algorithm, introducing the idea of Target-awareness REpresentation Disentanglement (TRED) TRED disentangles the relevant knowledge with respect to the target task from the original source model and used as a regularizer during fine-tuning the target model. Experiments on various real world datasets show that our method stably improves the standard fine-tuning by more than 2% in average.
  arXiv  Detail & Related papers  (2020-10-16T17:45:08Z)
• Learning Calibrated Uncertainties for Domain Shift: A Distributionally Robust Learning Approach [150.8920602230832]
  We propose a framework for learning calibrated uncertainties under domain shifts. In particular, the density ratio estimation reflects the closeness of a target (test) sample to the source (training) distribution. We show that our proposed method generates calibrated uncertainties that benefit downstream tasks.
  arXiv  Detail & Related papers  (2020-10-08T02:10:54Z)
• Mutual Information Gradient Estimation for Representation Learning [56.08429809658762]
  Mutual Information (MI) plays an important role in representation learning. Recent advances establish tractable and scalable MI estimators to discover useful representation. We propose the Mutual Information Gradient Estimator (MIGE) for representation learning based on the score estimation of implicit distributions.
  arXiv  Detail & Related papers  (2020-05-03T16:05:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.