Discrete Bridges for Mutual Information Estimation

• URL: http://arxiv.org/abs/2602.08894v1
• Date: Mon, 09 Feb 2026 16:55:09 GMT
• Title: Discrete Bridges for Mutual Information Estimation
• Authors: Iryna Zabarianska, Sergei Kholkin, Grigoriy Ksenofontov, Ivan Butakov, Alexander Korotin,
• Abstract summary: We leverage the discrete state space formulation of bridge matching models to address the estimation of the mutual information between discrete random variables.<n>By neatly framing MI estimation as a domain transfer problem, we construct a Discrete Bridge Mutual Information (DBMI) estimator suitable for discrete data.<n>We showcase the performance of our estimator on two MI estimation settings: low-dimensional and image-based.
• Score: 48.80678813569798
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion bridge models in both continuous and discrete state spaces have recently become powerful tools in the field of generative modeling. In this work, we leverage the discrete state space formulation of bridge matching models to address another important problem in machine learning and information theory: the estimation of the mutual information (MI) between discrete random variables. By neatly framing MI estimation as a domain transfer problem, we construct a Discrete Bridge Mutual Information (DBMI) estimator suitable for discrete data, which poses difficulties for conventional MI estimators. We showcase the performance of our estimator on two MI estimation settings: low-dimensional and image-based.

Related papers

• Time-Correlated Video Bridge Matching [49.94768097995648]
  Time-Correlated Video Bridge Matching (TCVBM) is a framework that extends Bridge Matching (BM) to time-correlated data sequences in the video domain.<n>TCVBM achieves superior performance across multiple quantitative metrics, demonstrating enhanced generation quality and reconstruction fidelity.
  arXiv  Detail & Related papers  (2025-10-14T12:35:30Z)
• MMG: Mutual Information Estimation via the MMSE Gap in Diffusion [25.691925207007795]
  Mutual information (MI) is one of the most general ways to measure relationships between random variables.<n>Denoising diffusion models have recently set a new bar for density estimation.<n>We show the diffusion models can be used in a straightforward way to estimate MI.
  arXiv  Detail & Related papers  (2025-09-24T23:04:48Z)
• InfoBridge: Mutual Information estimation via Bridge Matching [70.71590371950948]
  Diffusion bridge models have recently become a powerful tool in the field of generative modeling.<n>We leverage their power to address the estimation of the mutual information (MI) between two random variables.<n>We construct an unbiased estimator for data posing difficulties for conventional MI estimators.
  arXiv  Detail & Related papers  (2025-02-03T14:18:37Z)
• Generating Origin-Destination Matrices in Neural Spatial Interaction Models [11.188781092933313]
  Agent-based models (ABMs) are proliferating as decision-making tools across policy areas in transportation, economics, and epidemiology. A central object of interest is the discrete origin-destination matrix which captures interactions and agent trip counts between locations. Existing approaches resort to continuous approximations of this matrix and subsequent ad-hoc discretisations in order to perform ABM simulation and calibration. This impedes conditioning on partially observed summary statistics, fails to explore the multimodal matrix distribution over a discrete support, and incurs discretisation errors.
  arXiv  Detail & Related papers  (2024-10-09T18:09:02Z)
• SMILE: Zero-Shot Sparse Mixture of Low-Rank Experts Construction From Pre-Trained Foundation Models [85.67096251281191]
  We present an innovative approach to model fusion called zero-shot Sparse MIxture of Low-rank Experts (SMILE) construction. SMILE allows for the upscaling of source models into an MoE model without extra data or further training. We conduct extensive experiments across diverse scenarios, such as image classification and text generation tasks, using full fine-tuning and LoRA fine-tuning.
  arXiv  Detail & Related papers  (2024-08-19T17:32:15Z)
• Diffusion posterior sampling for simulation-based inference in tall data settings [53.17563688225137]
  Simulation-based inference ( SBI) is capable of approximating the posterior distribution that relates input parameters to a given observation. In this work, we consider a tall data extension in which multiple observations are available to better infer the parameters of the model. We compare our method to recently proposed competing approaches on various numerical experiments and demonstrate its superiority in terms of numerical stability and computational cost.
  arXiv  Detail & Related papers  (2024-04-11T09:23:36Z)
• Max-Sliced Mutual Information [17.667315953598788]
  Quantifying the dependence between high-dimensional random variables is central to statistical learning and inference. Two classical methods are canonical correlation analysis (CCA), which identifies maximally correlated projected versions of the original variables, and Shannon's mutual information, which is a universal dependence measure. This work proposes a middle ground in the form of a scalable information-theoretic generalization of CCA, termed max-sliced mutual information (mSMI)
  arXiv  Detail & Related papers  (2023-09-28T06:49:25Z)
• DiME: Maximizing Mutual Information by a Difference of Matrix-Based Entropies [0.9053163124987534]
  We introduce an information-theoretic quantity with similar properties to mutual information that can be estimated from data. We show that a difference of a matrix-based entropies (DiME) is well suited for problems involving the reproducing of mutual information between random variables. We provide examples of use cases for DiME, such as latent factor disentanglement and a multiview representation learning problem.
  arXiv  Detail & Related papers  (2023-01-19T16:56:21Z)
• Neural Methods for Point-wise Dependency Estimation [129.93860669802046]
  We focus on estimating point-wise dependency (PD), which quantitatively measures how likely two outcomes co-occur. We demonstrate the effectiveness of our approaches in 1) MI estimation, 2) self-supervised representation learning, and 3) cross-modal retrieval task.
  arXiv  Detail & Related papers  (2020-06-09T23:26:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.