Constant-Target Energy Matching: A Unified Framework for Continuous and Discrete Density Estimation

• URL: http://arxiv.org/abs/2605.09085v1
• Date: Sat, 09 May 2026 17:50:46 GMT
• Title: Constant-Target Energy Matching: A Unified Framework for Continuous and Discrete Density Estimation
• Abstract summary: Constant-Target Energy Matching (CTEM) is a unified energy-based framework for density estimation on general state spaces.<n>CTEM substantially improves density estimation over competitive baselines and yields higher-quality samples under standard sampling procedures.
• Score: 5.922959609782139
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Density estimation is a central primitive in probabilistic modeling, yet continuous, discrete, and mixed-variable domains are often treated by separate objectives, limiting the ability to exploit a common statistical structure across data types. Continuous score-based methods rely on log-density gradients, while discrete extensions typically use concrete score whose unbounded targets become unstable near low-probability states. We introduce Constant-Target Energy Matching (CTEM), a unified energy-based framework for density estimation on general state spaces. CTEM replaces ordinary density-ratio regression with a bounded energy-difference transform and derives from it a sample-only training objective with the constant target 1. The learned scalar potential recovers log p without partition-function estimation or explicit unbounded ratio regression. Across continuous, discrete, and mixed-variable benchmarks, CTEM substantially improves density estimation over competitive baselines and yields higher-quality samples under standard sampling procedures.

Related papers

• Learning under Target Shift: Optimal Density Ratio Estimation and Importance-Weighted Regression [0.2578242050187029]
  We study density ratio estimation and importance-weighted regression under target shift with continuous outputs.<n>We propose a spectral regularization method in a reproducing kernel Hilbert space (RKHS) for estimating the continuous density ratio.<n>We then incorporate the estimated density ratio into importance-weighted regression and characterize the propagation of density-ratio estimation error to the final predictor.
  arXiv  Detail & Related papers  (2026-09-14T16:05:44Z)
• Generative Modeling under Non-Monotonic MAR Missingness via Approximate Wasserstein Gradient Flows [6.221876252052772]
  We propose a principled iterative method for generating a complete dataset from a dataset with values Missing at Random (MAR)<n>FLOWGEM minimizes the expected Kullback-Leibler (KL) divergence between the observed data distribution and the distribution of the generated sample over different missingness patterns.<n>It achieves state-of-the-art performance across a range of settings, including the challenging case of non-monotonic MAR mechanisms.
  arXiv  Detail & Related papers  (2026-04-06T09:56:08Z)
• A Generative Sampler for distributions with possible discrete parameter based on Reversibility [9.349846971147256]
  We propose a unified, target-gradient-free generative sampling framework applicable across diverse state spaces.<n>We minimize the Maximum Mean Discrepancy (MMD) between the joint distributions of forward and backward Markov trajectories.<n> Experiments show that our framework accurately reproduces thermodynamic observables and captures mode-switching behavior across all regimes.
  arXiv  Detail & Related papers  (2026-03-10T06:32:51Z)
• Flow-Based Density Ratio Estimation for Intractable Distributions with Applications in Genomics [80.05951561886123]
  We leverage condition-aware flow matching to derive a single dynamical formulation for tracking density ratios along generative trajectories.<n>We demonstrate competitive performance on simulated benchmarks for closed-form ratio estimation, and show that our method supports versatile tasks in single-cell genomics data analysis.
  arXiv  Detail & Related papers  (2026-02-27T17:27:55Z)
• Manifold-Aligned Generative Transport [11.857867207010981]
  We propose a flow-like generator that learns a one-shot, manifold-aligned transport from a low-dimensional base distribution to the data space.<n>We empirically improve fidelity and manifold concentration across synthetic and benchmark datasets while sampling substantially faster than diffusion models.
  arXiv  Detail & Related papers  (2026-02-23T08:42:40Z)
• Density estimation with atoms, and functional estimation for mixed discrete-continuous data [29.43493007296859]
  In density-functional estimation, it is standard to assume that the underlying distribution has a density with respect to the Lebesgue measure.<n>When the data distribution is a mixture of continuous and discrete components, the resulting methods are inconsistent in theory and perform poorly in practice.<n>We modify existing estimators for a broad class of functionals of the continuous component of the mixture.
  arXiv  Detail & Related papers  (2025-08-03T10:22:35Z)
• Convergence of Score-Based Discrete Diffusion Models: A Discrete-Time Analysis [56.442307356162864]
  We study the theoretical aspects of score-based discrete diffusion models under the Continuous Time Markov Chain (CTMC) framework.<n>We introduce a discrete-time sampling algorithm in the general state space $[S]d$ that utilizes score estimators at predefined time points.<n>Our convergence analysis employs a Girsanov-based method and establishes key properties of the discrete score function.
  arXiv  Detail & Related papers  (2024-10-03T09:07:13Z)
• Sobolev Space Regularised Pre Density Models [51.558848491038916]
  We propose a new approach to non-parametric density estimation that is based on regularizing a Sobolev norm of the density. This method is statistically consistent, and makes the inductive validation model clear and consistent.
  arXiv  Detail & Related papers  (2023-07-25T18:47:53Z)
• Concrete Score Matching: Generalized Score Matching for Discrete Data [109.12439278055213]
  "Concrete score" is a generalization of the (Stein) score for discrete settings. "Concrete Score Matching" is a framework to learn such scores from samples.
  arXiv  Detail & Related papers  (2022-11-02T00:41:37Z)
• Density Ratio Estimation via Infinitesimal Classification [85.08255198145304]
  We propose DRE-infty, a divide-and-conquer approach to reduce Density ratio estimation (DRE) to a series of easier subproblems. Inspired by Monte Carlo methods, we smoothly interpolate between the two distributions via an infinite continuum of intermediate bridge distributions. We show that our approach performs well on downstream tasks such as mutual information estimation and energy-based modeling on complex, high-dimensional datasets.
  arXiv  Detail & Related papers  (2021-11-22T06:26:29Z)
• TraDE: Transformers for Density Estimation [101.20137732920718]
  TraDE is a self-attention-based architecture for auto-regressive density estimation. We present a suite of tasks such as regression using generated samples, out-of-distribution detection, and robustness to noise in the training data.
  arXiv  Detail & Related papers  (2020-04-06T07:32:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.