Transforming Conditional Density Estimation Into a Single Nonparametric Regression Task

• URL: http://arxiv.org/abs/2511.18530v1
• Date: Sun, 23 Nov 2025 16:50:34 GMT
• Title: Transforming Conditional Density Estimation Into a Single Nonparametric Regression Task
• Authors: Alexander G. Reisach, Olivier Collier, Alex Luedtke, Antoine Chambaz,
• Abstract summary: We transform the problem of conditional density estimation into a single nonparametric regression task via the introduction of auxiliary samples.<n>We develop condensité, a method that implements this approach.<n>We evaluate our method on a large population survey dataset and on a satellite imaging dataset.
• Score: 42.066457491320115
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a way of transforming the problem of conditional density estimation into a single nonparametric regression task via the introduction of auxiliary samples. This allows leveraging regression methods that work well in high dimensions, such as neural networks and decision trees. Our main theoretical result characterizes and establishes the convergence of our estimator to the true conditional density in the data limit. We develop condensité, a method that implements this approach. We demonstrate the benefit of the auxiliary samples on synthetic data and showcase that condensité can achieve good out-of-the-box results. We evaluate our method on a large population survey dataset and on a satellite imaging dataset. In both cases, we find that condensité matches or outperforms the state of the art and yields conditional densities in line with established findings in the literature on each dataset. Our contribution opens up new possibilities for regression-based conditional density estimation and the empirical results indicate strong promise for applied research.

Related papers

• Flow-Based Density Ratio Estimation for Intractable Distributions with Applications in Genomics [80.05951561886123]
  We leverage condition-aware flow matching to derive a single dynamical formulation for tracking density ratios along generative trajectories.<n>We demonstrate competitive performance on simulated benchmarks for closed-form ratio estimation, and show that our method supports versatile tasks in single-cell genomics data analysis.
  arXiv  Detail & Related papers  (2026-02-27T17:27:55Z)
• A Likelihood Based Approach to Distribution Regression Using Conditional Deep Generative Models [8.862614615192578]
  We study the large-sample properties of a likelihood-based approach for estimating conditional deep generative models.<n>Our results lead to the convergence rate of a sieve maximum likelihood estimator for estimating the conditional distribution.
  arXiv  Detail & Related papers  (2024-10-02T20:46:21Z)
• Generative modeling of density regression through tree flows [3.0262553206264893]
  We propose a flow-based generative model tailored for the density regression task on tabular data. We introduce a training algorithm for fitting the tree-based transforms using a divide-and-conquer strategy. Our method consistently achieves comparable or superior performance at a fraction of the training and sampling budget.
  arXiv  Detail & Related papers  (2024-06-07T21:07:35Z)
• Collaborative Heterogeneous Causal Inference Beyond Meta-analysis [68.4474531911361]
  We propose a collaborative inverse propensity score estimator for causal inference with heterogeneous data. Our method shows significant improvements over the methods based on meta-analysis when heterogeneity increases.
  arXiv  Detail & Related papers  (2024-04-24T09:04:36Z)
• Investigating the Adversarial Robustness of Density Estimation Using the Probability Flow ODE [4.7818621660181595]
  We introduce and evaluate six gradient-based log-likelihood attacks, including a novel reverse integration attack. Our experimental evaluations on CIFAR-10 show that density estimation using the PF ODE is robust against high-complexity, high-likelihood attacks, and that in some cases adversarial samples are semantically meaningful, as expected from a robust estimator.
  arXiv  Detail & Related papers  (2023-10-10T23:58:53Z)
• Nonparametric Probabilistic Regression with Coarse Learners [1.8275108630751844]
  We show that we can compute precise conditional densities with minimal assumptions on the shape or form of the density. We demonstrate this approach on a variety of datasets and show competitive performance, particularly on larger datasets.
  arXiv  Detail & Related papers  (2022-10-28T16:25:26Z)
• CAFE: Learning to Condense Dataset by Aligning Features [72.99394941348757]
  We propose a novel scheme to Condense dataset by Aligning FEatures (CAFE) At the heart of our approach is an effective strategy to align features from the real and synthetic data across various scales. We validate the proposed CAFE across various datasets, and demonstrate that it generally outperforms the state of the art.
  arXiv  Detail & Related papers  (2022-03-03T05:58:49Z)
• Heavy-tailed Streaming Statistical Estimation [58.70341336199497]
  We consider the task of heavy-tailed statistical estimation given streaming $p$ samples. We design a clipped gradient descent and provide an improved analysis under a more nuanced condition on the noise of gradients.
  arXiv  Detail & Related papers  (2021-08-25T21:30:27Z)
• GELATO: Geometrically Enriched Latent Model for Offline Reinforcement Learning [54.291331971813364]
  offline reinforcement learning approaches can be divided into proximal and uncertainty-aware methods. In this work, we demonstrate the benefit of combining the two in a latent variational model. Our proposed metrics measure both the quality of out of distribution samples as well as the discrepancy of examples in the data.
  arXiv  Detail & Related papers  (2021-02-22T19:42:40Z)
• Conditional Density Estimation via Weighted Logistic Regressions [0.30458514384586394]
  We propose a novel parametric conditional density estimation method by showing the connection between the general density and the likelihood function of inhomogeneous process models. The maximum likelihood estimates can be obtained via weighted logistic regressions, and the computation can be significantly relaxed by combining a block-wise alternating scheme and local case-control sampling.
  arXiv  Detail & Related papers  (2020-10-21T11:08:25Z)
• TraDE: Transformers for Density Estimation [101.20137732920718]
  TraDE is a self-attention-based architecture for auto-regressive density estimation. We present a suite of tasks such as regression using generated samples, out-of-distribution detection, and robustness to noise in the training data.
  arXiv  Detail & Related papers  (2020-04-06T07:32:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.