Improved probabilistic regression using diffusion models

• URL: http://arxiv.org/abs/2510.04583v1
• Date: Mon, 06 Oct 2025 08:36:05 GMT
• Title: Improved probabilistic regression using diffusion models
• Authors: Carlo Kneissl, Christopher Bülte, Philipp Scholl, Gitta Kutyniok,
• Abstract summary: We propose a novel diffusion-based framework for probabilistic regression that learns predictive distributions in a non way.<n>We investigate different noise parameterizations, analyze their trade-offs, and evaluate our framework across a broad range of regression tasks.<n>For several experiments, our approach shows superior performance against existing baselines, while delivering calibrated uncertainty estimates.
• Score: 16.918373481904755
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Probabilistic regression models the entire predictive distribution of a response variable, offering richer insights than classical point estimates and directly allowing for uncertainty quantification. While diffusion-based generative models have shown remarkable success in generating complex, high-dimensional data, their usage in general regression tasks often lacks uncertainty-related evaluation and remains limited to domain-specific applications. We propose a novel diffusion-based framework for probabilistic regression that learns predictive distributions in a nonparametric way. More specifically, we propose to model the full distribution of the diffusion noise, enabling adaptation to diverse tasks and enhanced uncertainty quantification. We investigate different noise parameterizations, analyze their trade-offs, and evaluate our framework across a broad range of regression tasks, covering low- and high-dimensional settings. For several experiments, our approach shows superior performance against existing baselines, while delivering calibrated uncertainty estimates, demonstrating its versatility as a tool for probabilistic prediction.

Related papers

• Practical Deep Heteroskedastic Regression [15.023152666894049]
  In heteroskedastic regression, where the uncertainty of the target depends on the input, a common approach is to train a neural network that parameterizes the mean and the variance of the predictive distribution.<n>We propose a simple and efficient procedure that addresses these challenges jointly by post-hoc fitting a variance model across the intermediate layers of a pretrained network on a hold-out dataset.<n>We demonstrate that our method on-par or state-of-the-art uncertainty quantification on several molecular graph datasets, without compromising mean prediction accuracy and remaining cheap to use at prediction time.
  arXiv  Detail & Related papers  (2026-03-02T11:19:32Z)
• SimDiff: Simpler Yet Better Diffusion Model for Time Series Point Forecasting [8.141505251306622]
  Diffusion models have recently shown promise in time series forecasting.<n>They often fail to achieve state-of-the-art point estimation performance.<n>We propose SimDiff, a single-stage, end-to-end framework for point estimation.
  arXiv  Detail & Related papers  (2025-11-24T16:09:55Z)
• Zero-Shot Uncertainty Quantification using Diffusion Probabilistic Models [7.136205674624813]
  We conduct a study to evaluate the effectiveness of ensemble methods on solving different regression problems using diffusion models. We demonstrate that ensemble methods consistently improve model prediction accuracy across various regression tasks. Our study provides a comprehensive view of the utility of diffusion ensembles, serving as a useful reference for practitioners employing diffusion models in regression problem-solving.
  arXiv  Detail & Related papers  (2024-08-08T18:34:52Z)
• Conformalized Selective Regression [2.3964255330849356]
  We propose a novel approach to selective regression by leveraging conformal prediction. We show how our proposed approach, conformalized selective regression, demonstrates an advantage over multiple state-of-the-art baselines.
  arXiv  Detail & Related papers  (2024-02-26T04:43:50Z)
• Quantification of Predictive Uncertainty via Inference-Time Sampling [57.749601811982096]
  We propose a post-hoc sampling strategy for estimating predictive uncertainty accounting for data ambiguity. The method can generate different plausible outputs for a given input and does not assume parametric forms of predictive distributions.
  arXiv  Detail & Related papers  (2023-08-03T12:43:21Z)
• Uncertainty Modeling for Out-of-Distribution Generalization [56.957731893992495]
  We argue that the feature statistics can be properly manipulated to improve the generalization ability of deep learning models. Common methods often consider the feature statistics as deterministic values measured from the learned features. We improve the network generalization ability by modeling the uncertainty of domain shifts with synthesized feature statistics during training.
  arXiv  Detail & Related papers  (2022-02-08T16:09:12Z)
• Trustworthy Multimodal Regression with Mixture of Normal-inverse Gamma Distributions [91.63716984911278]
  We introduce a novel Mixture of Normal-Inverse Gamma distributions (MoNIG) algorithm, which efficiently estimates uncertainty in principle for adaptive integration of different modalities and produces a trustworthy regression result. Experimental results on both synthetic and different real-world data demonstrate the effectiveness and trustworthiness of our method on various multimodal regression tasks.
  arXiv  Detail & Related papers  (2021-11-11T14:28:12Z)
• Which Invariance Should We Transfer? A Causal Minimax Learning Approach [18.71316951734806]
  We present a comprehensive minimax analysis from a causal perspective. We propose an efficient algorithm to search for the subset with minimal worst-case risk. The effectiveness and efficiency of our methods are demonstrated on synthetic data and the diagnosis of Alzheimer's disease.
  arXiv  Detail & Related papers  (2021-07-05T09:07:29Z)
• Aleatoric uncertainty for Errors-in-Variables models in deep regression [0.48733623015338234]
  We show how the concept of Errors-in-Variables can be used in Bayesian deep regression. We discuss the approach along various simulated and real examples.
  arXiv  Detail & Related papers  (2021-05-19T12:37:02Z)
• Learning Probabilistic Ordinal Embeddings for Uncertainty-Aware Regression [91.3373131262391]
  Uncertainty is the only certainty there is. Traditionally, the direct regression formulation is considered and the uncertainty is modeled by modifying the output space to a certain family of probabilistic distributions. How to model the uncertainty within the present-day technologies for regression remains an open issue.
  arXiv  Detail & Related papers  (2021-03-25T06:56:09Z)
• Learning Interpretable Deep State Space Model for Probabilistic Time Series Forecasting [98.57851612518758]
  Probabilistic time series forecasting involves estimating the distribution of future based on its history. We propose a deep state space model for probabilistic time series forecasting whereby the non-linear emission model and transition model are parameterized by networks. We show in experiments that our model produces accurate and sharp probabilistic forecasts.
  arXiv  Detail & Related papers  (2021-01-31T06:49:33Z)
• Unlabelled Data Improves Bayesian Uncertainty Calibration under Covariate Shift [100.52588638477862]
  We develop an approximate Bayesian inference scheme based on posterior regularisation. We demonstrate the utility of our method in the context of transferring prognostic models of prostate cancer across globally diverse populations.
  arXiv  Detail & Related papers  (2020-06-26T13:50:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.