Score-based Outlier Generation via Controlling the Radon-Nikodym Derivative

• URL: http://arxiv.org/abs/2609.12113v1
• Date: Thu, 10 Sep 2026 18:39:10 GMT
• Title: Score-based Outlier Generation via Controlling the Radon-Nikodym Derivative
• Abstract summary: Outliers are important for stress-testing algorithms and understanding system behaviour under rare conditions.<n>We introduce a measure-theoretic notion of outliers based on the distribution of log-likelihood values.<n>We exploit the Ornstein-Uhlenbeck semigroup underlying diffusion models to motivate an exponentially interpolated controller.
• Score: 6.4704208663487295
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Outliers are important for stress-testing algorithms and understanding system behaviour under rare conditions. Despite being commonly described as low-likelihood events, existing generative approaches rarely control likelihood explicitly. In this work, we introduce a measure-theoretic notion of outliers based on the distribution of log-likelihood values, which is guaranteed to assign higher probability mass to low-likelihood events with a specifiable magnitude. Building on this formulation, we derive how likelihood reweighting modifies the diffusion score and use this relation to motivate a controlled modification of the reverse-time dynamics. In particular, likelihood reweighting implies a scaling of the score function with a control term derived from the Radon-Nikodym derivative of the likelihood distributions. Correspondingly, the updated score function can be obtained with no retraining of the diffusion model. We exploit the Ornstein-Uhlenbeck semigroup underlying diffusion models to motivate an exponentially interpolated controller which approximates the true control. Experiments demonstrate controlled generation of low-likelihood samples while remaining consistent with the data geometry.

Related papers

• Conservation Laws for Diffusion Models [10.98617584718584]
  We develop conservation laws for a class of memoryless noise processes.<n>We show that the data--model cross-entropy (CE) can be characterized exactly as an integral of local information-theoretic derivatives along the noise path.<n>We validate these predictions on synthetic Markov sources and standard benchmarks.
  arXiv  Detail & Related papers  (2026-07-11T01:37:52Z)
• From Scores to Gibbs Correctors: Accelerating Uniform-Rate Discrete Diffusion Models [58.02657021435213]
  We propose a novel Gibbs-based corrector for discrete diffusion models, termed Gibbs-Accelerated Discrete Diffusion (GADD)<n>We show GADD achieves an overall sampling complexity of $mathcalO(mathrmpolylog (varepsilon-1)$, yielding the first such rate for diffusion-based samplers for uniform-rate discrete diffusion models.
  arXiv  Detail & Related papers  (2026-05-26T17:52:28Z)
• Causality-Encoded Diffusion Models for Interventional Sampling and Edge Inference [12.531769483039405]
  We propose a causality-encoded diffusion framework that incorporates a known directed acyclic graph.<n>The resulting sampler approximately recovers the observational distribution and enables interventional sampling.<n>An application to flow data demonstrates practical utility of the proposed method in assessing disputed signalling linkages.
  arXiv  Detail & Related papers  (2026-04-23T16:32:49Z)
• Mitigating the Likelihood Paradox in Flow-based OOD Detection via Entropy Manipulation [24.066271161451425]
  We argue that entropy control increases the expected log-likelihood gap between in-distribution and OOD samples in favor of the in-distribution.<n>We then evaluate our method against likelihood-based OOD detectors on standard benchmarks and find consistent AUROC improvements over baselines.
  arXiv  Detail & Related papers  (2026-02-10T09:31:03Z)
• Provably Reliable Classifier Guidance via Cross-Entropy Control [4.298880233819988]
  We show that cross-entropy loss at each diffusion model step is sufficient to control the corresponding guidance error.<n>Our result yields an upper bound on the sampling error vectors-guided diffusion models and bears resemblance to a reverse log-Sobolev-type inequality.
  arXiv  Detail & Related papers  (2026-01-29T02:59:04Z)
• Rectified Diffusion Guidance for Conditional Generation [94.83538269086613]
  We revisit the theory behind CFG and rigorously confirm that the improper combination coefficients (textiti.e.) brings about expectation shift the generative distribution.<n>We show that our approach enjoys a textbftextitform solution given the strength.<n> Empirical evidence on real-world data demonstrate the compatibility of our design with existing state-of-the-art diffusion models.
  arXiv  Detail & Related papers  (2024-10-24T13:41:32Z)
• Theory on Score-Mismatched Diffusion Models and Zero-Shot Conditional Samplers [49.97755400231656]
  We present the first performance guarantee with explicit dimensional dependencies for general score-mismatched diffusion samplers.<n>We show that score mismatches result in an distributional bias between the target and sampling distributions, proportional to the accumulated mismatch between the target and training distributions.<n>This result can be directly applied to zero-shot conditional samplers for any conditional model, irrespective of measurement noise.
  arXiv  Detail & Related papers  (2024-10-17T16:42:12Z)
• Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data [55.54827581105283]
  We show that the concrete score in absorbing diffusion can be expressed as conditional probabilities of clean data.<n>We propose a dedicated diffusion model without time-condition that characterizes the time-independent conditional probabilities.<n>Our models achieve SOTA performance among diffusion models on 5 zero-shot language modeling benchmarks.
  arXiv  Detail & Related papers  (2024-06-06T04:22:11Z)
• Rejection via Learning Density Ratios [50.91522897152437]
  Classification with rejection emerges as a learning paradigm which allows models to abstain from making predictions.<n>We propose a different distributional perspective, where we seek to find an idealized data distribution which maximizes a pretrained model's performance.<n>Our framework is tested empirically over clean and noisy datasets.
  arXiv  Detail & Related papers  (2024-05-29T01:32:17Z)
• Theoretical Insights for Diffusion Guidance: A Case Study for Gaussian Mixture Models [59.331993845831946]
  Diffusion models benefit from instillation of task-specific information into the score function to steer the sample generation towards desired properties. This paper provides the first theoretical study towards understanding the influence of guidance on diffusion models in the context of Gaussian mixture models.
  arXiv  Detail & Related papers  (2024-03-03T23:15:48Z)
• Generative Diffusion From An Action Principle [0.0]
  We show that score matching can be derived from an action principle, like the ones commonly used in physics. We use this insight to demonstrate the connection between different classes of diffusion models.
  arXiv  Detail & Related papers  (2023-10-06T18:00:00Z)
• Deconfounding Scores: Feature Representations for Causal Effect Estimation with Weak Overlap [140.98628848491146]
  We introduce deconfounding scores, which induce better overlap without biasing the target of estimation. We show that deconfounding scores satisfy a zero-covariance condition that is identifiable in observed data. In particular, we show that this technique could be an attractive alternative to standard regularizations.
  arXiv  Detail & Related papers  (2021-04-12T18:50:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.