Test-Time Compositional Generalization in Diffusion Models via Concept Discovery

• URL: http://arxiv.org/abs/2605.07078v1
• Date: Fri, 08 May 2026 00:53:53 GMT
• Title: Test-Time Compositional Generalization in Diffusion Models via Concept Discovery
• Abstract summary: We show that a pretrained diffusion model can discover query-specific concepts from the time-indexed scores it learns.<n>Both the analytic PoE sampler and the low-rank adapted model outperform query-only and nearest trained-class baselines.
• Score: 6.379257030501549
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Compositional generalization requires models to produce novel configurations from familiar parts. In diffusion models, prior compositional generation methods typically assume that the relevant concepts or conditioning signals are already available. We instead ask whether a pretrained diffusion model can discover query-specific concepts from the time-indexed scores it learns for the noisy marginals $p_t(x_t)$ and compose them at test time. Given a single out-of-distribution query, our method performs gradient ascent on $s_θ(x_t,t) \approx \nabla_{x_t}\log p_t(x_t)$ at multiple noising timesteps to recover local density modes, maps these modes into clean-space Gaussians, greedily selects relevant prototypes with a submodular likelihood objective, and combines them into a product-of-experts (PoE) teacher model with an analytic score. This teacher model can be sampled directly through classifier-free guidance or used to generate a sample pool for training a new class embedding and low-rank adapter. On held-out composition benchmarks built from ColorMNIST and CelebA, both the analytic PoE sampler and the low-rank adapted model outperform query-only and nearest trained-class baselines. These results suggest that the time-indexed score geometry of the diffusion model contains reusable density-mode concepts that support test-time compositional generation without a predefined concept library.

Related papers

• CAST: Closed-form Analytic Semantic Transfer for Zero-Shot Classifier Extension [1.8882634075101061]
  We introduce CAST (Closed-form Analytic Semantic Transfer), a training-free, image-free framework for extending a pre-trained to previously unseen classes through weight injection.<n>Experiments on standard zero-shot learning benchmarks demonstrate that CAST matches or exceeds existing image-free approaches.
  arXiv  Detail & Related papers  (2026-08-13T20:20:07Z)
• GenZ: Foundational models as latent variable generators within traditional statistical models [7.74887919885246]
  We present GenZ, a hybrid model that bridges foundational models and statistical modeling through interpretable semantic features.<n>Our approach addresses this by discovering semantic feature descriptions through an iterative process.<n>For Netflix movie embeddings, our model predicts collaborative filtering representations with 0.59 cosine similarity purely from semantic descriptions.
  arXiv  Detail & Related papers  (2025-12-31T12:56:01Z)
• Distillation of Discrete Diffusion by Exact Conditional Distribution Matching [9.460409527892345]
  We propose a simple and principled distillation alternative based on emphconditional distribution matching.<n>We exploit this structure to define distillation objectives that directly match conditional distributions between a pre-trained teacher and a low-NFE student.
  arXiv  Detail & Related papers  (2025-12-15T00:16:10Z)
• DDB: Diffusion Driven Balancing to Address Spurious Correlations [24.940576844328408]
  Deep neural networks trained with Empirical Risk Minimization often fail to generalize to out-of-distribution samples.<n>We propose a Diffusion Driven Balancing (DDB) technique to generate training samples with text-to-image diffusion models.<n>Our experiments show that our technique achieves better worst-group accuracy than the existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-03-21T15:28:22Z)
• Feynman-Kac Correctors in Diffusion: Annealing, Guidance, and Product of Experts [64.34482582690927]
  We provide an efficient and principled method for sampling from a sequence of annealed, geometric-averaged, or product distributions derived from pretrained score-based models.<n>We propose Sequential Monte Carlo (SMC) resampling algorithms that leverage inference-time scaling to improve sampling quality.
  arXiv  Detail & Related papers  (2025-03-04T17:46:51Z)
• Generative Modeling with Bayesian Sample Inference [50.07758840675341]
  We derive a novel generative model from iterative Gaussian posterior inference.<n>Our model uses a sequence of prediction and posterior update steps to iteratively narrow down the unknown sample.<n>In experiments, we demonstrate that our model improves sample quality on ImageNet32 over both BFNs and the closely related Variational Diffusion Models.
  arXiv  Detail & Related papers  (2025-02-11T14:27:10Z)
• Accelerated Diffusion Models via Speculative Sampling [89.43940130493233]
  Speculative sampling is a popular technique for accelerating inference in Large Language Models.<n>We extend speculative sampling to diffusion models, which generate samples via continuous, vector-valued Markov chains.<n>We propose various drafting strategies, including a simple and effective approach that does not require training a draft model.
  arXiv  Detail & Related papers  (2025-01-09T16:50:16Z)
• Concurrent Misclassification and Out-of-Distribution Detection for Semantic Segmentation via Energy-Based Normalizing Flow [0.0]
  Recent semantic segmentation models accurately classify test-time examples that are similar to a training dataset distribution. We propose a generative model for concurrent in-distribution misclassification (IDM) and OOD detection that relies on a normalizing flow framework. FlowEneDet achieves promising results on Cityscapes, Cityscapes-C, FishyScapes and SegmentMeIfYouCan benchmarks in IDM/OOD detection when applied to pretrained DeepLabV3+ and SegFormer semantic segmentation models.
  arXiv  Detail & Related papers  (2023-05-16T17:02:57Z)
• Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMC [102.64648158034568]
  diffusion models have quickly become the prevailing approach to generative modeling in many domains. We propose an energy-based parameterization of diffusion models which enables the use of new compositional operators. We find these samplers lead to notable improvements in compositional generation across a wide set of problems.
  arXiv  Detail & Related papers  (2023-02-22T18:48:46Z)
• Normalizing Flow based Hidden Markov Models for Classification of Speech Phones with Explainability [25.543231171094384]
  In pursuit of explainability, we develop generative models for sequential data. We combine modern neural networks (normalizing flows) and traditional generative models (hidden Markov models - HMMs) The proposed generative models can compute likelihood of a data and hence directly suitable for maximum-likelihood (ML) classification approach.
  arXiv  Detail & Related papers  (2021-07-01T20:10:55Z)
• Continual Learning with Fully Probabilistic Models [70.3497683558609]
  We present an approach for continual learning based on fully probabilistic (or generative) models of machine learning. We propose a pseudo-rehearsal approach using a Gaussian Mixture Model (GMM) instance for both generator and classifier functionalities. We show that GMR achieves state-of-the-art performance on common class-incremental learning problems at very competitive time and memory complexity.
  arXiv  Detail & Related papers  (2021-04-19T12:26:26Z)
• Goal-directed Generation of Discrete Structures with Conditional Generative Models [85.51463588099556]
  We introduce a novel approach to directly optimize a reinforcement learning objective, maximizing an expected reward. We test our methodology on two tasks: generating molecules with user-defined properties and identifying short python expressions which evaluate to a given target value.
  arXiv  Detail & Related papers  (2020-10-05T20:03:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.