Don't be so negative! Score-based Generative Modeling with Oracle-assisted Guidance

• URL: http://arxiv.org/abs/2307.16463v2
• Date: Sat, 12 Jul 2025 19:40:25 GMT
• Title: Don't be so negative! Score-based Generative Modeling with Oracle-assisted Guidance
• Authors: Saeid Naderiparizi, Xiaoxuan Liang, Setareh Cohan, Berend Zwartsenberg, Frank Wood,
• Abstract summary: This work addresses model learning in a setting where there further exists side-information in the form of an oracle.<n>We develop a new denoising diffusion probabilistic modeling methodology, Gen-neG, that leverages this additional side-information.<n>We empirically establish the utility of Gen-neG in applications including collision avoidance in self-driving simulators and safety-guarded human motion generation.
• Score: 13.442829248992863
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Score-based diffusion models are a powerful class of generative models, widely utilized across diverse domains. Despite significant advancements in large-scale tasks such as text-to-image generation, their application to constrained domains has received considerably less attention. This work addresses model learning in a setting where, in addition to the training dataset, there further exists side-information in the form of an oracle that can label samples as being outside the support of the true data generating distribution. Specifically we develop a new denoising diffusion probabilistic modeling methodology, Gen-neG, that leverages this additional side-information. Gen-neG builds on classifier guidance in diffusion models to guide the generation process towards the positive support region indicated by the oracle. We empirically establish the utility of Gen-neG in applications including collision avoidance in self-driving simulators and safety-guarded human motion generation.

Related papers

• Spatial Reasoners for Continuous Variables in Any Domain [49.83744014336816]
  We present a framework to perform spatial reasoning over continuous variables with generative denoising models.<n>We provide interfaces to control variable mapping from arbitrary data domains, generative model paradigms, and inference strategies.
  arXiv  Detail & Related papers  (2025-07-14T19:46:54Z)
• Domain Guidance: A Simple Transfer Approach for a Pre-trained Diffusion Model [62.11981915549919]
  Domain Guidance is a transfer approach that leverages pre-trained knowledge to guide the sampling process toward the target domain.<n>We demonstrate its substantial effectiveness across various transfer benchmarks, achieving over a 19.6% improvement in FID and a 23.4% improvement in FD$_textDINOv2$ compared to standard fine-tuning.
  arXiv  Detail & Related papers  (2025-04-02T09:07:55Z)
• Single Domain Generalization with Model-aware Parametric Batch-wise Mixup [22.709796153794507]
  Single Domain Generalization remains a formidable challenge in the field of machine learning. We propose a novel data augmentation approach, named as Model-aware Parametric Batch-wise Mixup. By exploiting inter-feature correlations, the parameterized mixup generator introduces additional versatility in combining features across a batch of instances.
  arXiv  Detail & Related papers  (2025-02-22T03:45:18Z)
• Generative Data Mining with Longtail-Guided Diffusion [39.460272573196896]
  We develop a proactive longtail discovery process by imagining additional data during training. We leverage these signals as guidance to generate additional training data from a latent diffusion model. Data generated by LTG exhibit semantically meaningful variation, yield significant generalization improvements on image classification benchmarks.
  arXiv  Detail & Related papers  (2025-02-04T03:51:00Z)
• Influence Functions for Scalable Data Attribution in Diffusion Models [52.92223039302037]
  Diffusion models have led to significant advancements in generative modelling. Yet their widespread adoption poses challenges regarding data attribution and interpretability. In this paper, we aim to help address such challenges by developing an textitinfluence functions framework.
  arXiv  Detail & Related papers  (2024-10-17T17:59:02Z)
• Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Prediction [88.65168366064061]
  We introduce Discrete Denoising Posterior Prediction (DDPP), a novel framework that casts the task of steering pre-trained MDMs as a problem of probabilistic inference. Our framework leads to a family of three novel objectives that are all simulation-free, and thus scalable. We substantiate our designs via wet-lab validation, where we observe transient expression of reward-optimized protein sequences.
  arXiv  Detail & Related papers  (2024-10-10T17:18:30Z)
• Recommendation with Generative Models [35.029116616023586]
  Generative models are AI models capable of creating new instances of data by learning and sampling from their statistical distributions. These models have applications across various domains, such as image generation, text synthesis, and music composition. In recommender systems, generative models, referred to as Gen-RecSys, improve the accuracy and diversity of recommendations.
  arXiv  Detail & Related papers  (2024-09-18T18:29:15Z)
• Sub-graph Based Diffusion Model for Link Prediction [43.15741675617231]
  Denoising Diffusion Probabilistic Models (DDPMs) represent a contemporary class of generative models with exceptional qualities. We build a novel generative model for link prediction using a dedicated design to decompose the likelihood estimation process via the Bayesian formula. Our proposed method presents numerous advantages: (1) transferability across datasets without retraining, (2) promising generalization on limited training data, and (3) robustness against graph adversarial attacks.
  arXiv  Detail & Related papers  (2024-09-13T02:23:55Z)
• Deep Generative Models in Robotics: A Survey on Learning from Multimodal Demonstrations [52.11801730860999]
  In recent years, the robot learning community has shown increasing interest in using deep generative models to capture the complexity of large datasets. We present the different types of models that the community has explored, such as energy-based models, diffusion models, action value maps, or generative adversarial networks. We also present the different types of applications in which deep generative models have been used, from grasp generation to trajectory generation or cost learning.
  arXiv  Detail & Related papers  (2024-08-08T11:34:31Z)
• Generative Expansion of Small Datasets: An Expansive Graph Approach [13.053285552524052]
  We introduce an Expansive Synthesis model generating large-scale, information-rich datasets from minimal samples. An autoencoder with self-attention layers and optimal transport refines distributional consistency. Results show comparable performance, demonstrating the model's potential to augment training data effectively.
  arXiv  Detail & Related papers  (2024-06-25T02:59:02Z)
• Bridging Model-Based Optimization and Generative Modeling via Conservative Fine-Tuning of Diffusion Models [54.132297393662654]
  We introduce a hybrid method that fine-tunes cutting-edge diffusion models by optimizing reward models through RL. We demonstrate the capability of our approach to outperform the best designs in offline data, leveraging the extrapolation capabilities of reward models.
  arXiv  Detail & Related papers  (2024-05-30T03:57:29Z)
• Synthetic location trajectory generation using categorical diffusion models [50.809683239937584]
  Diffusion models (DPMs) have rapidly evolved to be one of the predominant generative models for the simulation of synthetic data. We propose using DPMs for the generation of synthetic individual location trajectories (ILTs) which are sequences of variables representing physical locations visited by individuals.
  arXiv  Detail & Related papers  (2024-02-19T15:57:39Z)
• Image is All You Need to Empower Large-scale Diffusion Models for In-Domain Generation [7.1629002695210024]
  In-domain generation aims to perform a variety of tasks within a specific domain, such as unconditional generation, text-to-image, image editing, 3D generation, and more.<n>Early research typically required training specialized generators for each unique task and domain, often relying on fully-labeled data.<n>Motivated by the powerful generative capabilities and broad applications of diffusion models, we are driven to explore leveraging label-free data to empower these models for in-domain generation.
  arXiv  Detail & Related papers  (2023-12-13T14:59:49Z)
• ScribbleGen: Generative Data Augmentation Improves Scribble-supervised Semantic Segmentation [10.225021032417589]
  We propose ScribbleGen, a generative data augmentation method for scribble-supervised semantic segmentation. We leverage a ControlNet diffusion model conditioned on semantic scribbles to produce high-quality training data. We show that our framework significantly improves segmentation performance on small datasets, even surpassing fully-supervised segmentation.
  arXiv  Detail & Related papers  (2023-11-28T13:44:33Z)
• Improving Out-of-Distribution Robustness of Classifiers via Generative Interpolation [56.620403243640396]
  Deep neural networks achieve superior performance for learning from independent and identically distributed (i.i.d.) data. However, their performance deteriorates significantly when handling out-of-distribution (OoD) data. We develop a simple yet effective method called Generative Interpolation to fuse generative models trained from multiple domains for synthesizing diverse OoD samples.
  arXiv  Detail & Related papers  (2023-07-23T03:53:53Z)
• Learning to Jump: Thinning and Thickening Latent Counts for Generative Modeling [69.60713300418467]
  Learning to jump is a general recipe for generative modeling of various types of data. We demonstrate when learning to jump is expected to perform comparably to learning to denoise, and when it is expected to perform better.
  arXiv  Detail & Related papers  (2023-05-28T05:38:28Z)
• Generative Adversarial Reduced Order Modelling [0.0]
  We present GAROM, a new approach for reduced order modelling (ROM) based on generative adversarial networks (GANs) GANs have the potential to learn data distribution and generate more realistic data. In this work, we combine the GAN and ROM framework, by introducing a data-driven generative adversarial model able to learn solutions to parametric differential equations.
  arXiv  Detail & Related papers  (2023-05-25T09:23:33Z)
• VTAE: Variational Transformer Autoencoder with Manifolds Learning [144.0546653941249]
  Deep generative models have demonstrated successful applications in learning non-linear data distributions through a number of latent variables. The nonlinearity of the generator implies that the latent space shows an unsatisfactory projection of the data space, which results in poor representation learning. We show that geodesics and accurate computation can substantially improve the performance of deep generative models.
  arXiv  Detail & Related papers  (2023-04-03T13:13:19Z)
• Generative Causal Representation Learning for Out-of-Distribution Motion Forecasting [13.99348653165494]
  We propose Generative Causal Learning Representation to facilitate knowledge transfer under distribution shifts. While we evaluate the effectiveness of our proposed method in human trajectory prediction models, GCRL can be applied to other domains as well.
  arXiv  Detail & Related papers  (2023-02-17T00:30:44Z)
• A Survey on Generative Diffusion Model [75.93774014861978]
  Diffusion models are an emerging class of deep generative models. They have certain limitations, including a time-consuming iterative generation process and confinement to high-dimensional Euclidean space. This survey presents a plethora of advanced techniques aimed at enhancing diffusion models.
  arXiv  Detail & Related papers  (2022-09-06T16:56:21Z)
• Deep Generative Modeling on Limited Data with Regularization by Nontransferable Pre-trained Models [32.52492468276371]
  We propose regularized deep generative model (Reg-DGM) to reduce the variance of generative modeling with limited data. Reg-DGM uses a pre-trained model to optimize a weighted sum of a certain divergence and the expectation of an energy function. Empirically, with various pre-trained feature extractors and a data-dependent energy function, Reg-DGM consistently improves the generation performance of strong DGMs with limited data.
  arXiv  Detail & Related papers  (2022-08-30T10:28:50Z)
• Correcting Model Bias with Sparse Implicit Processes [0.9187159782788579]
  We show that Sparse Implicit Processes (SIP) is capable of correcting model bias when the data generating mechanism differs strongly from the one implied by the model. We use synthetic datasets to show that SIP is capable of providing predictive distributions that reflect the data better than the exact predictions of the initial, but wrongly assumed model.
  arXiv  Detail & Related papers  (2022-07-21T18:00:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.