Data-Dependent Smoothing for Protein Discovery with Walk-Jump Sampling

• URL: http://arxiv.org/abs/2509.02069v1
• Date: Tue, 02 Sep 2025 08:17:59 GMT
• Title: Data-Dependent Smoothing for Protein Discovery with Walk-Jump Sampling
• Authors: Srinivas Anumasa, Barath Chandran. C, Tingting Chen, Dianbo Liu,
• Abstract summary: Diffusion models have emerged as a powerful class of generative models by learning to iteratively reverse the noising process.<n>Their ability to generate high-quality samples has extended beyond high-dimensional image data to other complex domains such as proteins.<n>We introduce a Data-Dependent Smoothing Walk-Jump framework that employs kernel density estimation (KDE) as a preprocessing step to estimate the noise scale $sigma$ for each data point.<n>By incorporating local data geometry into the denoising process, our method accounts for the heterogeneous distribution of protein data.
• Score: 7.278972126771497
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diffusion models have emerged as a powerful class of generative models by learning to iteratively reverse the noising process. Their ability to generate high-quality samples has extended beyond high-dimensional image data to other complex domains such as proteins, where data distributions are typically sparse and unevenly spread. Importantly, the sparsity itself is uneven. Empirically, we observed that while a small fraction of samples lie in dense clusters, the majority occupy regions of varying sparsity across the data space. Existing approaches largely ignore this data-dependent variability. In this work, we introduce a Data-Dependent Smoothing Walk-Jump framework that employs kernel density estimation (KDE) as a preprocessing step to estimate the noise scale $\sigma$ for each data point, followed by training a score model with these data-dependent $\sigma$ values. By incorporating local data geometry into the denoising process, our method accounts for the heterogeneous distribution of protein data. Empirical evaluations demonstrate that our approach yields consistent improvements across multiple metrics, highlighting the importance of data-aware sigma prediction for generative modeling in sparse, high-dimensional settings.

Related papers

• Multimodal Atmospheric Super-Resolution With Deep Generative Models [0.0]
  Score-based diffusion modeling is a generative machine learning algorithm that can be used to sample from complex distributions.<n>In this article, we apply such a concept to the super-resolution of a high-dimensional dynamical system, given the real-time availability of low-resolution and experimentally observed sparse sensor measurements.
  arXiv  Detail & Related papers  (2025-06-28T06:47:09Z)
• Convergence Of Consistency Model With Multistep Sampling Under General Data Assumptions [11.317363635566517]
  We study the convergence of consistency models when the self-consistency property holds approximately under the training distribution.<n>Our analysis requires only mild data assumption and applies to a family of forward processes.
  arXiv  Detail & Related papers  (2025-05-06T05:31:10Z)
• Resolving Memorization in Empirical Diffusion Model for Manifold Data in High-Dimensional Spaces [5.716752583983991]
  When the data distribution consists of n points, empirical diffusion models tend to reproduce existing data points.<n>This work shows that the memorization issue can be solved simply by applying an inertia update at the end of the empirical diffusion simulation.<n>We demonstrate that the distribution of samples from this model approximates the true data distribution on a $C2$ manifold of dimension $d$, within a Wasserstein-1 distance of order $O(n-frac2d+4)$.
  arXiv  Detail & Related papers  (2025-05-05T09:40:41Z)
• Generative diffusion model with inverse renormalization group flows [0.0]
  Diffusion models produce data by denoising a sample corrupted by white noise.<n>We introduce a renormalization group-based diffusion model that leverages multiscale nature of data distributions.<n>We validate the versatility of the model through applications to protein structure prediction and image generation.
  arXiv  Detail & Related papers  (2025-01-15T19:00:01Z)
• Diffusion posterior sampling for simulation-based inference in tall data settings [53.17563688225137]
  Simulation-based inference ( SBI) is capable of approximating the posterior distribution that relates input parameters to a given observation. In this work, we consider a tall data extension in which multiple observations are available to better infer the parameters of the model. We compare our method to recently proposed competing approaches on various numerical experiments and demonstrate its superiority in terms of numerical stability and computational cost.
  arXiv  Detail & Related papers  (2024-04-11T09:23:36Z)
• Deep Generative Sampling in the Dual Divergence Space: A Data-efficient & Interpretative Approach for Generative AI [29.13807697733638]
  We build on the remarkable achievements in generative sampling of natural images. We propose an innovative challenge, potentially overly ambitious, which involves generating samples that resemble images. The statistical challenge lies in the small sample size, sometimes consisting of a few hundred subjects.
  arXiv  Detail & Related papers  (2024-04-10T22:35:06Z)
• Distribution-Aware Data Expansion with Diffusion Models [55.979857976023695]
  We propose DistDiff, a training-free data expansion framework based on the distribution-aware diffusion model. DistDiff consistently enhances accuracy across a diverse range of datasets compared to models trained solely on original data.
  arXiv  Detail & Related papers  (2024-03-11T14:07:53Z)
• Towards Theoretical Understandings of Self-Consuming Generative Models [56.84592466204185]
  This paper tackles the emerging challenge of training generative models within a self-consuming loop. We construct a theoretical framework to rigorously evaluate how this training procedure impacts the data distributions learned by future models. We present results for kernel density estimation, delivering nuanced insights such as the impact of mixed data training on error propagation.
  arXiv  Detail & Related papers  (2024-02-19T02:08:09Z)
• Score-based Diffusion Models in Function Space [137.70916238028306]
  Diffusion models have recently emerged as a powerful framework for generative modeling.<n>This work introduces a mathematically rigorous framework called Denoising Diffusion Operators (DDOs) for training diffusion models in function space.<n>We show that the corresponding discretized algorithm generates accurate samples at a fixed cost independent of the data resolution.
  arXiv  Detail & Related papers  (2023-02-14T23:50:53Z)
• Learning from aggregated data with a maximum entropy model [73.63512438583375]
  We show how a new model, similar to a logistic regression, may be learned from aggregated data only by approximating the unobserved feature distribution with a maximum entropy hypothesis. We present empirical evidence on several public datasets that the model learned this way can achieve performances comparable to those of a logistic model trained with the full unaggregated data.
  arXiv  Detail & Related papers  (2022-10-05T09:17:27Z)
• Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learning [112.69497636932955]
  Federated learning aims to train models across different clients without the sharing of data for privacy considerations. We study how data heterogeneity affects the representations of the globally aggregated models. We propose sc FedDecorr, a novel method that can effectively mitigate dimensional collapse in federated learning.
  arXiv  Detail & Related papers  (2022-10-01T09:04:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.