Mitigating Exposure Bias in Score-Based Generation of Molecular Conformations

• URL: http://arxiv.org/abs/2409.14014v1
• Date: Sat, 21 Sep 2024 04:54:37 GMT
• Title: Mitigating Exposure Bias in Score-Based Generation of Molecular Conformations
• Authors: Sijia Wang, Chen Wang, Zhenhao Zhao, Jiqiang Zhang, Weiran Cai,
• Abstract summary: We propose a method for measuring exposure bias in Score-Based Generative Models used for molecular conformation generation. We design a new compensation algorithm Input Perturbation (IP), which is adapted from a method originally designed for DPMs only. We achieve new state-of-the-art performance on the GEOM-Drugs dataset and are on par with GEOM-QM9.
• Score: 6.442534896075223
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Molecular conformation generation poses a significant challenge in the field of computational chemistry. Recently, Diffusion Probabilistic Models (DPMs) and Score-Based Generative Models (SGMs) are effectively used due to their capacity for generating accurate conformations far beyond conventional physics-based approaches. However, the discrepancy between training and inference rises a critical problem known as the exposure bias. While this issue has been extensively investigated in DPMs, the existence of exposure bias in SGMs and its effective measurement remain unsolved, which hinders the use of compensation methods for SGMs, including ConfGF and Torsional Diffusion as the representatives. In this work, we first propose a method for measuring exposure bias in SGMs used for molecular conformation generation, which confirms the significant existence of exposure bias in these models and measures its value. We design a new compensation algorithm Input Perturbation (IP), which is adapted from a method originally designed for DPMs only. Experimental results show that by introducing IP, SGM-based molecular conformation models can significantly improve both the accuracy and diversity of the generated conformations. Especially by using the IP-enhanced Torsional Diffusion model, we achieve new state-of-the-art performance on the GEOM-Drugs dataset and are on par on GEOM-QM9. We provide the code publicly at https://github.com/jia-975/torsionalDiff-ip.

Related papers

• Bellman Diffusion: Generative Modeling as Learning a Linear Operator in the Distribution Space [72.52365911990935]
  We introduce Bellman Diffusion, a novel DGM framework that maintains linearity in MDPs through gradient and scalar field modeling. Our results show that Bellman Diffusion achieves accurate field estimations and is a capable image generator, converging 1.5x faster than the traditional histogram-based baseline in distributional RL tasks.
  arXiv  Detail & Related papers  (2024-10-02T17:53:23Z)
• Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling [47.82616476928464]
  Masked diffusion models (MDMs) have emerged as a popular research topic for generative modeling of discrete data. We show that both training and sampling of MDMs are theoretically free from the time variable. We identify, for the first time, an underlying numerical issue, even with the commonly used 32-bit floating-point precision.
  arXiv  Detail & Related papers  (2024-09-04T17:48:19Z)
• Toward the Identifiability of Comparative Deep Generative Models [7.5479347719819865]
  We propose a theory of identifiability for comparative Deep Generative Models (DGMs) We show that, while these models lack identifiability across a general class of mixing functions, they surprisingly become identifiable when the mixing function is piece-wise affine. We also investigate the impact of model misspecification, and empirically show that previously proposed regularization techniques for fitting comparative DGMs help with identifiability when the number of latent variables is not known in advance.
  arXiv  Detail & Related papers  (2024-01-29T06:10:54Z)
• Uncertainty-biased molecular dynamics for learning uniformly accurate interatomic potentials [25.091146216183144]
  Active learning uses biased or unbiased molecular dynamics to generate candidate pools. Existing biased and unbiased MD-simulation methods are prone to miss either rare events or extrapolative regions. This work demonstrates that MD, when biased by the MLIP's energy uncertainty, simultaneously captures extrapolative regions and rare events.
  arXiv  Detail & Related papers  (2023-12-03T14:39:14Z)
• Diversity-Measurable Anomaly Detection [106.07413438216416]
  We propose Diversity-Measurable Anomaly Detection (DMAD) framework to enhance reconstruction diversity. PDM essentially decouples deformation from embedding and makes the final anomaly score more reliable.
  arXiv  Detail & Related papers  (2023-03-09T05:52:42Z)
• Deep Generative Modeling on Limited Data with Regularization by Nontransferable Pre-trained Models [32.52492468276371]
  We propose regularized deep generative model (Reg-DGM) to reduce the variance of generative modeling with limited data. Reg-DGM uses a pre-trained model to optimize a weighted sum of a certain divergence and the expectation of an energy function. Empirically, with various pre-trained feature extractors and a data-dependent energy function, Reg-DGM consistently improves the generation performance of strong DGMs with limited data.
  arXiv  Detail & Related papers  (2022-08-30T10:28:50Z)
• ER: Equivariance Regularizer for Knowledge Graph Completion [107.51609402963072]
  We propose a new regularizer, namely, Equivariance Regularizer (ER) ER can enhance the generalization ability of the model by employing the semantic equivariance between the head and tail entities. The experimental results indicate a clear and substantial improvement over the state-of-the-art relation prediction methods.
  arXiv  Detail & Related papers  (2022-06-24T08:18:05Z)
• GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generation [102.85440102147267]
  We propose a novel generative model named GeoDiff for molecular conformation prediction. We show that GeoDiff is superior or comparable to existing state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-03-06T09:47:01Z)
• Riemannian Score-Based Generative Modeling [56.20669989459281]
  We introduce score-based generative models (SGMs) demonstrating remarkable empirical performance. Current SGMs make the underlying assumption that the data is supported on a Euclidean manifold with flat geometry. This prevents the use of these models for applications in robotics, geoscience or protein modeling.
  arXiv  Detail & Related papers  (2022-02-06T11:57:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.