D3MES: Diffusion Transformer with multihead equivariant self-attention for 3D molecule generation

• URL: http://arxiv.org/abs/2501.07077v1
• Date: Mon, 13 Jan 2025 06:16:11 GMT
• Title: D3MES: Diffusion Transformer with multihead equivariant self-attention for 3D molecule generation
• Authors: Zhejun Zhang, Yuanping Chen, Shibing Chu,
• Abstract summary: We introduce a diffusion model for 3D molecule generation that combines a classifiable diffusion model, Diffusion Transformer, with multihead equiheadvariant self-attention. This method addresses two key challenges: correctly attaching hydrogen atoms in generated molecules through learning representations of molecules after hydrogen atoms are removed; and overcoming the limitations of existing models that cannot generate molecules across multiple classes simultaneously.
• Score: 1.3791394805787949
• License:
• Abstract: Understanding and predicting the diverse conformational states of molecules is crucial for advancing fields such as chemistry, material science, and drug development. Despite significant progress in generative models, accurately generating complex and biologically or material-relevant molecular structures remains a major challenge. In this work, we introduce a diffusion model for three-dimensional (3D) molecule generation that combines a classifiable diffusion model, Diffusion Transformer, with multihead equivariant self-attention. This method addresses two key challenges: correctly attaching hydrogen atoms in generated molecules through learning representations of molecules after hydrogen atoms are removed; and overcoming the limitations of existing models that cannot generate molecules across multiple classes simultaneously. The experimental results demonstrate that our model not only achieves state-of-the-art performance across several key metrics but also exhibits robustness and versatility, making it highly suitable for early-stage large-scale generation processes in molecular design, followed by validation and further screening to obtain molecules with specific properties.

Related papers

• MolMiner: Transformer architecture for fragment-based autoregressive generation of molecular stories [7.366789601705544]
  Chemical validity, interpretability of the generation process and flexibility to variable molecular sizes are among some of the remaining challenges for generative models in computational materials design. We propose an autoregressive approach that decomposes molecular generation into a sequence of discrete and interpretable steps. Our results show that the model can effectively bias the generation distribution according to the prompted multi-target objective.
  arXiv  Detail & Related papers  (2024-11-10T22:00:55Z)
• Conditional Synthesis of 3D Molecules with Time Correction Sampler [58.0834973489875]
  Time-Aware Conditional Synthesis (TACS) is a novel approach to conditional generation on diffusion models. It integrates adaptively controlled plug-and-play "online" guidance into a diffusion model, driving samples toward the desired properties.
  arXiv  Detail & Related papers  (2024-11-01T12:59:25Z)
• Geometric-Facilitated Denoising Diffusion Model for 3D Molecule Generation [32.464905769094536]
  Existing diffusion-based generative methods on de novo 3D molecule generation face two major challenges. We introduce a Dual-Track Transformer Network (DTN) to fully excevate global spatial relationships and learn high quality representations. As for the second challenge, we design Geometric-Facilitated Loss (GFLoss) which intervenes the formation of bonds during the training period, instead of directly embedding edges into the latent space.
  arXiv  Detail & Related papers  (2024-01-05T07:29:21Z)
• Diffusing on Two Levels and Optimizing for Multiple Properties: A Novel Approach to Generating Molecules with Desirable Properties [33.2976176283611]
  We present a novel approach to generating molecules with desirable properties, which expands the diffusion model framework with multiple innovative designs. To get desirable molecular fragments, we develop a novel electronic effect based fragmentation method. We show that the molecules generated by our proposed method have better validity, uniqueness, novelty, Fr'echet ChemNet Distance (FCD), QED, and PlogP than those generated by current SOTA models.
  arXiv  Detail & Related papers  (2023-10-05T11:43:21Z)
• Molecule Design by Latent Space Energy-Based Modeling and Gradual Distribution Shifting [53.44684898432997]
  Generation of molecules with desired chemical and biological properties is critical for drug discovery. We propose a probabilistic generative model to capture the joint distribution of molecules and their properties. Our method achieves very strong performances on various molecule design tasks.
  arXiv  Detail & Related papers  (2023-06-09T03:04:21Z)
• MUDiff: Unified Diffusion for Complete Molecule Generation [104.7021929437504]
  We present a new model for generating a comprehensive representation of molecules, including atom features, 2D discrete molecule structures, and 3D continuous molecule coordinates. We propose a novel graph transformer architecture to denoise the diffusion process. Our model is a promising approach for designing stable and diverse molecules and can be applied to a wide range of tasks in molecular modeling.
  arXiv  Detail & Related papers  (2023-04-28T04:25:57Z)
• MDM: Molecular Diffusion Model for 3D Molecule Generation [19.386468094571725]
  Existing diffusion-based 3D molecule generation methods could suffer from unsatisfactory performances. Interatomic relations are not in molecules' 3D point cloud representations. Proposed model significantly outperforms existing methods for both unconditional and conditional generation tasks.
  arXiv  Detail & Related papers  (2022-09-13T03:40:18Z)
• Scalable Fragment-Based 3D Molecular Design with Reinforcement Learning [68.8204255655161]
  We introduce a novel framework for scalable 3D design that uses a hierarchical agent to build molecules. In a variety of experiments, we show that our agent, guided only by energy considerations, can efficiently learn to produce molecules with over 100 atoms.
  arXiv  Detail & Related papers  (2022-02-01T18:54:24Z)
• Learning Neural Generative Dynamics for Molecular Conformation Generation [89.03173504444415]
  We study how to generate molecule conformations (textiti.e., 3D structures) from a molecular graph. We propose a novel probabilistic framework to generate valid and diverse conformations given a molecular graph.
  arXiv  Detail & Related papers  (2021-02-20T03:17:58Z)
• Learning Latent Space Energy-Based Prior Model for Molecule Generation [59.875533935578375]
  We learn latent space energy-based prior model with SMILES representation for molecule modeling. Our method is able to generate molecules with validity and uniqueness competitive with state-of-the-art models.
  arXiv  Detail & Related papers  (2020-10-19T09:34:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.