All-atom Diffusion Transformers: Unified generative modelling of molecules and materials

• URL: http://arxiv.org/abs/2503.03965v1
• Date: Wed, 05 Mar 2025 23:35:44 GMT
• Title: All-atom Diffusion Transformers: Unified generative modelling of molecules and materials
• Authors: Chaitanya K. Joshi, Xiang Fu, Yi-Lun Liao, Vahe Gharakhanyan, Benjamin Kurt Miller, Anuroop Sriram, Zachary W. Ulissi,
• Abstract summary: All-atom Diffusion Transformer (ADiT) is a unified latent diffusion framework for jointly generating both periodic materials and non-periodic molecular systems.<n>Experiments on QM9 and MP20 datasets demonstrate that jointly trained ADiT generates realistic and valid molecules as well as materials.
• Score: 11.180029648567658
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diffusion models are the standard toolkit for generative modelling of 3D atomic systems. However, for different types of atomic systems - such as molecules and materials - the generative processes are usually highly specific to the target system despite the underlying physics being the same. We introduce the All-atom Diffusion Transformer (ADiT), a unified latent diffusion framework for jointly generating both periodic materials and non-periodic molecular systems using the same model: (1) An autoencoder maps a unified, all-atom representations of molecules and materials to a shared latent embedding space; and (2) A diffusion model is trained to generate new latent embeddings that the autoencoder can decode to sample new molecules or materials. Experiments on QM9 and MP20 datasets demonstrate that jointly trained ADiT generates realistic and valid molecules as well as materials, exceeding state-of-the-art results from molecule and crystal-specific models. ADiT uses standard Transformers for both the autoencoder and diffusion model, resulting in significant speedups during training and inference compared to equivariant diffusion models. Scaling ADiT up to half a billion parameters predictably improves performance, representing a step towards broadly generalizable foundation models for generative chemistry. Open source code: https://github.com/facebookresearch/all-atom-diffusion-transformer

Related papers

• Towards Unified Latent Space for 3D Molecular Latent Diffusion Modeling [80.59215359958934]
  3D molecule generation is crucial for drug discovery and material science. Existing approaches typically maintain separate latent spaces for invariant and equivariant modalities. We propose a multi-modal VAE that compresses 3D molecules into latent sequences from a unified latent space.
  arXiv  Detail & Related papers  (2025-03-19T08:56:13Z)
• D3MES: Diffusion Transformer with multihead equivariant self-attention for 3D molecule generation [1.3791394805787949]
  We introduce a diffusion model for 3D molecule generation that combines a classifiable diffusion model, Diffusion Transformer, with multihead equiheadvariant self-attention.<n>This method addresses two key challenges: correctly attaching hydrogen atoms in generated molecules through learning representations of molecules after hydrogen atoms are removed; and overcoming the limitations of existing models that cannot generate molecules across multiple classes simultaneously.
  arXiv  Detail & Related papers  (2025-01-13T06:16:11Z)
• Conditional Synthesis of 3D Molecules with Time Correction Sampler [58.0834973489875]
  Time-Aware Conditional Synthesis (TACS) is a novel approach to conditional generation on diffusion models. It integrates adaptively controlled plug-and-play "online" guidance into a diffusion model, driving samples toward the desired properties.
  arXiv  Detail & Related papers  (2024-11-01T12:59:25Z)
• LDMol: Text-to-Molecule Diffusion Model with Structurally Informative Latent Space [55.5427001668863]
  We present a novel latent diffusion model dubbed LDMol for text-conditioned molecule generation. LDMol comprises a molecule autoencoder that produces a learnable and structurally informative feature space. We show that LDMol can be applied to downstream tasks such as molecule-to-text retrieval and text-guided molecule editing.
  arXiv  Detail & Related papers  (2024-05-28T04:59:13Z)
• Learning Joint 2D & 3D Diffusion Models for Complete Molecule Generation [32.66694406638287]
  We propose a new joint 2D and 3D diffusion model (JODO) that generates molecules with atom types, formal charges, bond information, and 3D coordinates. Our model can also be extended for inverse molecular design targeting single or multiple quantum properties.
  arXiv  Detail & Related papers  (2023-05-21T04:49:53Z)
• MUDiff: Unified Diffusion for Complete Molecule Generation [104.7021929437504]
  We present a new model for generating a comprehensive representation of molecules, including atom features, 2D discrete molecule structures, and 3D continuous molecule coordinates. We propose a novel graph transformer architecture to denoise the diffusion process. Our model is a promising approach for designing stable and diverse molecules and can be applied to a wide range of tasks in molecular modeling.
  arXiv  Detail & Related papers  (2023-04-28T04:25:57Z)
• Unifying Diffusion Models' Latent Space, with Applications to CycleDiffusion and Guidance [95.12230117950232]
  We show that a common latent space emerges from two diffusion models trained independently on related domains. Applying CycleDiffusion to text-to-image diffusion models, we show that large-scale text-to-image diffusion models can be used as zero-shot image-to-image editors.
  arXiv  Detail & Related papers  (2022-10-11T15:53:52Z)
• Torsional Diffusion for Molecular Conformer Generation [28.225704750892795]
  torsional diffusion is a novel diffusion framework that operates on the space of torsion angles. On a standard benchmark of drug-like molecules, torsional diffusion generates superior conformer ensembles. Our model provides exact likelihoods, which we employ to build the first generalizable Boltzmann generator.
  arXiv  Detail & Related papers  (2022-06-01T04:30:41Z)
• BIGDML: Towards Exact Machine Learning Force Fields for Materials [55.944221055171276]
  Machine-learning force fields (MLFF) should be accurate, computationally and data efficient, and applicable to molecules, materials, and interfaces thereof. Here, we introduce the Bravais-Inspired Gradient-Domain Machine Learning approach and demonstrate its ability to construct reliable force fields using a training set with just 10-200 atoms.
  arXiv  Detail & Related papers  (2021-06-08T10:14:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.