All-atom Diffusion Transformers: Unified generative modelling of molecules and materials

• URL: http://arxiv.org/abs/2503.03965v2
• Date: Thu, 22 May 2025 08:08:43 GMT
• Title: All-atom Diffusion Transformers: Unified generative modelling of molecules and materials
• Authors: Chaitanya K. Joshi, Xiang Fu, Yi-Lun Liao, Vahe Gharakhanyan, Benjamin Kurt Miller, Anuroop Sriram, Zachary W. Ulissi,
• Abstract summary: All-atom Diffusion Transformer (ADiT) is a unified latent diffusion framework for jointly generating both periodic materials and non-periodic molecular systems.<n>ADiT generates realistic and valid molecules as well as materials, obtaining state-of-the-art results on par with molecule and crystal-specific models.
• Score: 11.180029648567658
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diffusion models are the standard toolkit for generative modelling of 3D atomic systems. However, for different types of atomic systems -- such as molecules and materials -- the generative processes are usually highly specific to the target system despite the underlying physics being the same. We introduce the All-atom Diffusion Transformer (ADiT), a unified latent diffusion framework for jointly generating both periodic materials and non-periodic molecular systems using the same model: (1) An autoencoder maps a unified, all-atom representations of molecules and materials to a shared latent embedding space; and (2) A diffusion model is trained to generate new latent embeddings that the autoencoder can decode to sample new molecules or materials. Experiments on MP20, QM9 and GEOM-DRUGS datasets demonstrate that jointly trained ADiT generates realistic and valid molecules as well as materials, obtaining state-of-the-art results on par with molecule and crystal-specific models. ADiT uses standard Transformers with minimal inductive biases for both the autoencoder and diffusion model, resulting in significant speedups during training and inference compared to equivariant diffusion models. Scaling ADiT up to half a billion parameters predictably improves performance, representing a step towards broadly generalizable foundation models for generative chemistry. Open source code: https://github.com/facebookresearch/all-atom-diffusion-transformer

Related papers

• Zatom-1: A Multimodal Flow Foundation Model for 3D Molecules and Materials [51.342983349686556]
  General-purpose 3D chemical modeling encompasses molecules and materials, requiring both generative and predictive capabilities.<n>We introduce Zatom-1, the first end-to-end, fully open-source foundation model that unifies generative and predictive learning of 3D molecules and materials.
  arXiv  Detail & Related papers  (2026-02-24T20:52:39Z)
• MolHIT: Advancing Molecular-Graph Generation with Hierarchical Discrete Diffusion Models [37.89307688620534]
  We introduce MolHIT, a powerful molecular graph generation framework that overcomes long-standing performance limitations in existing methods.<n>Overall, MolHIT achieves new state-of-the-art performance on the MOSES dataset with near-perfect validity for the first time in graph diffusion.
  arXiv  Detail & Related papers  (2026-02-19T18:27:11Z)
• Towards Unified Latent Space for 3D Molecular Latent Diffusion Modeling [80.59215359958934]
  3D molecule generation is crucial for drug discovery and material science. Existing approaches typically maintain separate latent spaces for invariant and equivariant modalities. We propose a multi-modal VAE that compresses 3D molecules into latent sequences from a unified latent space.
  arXiv  Detail & Related papers  (2025-03-19T08:56:13Z)
• D3MES: Diffusion Transformer with multihead equivariant self-attention for 3D molecule generation [1.3791394805787949]
  We introduce a diffusion model for 3D molecule generation that combines a classifiable diffusion model, Diffusion Transformer, with multihead equiheadvariant self-attention.<n>This method addresses two key challenges: correctly attaching hydrogen atoms in generated molecules through learning representations of molecules after hydrogen atoms are removed; and overcoming the limitations of existing models that cannot generate molecules across multiple classes simultaneously.
  arXiv  Detail & Related papers  (2025-01-13T06:16:11Z)
• Conditional Synthesis of 3D Molecules with Time Correction Sampler [58.0834973489875]
  Time-Aware Conditional Synthesis (TACS) is a novel approach to conditional generation on diffusion models. It integrates adaptively controlled plug-and-play "online" guidance into a diffusion model, driving samples toward the desired properties.
  arXiv  Detail & Related papers  (2024-11-01T12:59:25Z)
• LDMol: Text-to-Molecule Diffusion Model with Structurally Informative Latent Space [55.5427001668863]
  We present a novel latent diffusion model dubbed LDMol for text-conditioned molecule generation. LDMol comprises a molecule autoencoder that produces a learnable and structurally informative feature space. We show that LDMol can be applied to downstream tasks such as molecule-to-text retrieval and text-guided molecule editing.
  arXiv  Detail & Related papers  (2024-05-28T04:59:13Z)
• Learning Joint 2D & 3D Diffusion Models for Complete Molecule Generation [32.66694406638287]
  We propose a new joint 2D and 3D diffusion model (JODO) that generates molecules with atom types, formal charges, bond information, and 3D coordinates. Our model can also be extended for inverse molecular design targeting single or multiple quantum properties.
  arXiv  Detail & Related papers  (2023-05-21T04:49:53Z)
• MUDiff: Unified Diffusion for Complete Molecule Generation [104.7021929437504]
  We present a new model for generating a comprehensive representation of molecules, including atom features, 2D discrete molecule structures, and 3D continuous molecule coordinates. We propose a novel graph transformer architecture to denoise the diffusion process. Our model is a promising approach for designing stable and diverse molecules and can be applied to a wide range of tasks in molecular modeling.
  arXiv  Detail & Related papers  (2023-04-28T04:25:57Z)
• Unifying Diffusion Models' Latent Space, with Applications to CycleDiffusion and Guidance [95.12230117950232]
  We show that a common latent space emerges from two diffusion models trained independently on related domains. Applying CycleDiffusion to text-to-image diffusion models, we show that large-scale text-to-image diffusion models can be used as zero-shot image-to-image editors.
  arXiv  Detail & Related papers  (2022-10-11T15:53:52Z)
• Torsional Diffusion for Molecular Conformer Generation [28.225704750892795]
  torsional diffusion is a novel diffusion framework that operates on the space of torsion angles. On a standard benchmark of drug-like molecules, torsional diffusion generates superior conformer ensembles. Our model provides exact likelihoods, which we employ to build the first generalizable Boltzmann generator.
  arXiv  Detail & Related papers  (2022-06-01T04:30:41Z)
• Equivariant Diffusion for Molecule Generation in 3D [74.289191525633]
  This work introduces a diffusion model for molecule computation generation in 3D that is equivariant to Euclidean transformations. Experimentally, the proposed method significantly outperforms previous 3D molecular generative methods regarding the quality of generated samples and efficiency at training time.
  arXiv  Detail & Related papers  (2022-03-31T12:52:25Z)
• BIGDML: Towards Exact Machine Learning Force Fields for Materials [55.944221055171276]
  Machine-learning force fields (MLFF) should be accurate, computationally and data efficient, and applicable to molecules, materials, and interfaces thereof. Here, we introduce the Bravais-Inspired Gradient-Domain Machine Learning approach and demonstrate its ability to construct reliable force fields using a training set with just 10-200 atoms.
  arXiv  Detail & Related papers  (2021-06-08T10:14:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.