MolMiner: Transformer architecture for fragment-based autoregressive generation of molecular stories

• URL: http://arxiv.org/abs/2411.06608v1
• Date: Sun, 10 Nov 2024 22:00:55 GMT
• Title: MolMiner: Transformer architecture for fragment-based autoregressive generation of molecular stories
• Authors: Raul Ortega Ochoa, Tejs Vegge, Jes Frellsen,
• Abstract summary: Chemical validity, interpretability of the generation process and flexibility to variable molecular sizes are among some of the remaining challenges for generative models in computational materials design. We propose an autoregressive approach that decomposes molecular generation into a sequence of discrete and interpretable steps. Our results show that the model can effectively bias the generation distribution according to the prompted multi-target objective.
• Score: 7.366789601705544
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Deep generative models for molecular discovery have become a very popular choice in new high-throughput screening paradigms. These models have been developed inheriting from the advances in natural language processing and computer vision, achieving ever greater results. However, generative molecular modelling has unique challenges that are often overlooked. Chemical validity, interpretability of the generation process and flexibility to variable molecular sizes are among some of the remaining challenges for generative models in computational materials design. In this work, we propose an autoregressive approach that decomposes molecular generation into a sequence of discrete and interpretable steps using molecular fragments as units, a 'molecular story'. Enforcing chemical rules in the stories guarantees the chemical validity of the generated molecules, the discrete sequential steps of a molecular story makes the process transparent improving interpretability, and the autoregressive nature of the approach allows the size of the molecule to be a decision of the model. We demonstrate the validity of the approach in a multi-target inverse design of electroactive organic compounds, focusing on the target properties of solubility, redox potential, and synthetic accessibility. Our results show that the model can effectively bias the generation distribution according to the prompted multi-target objective.

Related papers

• Sampling 3D Molecular Conformers with Diffusion Transformers [13.536503487456622]
  Diffusion Transformers (DiTs) have demonstrated strong performance in generative modeling.<n>Applying DiTs to molecules introduces novel challenges, such as integrating discrete molecular graph information with continuous 3D geometry.<n>We propose DiTMC, a framework that adapts DiTs to address these challenges through a modular architecture.
  arXiv  Detail & Related papers  (2025-06-18T11:47:59Z)
• Mol-CADiff: Causality-Aware Autoregressive Diffusion for Molecule Generation [13.401822039640297]
  Mol-CADiff is a novel diffusion-based framework that uses causal attention mechanisms for text-conditional molecular generation. Our approach explicitly models the causal relationship between textual prompts and molecular structures, overcoming key limitations in existing methods. Our experiments demonstrate that Mol-CADiff outperforms state-of-the-art methods in generating diverse, novel, and chemically valid molecules.
  arXiv  Detail & Related papers  (2025-03-07T15:10:37Z)
• Diffusion Models for Molecules: A Survey of Methods and Tasks [56.44565051667812]
  Generative tasks about molecules are crucial for drug discovery and material design. Diffusion models have emerged as an impressive class of deep generative models. This paper conducts a comprehensive survey of diffusion model-based molecular generative methods.
  arXiv  Detail & Related papers  (2025-02-13T17:22:50Z)
• D3MES: Diffusion Transformer with multihead equivariant self-attention for 3D molecule generation [1.3791394805787949]
  We introduce a diffusion model for 3D molecule generation that combines a classifiable diffusion model, Diffusion Transformer, with multihead equiheadvariant self-attention. This method addresses two key challenges: correctly attaching hydrogen atoms in generated molecules through learning representations of molecules after hydrogen atoms are removed; and overcoming the limitations of existing models that cannot generate molecules across multiple classes simultaneously.
  arXiv  Detail & Related papers  (2025-01-13T06:16:11Z)
• Tokenizing 3D Molecule Structure with Quantized Spherical Coordinates [28.452581855002855]
  Mol-StrucTok is a novel method for tokenizing 3D molecular structures.<n>We design a line notation for 3D molecules by extracting local atomic coordinates in a spherical coordinate system.<n>We employ a Vector Quantized Variational Autoencoder (VQ-VAE) to tokenize these coordinates, treating them as generation descriptors.
  arXiv  Detail & Related papers  (2024-12-02T14:50:44Z)
• Generative Modeling of Molecular Dynamics Trajectories [12.255021091552441]
  We introduce generative modeling of molecular trajectories as a paradigm for learning flexible multi-task surrogate models of MD from data. We show such generative models can be adapted to diverse tasks such as forward simulation, transition path sampling, and trajectory upsampling.
  arXiv  Detail & Related papers  (2024-09-26T13:02:28Z)
• Rethinking Molecular Design: Integrating Latent Variable and Auto-Regressive Models for Goal Directed Generation [0.6800113478497425]
  We return to the simplest representation of molecules, and investigate overlooked limitations of classical generative approaches. We propose a hybrid model in the form of a novel regularizer that leverages the strengths of both to improve validity, conditional generation, and style transfer of molecular sequences.
  arXiv  Detail & Related papers  (2024-08-19T11:50:23Z)
• SemlaFlow -- Efficient 3D Molecular Generation with Latent Attention and Equivariant Flow Matching [43.56824843205882]
  Semla is a scalable E(3)-equivariant message passing architecture.<n>SemlaFlow is trained to generate a joint distribution over atom types, coordinates, bond types and formal charges.<n>Our model produces state-of-the-art results on benchmark datasets with as few as 20 sampling steps.
  arXiv  Detail & Related papers  (2024-06-11T13:51:51Z)
• LDMol: Text-to-Molecule Diffusion Model with Structurally Informative Latent Space [55.5427001668863]
  We present a novel latent diffusion model dubbed LDMol for text-conditioned molecule generation. LDMol comprises a molecule autoencoder that produces a learnable and structurally informative feature space. We show that LDMol can be applied to downstream tasks such as molecule-to-text retrieval and text-guided molecule editing.
  arXiv  Detail & Related papers  (2024-05-28T04:59:13Z)
• Diffusing on Two Levels and Optimizing for Multiple Properties: A Novel Approach to Generating Molecules with Desirable Properties [33.2976176283611]
  We present a novel approach to generating molecules with desirable properties, which expands the diffusion model framework with multiple innovative designs. To get desirable molecular fragments, we develop a novel electronic effect based fragmentation method. We show that the molecules generated by our proposed method have better validity, uniqueness, novelty, Fr'echet ChemNet Distance (FCD), QED, and PlogP than those generated by current SOTA models.
  arXiv  Detail & Related papers  (2023-10-05T11:43:21Z)
• Molecule Design by Latent Prompt Transformer [61.68502207071992]
  This paper proposes a latent prompt Transformer model for solving challenging optimization problems such as molecule design. The goal is to find molecules with optimal values of a target chemical or biological property that can be computed by an existing software.
  arXiv  Detail & Related papers  (2023-10-05T02:09:51Z)
• Molecule Design by Latent Space Energy-Based Modeling and Gradual Distribution Shifting [53.44684898432997]
  Generation of molecules with desired chemical and biological properties is critical for drug discovery. We propose a probabilistic generative model to capture the joint distribution of molecules and their properties. Our method achieves very strong performances on various molecule design tasks.
  arXiv  Detail & Related papers  (2023-06-09T03:04:21Z)
• An Equivariant Generative Framework for Molecular Graph-Structure Co-Design [54.92529253182004]
  We present MolCode, a machine learning-based generative framework for underlineMolecular graph-structure underlineCo-design. In MolCode, 3D geometric information empowers the molecular 2D graph generation, which in turn helps guide the prediction of molecular 3D structure. Our investigation reveals that the 2D topology and 3D geometry contain intrinsically complementary information in molecule design.
  arXiv  Detail & Related papers  (2023-04-12T13:34:22Z)
• Domain-Agnostic Molecular Generation with Chemical Feedback [44.063584808910896]
  MolGen is a pre-trained molecular language model tailored specifically for molecule generation. It internalizes structural and grammatical insights through the reconstruction of over 100 million molecular SELFIES. Our chemical feedback paradigm steers the model away from molecular hallucinations, ensuring alignment between the model's estimated probabilities and real-world chemical preferences.
  arXiv  Detail & Related papers  (2023-01-26T17:52:56Z)
• Equivariant Shape-Conditioned Generation of 3D Molecules for Ligand-Based Drug Design [3.032184156362992]
  We introduce a new 3D generative model that enables shape-conditioned 3D molecular design by encoding molecular shape and variationally encoding chemical identity. We evaluate our 3D generative model in tasks relevant to drug design including shape-conditioned generation of chemically diverse molecular structures and shape-constrained molecular property optimization.
  arXiv  Detail & Related papers  (2022-10-06T19:37:34Z)
• A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Language [63.60376252491507]
  We propose a molecular multimodal foundation model which is pretrained from molecular graphs and their semantically related textual data. We believe that our model would have a broad impact on AI-empowered fields across disciplines such as biology, chemistry, materials, environment, and medicine.
  arXiv  Detail & Related papers  (2022-09-12T00:56:57Z)
• Retrieval-based Controllable Molecule Generation [63.44583084888342]
  We propose a new retrieval-based framework for controllable molecule generation. We use a small set of molecules to steer the pre-trained generative model towards synthesizing molecules that satisfy the given design criteria. Our approach is agnostic to the choice of generative models and requires no task-specific fine-tuning.
  arXiv  Detail & Related papers  (2022-08-23T17:01:16Z)
• Interpretable Molecular Graph Generation via Monotonic Constraints [19.401468196146336]
  Deep graph generative models treat molecule design as graph generation problems. Existing models have many shortcomings, including poor interpretability and controllability toward desired molecular properties. This paper proposes new methodologies for molecule generation with interpretable and deep controllable models.
  arXiv  Detail & Related papers  (2022-02-28T08:35:56Z)
• Molecular Attributes Transfer from Non-Parallel Data [57.010952598634944]
  We formulate molecular optimization as a style transfer problem and present a novel generative model that could automatically learn internal differences between two groups of non-parallel data. Experiments on two molecular optimization tasks, toxicity modification and synthesizability improvement, demonstrate that our model significantly outperforms several state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-11-30T06:10:22Z)
• Learning Neural Generative Dynamics for Molecular Conformation Generation [89.03173504444415]
  We study how to generate molecule conformations (textiti.e., 3D structures) from a molecular graph. We propose a novel probabilistic framework to generate valid and diverse conformations given a molecular graph.
  arXiv  Detail & Related papers  (2021-02-20T03:17:58Z)
• MIMOSA: Multi-constraint Molecule Sampling for Molecule Optimization [51.00815310242277]
  generative models and reinforcement learning approaches made initial success, but still face difficulties in simultaneously optimizing multiple drug properties. We propose the MultI-constraint MOlecule SAmpling (MIMOSA) approach, a sampling framework to use input molecule as an initial guess and sample molecules from the target distribution.
  arXiv  Detail & Related papers  (2020-10-05T20:18:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.