MolMiner: Transformer architecture for fragment-based autoregressive generation of molecular stories

• URL: http://arxiv.org/abs/2411.06608v1
• Date: Sun, 10 Nov 2024 22:00:55 GMT
• Title: MolMiner: Transformer architecture for fragment-based autoregressive generation of molecular stories
• Authors: Raul Ortega Ochoa, Tejs Vegge, Jes Frellsen,
• Abstract summary: Chemical validity, interpretability of the generation process and flexibility to variable molecular sizes are among some of the remaining challenges for generative models in computational materials design. We propose an autoregressive approach that decomposes molecular generation into a sequence of discrete and interpretable steps. Our results show that the model can effectively bias the generation distribution according to the prompted multi-target objective.
• Score: 7.366789601705544
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Deep generative models for molecular discovery have become a very popular choice in new high-throughput screening paradigms. These models have been developed inheriting from the advances in natural language processing and computer vision, achieving ever greater results. However, generative molecular modelling has unique challenges that are often overlooked. Chemical validity, interpretability of the generation process and flexibility to variable molecular sizes are among some of the remaining challenges for generative models in computational materials design. In this work, we propose an autoregressive approach that decomposes molecular generation into a sequence of discrete and interpretable steps using molecular fragments as units, a 'molecular story'. Enforcing chemical rules in the stories guarantees the chemical validity of the generated molecules, the discrete sequential steps of a molecular story makes the process transparent improving interpretability, and the autoregressive nature of the approach allows the size of the molecule to be a decision of the model. We demonstrate the validity of the approach in a multi-target inverse design of electroactive organic compounds, focusing on the target properties of solubility, redox potential, and synthetic accessibility. Our results show that the model can effectively bias the generation distribution according to the prompted multi-target objective.

Related papers

• Mol-CADiff: Causality-Aware Autoregressive Diffusion for Molecule Generation [13.401822039640297]
  Mol-CADiff is a novel diffusion-based framework that uses causal attention mechanisms for text-conditional molecular generation. Our approach explicitly models the causal relationship between textual prompts and molecular structures, overcoming key limitations in existing methods. Our experiments demonstrate that Mol-CADiff outperforms state-of-the-art methods in generating diverse, novel, and chemically valid molecules.
  arXiv  Detail & Related papers  (2025-03-07T15:10:37Z)
• Diffusion Models for Molecules: A Survey of Methods and Tasks [56.44565051667812]
  Generative tasks about molecules are crucial for drug discovery and material design. Diffusion models have emerged as an impressive class of deep generative models. This paper conducts a comprehensive survey of diffusion model-based molecular generative methods.
  arXiv  Detail & Related papers  (2025-02-13T17:22:50Z)
• D3MES: Diffusion Transformer with multihead equivariant self-attention for 3D molecule generation [1.3791394805787949]
  We introduce a diffusion model for 3D molecule generation that combines a classifiable diffusion model, Diffusion Transformer, with multihead equiheadvariant self-attention. This method addresses two key challenges: correctly attaching hydrogen atoms in generated molecules through learning representations of molecules after hydrogen atoms are removed; and overcoming the limitations of existing models that cannot generate molecules across multiple classes simultaneously.
  arXiv  Detail & Related papers  (2025-01-13T06:16:11Z)
• Rethinking Molecular Design: Integrating Latent Variable and Auto-Regressive Models for Goal Directed Generation [0.6800113478497425]
  We return to the simplest representation of molecules, and investigate overlooked limitations of classical generative approaches. We propose a hybrid model in the form of a novel regularizer that leverages the strengths of both to improve validity, conditional generation, and style transfer of molecular sequences.
  arXiv  Detail & Related papers  (2024-08-19T11:50:23Z)
• LDMol: Text-to-Molecule Diffusion Model with Structurally Informative Latent Space [55.5427001668863]
  We present a novel latent diffusion model dubbed LDMol for text-conditioned molecule generation. LDMol comprises a molecule autoencoder that produces a learnable and structurally informative feature space. We show that LDMol can be applied to downstream tasks such as molecule-to-text retrieval and text-guided molecule editing.
  arXiv  Detail & Related papers  (2024-05-28T04:59:13Z)
• Diffusing on Two Levels and Optimizing for Multiple Properties: A Novel Approach to Generating Molecules with Desirable Properties [33.2976176283611]
  We present a novel approach to generating molecules with desirable properties, which expands the diffusion model framework with multiple innovative designs. To get desirable molecular fragments, we develop a novel electronic effect based fragmentation method. We show that the molecules generated by our proposed method have better validity, uniqueness, novelty, Fr'echet ChemNet Distance (FCD), QED, and PlogP than those generated by current SOTA models.
  arXiv  Detail & Related papers  (2023-10-05T11:43:21Z)
• Molecule Design by Latent Space Energy-Based Modeling and Gradual Distribution Shifting [53.44684898432997]
  Generation of molecules with desired chemical and biological properties is critical for drug discovery. We propose a probabilistic generative model to capture the joint distribution of molecules and their properties. Our method achieves very strong performances on various molecule design tasks.
  arXiv  Detail & Related papers  (2023-06-09T03:04:21Z)
• Domain-Agnostic Molecular Generation with Chemical Feedback [44.063584808910896]
  MolGen is a pre-trained molecular language model tailored specifically for molecule generation. It internalizes structural and grammatical insights through the reconstruction of over 100 million molecular SELFIES. Our chemical feedback paradigm steers the model away from molecular hallucinations, ensuring alignment between the model's estimated probabilities and real-world chemical preferences.
  arXiv  Detail & Related papers  (2023-01-26T17:52:56Z)
• A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Language [63.60376252491507]
  We propose a molecular multimodal foundation model which is pretrained from molecular graphs and their semantically related textual data. We believe that our model would have a broad impact on AI-empowered fields across disciplines such as biology, chemistry, materials, environment, and medicine.
  arXiv  Detail & Related papers  (2022-09-12T00:56:57Z)
• Retrieval-based Controllable Molecule Generation [63.44583084888342]
  We propose a new retrieval-based framework for controllable molecule generation. We use a small set of molecules to steer the pre-trained generative model towards synthesizing molecules that satisfy the given design criteria. Our approach is agnostic to the choice of generative models and requires no task-specific fine-tuning.
  arXiv  Detail & Related papers  (2022-08-23T17:01:16Z)
• Interpretable Molecular Graph Generation via Monotonic Constraints [19.401468196146336]
  Deep graph generative models treat molecule design as graph generation problems. Existing models have many shortcomings, including poor interpretability and controllability toward desired molecular properties. This paper proposes new methodologies for molecule generation with interpretable and deep controllable models.
  arXiv  Detail & Related papers  (2022-02-28T08:35:56Z)
• Molecular Attributes Transfer from Non-Parallel Data [57.010952598634944]
  We formulate molecular optimization as a style transfer problem and present a novel generative model that could automatically learn internal differences between two groups of non-parallel data. Experiments on two molecular optimization tasks, toxicity modification and synthesizability improvement, demonstrate that our model significantly outperforms several state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-11-30T06:10:22Z)
• Learning Neural Generative Dynamics for Molecular Conformation Generation [89.03173504444415]
  We study how to generate molecule conformations (textiti.e., 3D structures) from a molecular graph. We propose a novel probabilistic framework to generate valid and diverse conformations given a molecular graph.
  arXiv  Detail & Related papers  (2021-02-20T03:17:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.