Text-Guided Molecule Generation with Diffusion Language Model
- URL: http://arxiv.org/abs/2402.13040v1
- Date: Tue, 20 Feb 2024 14:29:02 GMT
- Title: Text-Guided Molecule Generation with Diffusion Language Model
- Authors: Haisong Gong, Qiang Liu, Shu Wu, Liang Wang
- Abstract summary: We propose the Text-Guided Molecule Generation with Diffusion Language Model (TGM-DLM)
TGM-DLM updates token embeddings within the SMILES string collectively and iteratively, using a two-phase diffusion generation process.
We demonstrate that TGM-DLM outperforms MolT5-Base, an autoregressive model, without the need for additional data resources.
- Score: 23.170313481324598
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Text-guided molecule generation is a task where molecules are generated to
match specific textual descriptions. Recently, most existing SMILES-based
molecule generation methods rely on an autoregressive architecture. In this
work, we propose the Text-Guided Molecule Generation with Diffusion Language
Model (TGM-DLM), a novel approach that leverages diffusion models to address
the limitations of autoregressive methods. TGM-DLM updates token embeddings
within the SMILES string collectively and iteratively, using a two-phase
diffusion generation process. The first phase optimizes embeddings from random
noise, guided by the text description, while the second phase corrects invalid
SMILES strings to form valid molecular representations. We demonstrate that
TGM-DLM outperforms MolT5-Base, an autoregressive model, without the need for
additional data resources. Our findings underscore the remarkable effectiveness
of TGM-DLM in generating coherent and precise molecules with specific
properties, opening new avenues in drug discovery and related scientific
domains. Code will be released at: https://github.com/Deno-V/tgm-dlm.
Related papers
- LDMol: Text-Conditioned Molecule Diffusion Model Leveraging Chemically Informative Latent Space [55.5427001668863]
We present a novel latent diffusion model dubbed LDMol, which enables a natural text-conditioned molecule generation.
Specifically, LDMol is composed of three building blocks: a molecule encoder that produces a chemically informative feature space, a natural language-conditioned latent diffusion model using a Diffusion Transformer (DiT), and an autoregressive decoder for molecule regressive.
arXiv Detail & Related papers (2024-05-28T04:59:13Z) - Data-Efficient Molecular Generation with Hierarchical Textual Inversion [48.816943690420224]
We introduce Hierarchical textual Inversion for Molecular generation (HI-Mol), a novel data-efficient molecular generation method.
HI-Mol is inspired by the importance of hierarchical information, e.g., both coarse- and fine-grained features, in understanding the molecule distribution.
Compared to the conventional textual inversion method in the image domain using a single-level token embedding, our multi-level token embeddings allow the model to effectively learn the underlying low-shot molecule distribution.
arXiv Detail & Related papers (2024-05-05T08:35:23Z) - 3M-Diffusion: Latent Multi-Modal Diffusion for Text-Guided Generation of
Molecular Graphs [20.84977867473101]
We propose 3M-Diffusion, a novel multi-modal molecular graph generation method.
We show that 3M-Diffusion can generate high-quality, novel and diverse molecular graphs that semantically match the textual description provided.
arXiv Detail & Related papers (2024-03-11T21:44:54Z) - Empowering Molecule Discovery for Molecule-Caption Translation with Large Language Models: A ChatGPT Perspective [53.300288393173204]
Large Language Models (LLMs) have shown remarkable performance in various cross-modal tasks.
In this work, we propose an In-context Few-Shot Molecule Learning paradigm for molecule-caption translation.
We evaluate the effectiveness of MolReGPT on molecule-caption translation, including molecule understanding and text-based molecule generation.
arXiv Detail & Related papers (2023-06-11T08:16:25Z) - MolXPT: Wrapping Molecules with Text for Generative Pre-training [141.0924452870112]
MolXPT is a unified language model of text and molecules pre-trained on SMILES wrapped by text.
MolXPT outperforms strong baselines of molecular property prediction on MoleculeNet.
arXiv Detail & Related papers (2023-05-18T03:58:19Z) - TESS: Text-to-Text Self-Conditioned Simplex Diffusion [56.881170312435444]
Text-to-text Self-conditioned Simplex Diffusion employs a new form of self-conditioning, and applies the diffusion process on the logit simplex space rather than the learned embedding space.
We demonstrate that TESS outperforms state-of-the-art non-autoregressive models, requires fewer diffusion steps with minimal drop in performance, and is competitive with pretrained autoregressive sequence-to-sequence models.
arXiv Detail & Related papers (2023-05-15T06:33:45Z) - Diffusion Models for Non-autoregressive Text Generation: A Survey [94.4634088113513]
Non-autoregressive (NAR) text generation has attracted much attention in the field of natural language processing.
Recently, diffusion models have been introduced into NAR text generation, showing an improved text generation quality.
arXiv Detail & Related papers (2023-03-12T05:11:09Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.