LDMol: Text-to-Molecule Diffusion Model with Structurally Informative Latent Space

• URL: http://arxiv.org/abs/2405.17829v2
• Date: Thu, 03 Oct 2024 15:14:29 GMT
• Title: LDMol: Text-to-Molecule Diffusion Model with Structurally Informative Latent Space
• Authors: Jinho Chang, Jong Chul Ye,
• Abstract summary: We present a novel latent diffusion model dubbed LDMol for text-conditioned molecule generation. LDMol comprises a molecule autoencoder that produces a learnable and structurally informative feature space. We show that LDMol can be applied to downstream tasks such as molecule-to-text retrieval and text-guided molecule editing.
• Score: 55.5427001668863
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With the emergence of diffusion models as the frontline of generative models, many researchers have proposed molecule generation techniques with conditional diffusion models. However, the unavoidable discreteness of a molecule makes it difficult for a diffusion model to connect raw data with highly complex conditions like natural language. To address this, we present a novel latent diffusion model dubbed LDMol for text-conditioned molecule generation. LDMol comprises a molecule autoencoder that produces a learnable and structurally informative feature space, and a natural language-conditioned latent diffusion model. In particular, recognizing that multiple SMILES notations can represent the same molecule, we employ a contrastive learning strategy to extract feature space that is aware of the unique characteristics of the molecule structure. LDMol outperforms the existing baselines on the text-to-molecule generation benchmark, suggesting a potential for diffusion models can outperform autoregressive models in text data generation with a better choice of the latent domain. Furthermore, we show that LDMol can be applied to downstream tasks such as molecule-to-text retrieval and text-guided molecule editing, demonstrating its versatility as a diffusion model.

Related papers

• Continuous Diffusion Model for Language Modeling [57.396578974401734]
  Existing continuous diffusion models for discrete data have limited performance compared to discrete approaches.<n>We propose a continuous diffusion model for language modeling that incorporates the geometry of the underlying categorical distribution.
  arXiv  Detail & Related papers  (2025-02-17T08:54:29Z)
• Diffusion Models for Molecules: A Survey of Methods and Tasks [56.44565051667812]
  Generative tasks about molecules are crucial for drug discovery and material design. Diffusion models have emerged as an impressive class of deep generative models. This paper conducts a comprehensive survey of diffusion model-based molecular generative methods.
  arXiv  Detail & Related papers  (2025-02-13T17:22:50Z)
• MolMiner: Transformer architecture for fragment-based autoregressive generation of molecular stories [7.366789601705544]
  Chemical validity, interpretability of the generation process and flexibility to variable molecular sizes are among some of the remaining challenges for generative models in computational materials design. We propose an autoregressive approach that decomposes molecular generation into a sequence of discrete and interpretable steps. Our results show that the model can effectively bias the generation distribution according to the prompted multi-target objective.
  arXiv  Detail & Related papers  (2024-11-10T22:00:55Z)
• Bridging the Gap between Learning and Inference for Diffusion-Based Molecule Generation [18.936142688346816]
  GapDiff is a training framework that mitigates the data distributional disparity between training and inference. We conduct experiments using a 3D molecular generation model on the CrossDocked 2020 dataset.
  arXiv  Detail & Related papers  (2024-11-08T10:53:39Z)
• Conditional Synthesis of 3D Molecules with Time Correction Sampler [58.0834973489875]
  Time-Aware Conditional Synthesis (TACS) is a novel approach to conditional generation on diffusion models. It integrates adaptively controlled plug-and-play "online" guidance into a diffusion model, driving samples toward the desired properties.
  arXiv  Detail & Related papers  (2024-11-01T12:59:25Z)
• MING: A Functional Approach to Learning Molecular Generative Models [46.189683355768736]
  This paper introduces a novel paradigm for learning molecule generative models based on functional representations. We propose Molecular Implicit Neural Generation (MING), a diffusion-based model that learns molecular distributions in function space.
  arXiv  Detail & Related papers  (2024-10-16T13:02:02Z)
• SubGDiff: A Subgraph Diffusion Model to Improve Molecular Representation Learning [14.338345772161102]
  We propose a novel diffusion model termed SubGDiff for involving the molecular subgraph information in diffusion. SubGDiff adopts three vital techniques: subgraph prediction, expectation state, and k-step same subgraph diffusion. Experimentally, extensive downstream tasks demonstrate the superior performance of our approach.
  arXiv  Detail & Related papers  (2024-05-09T10:37:33Z)
• Data-Efficient Molecular Generation with Hierarchical Textual Inversion [48.816943690420224]
  We introduce Hierarchical textual Inversion for Molecular generation (HI-Mol), a novel data-efficient molecular generation method. HI-Mol is inspired by the importance of hierarchical information, e.g., both coarse- and fine-grained features, in understanding the molecule distribution. Compared to the conventional textual inversion method in the image domain using a single-level token embedding, our multi-level token embeddings allow the model to effectively learn the underlying low-shot molecule distribution.
  arXiv  Detail & Related papers  (2024-05-05T08:35:23Z)
• Integrating Chemical Language and Molecular Graph in Multimodal Fused Deep Learning for Drug Property Prediction [9.388979080270103]
  We construct multimodal deep learning models to cover different molecular representations. Compared with mono-modal models, our multimodal fused deep learning (MMFDL) models outperform single models in accuracy, reliability, and resistance capability against noise.
  arXiv  Detail & Related papers  (2023-12-29T07:19:42Z)
• Molecule Design by Latent Space Energy-Based Modeling and Gradual Distribution Shifting [53.44684898432997]
  Generation of molecules with desired chemical and biological properties is critical for drug discovery. We propose a probabilistic generative model to capture the joint distribution of molecules and their properties. Our method achieves very strong performances on various molecule design tasks.
  arXiv  Detail & Related papers  (2023-06-09T03:04:21Z)
• A Survey of Diffusion Models in Natural Language Processing [11.233768932957771]
  Diffusion models capture the diffusion of information or signals across a network or manifold. This paper discusses the different formulations of diffusion models used in NLP, their strengths and limitations, and their applications.
  arXiv  Detail & Related papers  (2023-05-24T03:25:32Z)
• Diffusion Models for Non-autoregressive Text Generation: A Survey [94.4634088113513]
  Non-autoregressive (NAR) text generation has attracted much attention in the field of natural language processing. Recently, diffusion models have been introduced into NAR text generation, showing an improved text generation quality.
  arXiv  Detail & Related papers  (2023-03-12T05:11:09Z)
• Unifying Diffusion Models' Latent Space, with Applications to CycleDiffusion and Guidance [95.12230117950232]
  We show that a common latent space emerges from two diffusion models trained independently on related domains. Applying CycleDiffusion to text-to-image diffusion models, we show that large-scale text-to-image diffusion models can be used as zero-shot image-to-image editors.
  arXiv  Detail & Related papers  (2022-10-11T15:53:52Z)
• A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Language [63.60376252491507]
  We propose a molecular multimodal foundation model which is pretrained from molecular graphs and their semantically related textual data. We believe that our model would have a broad impact on AI-empowered fields across disciplines such as biology, chemistry, materials, environment, and medicine.
  arXiv  Detail & Related papers  (2022-09-12T00:56:57Z)
• Learning Latent Space Energy-Based Prior Model for Molecule Generation [59.875533935578375]
  We learn latent space energy-based prior model with SMILES representation for molecule modeling. Our method is able to generate molecules with validity and uniqueness competitive with state-of-the-art models.
  arXiv  Detail & Related papers  (2020-10-19T09:34:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.