SALSA: Semantically-Aware Latent Space Autoencoder

• URL: http://arxiv.org/abs/2310.02744v1
• Date: Wed, 4 Oct 2023 11:34:46 GMT
• Title: SALSA: Semantically-Aware Latent Space Autoencoder
• Abstract summary: We propose a transformer-autoencoder modified with a contrastive task to learn graph-to-graph similarity between molecules. We show that the composed training objective leads to a higher quality latent space that is more 1) structurally-aware, 2) semantically continuous, and 3) property-aware.
• Score: 45.39474752148712
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In deep learning for drug discovery, chemical data are often represented as simplified molecular-input line-entry system (SMILES) sequences which allow for straightforward implementation of natural language processing methodologies, one being the sequence-to-sequence autoencoder. However, we observe that training an autoencoder solely on SMILES is insufficient to learn molecular representations that are semantically meaningful, where semantics are defined by the structural (graph-to-graph) similarities between molecules. We demonstrate by example that autoencoders may map structurally similar molecules to distant codes, resulting in an incoherent latent space that does not respect the structural similarities between molecules. To address this shortcoming we propose Semantically-Aware Latent Space Autoencoder (SALSA), a transformer-autoencoder modified with a contrastive task, tailored specifically to learn graph-to-graph similarity between molecules. Formally, the contrastive objective is to map structurally similar molecules (separated by a single graph edit) to nearby codes in the latent space. To accomplish this, we generate a novel dataset comprised of sets of structurally similar molecules and opt for a supervised contrastive loss that is able to incorporate full sets of positive samples. We compare SALSA to its ablated counterparts, and show empirically that the composed training objective (reconstruction and contrastive task) leads to a higher quality latent space that is more 1) structurally-aware, 2) semantically continuous, and 3) property-aware.

Related papers

• What Does a Chemical Language Model Know About Molecules? [0.3499870393443268]
  Chemical language models (cLMs) are widely assumed to learn surface-level syntactic patterns rather than learning meaningful molecular semantics.<n>Here, we apply sparse autoencoders (SAEs) to MolFormer, an encoder-only cLM, to mechanistically examine how molecular representations are built across layers.
  arXiv  Detail & Related papers  (2026-06-22T14:59:57Z)
• Structure-guided molecular design with contrastive 3D protein-ligand learning [4.938502760488423]
  Structure-based drug discovery faces the dual challenge of accurately capturing 3D protein-ligand interactions and navigating ultra-large chemical spaces.<n>We present a unified framework that addresses these challenges by combining contrastive 3D structure encoding with autoregressive molecular generation conditioned on commercial compound spaces.
  arXiv  Detail & Related papers  (2026-04-21T15:13:41Z)
• Beyond Learning on Molecules by Weakly Supervising on Molecules [0.155963096715749]
  Our Adaptive Chemical Embedding Model (ACE-Mol) learns from weak motifs labeled with.<n>natural language descriptors that are cheap to compute trivial to scale.<n>ACE-Mol achieves state-of-the-art performance across molecular property prediction benchmarks with interpretable, chemically meaningful representations.
  arXiv  Detail & Related papers  (2026-02-04T16:03:20Z)
• Molecular Representations in Implicit Functional Space via Hyper-Networks [53.70982267248536]
  We argue that molecular learning can instead be formulated as learning in function space.<n>We instantiate this formulation with MolField, a hyper-network-based framework that learns distributions over molecular fields.<n>Our results show that treating molecules as continuous functions fundamentally changes how molecular representations generalize across tasks.
  arXiv  Detail & Related papers  (2026-01-29T21:13:37Z)
• DiffMS: Diffusion Generation of Molecules Conditioned on Mass Spectra [60.39311767532607]
  DiffMS is a formula-restricted encoder-decoder generative network. We develop a robust decoder that bridges latent embeddings and molecular structures. Experiments show DiffMS outperforms existing models on $textitde novo$ molecule generation.
  arXiv  Detail & Related papers  (2025-02-13T18:29:48Z)
• Exploring Optimal Transport-Based Multi-Grained Alignments for Text-Molecule Retrieval [24.061535843472427]
  We introduce the Optimal TRansport-based Multi-grained Alignments model (ORMA) ORMA is a novel approach that facilitates multi-grained alignments between textual descriptions and molecules. Experimental results on the ChEBI-20 and PCdes datasets demonstrate that ORMA significantly outperforms existing state-of-the-art (SOTA) models.
  arXiv  Detail & Related papers  (2024-11-04T06:30:52Z)
• GraphXForm: Graph transformer for computer-aided molecular design with application to extraction [73.1842164721868]
  We present GraphXForm, a decoder-only graph transformer architecture, which is pretrained on existing compounds and then fine-tuned. We evaluate it on two solvent design tasks for liquid-liquid extraction, showing that it outperforms four state-of-the-art molecular design techniques.
  arXiv  Detail & Related papers  (2024-11-03T19:45:15Z)
• Pre-trained Molecular Language Models with Random Functional Group Masking [54.900360309677794]
  We propose a SMILES-based underlineem Molecular underlineem Language underlineem Model, which randomly masking SMILES subsequences corresponding to specific molecular atoms. This technique aims to compel the model to better infer molecular structures and properties, thus enhancing its predictive capabilities.
  arXiv  Detail & Related papers  (2024-11-03T01:56:15Z)
• FARM: Functional Group-Aware Representations for Small Molecules [55.281754551202326]
  We introduce Functional Group-Aware Representations for Small Molecules (FARM)<n>FARM is a novel model designed to bridge the gap between SMILES, natural language, and molecular graphs.<n>We evaluate FARM on the MoleculeNet dataset, where it achieves state-of-the-art performance on 11 out of 13 tasks.
  arXiv  Detail & Related papers  (2024-10-02T23:04:58Z)
• Attention Based Molecule Generation via Hierarchical Variational Autoencoder [0.0]
  We show that by combining recurrent neural networks with convolutional networks in a hierarchical manner, we are able to both extract autoregressive information from SMILES strings. This allows for generations with very high validity rates on the order of 95% when reconstructing known molecules.
  arXiv  Detail & Related papers  (2024-01-18T21:45:12Z)
• CHA2: CHemistry Aware Convex Hull Autoencoder Towards Inverse Molecular Design [2.169755083801688]
  It is impossible to explore the entire search space comprehensively to exploit de novo structures with properties of interest. To address this challenge, reducing the intractable search space into a lower-dimensional latent volume helps examine molecular candidates more feasibly. We propose using a convex hall surrounding the top molecules in terms of high QEDs to ensnare a tight subspace in the latent representation as an efficient way to reveal novel molecules with high QEDs.
  arXiv  Detail & Related papers  (2023-02-21T21:05:31Z)
• Implicit Geometry and Interaction Embeddings Improve Few-Shot Molecular Property Prediction [53.06671763877109]
  We develop molecular embeddings that encode complex molecular characteristics to improve the performance of few-shot molecular property prediction. Our approach leverages large amounts of synthetic data, namely the results of molecular docking calculations. On multiple molecular property prediction benchmarks, training from the embedding space substantially improves Multi-Task, MAML, and Prototypical Network few-shot learning performance.
  arXiv  Detail & Related papers  (2023-02-04T01:32:40Z)
• Relative Molecule Self-Attention Transformer [4.020171169198032]
  Relative Molecule Attention Transformer (R-MAT) is a novel Transformer-based model based on the developed self-attention layer that achieves state-of-the-art or very competitive results across awide range of molecule property prediction tasks. Our main contribution is Relative Molecule Attention Transformer (R-MAT): a novel Transformer-based model based on the developed self-attention layer that achieves state-of-the-art or very competitive results across awide range of molecule property prediction tasks.
  arXiv  Detail & Related papers  (2021-10-12T09:05:26Z)
• Learning Latent Space Energy-Based Prior Model for Molecule Generation [59.875533935578375]
  We learn latent space energy-based prior model with SMILES representation for molecule modeling. Our method is able to generate molecules with validity and uniqueness competitive with state-of-the-art models.
  arXiv  Detail & Related papers  (2020-10-19T09:34:20Z)
• Continuous Representation of Molecules Using Graph Variational Autoencoder [8.726908273226584]
  We propose a generative model in the form of a VAE which is operating on the 2D-graph structure of molecules. A side predictor is employed to prune the latent space and help the decoder in generating meaningful adjacency tensor of molecules.
  arXiv  Detail & Related papers  (2020-04-17T10:19:55Z)
• Learning Autoencoders with Relational Regularization [89.53065887608088]
  A new framework is proposed for learning autoencoders of data distributions. We minimize the discrepancy between the model and target distributions, with a emphrelational regularization We implement the framework with two scalable algorithms, making it applicable for both probabilistic and deterministic autoencoders.
  arXiv  Detail & Related papers  (2020-02-07T17:27:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.