MolMix: A Simple Yet Effective Baseline for Multimodal Molecular Representation Learning

• URL: http://arxiv.org/abs/2410.07981v2
• Date: Thu, 24 Oct 2024 08:34:50 GMT
• Title: MolMix: A Simple Yet Effective Baseline for Multimodal Molecular Representation Learning
• Authors: Andrei Manolache, Dragos Tantaru, Mathias Niepert,
• Abstract summary: We propose a simple transformer-based baseline for multimodal molecular representation learning. We integrate three distinct modalities: SMILES strings, 2D graph representations, and 3D conformers of molecules. Despite its simplicity, our approach achieves state-of-the-art results across multiple datasets.
• Score: 17.93173928602627
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this work, we propose a simple transformer-based baseline for multimodal molecular representation learning, integrating three distinct modalities: SMILES strings, 2D graph representations, and 3D conformers of molecules. A key aspect of our approach is the aggregation of 3D conformers, allowing the model to account for the fact that molecules can adopt multiple conformations-an important factor for accurate molecular representation. The tokens for each modality are extracted using modality-specific encoders: a transformer for SMILES strings, a message-passing neural network for 2D graphs, and an equivariant neural network for 3D conformers. The flexibility and modularity of this framework enable easy adaptation and replacement of these encoders, making the model highly versatile for different molecular tasks. The extracted tokens are then combined into a unified multimodal sequence, which is processed by a downstream transformer for prediction tasks. To efficiently scale our model for large multimodal datasets, we utilize Flash Attention 2 and bfloat16 precision. Despite its simplicity, our approach achieves state-of-the-art results across multiple datasets, demonstrating its effectiveness as a strong baseline for multimodal molecular representation learning.

Related papers

• BindGPT: A Scalable Framework for 3D Molecular Design via Language Modeling and Reinforcement Learning [11.862370962277938]
  We present a novel generative model, BindGPT, which uses a conceptually simple but powerful approach to create 3D molecules within the protein's binding site. We show how such simple conceptual approach combined with pretraining and scaling can perform on par or better than the current best specialized diffusion models.
  arXiv  Detail & Related papers  (2024-06-06T02:10:50Z)
• SE3Set: Harnessing equivariant hypergraph neural networks for molecular representation learning [27.713870291922333]
  We develop an SE(3) equivariant hypergraph neural network architecture tailored for advanced molecular representation learning. SE3Set has shown performance on par with state-of-the-art (SOTA) models for small molecule datasets. It excels on the MD22 dataset, achieving a notable improvement of approximately 20% in accuracy across all molecules.
  arXiv  Detail & Related papers  (2024-05-26T10:43:16Z)
• MolBind: Multimodal Alignment of Language, Molecules, and Proteins [16.98169256565552]
  MolBind is a framework that trains encoders for multiple modalities through contrastive learning. MolBind shows superior zero-shot learning performance across a wide range of tasks.
  arXiv  Detail & Related papers  (2024-03-13T01:38:42Z)
• Integrating Chemical Language and Molecular Graph in Multimodal Fused Deep Learning for Drug Property Prediction [9.388979080270103]
  We construct multimodal deep learning models to cover different molecular representations. Compared with mono-modal models, our multimodal fused deep learning (MMFDL) models outperform single models in accuracy, reliability, and resistance capability against noise.
  arXiv  Detail & Related papers  (2023-12-29T07:19:42Z)
• MUDiff: Unified Diffusion for Complete Molecule Generation [104.7021929437504]
  We present a new model for generating a comprehensive representation of molecules, including atom features, 2D discrete molecule structures, and 3D continuous molecule coordinates. We propose a novel graph transformer architecture to denoise the diffusion process. Our model is a promising approach for designing stable and diverse molecules and can be applied to a wide range of tasks in molecular modeling.
  arXiv  Detail & Related papers  (2023-04-28T04:25:57Z)
• One Transformer Can Understand Both 2D & 3D Molecular Data [94.93514673086631]
  We develop a novel Transformer-based Molecular model called Transformer-M. It can take molecular data of 2D or 3D formats as input and generate meaningful semantic representations. All empirical results show that Transformer-M can simultaneously achieve strong performance on 2D and 3D tasks.
  arXiv  Detail & Related papers  (2022-10-04T17:30:31Z)
• Multimodal Masked Autoencoders Learn Transferable Representations [127.35955819874063]
  We propose a simple and scalable network architecture, the Multimodal Masked Autoencoder (M3AE) M3AE learns a unified encoder for both vision and language data via masked token prediction. We provide an empirical study of M3AE trained on a large-scale image-text dataset, and find that M3AE is able to learn generalizable representations that transfer well to downstream tasks.
  arXiv  Detail & Related papers  (2022-05-27T19:09:42Z)
• Multimodal Token Fusion for Vision Transformers [54.81107795090239]
  We propose a multimodal token fusion method (TokenFusion) for transformer-based vision tasks. To effectively fuse multiple modalities, TokenFusion dynamically detects uninformative tokens and substitutes these tokens with projected and aggregated inter-modal features. The design of TokenFusion allows the transformer to learn correlations among multimodal features, while the single-modal transformer architecture remains largely intact.
  arXiv  Detail & Related papers  (2022-04-19T07:47:50Z)
• Dual-view Molecule Pre-training [186.07333992384287]
  Dual-view molecule pre-training can effectively combine the strengths of both types of molecule representations. DMP is tested on nine molecular property prediction tasks and achieves state-of-the-art performances on seven of them.
  arXiv  Detail & Related papers  (2021-06-17T03:58:38Z)
• A Novel Graph-based Multi-modal Fusion Encoder for Neural Machine Translation [131.33610549540043]
  We propose a novel graph-based multi-modal fusion encoder for NMT. We first represent the input sentence and image using a unified multi-modal graph. We then stack multiple graph-based multi-modal fusion layers that iteratively perform semantic interactions to learn node representations.
  arXiv  Detail & Related papers  (2020-07-17T04:06:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.