MolNexTR: A Generalized Deep Learning Model for Molecular Image
Recognition
- URL: http://arxiv.org/abs/2403.03691v2
- Date: Fri, 8 Mar 2024 06:32:12 GMT
- Title: MolNexTR: A Generalized Deep Learning Model for Molecular Image
Recognition
- Authors: Yufan Chen, Ching Ting Leung, Yong Huang, Jianwei Sun, Hao Chen, Hanyu
Gao
- Abstract summary: MolNexTR is a novel image-to-graph model that collaborates to fuse the strengths of ConvNext and Vision-TRansformer.
It can predict atoms and bonds simultaneously and understand their layout rules.
MolNexTR has demonstrated superior performance, achieving an accuracy rate of 81-97%.
- Score: 4.7793786389946815
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: In the field of chemical structure recognition, the task of converting
molecular images into graph structures and SMILES string stands as a
significant challenge, primarily due to the varied drawing styles and
conventions prevalent in chemical literature. To bridge this gap, we proposed
MolNexTR, a novel image-to-graph deep learning model that collaborates to fuse
the strengths of ConvNext, a powerful Convolutional Neural Network variant, and
Vision-TRansformer. This integration facilitates a more nuanced extraction of
both local and global features from molecular images. MolNexTR can predict
atoms and bonds simultaneously and understand their layout rules. It also
excels at flexibly integrating symbolic chemistry principles to discern
chirality and decipher abbreviated structures. We further incorporate a series
of advanced algorithms, including improved data augmentation module, image
contamination module, and a post-processing module to get the final SMILES
output. These modules synergistically enhance the model's robustness against
the diverse styles of molecular imagery found in real literature. In our test
sets, MolNexTR has demonstrated superior performance, achieving an accuracy
rate of 81-97%, marking a significant advancement in the domain of molecular
structure recognition. Scientific contribution: MolNexTR is a novel
image-to-graph model that incorporates a unique dual-stream encoder to extract
complex molecular image features, and combines chemical rules to predict atoms
and bonds while understanding atom and bond layout rules. In addition, it
employs a series of novel augmentation algorithms to significantly enhance the
robustness and performance of the model.
Related papers
- Data-Efficient Molecular Generation with Hierarchical Textual Inversion [48.816943690420224]
We introduce Hierarchical textual Inversion for Molecular generation (HI-Mol), a novel data-efficient molecular generation method.
HI-Mol is inspired by the importance of hierarchical information, e.g., both coarse- and fine-grained features, in understanding the molecule distribution.
Compared to the conventional textual inversion method in the image domain using a single-level token embedding, our multi-level token embeddings allow the model to effectively learn the underlying low-shot molecule distribution.
arXiv Detail & Related papers (2024-05-05T08:35:23Z) - UAlign: Pushing the Limit of Template-free Retrosynthesis Prediction with Unsupervised SMILES Alignment [51.49238426241974]
This paper introduces UAlign, a template-free graph-to-sequence pipeline for retrosynthesis prediction.
By combining graph neural networks and Transformers, our method can more effectively leverage the inherent graph structure of molecules.
arXiv Detail & Related papers (2024-03-25T03:23:03Z) - MultiModal-Learning for Predicting Molecular Properties: A Framework Based on Image and Graph Structures [2.5563339057415218]
MolIG is a novel MultiModaL molecular pre-training framework for predicting molecular properties based on Image and Graph structures.
It amalgamates the strengths of both molecular representation forms.
It exhibits enhanced performance in downstream tasks pertaining to molecular property prediction within benchmark groups.
arXiv Detail & Related papers (2023-11-28T10:28:35Z) - Atomic and Subgraph-aware Bilateral Aggregation for Molecular
Representation Learning [57.670845619155195]
We introduce a new model for molecular representation learning called the Atomic and Subgraph-aware Bilateral Aggregation (ASBA)
ASBA addresses the limitations of previous atom-wise and subgraph-wise models by incorporating both types of information.
Our method offers a more comprehensive way to learn representations for molecular property prediction and has broad potential in drug and material discovery applications.
arXiv Detail & Related papers (2023-05-22T00:56:00Z) - t-SMILES: A Scalable Fragment-based Molecular Representation Framework for De Novo Molecule Generation [9.116670221263753]
This study introduces a flexible, fragment-based, multiscale molecular representation framework called t-SMILES.
It describes molecules using SMILES-type strings obtained by performing a breadth-first search on a full binary tree formed from a fragmented molecular graph.
It significantly outperforms classical SMILES, DeepSMILES, SELFIES and baseline models in goal-directed tasks.
arXiv Detail & Related papers (2023-01-04T21:41:01Z) - A Molecular Multimodal Foundation Model Associating Molecule Graphs with
Natural Language [63.60376252491507]
We propose a molecular multimodal foundation model which is pretrained from molecular graphs and their semantically related textual data.
We believe that our model would have a broad impact on AI-empowered fields across disciplines such as biology, chemistry, materials, environment, and medicine.
arXiv Detail & Related papers (2022-09-12T00:56:57Z) - MolScribe: Robust Molecular Structure Recognition with Image-To-Graph
Generation [28.93523736883784]
MolScribe is an image-to-graph model that explicitly predicts atoms and bonds, along with their geometric layouts, to construct the molecular structure.
MolScribe significantly outperforms previous models, achieving 76-93% accuracy on public benchmarks.
arXiv Detail & Related papers (2022-05-28T03:03:45Z) - Image-to-Graph Transformers for Chemical Structure Recognition [4.180435324231826]
We present a deep learning model to extract molecular structures from images.
The proposed model is designed to transform the molecular image directly into the corresponding graph.
By end-to-end learning approach, it can fully utilize many open image-molecule pair data from various sources.
arXiv Detail & Related papers (2022-02-19T11:33:54Z) - Learning Neural Generative Dynamics for Molecular Conformation
Generation [89.03173504444415]
We study how to generate molecule conformations (textiti.e., 3D structures) from a molecular graph.
We propose a novel probabilistic framework to generate valid and diverse conformations given a molecular graph.
arXiv Detail & Related papers (2021-02-20T03:17:58Z) - Improved Conditional Flow Models for Molecule to Image Synthesis [37.886816307827196]
Mol2Image is a flow-based generative model for molecule to cell image synthesis.
To generate cell features at different resolutions and scale to high-resolution images, we develop a novel multi-scale flow architecture.
To maximize the mutual information between the generated images and the molecular interventions, we devise a training strategy based on contrastive learning.
arXiv Detail & Related papers (2020-06-15T16:39:50Z) - Multi-View Graph Neural Networks for Molecular Property Prediction [67.54644592806876]
We present Multi-View Graph Neural Network (MV-GNN), a multi-view message passing architecture.
In MV-GNN, we introduce a shared self-attentive readout component and disagreement loss to stabilize the training process.
We further boost the expressive power of MV-GNN by proposing a cross-dependent message passing scheme.
arXiv Detail & Related papers (2020-05-17T04:46:07Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.