ReactionT5: a large-scale pre-trained model towards application of
limited reaction data
- URL: http://arxiv.org/abs/2311.06708v1
- Date: Sun, 12 Nov 2023 02:25:00 GMT
- Title: ReactionT5: a large-scale pre-trained model towards application of
limited reaction data
- Authors: Tatsuya Sagawa and Ryosuke Kojima
- Abstract summary: Transformer-based deep neural networks have revolutionized the field of molecular-related prediction tasks by treating molecules as symbolic sequences.
We propose ReactionT5, a novel model that leverages pretraining on the Open Reaction Database (ORD), a publicly available large-scale resource.
We further fine-tune this model for yield prediction and product prediction tasks, demonstrating its impressive performance even with limited fine-tuning data.
- Score: 4.206175795966693
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Transformer-based deep neural networks have revolutionized the field of
molecular-related prediction tasks by treating molecules as symbolic sequences.
These models have been successfully applied in various organic chemical
applications by pretraining them with extensive compound libraries and
subsequently fine-tuning them with smaller in-house datasets for specific
tasks. However, many conventional methods primarily focus on single molecules,
with limited exploration of pretraining for reactions involving multiple
molecules. In this paper, we propose ReactionT5, a novel model that leverages
pretraining on the Open Reaction Database (ORD), a publicly available
large-scale resource. We further fine-tune this model for yield prediction and
product prediction tasks, demonstrating its impressive performance even with
limited fine-tuning data compared to traditional models. The pre-trained
ReactionT5 model is publicly accessible on the Hugging Face platform.
Related papers
- RGFN: Synthesizable Molecular Generation Using GFlowNets [51.33672611338754]
We propose Reaction-GFlowNet, an extension of the GFlowNet framework that operates directly in the space of chemical reactions.
RGFN allows out-of-the-box synthesizability while maintaining comparable quality of generated candidates.
We demonstrate the effectiveness of the proposed approach across a range of oracle models, including pretrained proxy models and GPU-accelerated docking.
arXiv Detail & Related papers (2024-06-01T13:11:11Z) - Specialising and Analysing Instruction-Tuned and Byte-Level Language Models for Organic Reaction Prediction [0.0]
Transformer-based encoder-decoder models have demonstrated impressive results in chemical reaction prediction tasks.
These models typically rely on pretraining using tens of millions of unlabelled molecules.
Can FlanT5 and ByT5 be effectively specialised for organic reaction prediction through task-specific fine-tuning?
arXiv Detail & Related papers (2024-05-17T08:39:56Z) - UAlign: Pushing the Limit of Template-free Retrosynthesis Prediction with Unsupervised SMILES Alignment [51.49238426241974]
This paper introduces UAlign, a template-free graph-to-sequence pipeline for retrosynthesis prediction.
By combining graph neural networks and Transformers, our method can more effectively leverage the inherent graph structure of molecules.
arXiv Detail & Related papers (2024-03-25T03:23:03Z) - Contextual Molecule Representation Learning from Chemical Reaction
Knowledge [24.501564702095937]
We introduce REMO, a self-supervised learning framework that takes advantage of well-defined atom-combination rules in common chemistry.
REMO pre-trains graph/Transformer encoders on 1.7 million known chemical reactions in the literature.
arXiv Detail & Related papers (2024-02-21T12:58:40Z) - Retrosynthesis prediction enhanced by in-silico reaction data
augmentation [66.5643280109899]
We present RetroWISE, a framework that employs a base model inferred from real paired data to perform in-silico reaction generation and augmentation.
On three benchmark datasets, RetroWISE achieves the best overall performance against state-of-the-art models.
arXiv Detail & Related papers (2024-01-31T07:40:37Z) - Molecule-Edit Templates for Efficient and Accurate Retrosynthesis
Prediction [0.16070833439280313]
We introduce METRO, a machine-learning model that predicts reactions using minimal templates.
We achieve state-of-the-art results on standard benchmarks.
arXiv Detail & Related papers (2023-10-11T09:00:02Z) - Retrieval-based Controllable Molecule Generation [63.44583084888342]
We propose a new retrieval-based framework for controllable molecule generation.
We use a small set of molecules to steer the pre-trained generative model towards synthesizing molecules that satisfy the given design criteria.
Our approach is agnostic to the choice of generative models and requires no task-specific fine-tuning.
arXiv Detail & Related papers (2022-08-23T17:01:16Z) - Improving Molecular Representation Learning with Metric
Learning-enhanced Optimal Transport [49.237577649802034]
We develop a novel optimal transport-based algorithm termed MROT to enhance their generalization capability for molecular regression problems.
MROT significantly outperforms state-of-the-art models, showing promising potential in accelerating the discovery of new substances.
arXiv Detail & Related papers (2022-02-13T04:56:18Z) - Data Transfer Approaches to Improve Seq-to-Seq Retrosynthesis [1.6449390849183363]
Retrosynthesis is a problem to infer reactant compounds to synthesize a given product compound through chemical reactions.
Recent studies on retrosynthesis focus on proposing more sophisticated prediction models.
The dataset to feed the models also plays an essential role in achieving the best generalizing models.
arXiv Detail & Related papers (2020-10-02T05:27:51Z) - Learning Graph Models for Retrosynthesis Prediction [90.15523831087269]
Retrosynthesis prediction is a fundamental problem in organic synthesis.
This paper introduces a graph-based approach that capitalizes on the idea that the graph topology of precursor molecules is largely unaltered during a chemical reaction.
Our model achieves a top-1 accuracy of $53.7%$, outperforming previous template-free and semi-template-based methods.
arXiv Detail & Related papers (2020-06-12T09:40:42Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.