Substrate Scope Contrastive Learning: Repurposing Human Bias to Learn Atomic Representations

• URL: http://arxiv.org/abs/2402.16882v1
• Date: Mon, 19 Feb 2024 02:21:20 GMT
• Title: Substrate Scope Contrastive Learning: Repurposing Human Bias to Learn Atomic Representations
• Authors: Wenhao Gao, Priyanka Raghavan, Ron Shprints, Connor W. Coley
• Abstract summary: We introduce a novel pre-training strategy, substrate scope contrastive learning, which learns atomic representations tailored to chemical reactivity. We focus on 20,798 aryl halides in the CAS Content Collection spanning thousands of publications to learn a representation of aryl halide reactivity. This work not only presents a chemistry-tailored neural network pre-training strategy to learn reactivity-aligned atomic representations, but also marks a first-of-its-kind approach to benefit from the human bias in substrate scope design.
• Score: 14.528429119352328
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Learning molecular representation is a critical step in molecular machine learning that significantly influences modeling success, particularly in data-scarce situations. The concept of broadly pre-training neural networks has advanced fields such as computer vision, natural language processing, and protein engineering. However, similar approaches for small organic molecules have not achieved comparable success. In this work, we introduce a novel pre-training strategy, substrate scope contrastive learning, which learns atomic representations tailored to chemical reactivity. This method considers the grouping of substrates and their yields in published substrate scope tables as a measure of their similarity or dissimilarity in terms of chemical reactivity. We focus on 20,798 aryl halides in the CAS Content Collection spanning thousands of publications to learn a representation of aryl halide reactivity. We validate our pre-training approach through both intuitive visualizations and comparisons to traditional reactivity descriptors and physical organic chemistry principles. The versatility of these embeddings is further evidenced in their application to yield prediction, regioselectivity prediction, and the diverse selection of new substrates. This work not only presents a chemistry-tailored neural network pre-training strategy to learn reactivity-aligned atomic representations, but also marks a first-of-its-kind approach to benefit from the human bias in substrate scope design.

Related papers

• Challenging reaction prediction models to generalize to novel chemistry [12.33727805025678]
  We report a series of evaluations of a prototypical SMILES-based deep learning model. First, we illustrate how performance on randomly sampled datasets is overly optimistic compared to performance when generalizing to new patents or new authors. Second, we conduct time splits that evaluate how models perform when tested on reactions published in years after those in their training set, mimicking real-world deployment.
  arXiv  Detail & Related papers  (2025-01-11T23:49:14Z)
• Learning Chemical Reaction Representation with Reactant-Product Alignment [50.28123475356234]
  This paper introduces modelname, a novel chemical reaction representation learning model tailored for a variety of organic-reaction-related tasks. By integrating atomic correspondence between reactants and products, our model discerns the molecular transformations that occur during the reaction, thereby enhancing the comprehension of the reaction mechanism. We have designed an adapter structure to incorporate reaction conditions into the chemical reaction representation, allowing the model to handle diverse reaction conditions and adapt to various datasets and downstream tasks, e.g., reaction performance prediction.
  arXiv  Detail & Related papers  (2024-11-26T17:41:44Z)
• Dynamic Post-Hoc Neural Ensemblers [55.15643209328513]
  In this study, we explore employing neural networks as ensemble methods. Motivated by the risk of learning low-diversity ensembles, we propose regularizing the model by randomly dropping base model predictions. We demonstrate this approach lower bounds the diversity within the ensemble, reducing overfitting and improving generalization capabilities.
  arXiv  Detail & Related papers  (2024-10-06T15:25:39Z)
• NeuralCRNs: A Natural Implementation of Learning in Chemical Reaction Networks [0.0]
  We present a novel supervised learning framework constructed as a collection of deterministic chemical reaction networks (CRNs) Unlike prior works, the NeuralCRNs framework is founded on dynamical system-based learning implementations and, thus, results in chemically compatible computations.
  arXiv  Detail & Related papers  (2024-08-18T01:43:26Z)
• Active Causal Learning for Decoding Chemical Complexities with Targeted Interventions [0.0]
  We introduce an active learning approach that discerns underlying cause-effect relationships through strategic sampling. This method identifies the smallest subset of the dataset capable of encoding the most information representative of a much larger chemical space. The identified causal relations are then leveraged to conduct systematic interventions, optimizing the design task within a chemical space that the models have not encountered previously.
  arXiv  Detail & Related papers  (2024-04-05T17:15:48Z)
• UAlign: Pushing the Limit of Template-free Retrosynthesis Prediction with Unsupervised SMILES Alignment [51.49238426241974]
  This paper introduces UAlign, a template-free graph-to-sequence pipeline for retrosynthesis prediction. By combining graph neural networks and Transformers, our method can more effectively leverage the inherent graph structure of molecules.
  arXiv  Detail & Related papers  (2024-03-25T03:23:03Z)
• Seeing Unseen: Discover Novel Biomedical Concepts via Geometry-Constrained Probabilistic Modeling [53.7117640028211]
  We present a geometry-constrained probabilistic modeling treatment to resolve the identified issues. We incorporate a suite of critical geometric properties to impose proper constraints on the layout of constructed embedding space. A spectral graph-theoretic method is devised to estimate the number of potential novel classes.
  arXiv  Detail & Related papers  (2024-03-02T00:56:05Z)
• Contextual Molecule Representation Learning from Chemical Reaction Knowledge [24.501564702095937]
  We introduce REMO, a self-supervised learning framework that takes advantage of well-defined atom-combination rules in common chemistry. REMO pre-trains graph/Transformer encoders on 1.7 million known chemical reactions in the literature.
  arXiv  Detail & Related papers  (2024-02-21T12:58:40Z)
• A large dataset curation and benchmark for drug target interaction [0.7699646945563469]
  Bioactivity data plays a key role in drug discovery and repurposing. We propose a way to standardize and represent efficiently a very large dataset curated from multiple public sources.
  arXiv  Detail & Related papers  (2024-01-30T17:06:25Z)
• Data Attribution for Diffusion Models: Timestep-induced Bias in Influence Estimation [53.27596811146316]
  Diffusion models operate over a sequence of timesteps instead of instantaneous input-output relationships in previous contexts. We present Diffusion-TracIn that incorporates this temporal dynamics and observe that samples' loss gradient norms are highly dependent on timestep. We introduce Diffusion-ReTrac as a re-normalized adaptation that enables the retrieval of training samples more targeted to the test sample of interest.
  arXiv  Detail & Related papers  (2024-01-17T07:58:18Z)
• Atomic and Subgraph-aware Bilateral Aggregation for Molecular Representation Learning [57.670845619155195]
  We introduce a new model for molecular representation learning called the Atomic and Subgraph-aware Bilateral Aggregation (ASBA) ASBA addresses the limitations of previous atom-wise and subgraph-wise models by incorporating both types of information. Our method offers a more comprehensive way to learn representations for molecular property prediction and has broad potential in drug and material discovery applications.
  arXiv  Detail & Related papers  (2023-05-22T00:56:00Z)
• Discovery of structure-property relations for molecules via hypothesis-driven active learning over the chemical space [0.0]
  We introduce a novel approach for the active learning over the chemical spaces based on hypothesis learning. We construct the hypotheses on the possible relationships between structures and functionalities of interest based on a small subset of data. This approach combines the elements from the symbolic regression methods such as SISSO and active learning into a single framework.
  arXiv  Detail & Related papers  (2023-01-06T14:22:43Z)
• Feature-Level Debiased Natural Language Understanding [86.8751772146264]
  Existing natural language understanding (NLU) models often rely on dataset biases to achieve high performance on specific datasets. We propose debiasing contrastive learning (DCT) to mitigate biased latent features and neglect the dynamic nature of bias. DCT outperforms state-of-the-art baselines on out-of-distribution datasets while maintaining in-distribution performance.
  arXiv  Detail & Related papers  (2022-12-11T06:16:14Z)
• MetaRF: Differentiable Random Forest for Reaction Yield Prediction with a Few Trails [58.47364143304643]
  In this paper, we focus on the reaction yield prediction problem. We first put forth MetaRF, an attention-based differentiable random forest model specially designed for the few-shot yield prediction. To improve the few-shot learning performance, we further introduce a dimension-reduction based sampling method.
  arXiv  Detail & Related papers  (2022-08-22T06:40:13Z)
• Tyger: Task-Type-Generic Active Learning for Molecular Property Prediction [121.97742787439546]
  How to accurately predict the properties of molecules is an essential problem in AI-driven drug discovery. To reduce annotation cost, deep Active Learning methods are developed to select only the most representative and informative data for annotating. We propose a Task-type-generic active learning framework (termed Tyger) that is able to handle different types of learning tasks in a unified manner.
  arXiv  Detail & Related papers  (2022-05-23T12:56:12Z)
• Selection of pseudo-annotated data for adverse drug reaction classification across drug groups [12.259552039796027]
  We assess the robustness of state-of-the-art neural architectures across different drug groups. We investigate several strategies to use pseudo-labeled data in addition to a manually annotated train set.
  arXiv  Detail & Related papers  (2021-11-24T13:11:05Z)
• Machine learning modeling of family wide enzyme-substrate specificity screens [2.276367922551686]
  Biocatalysis is a promising approach to synthesize pharmaceuticals, complex natural products, and commodity chemicals at scale. The adoption of biocatalysis is limited by our ability to select enzymes that will catalyze their natural chemical transformation on non-natural substrates.
  arXiv  Detail & Related papers  (2021-09-08T19:44:42Z)
• RetCL: A Selection-based Approach for Retrosynthesis via Contrastive Learning [107.64562550844146]
  Retrosynthesis is an emerging research area of deep learning. We propose a new approach that reformulating retrosynthesis into a selection problem of reactants from a candidate set of commercially available molecules. For learning the score functions, we also propose a novel contrastive training scheme with hard negative mining.
  arXiv  Detail & Related papers  (2021-05-03T12:47:57Z)
• Chemical Property Prediction Under Experimental Biases [26.407895054724452]
  This study focuses on mitigating bias in the experimental datasets. We adopted two techniques from causal inference combined with graph neural networks that can represent molecular structures. The experimental results in four possible bias scenarios indicated that the inverse propensity scoring-based method and the counter-factual regression-based method made solid improvements.
  arXiv  Detail & Related papers  (2020-09-18T08:40:57Z)
• Energy-based View of Retrosynthesis [70.66156081030766]
  We propose a framework that unifies sequence- and graph-based methods as energy-based models. We present a novel dual variant within the framework that performs consistent training over Bayesian forward- and backward-prediction. This model improves state-of-the-art performance by 9.6% for template-free approaches where the reaction type is unknown.
  arXiv  Detail & Related papers  (2020-07-14T18:51:06Z)
• Graph Neural Networks for the Prediction of Substrate-Specific Organic Reaction Conditions [79.45090959869124]
  We present a systematic investigation using graph neural networks (GNNs) to model organic chemical reactions. We evaluate seven different GNN architectures for classification tasks pertaining to the identification of experimental reagents and conditions.
  arXiv  Detail & Related papers  (2020-07-08T17:21:00Z)
• Multi-View Self-Attention for Interpretable Drug-Target Interaction Prediction [4.307720252429733]
  In machine learning approaches, the numerical representation of molecules is critical to the performance of the model. We propose a self-attention-based multi-view representation learning approach for modeling drug-target interactions.
  arXiv  Detail & Related papers  (2020-05-01T14:28:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.