CLDR: Contrastive Learning Drug Response Models from Natural Language Supervision

• URL: http://arxiv.org/abs/2312.10707v1
• Date: Sun, 17 Dec 2023 12:51:49 GMT
• Title: CLDR: Contrastive Learning Drug Response Models from Natural Language Supervision
• Authors: Kun Li, Wenbin Hu
• Abstract summary: We propose MN, a contrastive learning framework with natural language supervision. We validated several hundred thousand samples from the Genomics of Drug Sensitivity in Cancer dataset. Experiments prove that the MNeffectively constrains the samples to a continuous distribution in the representation space.
• Score: 15.330792768046615
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep learning-based drug response prediction (DRP) methods can accelerate the drug discovery process and reduce R\&D costs. Although the mainstream methods achieve high accuracy in predicting response regression values, the regression-aware representations of these methods are fragmented and fail to capture the continuity of the sample order. This phenomenon leads to models optimized to sub-optimal solution spaces, reducing generalization ability and may result in significant wasted costs in the drug discovery phase. In this paper, we propose \MN, a contrastive learning framework with natural language supervision for the DRP. The \MN~converts regression labels into text, which is merged with the captions text of the drug response as a second modality of the samples compared to the traditional modalities (graph, sequence). In each batch, two modalities of one sample are considered positive pairs and the other pairs are considered negative pairs. At the same time, in order to enhance the continuous representation capability of the numerical text, a common-sense numerical knowledge graph is introduced. We validated several hundred thousand samples from the Genomics of Drug Sensitivity in Cancer dataset, observing the average improvement of the DRP method ranges from 7.8\% to 31.4\% with the application of our framework. The experiments prove that the \MN~effectively constrains the samples to a continuous distribution in the representation space, and achieves impressive prediction performance with only a few epochs of fine-tuning after pre-training. The code is available at: \url{https://gitee.com/xiaoyibang/clipdrug.git}.

Related papers

• ROTI-GCV: Generalized Cross-Validation for right-ROTationally Invariant Data [1.194799054956877]
  Two key tasks in high-dimensional regularized regression are tuning the regularization strength for accurate predictions and estimating the out-of-sample risk. We introduce a new framework, ROTI-GCV, for reliably performing cross-validation under challenging conditions.
  arXiv  Detail & Related papers  (2024-06-17T15:50:00Z)
• Flexible Heteroscedastic Count Regression with Deep Double Poisson Networks [4.58556584533865]
  We propose the Deep Double Poisson Network (DDPN) to produce accurate, input-conditional uncertainty representations. DDPN vastly outperforms existing discrete models. It can be applied to a variety of count regression datasets.
  arXiv  Detail & Related papers  (2024-06-13T16:02:03Z)
• Regressor-free Molecule Generation to Support Drug Response Prediction [83.25894107956735]
  Conditional generation based on the target IC50 score can obtain a more effective sampling space. Regressor-free guidance combines a diffusion model's score estimation with a regression controller model's gradient based on number labels.
  arXiv  Detail & Related papers  (2024-05-23T13:22:17Z)
• Embracing Uncertainty Flexibility: Harnessing a Supervised Tree Kernel to Empower Ensemble Modelling for 2D Echocardiography-Based Prediction of Right Ventricular Volume [0.5492530316344587]
  The right ventricular (RV) function deterioration strongly predicts clinical outcomes in numerous circumstances. We propose to complement the volume predictions with uncertainty scores. The proposed framework can be used to enhance the decision-making process and reduce risks.
  arXiv  Detail & Related papers  (2024-03-04T12:36:31Z)
• Boosting Differentiable Causal Discovery via Adaptive Sample Reweighting [62.23057729112182]
  Differentiable score-based causal discovery methods learn a directed acyclic graph from observational data. We propose a model-agnostic framework to boost causal discovery performance by dynamically learning the adaptive weights for the Reweighted Score function, ReScore.
  arXiv  Detail & Related papers  (2023-03-06T14:49:59Z)
• Tailoring Language Generation Models under Total Variation Distance [55.89964205594829]
  The standard paradigm of neural language generation adopts maximum likelihood estimation (MLE) as the optimizing method. We develop practical bounds to apply it to language generation. We introduce the TaiLr objective that balances the tradeoff of estimating TVD.
  arXiv  Detail & Related papers  (2023-02-26T16:32:52Z)
• MetaRF: Differentiable Random Forest for Reaction Yield Prediction with a Few Trails [58.47364143304643]
  In this paper, we focus on the reaction yield prediction problem. We first put forth MetaRF, an attention-based differentiable random forest model specially designed for the few-shot yield prediction. To improve the few-shot learning performance, we further introduce a dimension-reduction based sampling method.
  arXiv  Detail & Related papers  (2022-08-22T06:40:13Z)
• Saliency Grafting: Innocuous Attribution-Guided Mixup with Calibrated Label Mixing [104.630875328668]
  Mixup scheme suggests mixing a pair of samples to create an augmented training sample. We present a novel, yet simple Mixup-variant that captures the best of both worlds.
  arXiv  Detail & Related papers  (2021-12-16T11:27:48Z)
• SCEHR: Supervised Contrastive Learning for Clinical Risk Prediction using Electronic Health Records [24.35874264767865]
  We extend the supervised contrastive learning framework to clinical risk prediction problems based on longitudinal electronic health records (EHR) Our proposed loss functions show benefits in improving the performance of strong baselines and even state-of-the-art models on benchmarking tasks for clinical risk predictions. Our loss functions can be easily used to replace (binary or multi-label) cross-entropy loss adopted in existing clinical predictive models.
  arXiv  Detail & Related papers  (2021-10-11T00:32:17Z)
• Imputation-Free Learning from Incomplete Observations [73.15386629370111]
  We introduce the importance of guided gradient descent (IGSGD) method to train inference from inputs containing missing values without imputation. We employ reinforcement learning (RL) to adjust the gradients used to train the models via back-propagation. Our imputation-free predictions outperform the traditional two-step imputation-based predictions using state-of-the-art imputation methods.
  arXiv  Detail & Related papers  (2021-07-05T12:44:39Z)
• Stochastic Gradient Descent Meets Distribution Regression [0.0]
  gradient descent (SGD) provides a simple and efficient way to solve a broad range of machine learning problems. We focus on distribution regression (DR), involving two stages of sampling: Firstly, we regress from probability measures to real-valued responses.
  arXiv  Detail & Related papers  (2020-10-24T09:03:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.