From Theory to Therapy: Reframing SBDD Model Evaluation via Practical Metrics
- URL: http://arxiv.org/abs/2406.08980v1
- Date: Thu, 13 Jun 2024 10:23:52 GMT
- Title: From Theory to Therapy: Reframing SBDD Model Evaluation via Practical Metrics
- Authors: Bowen Gao, Haichuan Tan, Yanwen Huang, Minsi Ren, Xiao Huang, Wei-Ying Ma, Ya-Qin Zhang, Yanyan Lan,
- Abstract summary: The reliability of the Vina docking score is increasingly questioned due to its susceptibility to overfitting.
We propose a comprehensive evaluation framework that includes assessing the similarity of generated molecules to known active compounds.
Our proposed metrics and dataset aim to bridge this gap, enhancing the practical applicability of future SBDD models.
- Score: 21.78568415483299
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Recent advancements in structure-based drug design (SBDD) have significantly enhanced the efficiency and precision of drug discovery by generating molecules tailored to bind specific protein pockets. Despite these technological strides, their practical application in real-world drug development remains challenging due to the complexities of synthesizing and testing these molecules. The reliability of the Vina docking score, the current standard for assessing binding abilities, is increasingly questioned due to its susceptibility to overfitting. To address these limitations, we propose a comprehensive evaluation framework that includes assessing the similarity of generated molecules to known active compounds, introducing a virtual screening-based metric for practical deployment capabilities, and re-evaluating binding affinity more rigorously. Our experiments reveal that while current SBDD models achieve high Vina scores, they fall short in practical usability metrics, highlighting a significant gap between theoretical predictions and real-world applicability. Our proposed metrics and dataset aim to bridge this gap, enhancing the practical applicability of future SBDD models and aligning them more closely with the needs of pharmaceutical research and development.
Related papers
- SPIN: SE(3)-Invariant Physics Informed Network for Binding Affinity Prediction [3.406882192023597]
Accurate prediction of protein-ligand binding affinity is crucial for drug development.
Traditional methods often fail to accurately model the complex's spatial information.
We propose SPIN, a model that incorporates various inductive biases applicable to this task.
arXiv Detail & Related papers (2024-07-10T08:40:07Z) - TAGMol: Target-Aware Gradient-guided Molecule Generation [19.977071499171903]
3D generative models have shown significant promise in structure-based drug design (SBDD)
We decouple the problem into molecular generation and property prediction.
The latter synergistically guides the diffusion sampling process, facilitating guided diffusion and resulting in the creation of meaningful molecules with the desired properties.
We call this guided molecular generation process as TAGMol.
arXiv Detail & Related papers (2024-06-03T14:43:54Z) - Rethinking Specificity in SBDD: Leveraging Delta Score and Energy-Guided Diffusion [18.269722389716165]
We introduce the Delta Score, a new metric for evaluating the specificity of molecular binding.
We develop an innovative energy-guided approach using contrastive learning, with active compounds as decoys, to direct generative models toward creating molecules with high specificity.
arXiv Detail & Related papers (2024-03-04T07:40:25Z) - Delta Score: Improving the Binding Assessment of Structure-Based Drug
Design Methods [14.272327734087598]
We introduce the delta score, a novel evaluation metric grounded in tangible pharmaceutical requisites.
Our experiments reveal that molecules produced by current deep generative models significantly lag behind ground reference truth when assessed with the delta score.
arXiv Detail & Related papers (2023-11-01T08:37:39Z) - PGraphDTA: Improving Drug Target Interaction Prediction using Protein
Language Models and Contact Maps [4.590060921188914]
Key aspect of drug discovery involves identifying novel drug-target (DT) interactions.
Protein-ligand interactions exhibit a continuum of binding strengths, known as binding affinity.
We propose novel enhancements to enhance their performance.
arXiv Detail & Related papers (2023-10-06T05:00:25Z) - Validation Diagnostics for SBI algorithms based on Normalizing Flows [55.41644538483948]
This work proposes easy to interpret validation diagnostics for multi-dimensional conditional (posterior) density estimators based on NF.
It also offers theoretical guarantees based on results of local consistency.
This work should help the design of better specified models or drive the development of novel SBI-algorithms.
arXiv Detail & Related papers (2022-11-17T15:48:06Z) - Differentiable Agent-based Epidemiology [71.81552021144589]
We introduce GradABM: a scalable, differentiable design for agent-based modeling that is amenable to gradient-based learning with automatic differentiation.
GradABM can quickly simulate million-size populations in few seconds on commodity hardware, integrate with deep neural networks and ingest heterogeneous data sources.
arXiv Detail & Related papers (2022-07-20T07:32:02Z) - SSM-DTA: Breaking the Barriers of Data Scarcity in Drug-Target Affinity
Prediction [127.43571146741984]
Drug-Target Affinity (DTA) is of vital importance in early-stage drug discovery.
wet experiments remain the most reliable method, but they are time-consuming and resource-intensive.
Existing methods have primarily focused on developing techniques based on the available DTA data, without adequately addressing the data scarcity issue.
We present the SSM-DTA framework, which incorporates three simple yet highly effective strategies.
arXiv Detail & Related papers (2022-06-20T14:53:25Z) - Pathloss modeling for in-body optical wireless communications [65.33908037519238]
Optical wireless communications (OWCs) have been recognized as a candidate enabler of next generation in-body nano-scale networks and implants.
This paper focuses on presenting a general pathloss model for in-body OWCs.
arXiv Detail & Related papers (2021-05-06T17:17:45Z) - Deep Learning for Virtual Screening: Five Reasons to Use ROC Cost
Functions [80.12620331438052]
deep learning has become an important tool for rapid screening of billions of molecules in silico for potential hits containing desired chemical features.
Despite its importance, substantial challenges persist in training these models, such as severe class imbalance, high decision thresholds, and lack of ground truth labels in some datasets.
We argue in favor of directly optimizing the receiver operating characteristic (ROC) in such cases, due to its robustness to class imbalance.
arXiv Detail & Related papers (2020-06-25T08:46:37Z) - Learning To Navigate The Synthetically Accessible Chemical Space Using
Reinforcement Learning [75.95376096628135]
We propose a novel forward synthesis framework powered by reinforcement learning (RL) for de novo drug design.
In this setup, the agent learns to navigate through the immense synthetically accessible chemical space.
We describe how the end-to-end training in this study represents an important paradigm in radically expanding the synthesizable chemical space.
arXiv Detail & Related papers (2020-04-26T21:40:03Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.