From Theory to Therapy: Reframing SBDD Model Evaluation via Practical Metrics

• URL: http://arxiv.org/abs/2406.08980v1
• Date: Thu, 13 Jun 2024 10:23:52 GMT
• Title: From Theory to Therapy: Reframing SBDD Model Evaluation via Practical Metrics
• Authors: Bowen Gao, Haichuan Tan, Yanwen Huang, Minsi Ren, Xiao Huang, Wei-Ying Ma, Ya-Qin Zhang, Yanyan Lan,
• Abstract summary: The reliability of the Vina docking score is increasingly questioned due to its susceptibility to overfitting. We propose a comprehensive evaluation framework that includes assessing the similarity of generated molecules to known active compounds. Our proposed metrics and dataset aim to bridge this gap, enhancing the practical applicability of future SBDD models.
• Score: 21.78568415483299
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advancements in structure-based drug design (SBDD) have significantly enhanced the efficiency and precision of drug discovery by generating molecules tailored to bind specific protein pockets. Despite these technological strides, their practical application in real-world drug development remains challenging due to the complexities of synthesizing and testing these molecules. The reliability of the Vina docking score, the current standard for assessing binding abilities, is increasingly questioned due to its susceptibility to overfitting. To address these limitations, we propose a comprehensive evaluation framework that includes assessing the similarity of generated molecules to known active compounds, introducing a virtual screening-based metric for practical deployment capabilities, and re-evaluating binding affinity more rigorously. Our experiments reveal that while current SBDD models achieve high Vina scores, they fall short in practical usability metrics, highlighting a significant gap between theoretical predictions and real-world applicability. Our proposed metrics and dataset aim to bridge this gap, enhancing the practical applicability of future SBDD models and aligning them more closely with the needs of pharmaceutical research and development.

Related papers

• Causal Representation Learning from Multimodal Biological Observations [57.00712157758845]
  We aim to develop flexible identification conditions for multimodal data. We establish identifiability guarantees for each latent component, extending the subspace identification results from prior work. Our key theoretical ingredient is the structural sparsity of the causal connections among distinct modalities.
  arXiv  Detail & Related papers  (2024-11-10T16:40:27Z)
• Evaluation Framework for AI-driven Molecular Design of Multi-target Drugs: Brain Diseases as a Case Study [0.0]
  Multi-target Drug Discovery (MTDD) is an emerging paradigm for discovering drugs against complex disorders. This work proposes an evaluation framework for molecule generation techniques in MTDD scenarios.
  arXiv  Detail & Related papers  (2024-08-20T01:42:16Z)
• SPIN: SE(3)-Invariant Physics Informed Network for Binding Affinity Prediction [3.406882192023597]
  Accurate prediction of protein-ligand binding affinity is crucial for drug development. Traditional methods often fail to accurately model the complex's spatial information. We propose SPIN, a model that incorporates various inductive biases applicable to this task.
  arXiv  Detail & Related papers  (2024-07-10T08:40:07Z)
• TAGMol: Target-Aware Gradient-guided Molecule Generation [19.977071499171903]
  3D generative models have shown significant promise in structure-based drug design (SBDD) We decouple the problem into molecular generation and property prediction. The latter synergistically guides the diffusion sampling process, facilitating guided diffusion and resulting in the creation of meaningful molecules with the desired properties. We call this guided molecular generation process as TAGMol.
  arXiv  Detail & Related papers  (2024-06-03T14:43:54Z)
• Rethinking Specificity in SBDD: Leveraging Delta Score and Energy-Guided Diffusion [18.269722389716165]
  We introduce the Delta Score, a new metric for evaluating the specificity of molecular binding. We develop an innovative energy-guided approach using contrastive learning, with active compounds as decoys, to direct generative models toward creating molecules with high specificity.
  arXiv  Detail & Related papers  (2024-03-04T07:40:25Z)
• Delta Score: Improving the Binding Assessment of Structure-Based Drug Design Methods [14.272327734087598]
  We introduce the delta score, a novel evaluation metric grounded in tangible pharmaceutical requisites. Our experiments reveal that molecules produced by current deep generative models significantly lag behind ground reference truth when assessed with the delta score.
  arXiv  Detail & Related papers  (2023-11-01T08:37:39Z)
• Geometric Deep Learning for Structure-Based Drug Design: A Survey [83.87489798671155]
  Structure-based drug design (SBDD) leverages the three-dimensional geometry of proteins to identify potential drug candidates. Recent advancements in geometric deep learning, which effectively integrate and process 3D geometric data, have significantly propelled the field forward.
  arXiv  Detail & Related papers  (2023-06-20T14:21:58Z)
• Validation Diagnostics for SBI algorithms based on Normalizing Flows [55.41644538483948]
  This work proposes easy to interpret validation diagnostics for multi-dimensional conditional (posterior) density estimators based on NF. It also offers theoretical guarantees based on results of local consistency. This work should help the design of better specified models or drive the development of novel SBI-algorithms.
  arXiv  Detail & Related papers  (2022-11-17T15:48:06Z)
• SSM-DTA: Breaking the Barriers of Data Scarcity in Drug-Target Affinity Prediction [127.43571146741984]
  Drug-Target Affinity (DTA) is of vital importance in early-stage drug discovery. wet experiments remain the most reliable method, but they are time-consuming and resource-intensive. Existing methods have primarily focused on developing techniques based on the available DTA data, without adequately addressing the data scarcity issue. We present the SSM-DTA framework, which incorporates three simple yet highly effective strategies.
  arXiv  Detail & Related papers  (2022-06-20T14:53:25Z)
• Pathloss modeling for in-body optical wireless communications [65.33908037519238]
  Optical wireless communications (OWCs) have been recognized as a candidate enabler of next generation in-body nano-scale networks and implants. This paper focuses on presenting a general pathloss model for in-body OWCs.
  arXiv  Detail & Related papers  (2021-05-06T17:17:45Z)
• Deep Learning for Virtual Screening: Five Reasons to Use ROC Cost Functions [80.12620331438052]
  deep learning has become an important tool for rapid screening of billions of molecules in silico for potential hits containing desired chemical features. Despite its importance, substantial challenges persist in training these models, such as severe class imbalance, high decision thresholds, and lack of ground truth labels in some datasets. We argue in favor of directly optimizing the receiver operating characteristic (ROC) in such cases, due to its robustness to class imbalance.
  arXiv  Detail & Related papers  (2020-06-25T08:46:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.