ISAAC: Auditing Causal Reasoning in Deep Models for Drug-Target Interaction

• URL: http://arxiv.org/abs/2605.02962v1
• Date: Sun, 03 May 2026 06:36:40 GMT
• Title: ISAAC: Auditing Causal Reasoning in Deep Models for Drug-Target Interaction
• Abstract summary: We introduce ISAAC, a post-hoc framework that evaluates prior-relative structural sensitivity by probing frozen models.<n>ISAAC reveals approximately 25% relative differences in reasoning scores across models with comparable AUROC.<n>These discrepancies, undetectable under conventional accuracy metrics, motivate the use of post-hoc structural auditing.
• Score: 11.805780759216134
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep learning models for drug--target interaction (DTI) prediction often achieve strong benchmark performance without necessarily relying on mechanistically meaningful molecular features, a limitation that standard accuracy-based evaluation cannot detect. We introduce ISAAC (Intervention-based Structural Auditing Approach for Causal Reasoning), a post-hoc framework that evaluates prior-relative structural sensitivity by probing frozen models through matched mechanistic and spurious input-level interventions, independently of predictive accuracy. Applied to three sequence-based DTI architectures on the Davis benchmark, ISAAC reveals approximately 25\% relative differences in reasoning scores across models with comparable AUROC (within around 3\%), stable across training and intervention seeds and two distinct perturbation operators. These discrepancies, undetectable under conventional accuracy metrics, motivate the use of post-hoc structural auditing as a complement to standard performance evaluation in scientific machine learning for molecular modeling.

Related papers

• I-SAFE: Wasserstein Coherence Metrics for Structural Auditing of Scientific AI Models [5.562984399879219]
  We introduce the textscI-SAFE (Interventional Secure, Accurate, Fair and Explainable) framework for scientific AI models.<n>textscI-SAFE evaluates raw model outputs under structurally guided perturbations of the input.<n>We instantiate textscI-SAFE on drug--target interaction (DTI) prediction using the Davis kinase benchmark.
  arXiv  Detail & Related papers  (2026-05-20T20:44:30Z)
• K-Models: a Flexible and Interpretable Method for Ordinal Clustering with Application to Antigen-Antibody Interaction Profiles [65.21921642886367]
  K-Models is a novel framework that integrates ordinal constraints and estimates key underlying elements of the random process generating the observed functional profiles.<n>The proposed method is evaluated through simulations and real-world applications.
  arXiv  Detail & Related papers  (2026-05-14T13:35:44Z)
• Evaluating Uplift Modeling under Structural Biases: Insights into Metric Stability and Model Robustness [8.135022024189306]
  In personalized marketing, uplift models estimate incremental effects by modeling how customer behavior changes under alternative treatments.<n>Real-world data often exhibit biases, such as selection bias, spillover effects, and unobserved confounding.
  arXiv  Detail & Related papers  (2026-03-21T11:54:25Z)
• STAR : Bridging Statistical and Agentic Reasoning for Large Model Performance Prediction [78.0692157478247]
  We propose STAR, a framework that bridges data-driven STatistical expectations with knowledge-driven Agentic Reasoning.<n>We show that STAR consistently outperforms all baselines on both score-based and rank-based metrics.
  arXiv  Detail & Related papers  (2026-02-12T16:30:07Z)
• EvalQReason: A Framework for Step-Level Reasoning Evaluation in Large Language Models [0.8399688944263844]
  We present EvalQReason, a framework that quantifies LLM reasoning quality through step-level probability distribution analysis.<n>The framework introduces two complementary algorithms: Consecutive Step Divergence (CSD), which measures local coherence between adjacent reasoning steps, and Step-to-Final Convergence (SFC), which assesses global alignment with final answers.
  arXiv  Detail & Related papers  (2026-02-02T16:32:40Z)
• Analysis of Transferability Estimation Metrics for Surgical Phase Recognition [3.3285108719932555]
  Fine-tuning pre-trained models has become a cornerstone of modern machine learning, allowing practitioners to achieve high performance with limited labeled data.<n>In surgical video analysis, where expert annotations are especially time-consuming and costly, identifying the most suitable pre-trained model for a downstream task is both critical and challenging.<n>We provide the first comprehensive benchmark of three representative metrics, LogME, H-Score, and TransRate, on two diverse datasets.
  arXiv  Detail & Related papers  (2025-08-22T18:05:33Z)
• DISPROTBENCH: A Disorder-Aware, Task-Rich Benchmark for Evaluating Protein Structure Prediction in Realistic Biological Contexts [76.59606029593085]
  DisProtBench is a benchmark for evaluating protein structure prediction models (PSPMs) under structural disorder and complex biological conditions.<n>DisProtBench spans three key axes: data complexity, task diversity, and Interpretability.<n>Results reveal significant variability in model robustness under disorder, with low-confidence regions linked to functional prediction failures.
  arXiv  Detail & Related papers  (2025-06-18T23:58:22Z)
• CaliciBoost: Performance-Driven Evaluation of Molecular Representations for Caco-2 Permeability Prediction [2.6621416538504827]
  Caco-2 permeability serves as a critical in vitro indicator for predicting the oral absorption of drug candidates during early-stage drug discovery.<n>We investigated the impact of eight molecular feature representation types including 2D/3D descriptors, structural fingerprints, and deep learning-based embeddings.<n>For both PaDEL and Mordred representations, the incorporation of 3D descriptors resulted in a 15.73% reduction in MAE compared to using 2D features alone.
  arXiv  Detail & Related papers  (2025-06-09T07:30:28Z)
• Rethinking Robustness in Machine Learning: A Posterior Agreement Approach [41.50777631705435]
  Posterior Agreement (PA) theory of model validation provides a principled framework for robustness evaluation.<n>We show that PA offers a reliable analysis of the vulnerabilities in learning algorithms across different shift conditions.<n>Results show that PA provides higher discriminability than accuracy-based measures, while requiring no supervision.
  arXiv  Detail & Related papers  (2025-03-20T16:03:39Z)
• Language Models as Causal Effect Generators [48.696932388555894]
  We present sequence-driven structural causal models (SD-SCMs)<n>An SD-SCM enables sampling from observational, interventional, and counterfactual distributions according to the desired causal structure.<n>We propose a new type of benchmark for causal inference methods, generating individual-level counterfactual data to test treatment effect estimation.
  arXiv  Detail & Related papers  (2024-11-12T18:50:35Z)
• Scalable Intervention Target Estimation in Linear Models [52.60799340056917]
  Current approaches to causal structure learning either work with known intervention targets or use hypothesis testing to discover the unknown intervention targets. This paper proposes a scalable and efficient algorithm that consistently identifies all intervention targets. The proposed algorithm can be used to also update a given observational Markov equivalence class into the interventional Markov equivalence class.
  arXiv  Detail & Related papers  (2021-11-15T03:16:56Z)
• Machine learning for causal inference: on the use of cross-fit estimators [77.34726150561087]
  Doubly-robust cross-fit estimators have been proposed to yield better statistical properties. We conducted a simulation study to assess the performance of several estimators for the average causal effect (ACE) When used with machine learning, the doubly-robust cross-fit estimators substantially outperformed all of the other estimators in terms of bias, variance, and confidence interval coverage.
  arXiv  Detail & Related papers  (2020-04-21T23:09:55Z)
• Estimating the Effects of Continuous-valued Interventions using Generative Adversarial Networks [103.14809802212535]
  We build on the generative adversarial networks (GANs) framework to address the problem of estimating the effect of continuous-valued interventions. Our model, SCIGAN, is flexible and capable of simultaneously estimating counterfactual outcomes for several different continuous interventions. To address the challenges presented by shifting to continuous interventions, we propose a novel architecture for our discriminator.
  arXiv  Detail & Related papers  (2020-02-27T18:46:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.