I-SAFE: Wasserstein Coherence Metrics for Structural Auditing of Scientific AI Models

• URL: http://arxiv.org/abs/2605.21731v1
• Date: Wed, 20 May 2026 20:44:30 GMT
• Title: I-SAFE: Wasserstein Coherence Metrics for Structural Auditing of Scientific AI Models
• Abstract summary: We introduce the textscI-SAFE (Interventional Secure, Accurate, Fair and Explainable) framework for scientific AI models.<n>textscI-SAFE evaluates raw model outputs under structurally guided perturbations of the input.<n>We instantiate textscI-SAFE on drug--target interaction (DTI) prediction using the Davis kinase benchmark.
• Score: 5.562984399879219
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep learning models are increasingly used in scientific prediction tasks where strong benchmark performance is often interpreted as evidence of scientifically meaningful behavior. This interpretation is fragile, as models may exploit shortcut features, dataset-specific regularities, or distributional biases that are predictive on held-out data but not aligned with domain-relevant structure. To address this limitation, we introduce the \textsc{I-SAFE} (Interventional Secure, Accurate, Fair and Explainable) framework, a post-hoc distributional auditing framework for scientific AI models centered on the Wasserstein Coherence Metric (WCM). Given a trained black-box predictor and an external structural prior encoding domain knowledge about task-relevant input structure, \textsc{I-SAFE} evaluates raw model outputs under structurally guided perturbations of the input. The proposed audit measures output-distribution coherence through three complementary metrics: a Quantile-Based Metric (QBM) for location-level coherence, the WCM for ordinal coherence, and a translation-invariant WCM variant for shape coherence. We instantiate \textsc{I-SAFE} on drug--target interaction (DTI) prediction using the Davis kinase benchmark, KLIFS (Kinase--Ligand Interaction Fingerprints and Structures) binding-pocket annotations, and three sequence-based DTI models: DeepConvDTI, DeepDTA, and TAPB. Although the models operate in a comparable predictive regime, \textsc{I-SAFE} reveals substantially different distributional response profiles, a distinction invisible to accuracy-based evaluation. The framework is model-agnostic and applicable to any domain where inputs admit a structured decomposition and an external prior is available.

Related papers

• APO: Unsupervised Atomic Policy Optimization for 3D Structure Prediction of Atomic Systems [64.56252173115246]
  We propose (Atomic Policy Optimization) a fully unsupervised alignment framework that eliminates the need for ground-truth reference structures.<n>APO adapts group-relative policy optimization to 3D atomic environments, utilizing a novel dual-reward mechanism.<n>Our results suggest that intrinsic physical consistency can serve as a superior guide for alignment compared to noisy, supervised coordinate matching.
  arXiv  Detail & Related papers  (2026-07-30T17:21:58Z)
• ISAAC: Auditing Causal Reasoning in Deep Models for Drug-Target Interaction [11.805780759216134]
  We introduce ISAAC, a post-hoc framework that evaluates prior-relative structural sensitivity by probing frozen models.<n>ISAAC reveals approximately 25% relative differences in reasoning scores across models with comparable AUROC.<n>These discrepancies, undetectable under conventional accuracy metrics, motivate the use of post-hoc structural auditing.
  arXiv  Detail & Related papers  (2026-05-03T06:36:40Z)
• Trust in One Round: Confidence Estimation for Large Language Models via Structural Signals [13.89434979851652]
  Large language models (LLMs) are increasingly deployed in domains where errors carry high social, scientific, or safety costs.<n>We present Structural Confidence, a single-pass, model-agnostic framework that enhances output correctness prediction.
  arXiv  Detail & Related papers  (2026-02-01T02:35:59Z)
• Table-BiEval: A Self-Supervised, Dual-Track Framework for Decoupling Structure and Content in LLM Evaluation [11.450834626205676]
  Table-BiEval is a novel approach based on a human-free, self-supervised evaluation framework.<n>It calculates Content Semantic Accuracy and Normalized Tree Edit Distance to decouple structure from content.<n>Results reveal substantial variability, highlighting that mid-sized models can surprisingly outperform larger counterparts in structural efficiency.
  arXiv  Detail & Related papers  (2026-01-09T07:38:27Z)
• Structural Equation-VAE: Disentangled Latent Representations for Tabular Data [4.101599614979332]
  We introduce SE-VAE (Structural Equation-Variational Autoencoder), a novel architecture that embeds measurement structure directly into the design of a variational autoencoder.<n>Inspired by structural equation modeling, SE-VAE aligns latent subspaces with known indicator groupings and introduces a global nuisance latent to isolate construct-specific confounding variation.<n> SE-VAE consistently outperforms alternatives in factor recovery, interpretability, and robustness to nuisance variation.
  arXiv  Detail & Related papers  (2025-08-08T14:21:20Z)
• SEED: A Structural Encoder for Embedding-Driven Decoding in Time Series Prediction with LLMs [3.036179638516407]
  We introduce SEED, a structural encoder for embedding-driven decoding, which integrates four stages: a token-aware encoder for patch extraction, a projection module that aligns patches with language model embeddings, and a semantic reprogramming mechanism that maps patches to task-aware prototypes.<n>This modular architecture decouples representation learning from inference, enabling efficient alignment between numerical patterns and semantic reasoning.
  arXiv  Detail & Related papers  (2025-06-25T06:40:14Z)
• Learning Identifiable Structures Helps Avoid Bias in DNN-based Supervised Causal Learning [56.22841701016295]
  Supervised Causal Learning (SCL) is an emerging paradigm in this field.<n>Existing Deep Neural Network (DNN)-based methods commonly adopt the "Node-Edge approach"
  arXiv  Detail & Related papers  (2025-02-15T19:10:35Z)
• GenBench: A Benchmarking Suite for Systematic Evaluation of Genomic Foundation Models [56.63218531256961]
  We introduce GenBench, a benchmarking suite specifically tailored for evaluating the efficacy of Genomic Foundation Models. GenBench offers a modular and expandable framework that encapsulates a variety of state-of-the-art methodologies. We provide a nuanced analysis of the interplay between model architecture and dataset characteristics on task-specific performance.
  arXiv  Detail & Related papers  (2024-06-01T08:01:05Z)
• Coherent Entity Disambiguation via Modeling Topic and Categorical Dependency [87.16283281290053]
  Previous entity disambiguation (ED) methods adopt a discriminative paradigm, where prediction is made based on matching scores between mention context and candidate entities. We propose CoherentED, an ED system equipped with novel designs aimed at enhancing the coherence of entity predictions. We achieve new state-of-the-art results on popular ED benchmarks, with an average improvement of 1.3 F1 points.
  arXiv  Detail & Related papers  (2023-11-06T16:40:13Z)
• Prototype-based Aleatoric Uncertainty Quantification for Cross-modal Retrieval [139.21955930418815]
  Cross-modal Retrieval methods build similarity relations between vision and language modalities by jointly learning a common representation space. However, the predictions are often unreliable due to the Aleatoric uncertainty, which is induced by low-quality data, e.g., corrupt images, fast-paced videos, and non-detailed texts. We propose a novel Prototype-based Aleatoric Uncertainty Quantification (PAU) framework to provide trustworthy predictions by quantifying the uncertainty arisen from the inherent data ambiguity.
  arXiv  Detail & Related papers  (2023-09-29T09:41:19Z)
• Geometric Deep Learning for Structure-Based Drug Design: A Survey [83.87489798671155]
  Structure-based drug design (SBDD) leverages the three-dimensional geometry of proteins to identify potential drug candidates. Recent advancements in geometric deep learning, which effectively integrate and process 3D geometric data, have significantly propelled the field forward.
  arXiv  Detail & Related papers  (2023-06-20T14:21:58Z)
• Incorporating Domain Knowledge in Deep Neural Networks for Discrete Choice Models [0.5801044612920815]
  This paper proposes a framework that expands the potential of data-driven approaches for DCM. It includes pseudo data samples that represent required relationships and a loss function that measures their fulfillment. A case study demonstrates the potential of this framework for discrete choice analysis.
  arXiv  Detail & Related papers  (2023-05-30T12:53:55Z)
• Validation Diagnostics for SBI algorithms based on Normalizing Flows [55.41644538483948]
  This work proposes easy to interpret validation diagnostics for multi-dimensional conditional (posterior) density estimators based on NF. It also offers theoretical guarantees based on results of local consistency. This work should help the design of better specified models or drive the development of novel SBI-algorithms.
  arXiv  Detail & Related papers  (2022-11-17T15:48:06Z)
• Interpretable Mixture of Experts [71.55701784196253]
  Interpretable Mixture of Experts (IME) is an inherently-interpretable modeling framework. IME is demonstrated to be more accurate than single interpretable models and perform comparably with existing state-of-the-art Deep Neural Networks (DNNs) in accuracy. IME's explanations are compared to commonly-used post-hoc explanations methods through a user study.
  arXiv  Detail & Related papers  (2022-06-05T06:40:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.