Robustness tests for biomedical foundation models should tailor to specification

• URL: http://arxiv.org/abs/2502.10374v1
• Date: Fri, 14 Feb 2025 18:52:10 GMT
• Title: Robustness tests for biomedical foundation models should tailor to specification
• Abstract summary: We suggest a priority-based, task-oriented approach to tailor robustness evaluation objectives to a predefined specification.<n>We urge concrete policies to adopt a granular categorization of robustness concepts in the specification.
• Score: 16.66048720047442
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Existing regulatory frameworks for biomedical AI include robustness as a key component but lack detailed implementational guidance. The recent rise of biomedical foundation models creates new hurdles in testing and certification given their broad capabilities and susceptibility to complex distribution shifts. To balance test feasibility and effectiveness, we suggest a priority-based, task-oriented approach to tailor robustness evaluation objectives to a predefined specification. We urge concrete policies to adopt a granular categorization of robustness concepts in the specification. Our approach promotes the standardization of risk assessment and monitoring, which guides technical developments and mitigation efforts.

Related papers

• Persistent Convolution: A Topological Framework for AI Alignment Testing and Semantic Space Characterization [0.0]
  This work develops and demonstrates a topology-based multi-modal alignment test to make deployment, selection, and comparison of opaque models more interpretable.<n>These methods also offer an intuitive connection to possibility theory and a unified decision theoretic framework from data to deployment.
  arXiv  Detail & Related papers  (2026-07-31T04:12:21Z)
• Dementia Etiology Diagnosis via Collaborative Meta Knowledge Enhancement [81.27196033392559]
  We propose a Collaborative Meta Knowledge Enhancement (COME) framework for dementia etiology diagnosis.<n>COME injects multi-center acquisition semantics, source identifiers, and modality indicators as heterogeneity-aware embeddings into a unified Transformer architecture for scale-up training.<n>Our method achieves state-of-the-art in-domain performance with a mean macro-averaged AUC of 85.62% and a 4.29-point gain over the strongest baseline.
  arXiv  Detail & Related papers  (2026-07-24T04:26:47Z)
• Position: Anthropomorphic Misalignment Research Needs Stronger Evidence [52.76826423649968]
  We show how conceptual ambiguity, non-robust datasets, experimental design, and insufficient causal interventions can lead to overinterpretation of model behaviors.<n>This position paper aims to offer guidance on evidentiary considerations that can help improve methodological rigor in AMR.
  arXiv  Detail & Related papers  (2026-05-29T16:38:53Z)
• Structure-Adaptive Conformal Inference for Large-Scale Out-of-Distribution Testing [1.867982979635437]
  This paper addresses structured out-of-distribution (OOD) testing in high-stakes machine learning applications.<n>We propose the structure-adaptive conformal q-value (SCQ) and pseudo-score-guided automated model selection (P-TAMS)<n>P-TAMS adapts conformalized model selection to structured OOD testing across a toolbox of candidate models.
  arXiv  Detail & Related papers  (2026-05-26T01:28:16Z)
• QuantumQA: Enhancing Scientific Reasoning via Physics-Consistent Dataset and Verification-Aware Reinforcement Learning [19.224968023943052]
  We introduce QuantumQA, a large-scale dataset constructed via a task-adaptive strategy.<n>We propose the verification-aware reward model (VRM) tailored for Reinforcement Learning with Verifiable Rewards (RLVR)
  arXiv  Detail & Related papers  (2026-04-20T12:33:50Z)
• Beyond Prompt: Fine-grained Simulation of Cognitively Impaired Standardized Patients via Stochastic Steering [58.85421622061983]
  We propose StsPatient for the fine-grained simulation of cognitively impaired patients.<n>We capture domain-specific features by extracting steering vectors from contrastive pairs of instructions and responses.<n>StsPatient significantly outperforms baselines in both clinical authenticity and severity controllability.
  arXiv  Detail & Related papers  (2026-04-14T02:37:46Z)
• ProMedical: Hierarchical Fine-Grained Criteria Modeling for Medical LLM Alignment via Explicit Injection [1.1273389819207937]
  We introduce ProMedical, a unified alignment framework grounded in fine-grained clinical criteria.<n>We first construct ProMedical-Preference-50k, a dataset generated via a human-in-the-loop pipeline.<n>Unlike traditional scalar reward models, our approach explicitly disentangles safety constraints from general proficiency.
  arXiv  Detail & Related papers  (2026-04-09T14:57:33Z)
• POET: Protocol Optimization via Eligibility Tuning [1.4267510278572033]
  We propose a guided generation framework that introduces interpretable semantic axes to steer EC generation.<n>These axes offer a middle ground between specificity and usability, enabling clinicians to guide generation without specifying exact entities.<n>Our results show that our guided generation approach consistently outperforms unguided generation in both automatic, rubric-based and clinician evaluations.
  arXiv  Detail & Related papers  (2026-01-30T22:32:43Z)
• An Integrated Fusion Framework for Ensemble Learning Leveraging Gradient Boosting and Fuzzy Rule-Based Models [59.13182819190547]
  Fuzzy rule-based models excel in interpretability and have seen widespread application across diverse fields.<n>They face challenges such as complex design specifications and scalability issues with large datasets.<n>This paper proposes an Integrated Fusion Framework that merges the strengths of both paradigms to enhance model performance and interpretability.
  arXiv  Detail & Related papers  (2025-11-11T10:28:23Z)
• Automated Genomic Interpretation via Concept Bottleneck Models for Medical Robotics [26.40159044419644]
  We propose an automated genomic interpretation module that transforms raw DNA sequences into actionable, interpretable decisions.<n>Our framework combines Chaos Game Representation with a Concept Bottleneck Model (CBM), enforcing predictions to flow through biologically meaningful concepts.<n>By bridging the gap between interpretable genomic modeling and automated decision-making, this work establishes a reliable foundation for robotic and clinical automation in genomic medicine.
  arXiv  Detail & Related papers  (2025-10-02T02:51:34Z)
• Score-based Generative Modeling for Conditional Independence Testing [35.0533359302886]
  We propose a novel CI testing method via score-based generative modeling, which achieves precise Type I error control and strong testing power.<n>We theoretically establish the error bound of conditional distributions modeled by score-based generative models and prove the validity of our CI tests.
  arXiv  Detail & Related papers  (2025-05-29T10:10:46Z)
• Advancing Embodied Agent Security: From Safety Benchmarks to Input Moderation [52.83870601473094]
  Embodied agents exhibit immense potential across a multitude of domains. Existing research predominantly concentrates on the security of general large language models. This paper introduces a novel input moderation framework, meticulously designed to safeguard embodied agents.
  arXiv  Detail & Related papers  (2025-04-22T08:34:35Z)
• Beyond Black-Box Benchmarking: Observability, Analytics, and Optimization of Agentic Systems [1.415098516077151]
  The rise of agentic AI systems, where agents collaborate to perform diverse tasks, poses new challenges with observing, analyzing and optimizing their behavior. Traditional evaluation and benchmarking approaches struggle to handle the non-deterministic, context-sensitive, and dynamic nature of these systems. This paper explores key challenges and opportunities in analyzing and optimizing agentic systems across development, testing, and maintenance.
  arXiv  Detail & Related papers  (2025-03-09T20:02:04Z)
• SMRS: advocating a unified reporting standard for surrogate models in the artificial intelligence era [1.4835379864550937]
  We argue for the urgent need to establish a structured reporting standard for surrogate models.<n>By promoting a standardised yet flexible framework, we aim to improve the reliability of surrogate modelling.
  arXiv  Detail & Related papers  (2025-02-10T18:31:15Z)
• Pitfalls of topology-aware image segmentation [81.19923502845441]
  We identify critical pitfalls in model evaluation that include inadequate connectivity choices, overlooked topological artifacts, and inappropriate use of evaluation metrics.<n>We propose a set of actionable recommendations to establish fair and robust evaluation standards for topology-aware medical image segmentation methods.
  arXiv  Detail & Related papers  (2024-12-19T08:11:42Z)
• Exposing Assumptions in AI Benchmarks through Cognitive Modelling [0.0]
  Cultural AI benchmarks often rely on implicit assumptions about measured constructs, leading to vague formulations with poor validity and unclear interrelations. We propose exposing these assumptions using explicit cognitive models formulated as Structural Equation Models.
  arXiv  Detail & Related papers  (2024-09-25T11:55:02Z)
• Beyond One-Time Validation: A Framework for Adaptive Validation of Prognostic and Diagnostic AI-based Medical Devices [55.319842359034546]
  Existing approaches often fall short in addressing the complexity of practically deploying these devices. The presented framework emphasizes the importance of repeating validation and fine-tuning during deployment. It is positioned within the current US and EU regulatory landscapes.
  arXiv  Detail & Related papers  (2024-09-07T11:13:52Z)
• FEDKIM: Adaptive Federated Knowledge Injection into Medical Foundation Models [54.09244105445476]
  This study introduces a novel knowledge injection approach, FedKIM, to scale the medical foundation model within a federated learning framework.<n>FedKIM leverages lightweight local models to extract healthcare knowledge from private data and integrates this knowledge into a centralized foundation model.<n>Our experiments across twelve tasks in seven modalities demonstrate the effectiveness of FedKIM in various settings.
  arXiv  Detail & Related papers  (2024-08-17T15:42:29Z)
• Benchmarks as Microscopes: A Call for Model Metrology [76.64402390208576]
  Modern language models (LMs) pose a new challenge in capability assessment. To be confident in our metrics, we need a new discipline of model metrology.
  arXiv  Detail & Related papers  (2024-07-22T17:52:12Z)
• GenBench: A Benchmarking Suite for Systematic Evaluation of Genomic Foundation Models [56.63218531256961]
  We introduce GenBench, a benchmarking suite specifically tailored for evaluating the efficacy of Genomic Foundation Models. GenBench offers a modular and expandable framework that encapsulates a variety of state-of-the-art methodologies. We provide a nuanced analysis of the interplay between model architecture and dataset characteristics on task-specific performance.
  arXiv  Detail & Related papers  (2024-06-01T08:01:05Z)
• Assessing biomedical knowledge robustness in large language models by query-efficient sampling attacks [0.6282171844772422]
  An increasing depth of parametric domain knowledge in large language models (LLMs) is fueling their rapid deployment in real-world applications.<n>The recent discovery of named entities as adversarial examples in natural language processing tasks raises questions about their potential impact on the knowledge robustness of pre-trained and finetuned LLMs.<n>We developed an embedding-space attack based on powerscaled distance-weighted sampling to assess the robustness of their biomedical knowledge.
  arXiv  Detail & Related papers  (2024-02-16T09:29:38Z)
• Evaluating General-Purpose AI with Psychometrics [43.85432514910491]
  We discuss the need for a comprehensive and accurate evaluation of general-purpose AI systems such as large language models. Current evaluation methodology, mostly based on benchmarks of specific tasks, falls short of adequately assessing these versatile AI systems. To tackle these challenges, we suggest transitioning from task-oriented evaluation to construct-oriented evaluation.
  arXiv  Detail & Related papers  (2023-10-25T05:38:38Z)
• Benchmarking Scalable Epistemic Uncertainty Quantification in Organ Segmentation [7.313010190714819]
  quantifying uncertainty associated with model predictions is crucial in critical clinical applications. Deep learning based methods for automatic organ segmentation have shown promise in aiding diagnosis and treatment planning. It is unclear which method is preferred in the medical image analysis setting.
  arXiv  Detail & Related papers  (2023-08-15T00:09:33Z)
• Geometric Deep Learning for Structure-Based Drug Design: A Survey [83.87489798671155]
  Structure-based drug design (SBDD) leverages the three-dimensional geometry of proteins to identify potential drug candidates. Recent advancements in geometric deep learning, which effectively integrate and process 3D geometric data, have significantly propelled the field forward.
  arXiv  Detail & Related papers  (2023-06-20T14:21:58Z)
• From Static Benchmarks to Adaptive Testing: Psychometrics in AI Evaluation [60.14902811624433]
  We discuss a paradigm shift from static evaluation methods to adaptive testing. This involves estimating the characteristics and value of each test item in the benchmark and dynamically adjusting items in real-time. We analyze the current approaches, advantages, and underlying reasons for adopting psychometrics in AI evaluation.
  arXiv  Detail & Related papers  (2023-06-18T09:54:33Z)
• Benchmarking Model Predictive Control Algorithms in Building Optimization Testing Framework (BOPTEST) [40.17692290400862]
  We present a data-driven modeling and control framework for physics-based building emulators. Our approach consists of: (a) Offline training of differentiable surrogate models that accelerate model evaluations, provide cost-effective gradients, and maintain good predictive accuracy for the receding horizon in Model Predictive Control (MPC) We extensively evaluate the modeling and control performance using multiple surrogate models and optimization frameworks across various test cases available in the Building Optimization Testing Framework (BOPTEST)
  arXiv  Detail & Related papers  (2023-01-31T06:55:19Z)
• Validation Diagnostics for SBI algorithms based on Normalizing Flows [55.41644538483948]
  This work proposes easy to interpret validation diagnostics for multi-dimensional conditional (posterior) density estimators based on NF. It also offers theoretical guarantees based on results of local consistency. This work should help the design of better specified models or drive the development of novel SBI-algorithms.
  arXiv  Detail & Related papers  (2022-11-17T15:48:06Z)
• Estimating Test Performance for AI Medical Devices under Distribution Shift with Conformal Prediction [4.395519864600419]
  We consider the task of predicting the test accuracy of an arbitrary black-box model on an unlabeled target domain. We propose a "black-box" test estimation technique based on conformal prediction and evaluate it against other methods.
  arXiv  Detail & Related papers  (2022-07-12T19:25:21Z)
• Extending Process Discovery with Model Complexity Optimization and Cyclic States Identification: Application to Healthcare Processes [62.997667081978825]
  The paper presents an approach to process mining providing semi-automatic support to model optimization. A model simplification approach is proposed, which essentially abstracts the raw model at the desired granularity. We aim to demonstrate the capabilities of the technological solution using three datasets from different applications in the healthcare domain.
  arXiv  Detail & Related papers  (2022-06-10T16:20:59Z)
• Calibrating Over-Parametrized Simulation Models: A Framework via Eligibility Set [3.862247454265944]
  We develop a framework to develop calibration schemes that satisfy rigorous frequentist statistical guarantees. We demonstrate our methodology on several numerical examples, including an application to calibration of a limit order book market simulator.
  arXiv  Detail & Related papers  (2021-05-27T00:59:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.