Metamorphic Testing for Clinical ML Models: A Framework Proposal and Pilot Study

• URL: http://arxiv.org/abs/2607.22984v1
• Date: Sat, 25 Jul 2026 01:42:45 GMT
• Title: Metamorphic Testing for Clinical ML Models: A Framework Proposal and Pilot Study
• Abstract summary: Machine learning models for clinical prediction tasks routinely achieve high AUROC scores.<n> AUROC measures ranking performance rather than clinical sensibility.<n>This paper proposes applying metamorphic testing to machine learning models to evaluate behavioral correctness.
• Score: 3.4905170154782343
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Machine learning models for clinical prediction tasks, such as in-hospital mortality and sepsis onset, routinely achieve high AUROC scores. However, AUROC measures ranking performance rather than clinical sensibility. A model may rank patients correctly overall while predicting a lower mortality risk when a patient's SOFA score worsens, contradicting established medical knowledge. This paper proposes applying metamorphic testing (MT) to clinical machine learning models to evaluate behavioral correctness without requiring ground-truth labels for individual predictions. We design a catalog of 12 candidate metamorphic relations (MRs) for three ICU prediction tasks using the MIMIC-III and MIMIC-IV datasets, with each MR grounded in an authoritative clinical guideline. We further propose a five-layer validation strategy to ensure that MRs are clinically sound before deployment. As a feasibility study, we evaluate the approach on the UCI Heart Disease dataset. Although the three clinical models achieve strong predictive performance (AUROC = 0.849-0.900), they exhibit MT violation rates ranging from 27% to 87% across five pilot MRs. An injected-fault experiment further shows that a sign-negation error in a blood pressure feature remains undetected by AUROC but increases the MT violation rate by 31-67 percentage points. These findings suggest that metamorphic testing provides a valuable complement to conventional performance metrics for assessing the behavioral correctness of clinical prediction models.

Related papers

• Beyond Scalar Scores: Exploring LLM-based Metrics for Clinical Significance Evaluation in Radiology Reports [49.5225801722164]
  Existing metrics obscure this requirement by reducing report quality to a medically ungrounded scalar.<n>We study this boundary using ReEvalMed benchmark as testbed and evaluate metric-level clinical significance.
  arXiv  Detail & Related papers  (2026-06-17T08:10:30Z)
• Estimating Individualized Treatment Effects in Acute Ischemic Stroke with Causal Transformation Models (TRAM-DAG): A Multi-Centre Observational Study with External RCT Validation [0.5359378066251386]
  Mechanical thrombectomy has been shown to be more effective on average than lysis in randomized controlled trials (RCTs)<n>We aim to identify which individual patients benefit most from mechanical thrombectomy compared to lysis.<n>We demonstrate that causal transformation models on directed acyclic graphs (TRAM-DAG) can be used for ITE estimation after being fitted on observational stroke patient data.
  arXiv  Detail & Related papers  (2026-06-10T19:29:43Z)
• Multimodal Ordinal Modeling of Alzheimer's Disease Severity Using Structural MRI and Clinical Data [0.9786690381850356]
  Alzheimer's disease (AD) requires accurate and scalable tools for assessing disease severity.<n>We propose an attention-enhanced machine learning framework with ordinal regression for automated and interpretable AD severity staging.
  arXiv  Detail & Related papers  (2026-06-10T08:26:11Z)
• ReMedi: Reasoner for Medical Clinical Prediction [70.84466325266068]
  We present ReMedi, a framework for improving clinical outcome prediction from EHR.<n>ReMedi generates rationale-answer pairs using a challenging sample regeneration mechanism for complex clinical questions.<n> Experiments on multiple EHR prediction tasks demonstrate substantial gains of up to 19.9 percent over state-of-the-art baselines in terms of F1 score.
  arXiv  Detail & Related papers  (2026-05-02T14:44:49Z)
• Explainable Machine Learning for Sepsis Outcome Prediction Using a Novel Romanian Electronic Health Record Dataset [0.5277756703318045]
  This study aims to identify clinically strong predictors while achieving state-of-the-art results across three classification tasks.<n>Eosinopenia emerged as a top predictor, highlighting its value as an underutilized marker that is not included in current assessment standards.
  arXiv  Detail & Related papers  (2026-04-06T14:07:23Z)
• Prediction of Lung Metastasis from Hepatocellular Carcinoma using the SEER Database [0.9055332067000195]
  Hepatocellular carcinoma (HCC) is a leading cause of cancer-related mortality.<n> predictive models for lung metastasis inHCC remain limited in scope and clinical applicability.<n>We develop and validate an end-to-end machine learning pipeline using data from the Surveillance, Epidemiology, and End Results (SEER) database.
  arXiv  Detail & Related papers  (2025-01-20T20:06:31Z)
• CRTRE: Causal Rule Generation with Target Trial Emulation Framework [47.2836994469923]
  We introduce a novel method called causal rule generation with target trial emulation framework (CRTRE) CRTRE applies randomize trial design principles to estimate the causal effect of association rules. We then incorporate such association rules for the downstream applications such as prediction of disease onsets.
  arXiv  Detail & Related papers  (2024-11-10T02:40:06Z)
• Advancements In Heart Disease Prediction: A Machine Learning Approach For Early Detection And Risk Assessment [0.0]
  This paper comprehends, assess, and analyze the role, relevance, and efficiency of machine learning models in predicting heart disease risks using clinical data. The Support Vector Machine (SVM) demonstrates the highest accuracy at 91.51%, confirming its superiority among the evaluated models in terms of predictive capability.
  arXiv  Detail & Related papers  (2024-10-16T22:32:19Z)
• Machine Learning for ALSFRS-R Score Prediction: Making Sense of the Sensor Data [44.99833362998488]
  Amyotrophic Lateral Sclerosis (ALS) is a rapidly progressive neurodegenerative disease that presents individuals with limited treatment options. The present investigation, spearheaded by the iDPP@CLEF 2024 challenge, focuses on utilizing sensor-derived data obtained through an app.
  arXiv  Detail & Related papers  (2024-07-10T19:17:23Z)
• Multimodal Pretraining of Medical Time Series and Notes [45.89025874396911]
  Deep learning models show promise in extracting meaningful patterns, but they require extensive labeled data. We propose a novel approach employing self-supervised pretraining, focusing on the alignment of clinical measurements and notes. In downstream tasks, including in-hospital mortality prediction and phenotyping, our model outperforms baselines in settings where only a fraction of the data is labeled.
  arXiv  Detail & Related papers  (2023-12-11T21:53:40Z)
• TREEMENT: Interpretable Patient-Trial Matching via Personalized Dynamic Tree-Based Memory Network [54.332862955411656]
  Clinical trials are critical for drug development but often suffer from expensive and inefficient patient recruitment. In recent years, machine learning models have been proposed for speeding up patient recruitment via automatically matching patients with clinical trials. We introduce a dynamic tree-based memory network model named TREEMENT to provide accurate and interpretable patient trial matching.
  arXiv  Detail & Related papers  (2023-07-19T12:35:09Z)
• MIA-Prognosis: A Deep Learning Framework to Predict Therapy Response [58.0291320452122]
  This paper aims at a unified deep learning approach to predict patient prognosis and therapy response. We formalize the prognosis modeling as a multi-modal asynchronous time series classification task. Our predictive model could further stratify low-risk and high-risk patients in terms of long-term survival.
  arXiv  Detail & Related papers  (2020-10-08T15:30:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.