Performance evaluation of deep learning models for image analysis: considerations for visual control and statistical metrics

• URL: http://arxiv.org/abs/2603.13557v1
• Date: Fri, 13 Mar 2026 19:49:39 GMT
• Title: Performance evaluation of deep learning models for image analysis: considerations for visual control and statistical metrics
• Abstract summary: Deep learning-based automated image analysis (DL-AIA) has been shown to outperform trained pathologists in tasks related to feature quantification.<n>The use of DL-AIA tools is currently extending from proof-of-principle studies to routine applications such as patient samples.<n>To ensure that DL-AIA applications are safe and reliable, it is critical to conduct a thorough and objective generalization performance assessment.
• Score: 38.007806456084296
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Deep learning-based automated image analysis (DL-AIA) has been shown to outperform trained pathologists in tasks related to feature quantification. Related to these capacities the use of DL-AIA tools is currently extending from proof-of-principle studies to routine applications such as patient samples (diagnostic pathology), regulatory safety assessment (toxicologic pathology), and recurrent research tasks. To ensure that DL-AIA applications are safe and reliable, it is critical to conduct a thorough and objective generalization performance assessment (i.e., the ability of the algorithm to accurately predict patterns of interest) and possibly evaluate model robustness (i.e., the algorithm's capacity to maintain predictive accuracy on images from different sources). In this article, we review the practices for performance assessment in veterinary pathology publications by which two approaches were identified: 1) Exclusive visual performance control (i.e. eyeballing of algorithmic predictions) plus validation of the models application utilizing secondary performance indices, and 2) Statistical performance control (alongside the other methods), which requires a dataset creation and separation of an hold-out test set prior to model training. This article compares the strengths and weaknesses of statistical and visual performance control methods. Furthermore, we discuss relevant considerations for rigorous statistical performance evaluation including metric selection, test dataset image composition, ground truth label quality, resampling methods such as bootstrapping, statistical comparison of multiple models, and evaluation of model stability. It is our conclusion that visual and statistical evaluation have complementary strength and a combination of both provides the greatest insight into the DL model's performance and sources of error.

Related papers

• STAR : Bridging Statistical and Agentic Reasoning for Large Model Performance Prediction [78.0692157478247]
  We propose STAR, a framework that bridges data-driven STatistical expectations with knowledge-driven Agentic Reasoning.<n>We show that STAR consistently outperforms all baselines on both score-based and rank-based metrics.
  arXiv  Detail & Related papers  (2026-02-12T16:30:07Z)
• A systematic evaluation of uncertainty quantification techniques in deep learning: a case study in photoplethysmography signal analysis [1.6690512882610855]
  Deep learning models can be used to continuously monitor physiological parameters outside of clinical settings.<n>There is risk of poor performance when deployed in practical measurement scenarios leading to negative patient outcomes.<n>Here we implement eight uncertainty (UQ) techniques to models trained on two clinically relevant prediction tasks.
  arXiv  Detail & Related papers  (2025-10-31T22:54:13Z)
• ARISE: An Adaptive Resolution-Aware Metric for Test-Time Scaling Evaluation in Large Reasoning Models [102.4511331368587]
  ARISE (Adaptive Resolution-aware Scaling Evaluation) is a novel metric designed to assess the test-time scaling effectiveness of large reasoning models.<n>We conduct comprehensive experiments evaluating state-of-the-art reasoning models across diverse domains.
  arXiv  Detail & Related papers  (2025-10-07T15:10:51Z)
• Testing and Improving the Robustness of Amortized Bayesian Inference for Cognitive Models [0.5223954072121659]
  Contaminant observations and outliers often cause problems when estimating the parameters of cognitive models.<n>In this study, we test and improve the robustness of parameter estimation using amortized Bayesian inference.<n>The proposed method is straightforward and practical to implement and has a broad applicability in fields where outlier detection or removal is challenging.
  arXiv  Detail & Related papers  (2024-12-29T21:22:24Z)
• How Deep is your Guess? A Fresh Perspective on Deep Learning for Medical Time-Series Imputation [6.547981908229007]
  We show how architectural and framework biases combine to influence model performance.<n>Experiments show imputation performance variations of up to 20% based on preprocessing and implementation choices.<n>We identify critical gaps between current deep imputation methods and medical requirements.
  arXiv  Detail & Related papers  (2024-07-11T12:33:28Z)
• Opinion-Unaware Blind Image Quality Assessment using Multi-Scale Deep Feature Statistics [54.08757792080732]
  We propose integrating deep features from pre-trained visual models with a statistical analysis model to achieve opinion-unaware BIQA (OU-BIQA) Our proposed model exhibits superior consistency with human visual perception compared to state-of-the-art BIQA models.
  arXiv  Detail & Related papers  (2024-05-29T06:09:34Z)
• Towards Better Modeling with Missing Data: A Contrastive Learning-based Visual Analytics Perspective [7.577040836988683]
  Missing data can pose a challenge for machine learning (ML) modeling. Current approaches are categorized into feature imputation and label prediction. This study proposes a Contrastive Learning framework to model observed data with missing values.
  arXiv  Detail & Related papers  (2023-09-18T13:16:24Z)
• Position: AI Evaluation Should Learn from How We Test Humans [65.36614996495983]
  We argue that psychometrics, a theory originating in the 20th century for human assessment, could be a powerful solution to the challenges in today's AI evaluations.
  arXiv  Detail & Related papers  (2023-06-18T09:54:33Z)
• Robustness and Generalization Performance of Deep Learning Models on Cyber-Physical Systems: A Comparative Study [71.84852429039881]
  Investigation focuses on the models' ability to handle a range of perturbations, such as sensor faults and noise. We test the generalization and transfer learning capabilities of these models by exposing them to out-of-distribution (OOD) samples.
  arXiv  Detail & Related papers  (2023-06-13T12:43:59Z)
• Explaining medical AI performance disparities across sites with confounder Shapley value analysis [8.785345834486057]
  Multi-site evaluations are key to diagnosing such disparities. Our framework provides a method for quantifying the marginal and cumulative effect of each type of bias on the overall performance difference. We demonstrate its usefulness in a case study of a deep learning model trained to detect the presence of pneumothorax.
  arXiv  Detail & Related papers  (2021-11-12T18:54:10Z)
• Semi-supervised Medical Image Classification with Relation-driven Self-ensembling Model [71.80319052891817]
  We present a relation-driven semi-supervised framework for medical image classification. It exploits the unlabeled data by encouraging the prediction consistency of given input under perturbations. Our method outperforms many state-of-the-art semi-supervised learning methods on both single-label and multi-label image classification scenarios.
  arXiv  Detail & Related papers  (2020-05-15T06:57:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.