Beyond Benchmarks: A Framework for Post Deployment Validation of CT Lung Nodule Detection AI

• URL: http://arxiv.org/abs/2603.26785v1
• Date: Wed, 25 Mar 2026 06:35:54 GMT
• Title: Beyond Benchmarks: A Framework for Post Deployment Validation of CT Lung Nodule Detection AI
• Abstract summary: Performance reported under benchmark conditions may not reflect real-world behavior.<n>Twenty-one cases from the publicly available LIDC-IDRI dataset were evaluated.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Background: Artificial intelligence (AI) assisted lung nodule detection systems are increasingly deployed in clinical settings without site-specific validation. Performance reported under benchmark conditions may not reflect real-world behavior when acquisition parameters differ from training data. Purpose: To propose and demonstrate a physics-guided framework for evaluating the sensitivity of a deployed lung nodule detection model to systematic variation in CT acquisition parameters. Methods: Twenty-one cases from the publicly available LIDC-IDRI dataset were evaluated using a MONAI RetinaNet model pretrained on LUNA16 (fold 0, no fine-tuning). Five imaging conditions were tested: baseline, 25% dose reduction, 50% dose reduction, 3 mm slice thickness, and 5 mm slice thickness. Dose reduction was simulated via image-domain Gaussian noise; slice thickness via moving average along the z-axis. Detection sensitivity was computed at a confidence threshold of 0.5 with a 15 mm matching criterion. Results: Baseline sensitivity was 45.2% (57/126 consensus nodules). Dose reduction produced slight degradation: 41.3% at 25% dose and 42.1% at 50% dose. The 5 mm slice thickness condition produced a marked drop to 26.2% - a 19 percentage point reduction representing a 42% relative decrease from baseline. This finding was consistent across confidence thresholds from 0.1 to 0.9. Per-case analysis revealed heterogeneous performance including two cases with complete detection failure at baseline. Conclusion: Slice thickness represents a more fundamental constraint on AI detection performance than image noise under the conditions tested. The proposed framework is reproducible, requires no proprietary scanner data, and is designed to serve as the basis for ongoing post-deployment QA in resource-constrained environment.

Related papers

• Parameter-Efficient pretrained-CT-to-MRI Transfer for Rectal Cancer Segmentation: Performance-Calibration Trade-offs [3.0748861313822995]
  We introduce SWIFT, a pretrained model wIth parameter-eF and Tumor-aware fine-tuning for rectal cancer segmentation.<n>A Swin V2 encoder pretrained on 10,444 public 3D CT volumes was adapted to T2-weighted MRI through four cumulative configurations.<n> Geometric accuracy, tumor detection, radiomic agreement, and probability calibration were evaluated.
  arXiv  Detail & Related papers  (2026-08-27T14:27:49Z)
• Luminosity-Adaptive Contrast Enhancement Using CLAHE for Retinal Fundus Images with Quantitative Validation and Comparative Analysis [0.0]
  Clinical utility of fundus images is routinely compromised by non-uniform illumination, motion blur, and low contrast.<n>This study proposes a two-stage image enhancement pipeline combining luminosity correction via HSV colour space decomposition with Contrast Limited Adaptive Histogram Equalization.
  arXiv  Detail & Related papers  (2026-07-20T08:39:13Z)
• Acquisition state behaves as a structured, measurable variable governing lung-nodule AI: kernel-driven measurement instability and noise-driven detection fragility, invisible to DICOM metadata [0.0]
  The 2026 ACR-SIIM Practice recommends local acceptance testing and ongoing drift monitoring.<n>We argue that a necessary, currently unmonitored layer sits beneath output metrics: whether incoming studies remain within the acquisition envelope a model was validated on.
  arXiv  Detail & Related papers  (2026-06-11T02:38:37Z)
• AortaDiff: A Unified Multitask Diffusion Framework For Contrast-Free AAA Imaging [7.769952833857425]
  We propose a unified deep learning framework that generates synthetic CECT images from non-contrast CT scans.<n>Our approach integrates conditional diffusion models (CDM) with multi-task learning, enabling end-to-end joint optimization of image synthesis and anatomical segmentation.
  arXiv  Detail & Related papers  (2025-10-01T22:19:27Z)
• A Unified Model for Compressed Sensing MRI Across Undersampling Patterns [69.19631302047569]
  We propose a unified MRI reconstruction model robust to various measurement undersampling patterns and image resolutions.<n>Our model improves SSIM by 11% and PSNR by 4 dB over a state-of-the-art CNN (End-to-End VarNet) with 600$times$ faster inference than diffusion methods.
  arXiv  Detail & Related papers  (2024-10-05T20:03:57Z)
• Controllable retinal image synthesis using conditional StyleGAN and latent space manipulation for improved diagnosis and grading of diabetic retinopathy [0.0]
  This paper proposes a framework for controllably generating high-fidelity and diverse DR fundus images. We achieve comprehensive control over DR severity and visual features within generated images. We manipulate the DR images generated conditionally on grades, further enhancing the dataset diversity.
  arXiv  Detail & Related papers  (2024-09-11T17:08:28Z)
• Enhancing Diagnostic Reliability of Foundation Model with Uncertainty Estimation in OCT Images [41.002573031087856]
  We developed a foundation model with uncertainty estimation (FMUE) to detect 11 retinal conditions on optical coherence tomography ( OCT) FMUE achieved a higher F1 score of 96.76% than two state-of-the-art algorithms, RETFound and UIOS, and got further improvement with thresholding strategy to 98.44%. Our model is superior to two ophthalmologists with a higher F1 score (95.17% vs. 61.93% &71.72%)
  arXiv  Detail & Related papers  (2024-06-18T03:04:52Z)
• A quality assurance framework for real-time monitoring of deep learning segmentation models in radiotherapy [3.5752677591512487]
  This work uses cardiac substructure segmentation as an example task to establish a quality assurance framework. A benchmark dataset consisting of Computed Tomography (CT) images along with manual cardiac delineations of 241 patients was collected. An image domain shift detector was developed by utilizing a trained Denoising autoencoder (DAE) and two hand-engineered features. A regression model was trained to predict the per-patient segmentation accuracy, measured by Dice similarity coefficient (DSC)
  arXiv  Detail & Related papers  (2023-05-19T14:51:05Z)
• Task-Based Assessment for Neural Networks: Evaluating Undersampled MRI Reconstructions based on Human Observer Signal Detection [45.82374977939355]
  Common metrics for evaluating image quality like the normalized root mean squared error (NRMSE) and structural similarity (SSIM) are global metrics which average out impact of subtle features in the images. We used measures of image quality which incorporate a subtle signal for a specific task allow for image quality assessment which locally evaluates the effect of undersampling on a signal.
  arXiv  Detail & Related papers  (2022-10-21T16:39:04Z)
• Automated SSIM Regression for Detection and Quantification of Motion Artefacts in Brain MR Images [54.739076152240024]
  Motion artefacts in magnetic resonance brain images are a crucial issue. The assessment of MR image quality is fundamental before proceeding with the clinical diagnosis. An automated image quality assessment based on the structural similarity index (SSIM) regression has been proposed here.
  arXiv  Detail & Related papers  (2022-06-14T10:16:54Z)
• The Report on China-Spain Joint Clinical Testing for Rapid COVID-19 Risk Screening by Eye-region Manifestations [59.48245489413308]
  We developed and tested a COVID-19 rapid prescreening model using the eye-region images captured in China and Spain with cellphone cameras. The performance was measured using area under receiver-operating-characteristic curve (AUC), sensitivity, specificity, accuracy, and F1.
  arXiv  Detail & Related papers  (2021-09-18T02:28:01Z)
• Quantification of pulmonary involvement in COVID-19 pneumonia by means of a cascade oftwo U-nets: training and assessment on multipledatasets using different annotation criteria [83.83783947027392]
  This study aims at exploiting Artificial intelligence (AI) for the identification, segmentation and quantification of COVID-19 pulmonary lesions. We developed an automated analysis pipeline, the LungQuant system, based on a cascade of two U-nets. The accuracy in predicting the CT-Severity Score (CT-SS) of the LungQuant system has been also evaluated.
  arXiv  Detail & Related papers  (2021-05-06T10:21:28Z)
• FLANNEL: Focal Loss Based Neural Network Ensemble for COVID-19 Detection [61.04937460198252]
  We construct the X-ray imaging data from 2874 patients with four classes: normal, bacterial pneumonia, non-COVID-19 viral pneumonia, and COVID-19. To identify COVID-19, we propose a Focal Loss Based Neural Ensemble Network (FLANNEL) FLANNEL consistently outperforms baseline models on COVID-19 identification task in all metrics.
  arXiv  Detail & Related papers  (2020-10-30T03:17:31Z)
• OCT-GAN: Single Step Shadow and Noise Removal from Optical Coherence Tomography Images of the Human Optic Nerve Head [47.812972855826985]
  We developed a single process that successfully removed both noise and retinal shadows from unseen single-frame B-scans within 10.4ms. The proposed algorithm reduces the necessity for long image acquisition times, minimizes expensive hardware requirements and reduces motion artifacts in OCT images.
  arXiv  Detail & Related papers  (2020-10-06T08:32:32Z)
• Deep Learning Based Detection and Localization of Intracranial Aneurysms in Computed Tomography Angiography [5.973882600944421]
  A two-step model was implemented: a 3D region proposal network for initial aneurysm detection and 3D DenseNetsfor false-positive reduction. Our model showed statistically higher accuracy, sensitivity, and specificity when compared to the available model at 0.25 FPPV and the best F-1 score.
  arXiv  Detail & Related papers  (2020-05-22T10:49:23Z)
• Using a Generative Adversarial Network for CT Normalization and its Impact on Radiomic Features [3.4548443472506194]
  Radiomic features are sensitive to differences in acquisitions due to variations in dose levels and slice thickness. A 3D generative adversarial network (GAN) was used to normalize reduced dose, thick slice (2.0mm) images to normal dose (100%), thinner slice (1.0mm) images.
  arXiv  Detail & Related papers  (2020-01-22T23:41:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.