Need for Objective Task-based Evaluation of Deep Learning-Based Denoising Methods: A Study in the Context of Myocardial Perfusion SPECT

• URL: http://arxiv.org/abs/2303.02110v5
• Date: Sun, 2 Apr 2023 00:18:22 GMT
• Title: Need for Objective Task-based Evaluation of Deep Learning-Based Denoising Methods: A Study in the Context of Myocardial Perfusion SPECT
• Authors: Zitong Yu, Md Ashequr Rahman, Richard Laforest, Thomas H. Schindler, Robert J. Gropler, Richard L. Wahl, Barry A. Siegel, Abhinav K. Jha
• Abstract summary: This study investigates whether evaluation with figures of merit (FoMs) is consistent with objective clinical-task-based evaluation. The impact of DL-based denoising was evaluated using fidelity-based FoMs and AUC. The results motivate the need for objective task-based evaluation of DL-based denoising approaches.
• Score: 11.559405600109415
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Artificial intelligence-based methods have generated substantial interest in nuclear medicine. An area of significant interest has been using deep-learning (DL)-based approaches for denoising images acquired with lower doses, shorter acquisition times, or both. Objective evaluation of these approaches is essential for clinical application. DL-based approaches for denoising nuclear-medicine images have typically been evaluated using fidelity-based figures of merit (FoMs) such as RMSE and SSIM. However, these images are acquired for clinical tasks and thus should be evaluated based on their performance in these tasks. Our objectives were to (1) investigate whether evaluation with these FoMs is consistent with objective clinical-task-based evaluation; (2) provide a theoretical analysis for determining the impact of denoising on signal-detection tasks; (3) demonstrate the utility of virtual clinical trials (VCTs) to evaluate DL-based methods. A VCT to evaluate a DL-based method for denoising myocardial perfusion SPECT (MPS) images was conducted. The impact of DL-based denoising was evaluated using fidelity-based FoMs and AUC, which quantified performance on detecting perfusion defects in MPS images as obtained using a model observer with anthropomorphic channels. Based on fidelity-based FoMs, denoising using the considered DL-based method led to significantly superior performance. However, based on ROC analysis, denoising did not improve, and in fact, often degraded detection-task performance. The results motivate the need for objective task-based evaluation of DL-based denoising approaches. Further, this study shows how VCTs provide a mechanism to conduct such evaluations using VCTs. Finally, our theoretical treatment reveals insights into the reasons for the limited performance of the denoising approach.

Related papers

• WIA-LD2ND: Wavelet-based Image Alignment for Self-supervised Low-Dose CT Denoising [74.14134385961775]
  We introduce a novel self-supervised CT image denoising method called WIA-LD2ND, only using NDCT data. WIA-LD2ND comprises two modules: Wavelet-based Image Alignment (WIA) and Frequency-Aware Multi-scale Loss (FAM)
  arXiv  Detail & Related papers  (2024-03-18T11:20:11Z)
• KIEval: A Knowledge-grounded Interactive Evaluation Framework for Large Language Models [53.84677081899392]
  KIEval is a Knowledge-grounded Interactive Evaluation framework for large language models. It incorporates an LLM-powered "interactor" role for the first time to accomplish a dynamic contamination-resilient evaluation. Extensive experiments on seven leading LLMs across five datasets validate KIEval's effectiveness and generalization.
  arXiv  Detail & Related papers  (2024-02-23T01:30:39Z)
• Self-supervised OCT Image Denoising with Slice-to-Slice Registration and Reconstruction [5.972377737617966]
  Learning-based self-supervised methods for structure-preserving noise reduction have demonstrated superior performance over traditional methods. We introduce a new end-to-end self-supervised learning framework specifically tailored for OCT image denoising.
  arXiv  Detail & Related papers  (2023-11-26T02:45:16Z)
• DEMIST: A deep-learning-based task-specific denoising approach for myocardial perfusion SPECT [17.994633874783144]
  We propose a Detection task-specific deep-learning-based approach for denoising MPI SPECT images (DEMIST) The approach, while performing denoising, is designed to preserve features that influence observer performance on detection tasks. The results provide strong evidence for further clinical evaluation of DEMIST to denoise low-count images in MPI SPECT.
  arXiv  Detail & Related papers  (2023-06-07T08:40:25Z)
• A task-specific deep-learning-based denoising approach for myocardial perfusion SPECT [15.07522345889704]
  We propose a DL-based denoising approach designed to preserve observer-related information for detection tasks. Our results demonstrate that the proposed method yields improved performance on this detection task compared to using low-dose images.
  arXiv  Detail & Related papers  (2023-03-01T03:33:12Z)
• The role of noise in denoising models for anomaly detection in medical images [62.0532151156057]
  Pathological brain lesions exhibit diverse appearance in brain images. Unsupervised anomaly detection approaches have been proposed using only normal data for training. We show that optimization of the spatial resolution and magnitude of the noise improves the performance of different model training regimes.
  arXiv  Detail & Related papers  (2023-01-19T21:39:38Z)
• Ontology-aware Learning and Evaluation for Audio Tagging [56.59107110017436]
  Mean average precision (mAP) metric treats different kinds of sound as independent classes without considering their relations. Ontology-aware mean average precision (OmAP) addresses the weaknesses of mAP by utilizing the AudioSet ontology information during the evaluation. We conduct human evaluations and demonstrate that OmAP is more consistent with human perception than mAP.
  arXiv  Detail & Related papers  (2022-11-22T11:35:14Z)
• Benchmarking Heterogeneous Treatment Effect Models through the Lens of Interpretability [82.29775890542967]
  Estimating personalized effects of treatments is a complex, yet pervasive problem. Recent developments in the machine learning literature on heterogeneous treatment effect estimation gave rise to many sophisticated, but opaque, tools. We use post-hoc feature importance methods to identify features that influence the model's predictions.
  arXiv  Detail & Related papers  (2022-06-16T17:59:05Z)
• Investigating the limited performance of a deep-learning-based SPECT denoising approach: An observer-study-based characterization [16.943040406235024]
  We conducted a task-based characterization of a DL-based denoising approach for individual signal properties. A CNN-based denoiser was trained to process the low-count images. As in previous studies, we observed that the DL-based denoising method did not improve performance on signal-detection tasks.
  arXiv  Detail & Related papers  (2022-03-03T18:51:59Z)
• Explaining Clinical Decision Support Systems in Medical Imaging using Cycle-Consistent Activation Maximization [112.2628296775395]
  Clinical decision support using deep neural networks has become a topic of steadily growing interest. clinicians are often hesitant to adopt the technology because its underlying decision-making process is considered to be intransparent and difficult to comprehend. We propose a novel decision explanation scheme based on CycleGAN activation which generates high-quality visualizations of classifier decisions even in smaller data sets.
  arXiv  Detail & Related papers  (2020-10-09T14:39:27Z)
• Self-supervised Dynamic CT Perfusion Image Denoising with Deep Neural Networks [6.167259271197635]
  Dynamic computed tomography (CTP) imaging is a promising approach for acute ischemic stroke diagnosis and evaluation. Hemodynamic parametric maps of cerebral parenchyma are calculated from repeated CT scans of the first pass of iodinated contrast through the brain. It is necessary to reduce the dose of perfusion for routine applications due to the high radiation exposure from the repeated scans, where image denoising is necessary to achieve a reliable diagnosis.
  arXiv  Detail & Related papers  (2020-05-19T21:44:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.