Evaluating the Safety of Deep Learning-Based Brain MRI Reconstruction

• URL: http://arxiv.org/abs/2608.28714v1
• Date: Fri, 28 Aug 2026 08:08:14 GMT
• Title: Evaluating the Safety of Deep Learning-Based Brain MRI Reconstruction
• Abstract summary: Deep learning can erase lesions or synthesize false tissue, but failures pixel-averaged metrics like PSNR and SSIM miss.<n>Following PRISMA 2020, we searched seven databases without date limits, including 263 studies (1995-2026)<n>Only 18 of 263 studies (6.8%) recorded both a fidelity metric and reader assessment on identical data, leaving the central surrogate unmeasured.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Objective: Deep learning accelerates brain MRI four- to tenfold, but models can erase lesions or synthesize false tissue - failures pixel-averaged metrics like PSNR and SSIM miss. We review whether current evaluation practices detect this blind spot. Methods: Following PRISMA 2020, we searched seven databases without date limits, including 263 studies (1995-2026), appraised them using QUADAS-2 and matched instruments, and synthesized narratively. Categories were derived from titles, abstracts, and controlled vocabulary; reported prevalence figures represent floors. Duplicate screening achieved high agreement (Fleiss kappa = 0.877), as did appraisal (0.788; 0.390 where observable). Extraction is unaudited. Results: Only 18 of 263 studies (6.8%) recorded both a fidelity metric and reader assessment on identical data, leaving the central surrogate unmeasured. Reader studies mostly measured inter-reader agreement, which was weak: fastMRI 2020 concordance reached 0.457 and 0.386 (Kendall W), improving only where SSIM diverged. Erasing a 100 mm3 lacunar infarct shifts global PSNR by 0.03 dB under the stated error model. As the corpus grew fivefold, reader assessments dropped from 32% to 18%, recovering to 21%. Generative models - most associated with hallucination (39%) - were among the least reader-evaluated (11.3%), while self-supervised models reached 47% with zero reader evaluation. Only 5% released code and ran reader studies; none evaluated a model observer; no named dataset covered acute stroke or hemorrhage. Conclusions: On these floors, current evaluation practices cannot certify diagnostic safety. We derive five requirements safety-oriented evaluations must meet.

Related papers

• ImplicitMemBench: Measuring Unconscious Behavioral Adaptation in Large Language Models [60.14219417402433]
  Existing memory benchmarks for LLM agents evaluate explicit recall of facts, yet overlook implicit memory where experience becomes automated behavior without conscious retrieval.<n>We introduce ImplicitMemBench, the first systematic benchmark evaluating implicit memory through three constructs drawn from standard cognitive-science accounts of non-declarative memory.<n>Our 300-item suite employs a unified Learning/Priming-Interfere-Test protocol with first-attempt scoring.
  arXiv  Detail & Related papers  (2026-04-09T10:26:32Z)
• LoV3D: Grounding Cognitive Prognosis Reasoning in Longitudinal 3D Brain MRI via Regional Volume Assessments [14.481985722970238]
  We present LoV3D, a pipeline for training 3D vision-language models, which reads longitudinal T1-weighted brain MRI.<n>The pipeline grounds the final diagnosis by enforcing label consistency, longitudinal coherence, and biological plausibility.<n>On a subject-level held-out ADNI test set, LoV3D achieves 93.7% three-class diagnostic accuracy.
  arXiv  Detail & Related papers  (2026-03-12T15:40:59Z)
• PanCanBench: A Comprehensive Benchmark for Evaluating Large Language Models in Pancreatic Oncology [48.732366302949515]
  Large language models (LLMs) have achieved expert-level performance on standardized examinations, yet multiple-choice accuracy poorly reflects real-world clinical utility and safety.<n>We developed a human-in-the-loop pipeline to create expert rubrics for de-identified patient questions.<n>We evaluated 22 proprietary and open-source LLMs using an LLM-as-a-judge framework, measuring clinical completeness, factual accuracy, and web-search integration.
  arXiv  Detail & Related papers  (2026-03-02T00:50:39Z)
• From Scores to Steps: Diagnosing and Improving LLM Performance in Evidence-Based Medical Calculations [45.414878840652115]
  Large language models (LLMs) have demonstrated promising performance on medical benchmarks.<n>However, their ability to perform medical calculations remains underexplored and poorly evaluated.<n>In this work, we revisit medical calculation evaluation with a stronger focus on clinical trustworthiness.
  arXiv  Detail & Related papers  (2025-09-20T09:10:26Z)
• Evaluation of Machine Learning Reconstruction Techniques for Accelerated Brain MRI Scans [0.0]
  DeepFoqus-Accelerate enables robust fourfold brain MRI acceleration with 75% reduced scan time.<n>These findings demonstrate that DeepFoqus-Accelerate enables robust fourfold brain MRI acceleration with 75% reduced scan time.
  arXiv  Detail & Related papers  (2025-09-08T20:20:24Z)
• Utilizing Machine Learning Models to Predict Acute Kidney Injury in Septic Patients from MIMIC-III Database [0.0]
  Sepsis is a severe condition that causes the body to respond incorrectly to an infection.<n>For septic patients, approximately 50% develop acute kidney injury (AKI)<n>Models that can accurately predict AKI based on specific qualities of septic patients are crucial for early detection and intervention.
  arXiv  Detail & Related papers  (2024-12-04T22:05:35Z)
• Artificial Intelligence-Based Triaging of Cutaneous Melanocytic Lesions [0.8864540224289991]
  Pathologists are facing an increasing workload due to a growing volume of cases and the need for more comprehensive diagnoses. We developed an artificial intelligence (AI) model for triaging cutaneous melanocytic lesions based on whole slide images.
  arXiv  Detail & Related papers  (2024-10-14T13:49:04Z)
• Systematic Evaluation of Neural Retrieval Models on the Touché 2020 Argument Retrieval Subset of BEIR [99.13855300096925]
  We run a study on the Touch'e 2020 data to explore the potential limits of neural retrieval models. Our black-box evaluation reveals an inherent bias of neural models towards retrieving short passages. As many of the short Touch'e passages are not argumentative and thus non-relevant per se, we denoise the Touch'e 2020 data by excluding very short passages.
  arXiv  Detail & Related papers  (2024-07-10T16:07:51Z)
• Towards a Holistic Framework for Multimodal Large Language Models in Three-dimensional Brain CT Report Generation [42.06416052431378]
  2D radiology captioning is incompetent to reflect the real-world diagnostic challenge in the volumetric 3D anatomy. We collected an 18,885 text-scan pairs 3D-BrainCT dataset and applied clinical visual instruction tuning to train BrainGPT models to generate radiology-adherent 3D brain CT reports. Our work embodies a holistic framework that showcased the first-hand experience of curating a 3D brain CT dataset, fine-tuning anatomy-sensible language models, and proposing robust radiology evaluation metrics.
  arXiv  Detail & Related papers  (2024-07-02T12:58:35Z)
• A self-supervised learning strategy for postoperative brain cavity segmentation simulating resections [46.414990784180546]
  Convolutional neural networks (CNNs) are the state-of-the-art image segmentation technique. CNNs require large annotated datasets for training. Self-supervised learning strategies can leverage unlabeled data for training.
  arXiv  Detail & Related papers  (2021-05-24T12:27:06Z)
• Handling Missing MRI Input Data in Deep Learning Segmentation of Brain Metastases: A Multi-Center Study [1.4463443378902883]
  A deep learning based segmentation model for automatic segmentation of brain metastases, named DropOut, was trained on multi-sequence MRI. The segmentation results were compared with the performance of a state-of-the-art DeepLabV3 model. The DropOut model showed a significantly higher score compared to the DeepLabV3 model.
  arXiv  Detail & Related papers  (2019-12-27T02:49:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.