Rethinking Evaluation of Multiple Sclerosis (MS) Lesion Segmentation Models

• URL: http://arxiv.org/abs/2605.09666v1
• Date: Sun, 10 May 2026 17:31:21 GMT
• Title: Rethinking Evaluation of Multiple Sclerosis (MS) Lesion Segmentation Models
• Abstract summary: Multiple Sclerosis (MS) is a chronic autoimmune disease that can significantly reduce the quality of life of a patient.<n>Deep learning offers state-of-the-art models for detecting and segmenting MS lesions in brain MRI scans.
• Score: 4.724523760253738
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multiple Sclerosis (MS) is a chronic autoimmune disease that can significantly reduce the quality of life of a patient. Existing treatment options can only help slow down the progression of the disease. Therefore, early detection and precise monitoring of disease progression are important. Deep learning offers state-of-the-art models for detecting and segmenting MS lesions in brain MRI scans. However, most of these models are evaluated using the Dice score, without accounting for lesion-wise detection and segmentation performance or other metrics that quantify model performance in cases that are complex or confusing for human annotators, or in cases that are essential for disease detection and progression monitoring. In this paper, we highlight the need to rethink the evaluation of MS lesion segmentation models. In this context, we first present problem fingerprinting in detail to highlight what neurologists look for in brain MRI scans for MS detection and progression monitoring, and which metrics are required to properly quantify model performance in these contexts. Additionally, we present an analysis of state-of-the-art models on two open-source datasets using these metrics to highlight their usability for real-world deployment in hospitals.

Related papers

• BenchX: Benchmarking AI Models for Cancer Detection and Localization with Demographic and Protocol Biases [59.19934491064188]
  Artificial intelligence (AI) has achieved remarkable success in medical imaging.<n>These models often perform inconsistently across real-world clinical settings.<n>We develop a benchmark of 85,355 CT scans that systematically evaluates 12 tumor-detection AI models.
  arXiv  Detail & Related papers  (2026-06-23T17:58:59Z)
• UCSF-PDGM-VQA: Visual Question Answering dataset for brain tumor MRI interpretation [1.2006896500048554]
  Brain tumor diagnosis is largely dependent on Magnetic Resonance Imaging (MRI) evaluation.<n>Despite increasing demands in radiology, this expertise is difficult to scale, straining the current health systems.<n>Vision-Language Models (VLMs) provide an opportunity to reduce this burden through a semi-automated, interactive interpretation of complex brain MRIs.
  arXiv  Detail & Related papers  (2026-05-16T20:10:19Z)
• Automated Lesion Segmentation of Stroke MRI Using nnU-Net: A Comprehensive External Validation Across Acute and Chronic Lesions [0.0]
  We evaluate stroke lesion segmentation using the nnU-Net framework across multiple publicly available MRI datasets.<n>Across stroke stages, models showed robust generalisation, with segmentation accuracy approaching reported inter-rater reliability.<n>In acute stroke, DWI-trained models consistently outperformed FLAIR-based models, with only modest gains from multimodal combinations.<n>In chronic stroke, increasing training set size improved performance, with diminishing returns beyond several hundred cases.
  arXiv  Detail & Related papers  (2026-01-13T16:29:20Z)
• An Explainable Hybrid AI Framework for Enhanced Tuberculosis and Symptom Detection [55.35661671061754]
  Tuberculosis remains a critical global health issue, particularly in resource-limited and remote areas.<n>We propose a framework which enhances disease and symptom detection on chest X-rays by integrating two supervised heads and a self-supervised head.<n>Our model achieves an accuracy of 98.85% for distinguishing between COVID-19, tuberculosis, and normal cases, and a macro-F1 score of 90.09% for multilabel symptom detection.
  arXiv  Detail & Related papers  (2025-10-21T17:18:55Z)
• Metrics that matter: Evaluating image quality metrics for medical image generation [48.85783422900129]
  This study comprehensively assesses commonly used no-reference image quality metrics using brain MRI data.<n>We evaluate metric sensitivity to a range of challenges, including noise, distribution shifts, and, critically, morphological alterations designed to mimic clinically relevant inaccuracies.
  arXiv  Detail & Related papers  (2025-05-12T01:57:25Z)
• Anatomical Foundation Models for Brain MRIs [6.993491018326816]
  We propose AnatCL, an anatomical foundation model for brain MRIs.<n>We consider 12 different downstream tasks for the diagnosis of different conditions.<n>Our findings show that incorporating anatomical information during pre-training leads to more robust and general representations.
  arXiv  Detail & Related papers  (2024-08-07T14:04:50Z)
• Potential of Multimodal Large Language Models for Data Mining of Medical Images and Free-text Reports [51.45762396192655]
  Multimodal large language models (MLLMs) have recently transformed many domains, significantly affecting the medical field. Notably, Gemini-Vision-series (Gemini) and GPT-4-series (GPT-4) models have epitomized a paradigm shift in Artificial General Intelligence for computer vision. This study evaluated the performance of the Gemini, GPT-4, and 4 popular large models for an exhaustive evaluation across 14 medical imaging datasets.
  arXiv  Detail & Related papers  (2024-07-08T09:08:42Z)
• TADM: Temporally-Aware Diffusion Model for Neurodegenerative Progression on Brain MRI [4.414541804340033]
  Temporally-Aware Diffusion Model (TADM) learns the distribution of structural changes in terms of intensity differences between scans. During training, we propose to leverage a pre-trained Brain-Age Estimator (BAE) to refine the model's training process. Our approach will benefit applications, such as predicting patient outcomes or improving treatments for patients.
  arXiv  Detail & Related papers  (2024-06-18T09:00:49Z)
• Cross-modality Guidance-aided Multi-modal Learning with Dual Attention for MRI Brain Tumor Grading [47.50733518140625]
  Brain tumor represents one of the most fatal cancers around the world, and is very common in children and the elderly. We propose a novel cross-modality guidance-aided multi-modal learning with dual attention for addressing the task of MRI brain tumor grading.
  arXiv  Detail & Related papers  (2024-01-17T07:54:49Z)
• Patched Diffusion Models for Unsupervised Anomaly Detection in Brain MRI [55.78588835407174]
  We propose a method that reformulates the generation task of diffusion models as a patch-based estimation of healthy brain anatomy. We evaluate our approach on data of tumors and multiple sclerosis lesions and demonstrate a relative improvement of 25.1% compared to existing baselines.
  arXiv  Detail & Related papers  (2023-03-07T09:40:22Z)
• Benchmarking Continuous Time Models for Predicting Multiple Sclerosis Progression [46.394865849252696]
  Multiple sclerosis is a disease that affects the brain and spinal cord, it can lead to severe disability and has no known cure. In a recent paper it was shown that disease progression can be predicted effectively using performance outcome measures and demographic data. We benchmark four continuous time models using a publicly available multiple sclerosis dataset. We find that the best continuous model is often able to outperform the best benchmarked discrete time model.
  arXiv  Detail & Related papers  (2023-02-15T18:45:32Z)
• Lesion-Specific Prediction with Discriminator-Based Supervised Guided Attention Module Enabled GANs in Multiple Sclerosis [0.0]
  Multiple Sclerosis (MS) is a chronic neurological condition characterized by the development of lesions in the white matter of the brain. In this study, we propose a novel modification to generative adversarial networks (GANs) to predict future lesion-specific FLAIR MRI for MS at fixed time intervals.
  arXiv  Detail & Related papers  (2022-08-30T20:37:38Z)
• Data and Physics Driven Learning Models for Fast MRI -- Fundamentals and Methodologies from CNN, GAN to Attention and Transformers [72.047680167969]
  This article aims to introduce the deep learning based data driven techniques for fast MRI including convolutional neural network and generative adversarial network based methods. We will detail the research in coupling physics and data driven models for MRI acceleration. Finally, we will demonstrate through a few clinical applications, explain the importance of data harmonisation and explainable models for such fast MRI techniques in multicentre and multi-scanner studies.
  arXiv  Detail & Related papers  (2022-04-01T22:48:08Z)
• Multiple Sclerosis Lesion Analysis in Brain Magnetic Resonance Images: Techniques and Clinical Applications [22.410543483471915]
  Multiple sclerosis (MS) is a chronic inflammatory and degenerative disease of the central nervous system. Traditionally, MS lesions have been manually annotated on 2D MRI slices. Deep learning techniques have achieved remarkable breakthroughs in the MS lesion segmentation task.
  arXiv  Detail & Related papers  (2021-04-20T15:08:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.