Optimization for Medical Image Segmentation: Theory and Practice when evaluating with Dice Score or Jaccard Index

• URL: http://arxiv.org/abs/2010.13499v1
• Date: Mon, 26 Oct 2020 11:45:55 GMT
• Title: Optimization for Medical Image Segmentation: Theory and Practice when evaluating with Dice Score or Jaccard Index
• Authors: Tom Eelbode, Jeroen Bertels, Maxim Berman, Dirk Vandermeulen, Frederik Maes, Raf Bisschops, Matthew B. Blaschko
• Abstract summary: We investigate the relation within the group of metric-sensitive loss functions. We find that the Dice score and Jaccard index approximate each other relatively and absolutely. We verify these results empirically in an extensive validation on six medical segmentation tasks.
• Score: 25.04858968806884
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In many medical imaging and classical computer vision tasks, the Dice score and Jaccard index are used to evaluate the segmentation performance. Despite the existence and great empirical success of metric-sensitive losses, i.e. relaxations of these metrics such as soft Dice, soft Jaccard and Lovasz-Softmax, many researchers still use per-pixel losses, such as (weighted) cross-entropy to train CNNs for segmentation. Therefore, the target metric is in many cases not directly optimized. We investigate from a theoretical perspective, the relation within the group of metric-sensitive loss functions and question the existence of an optimal weighting scheme for weighted cross-entropy to optimize the Dice score and Jaccard index at test time. We find that the Dice score and Jaccard index approximate each other relatively and absolutely, but we find no such approximation for a weighted Hamming similarity. For the Tversky loss, the approximation gets monotonically worse when deviating from the trivial weight setting where soft Tversky equals soft Dice. We verify these results empirically in an extensive validation on six medical segmentation tasks and can confirm that metric-sensitive losses are superior to cross-entropy based loss functions in case of evaluation with Dice Score or Jaccard Index. This further holds in a multi-class setting, and across different object sizes and foreground/background ratios. These results encourage a wider adoption of metric-sensitive loss functions for medical segmentation tasks where the performance measure of interest is the Dice score or Jaccard index.

Related papers

• Semiparametric conformal prediction [79.6147286161434]
  Risk-sensitive applications require well-calibrated prediction sets over multiple, potentially correlated target variables. We treat the scores as random vectors and aim to construct the prediction set accounting for their joint correlation structure. We report desired coverage and competitive efficiency on a range of real-world regression problems.
  arXiv  Detail & Related papers  (2024-11-04T14:29:02Z)
• Revisiting Evaluation Metrics for Semantic Segmentation: Optimization and Evaluation of Fine-grained Intersection over Union [113.20223082664681]
  We propose the use of fine-grained mIoUs along with corresponding worst-case metrics. These fine-grained metrics offer less bias towards large objects, richer statistical information, and valuable insights into model and dataset auditing. Our benchmark study highlights the necessity of not basing evaluations on a single metric and confirms that fine-grained mIoUs reduce the bias towards large objects.
  arXiv  Detail & Related papers  (2023-10-30T03:45:15Z)
• Robust T-Loss for Medical Image Segmentation [56.524774292536264]
  This paper presents a new robust loss function, the T-Loss, for medical image segmentation. The proposed loss is based on the negative log-likelihood of the Student-t distribution and can effectively handle outliers in the data. Our experiments show that the T-Loss outperforms traditional loss functions in terms of dice scores on two public medical datasets.
  arXiv  Detail & Related papers  (2023-06-01T14:49:40Z)
• A Generalized Surface Loss for Reducing the Hausdorff Distance in Medical Imaging Segmentation [1.2289361708127877]
  We propose a novel loss function to minimize Hausdorff-based metrics with more desirable numerical properties than current methods. Our loss function outperforms other losses when tested on the LiTS and BraTS datasets using the state-of-the-art nnUNet architecture.
  arXiv  Detail & Related papers  (2023-02-08T04:01:42Z)
• Theoretical analysis and experimental validation of volume bias of soft Dice optimized segmentation maps in the context of inherent uncertainty [6.692460499366963]
  Recent segmentation methods use a differentiable surrogate metric, such as soft Dice, as part of the loss function during the learning phase. We first briefly describe how to derive volume estimates from a segmentation that is, potentially, inherently uncertain or ambiguous. We find that, even though soft Dice optimization leads to an improved performance with respect to the Dice score and other measures, it may introduce a volume bias for tasks with high inherent uncertainty.
  arXiv  Detail & Related papers  (2022-11-08T11:04:52Z)
• Optimizing Partial Area Under the Top-k Curve: Theory and Practice [151.5072746015253]
  We develop a novel metric named partial Area Under the top-k Curve (AUTKC) AUTKC has a better discrimination ability, and its Bayes optimal score function could give a correct top-K ranking with respect to the conditional probability. We present an empirical surrogate risk minimization framework to optimize the proposed metric.
  arXiv  Detail & Related papers  (2022-09-03T11:09:13Z)
• Impact of loss function in Deep Learning methods for accurate retinal vessel segmentation [1.1470070927586016]
  We compare Binary Cross Entropy, Dice, Tversky, and Combo loss using the deep learning architectures (i.e. U-Net, Attention U-Net, and Nested UNet) with the DRIVE dataset. The results showed that there is a significant difference in the selection of loss function.
  arXiv  Detail & Related papers  (2022-06-01T14:47:18Z)
• Recall@k Surrogate Loss with Large Batches and Similarity Mixup [62.67458021725227]
  Direct optimization, by gradient descent, of an evaluation metric is not possible when it is non-differentiable. In this work, a differentiable surrogate loss for the recall is proposed. The proposed method achieves state-of-the-art results in several image retrieval benchmarks.
  arXiv  Detail & Related papers  (2021-08-25T11:09:11Z)
• Segmentation with Multiple Acceptable Annotations: A Case Study of Myocardial Segmentation in Contrast Echocardiography [12.594060034146125]
  We propose a new extended Dice to evaluate segmentation performance when multiple accepted ground truth is available. We then solve the second problem by further incorporating the new metric into a loss function that enables neural networks to learn general features of myocardium. Experiment results on our clinical MCE data set demonstrate that the neural network trained with the proposed loss function outperforms those existing ones.
  arXiv  Detail & Related papers  (2021-06-29T17:32:24Z)
• Matthews Correlation Coefficient Loss for Deep Convolutional Networks: Application to Skin Lesion Segmentation [19.673662082910766]
  Deep learning-based models are susceptible to class imbalance in the data. We propose a novel metric-based loss function using the Matthews correlation coefficient, a metric that has been shown to be efficient in scenarios with skewed class distributions. We show that the proposed loss function outperform those trained using Dice loss by 11.25%, 4.87%, and 0.76% respectively in the mean Jaccard index.
  arXiv  Detail & Related papers  (2020-10-26T09:50:25Z)
• DONet: Dual Objective Networks for Skin Lesion Segmentation [77.9806410198298]
  We propose a simple yet effective framework, named Dual Objective Networks (DONet), to improve the skin lesion segmentation. Our DONet adopts two symmetric decoders to produce different predictions for approaching different objectives. To address the challenge of large variety of lesion scales and shapes in dermoscopic images, we additionally propose a recurrent context encoding module (RCEM)
  arXiv  Detail & Related papers  (2020-08-19T06:02:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.