Generalized Wasserstein Dice Score, Distributionally Robust Deep
Learning, and Ranger for brain tumor segmentation: BraTS 2020 challenge
- URL: http://arxiv.org/abs/2011.01614v2
- Date: Mon, 25 Jan 2021 11:41:03 GMT
- Title: Generalized Wasserstein Dice Score, Distributionally Robust Deep
Learning, and Ranger for brain tumor segmentation: BraTS 2020 challenge
- Authors: Lucas Fidon and Sebastien Ourselin and Tom Vercauteren
- Abstract summary: Training a deep neural network is an optimization problem with four main ingredients: the design of the deep neural network, the per-sample loss function, the population loss function, and the architectures.
We experimented with adopting the opposite approach, focusing only on the design of the deep neural network.
With an ensemble of three deep neural networks trained with various optimization procedures, we achieved promising results on the validation of the BraTS 2020 challenge.
- Score: 4.787559241352888
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Training a deep neural network is an optimization problem with four main
ingredients: the design of the deep neural network, the per-sample loss
function, the population loss function, and the optimizer. However, methods
developed to compete in recent BraTS challenges tend to focus only on the
design of deep neural network architectures, while paying less attention to the
three other aspects. In this paper, we experimented with adopting the opposite
approach. We stuck to a generic and state-of-the-art 3D U-Net architecture and
experimented with a non-standard per-sample loss function, the generalized
Wasserstein Dice loss, a non-standard population loss function, corresponding
to distributionally robust optimization, and a non-standard optimizer, Ranger.
Those variations were selected specifically for the problem of multi-class
brain tumor segmentation. The generalized Wasserstein Dice loss is a per-sample
loss function that allows taking advantage of the hierarchical structure of the
tumor regions labeled in BraTS. Distributionally robust optimization is a
generalization of empirical risk minimization that accounts for the presence of
underrepresented subdomains in the training dataset. Ranger is a generalization
of the widely used Adam optimizer that is more stable with small batch size and
noisy labels. We found that each of those variations of the optimization of
deep neural networks for brain tumor segmentation leads to improvements in
terms of Dice scores and Hausdorff distances. With an ensemble of three deep
neural networks trained with various optimization procedures, we achieved
promising results on the validation dataset of the BraTS 2020 challenge. Our
ensemble ranked fourth out of the 693 registered teams for the segmentation
task of the BraTS 2020 challenge.
Related papers
- DiTMoS: Delving into Diverse Tiny-Model Selection on Microcontrollers [34.282971510732736]
We introduce DiTMoS, a novel DNN training and inference framework with a selector-classifiers architecture.
A composition of weak models can exhibit high diversity and the union of them can significantly boost the accuracy upper bound.
We deploy DiTMoS on the Neucleo STM32F767ZI board and evaluate it based on three time-series datasets for human activity recognition, keywords spotting, and emotion recognition.
arXiv Detail & Related papers (2024-03-14T02:11:38Z) - Universal Consistency of Wide and Deep ReLU Neural Networks and Minimax
Optimal Convergence Rates for Kolmogorov-Donoho Optimal Function Classes [7.433327915285969]
We prove the universal consistency of wide and deep ReLU neural network classifiers trained on the logistic loss.
We also give sufficient conditions for a class of probability measures for which classifiers based on neural networks achieve minimax optimal rates of convergence.
arXiv Detail & Related papers (2024-01-08T23:54:46Z) - Pruning Convolutional Filters via Reinforcement Learning with Entropy
Minimization [0.0]
We introduce a novel information-theoretic reward function which minimizes the spatial entropy of convolutional activations.
Our method shows that there is another possibility to preserve accuracy without the need to directly optimize it in the agent's reward function.
arXiv Detail & Related papers (2023-12-08T09:34:57Z) - A Continuous Optimisation Benchmark Suite from Neural Network Regression [0.0]
Training neural networks is an optimisation task that has gained prominence with the recent successes of deep learning.
gradient descent variants are by far the most common choice with their trusted good performance on large-scale machine learning tasks.
We contribute CORNN, a suite for benchmarking the performance of any continuous black-box algorithm on neural network training problems.
arXiv Detail & Related papers (2021-09-12T20:24:11Z) - Understanding the Generalization of Adam in Learning Neural Networks
with Proper Regularization [118.50301177912381]
We show that Adam can converge to different solutions of the objective with provably different errors, even with weight decay globalization.
We show that if convex, and the weight decay regularization is employed, any optimization algorithms including Adam will converge to the same solution.
arXiv Detail & Related papers (2021-08-25T17:58:21Z) - Non-Gradient Manifold Neural Network [79.44066256794187]
Deep neural network (DNN) generally takes thousands of iterations to optimize via gradient descent.
We propose a novel manifold neural network based on non-gradient optimization.
arXiv Detail & Related papers (2021-06-15T06:39:13Z) - Topological obstructions in neural networks learning [67.8848058842671]
We study global properties of the loss gradient function flow.
We use topological data analysis of the loss function and its Morse complex to relate local behavior along gradient trajectories with global properties of the loss surface.
arXiv Detail & Related papers (2020-12-31T18:53:25Z) - Attribute-Guided Adversarial Training for Robustness to Natural
Perturbations [64.35805267250682]
We propose an adversarial training approach which learns to generate new samples so as to maximize exposure of the classifier to the attributes-space.
Our approach enables deep neural networks to be robust against a wide range of naturally occurring perturbations.
arXiv Detail & Related papers (2020-12-03T10:17:30Z) - Brain tumor segmentation with self-ensembled, deeply-supervised 3D U-net
neural networks: a BraTS 2020 challenge solution [56.17099252139182]
We automate and standardize the task of brain tumor segmentation with U-net like neural networks.
Two independent ensembles of models were trained, and each produced a brain tumor segmentation map.
Our solution achieved a Dice of 0.79, 0.89 and 0.84, as well as Hausdorff 95% of 20.4, 6.7 and 19.5mm on the final test dataset.
arXiv Detail & Related papers (2020-10-30T14:36:10Z) - The Hidden Convex Optimization Landscape of Two-Layer ReLU Neural
Networks: an Exact Characterization of the Optimal Solutions [51.60996023961886]
We prove that finding all globally optimal two-layer ReLU neural networks can be performed by solving a convex optimization program with cone constraints.
Our analysis is novel, characterizes all optimal solutions, and does not leverage duality-based analysis which was recently used to lift neural network training into convex spaces.
arXiv Detail & Related papers (2020-06-10T15:38:30Z) - ODEN: A Framework to Solve Ordinary Differential Equations using
Artificial Neural Networks [0.0]
We prove a specific loss function, which does not require knowledge of the exact solution, to evaluate neural networks' performance.
Neural networks are shown to be proficient at approximating continuous solutions within their training domains.
A user-friendly and adaptable open-source code (ODE$mathcalN$) is provided on GitHub.
arXiv Detail & Related papers (2020-05-28T15:34:10Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.