Applications of Sequential Learning for Medical Image Classification
- URL: http://arxiv.org/abs/2309.14591v1
- Date: Tue, 26 Sep 2023 00:46:25 GMT
- Title: Applications of Sequential Learning for Medical Image Classification
- Authors: Sohaib Naim and Brian Caffo and Haris I Sair and Craig K Jones
- Abstract summary: We develop a neural network training framework for continual training of small amounts of medical imaging data.
We address problems that impede sequential learning such as overfitting, catastrophic forgetting, and concept drift.
- Score: 0.13191970195165517
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Purpose: The aim of this work is to develop a neural network training
framework for continual training of small amounts of medical imaging data and
create heuristics to assess training in the absence of a hold-out validation or
test set.
Materials and Methods: We formulated a retrospective sequential learning
approach that would train and consistently update a model on mini-batches of
medical images over time. We address problems that impede sequential learning
such as overfitting, catastrophic forgetting, and concept drift through PyTorch
convolutional neural networks (CNN) and publicly available Medical MNIST and
NIH Chest X-Ray imaging datasets. We begin by comparing two methods for a
sequentially trained CNN with and without base pre-training. We then transition
to two methods of unique training and validation data recruitment to estimate
full information extraction without overfitting. Lastly, we consider an example
of real-life data that shows how our approach would see mainstream research
implementation.
Results: For the first experiment, both approaches successfully reach a ~95%
accuracy threshold, although the short pre-training step enables sequential
accuracy to plateau in fewer steps. The second experiment comparing two methods
showed better performance with the second method which crosses the ~90%
accuracy threshold much sooner. The final experiment showed a slight advantage
with a pre-training step that allows the CNN to cross ~60% threshold much
sooner than without pre-training.
Conclusion: We have displayed sequential learning as a serviceable
multi-classification technique statistically comparable to traditional CNNs
that can acquire data in small increments feasible for clinically realistic
scenarios.
Related papers
- Boosting Few-Shot Learning with Disentangled Self-Supervised Learning and Meta-Learning for Medical Image Classification [8.975676404678374]
We present a strategy for improving the performance and generalization capabilities of models trained in low-data regimes.
The proposed method starts with a pre-training phase, where features learned in a self-supervised learning setting are disentangled to improve the robustness of the representations for downstream tasks.
We then introduce a meta-fine-tuning step, leveraging related classes between meta-training and meta-testing phases but varying the level.
arXiv Detail & Related papers (2024-03-26T09:36:20Z) - Are Deep Learning Classification Results Obtained on CT Scans Fair and
Interpretable? [0.0]
Most lung nodule classification papers using deep learning randomly shuffle data and split it into training, validation, and test sets.
In contrast, deep neural networks trained with strict patient-level separation maintain their accuracy rates even when new patient images are tested.
Heat-map visualizations of the activations of the deep neural networks trained with strict patient-level separation indicate a higher degree of focus on the relevant nodules.
arXiv Detail & Related papers (2023-09-22T05:57:25Z) - Self-Supervised Pre-Training with Contrastive and Masked Autoencoder
Methods for Dealing with Small Datasets in Deep Learning for Medical Imaging [8.34398674359296]
Deep learning in medical imaging has the potential to minimize the risk of diagnostic errors, reduce radiologist workload, and accelerate diagnosis.
Training such deep learning models requires large and accurate datasets, with annotations for all training samples.
To address this challenge, deep learning models can be pre-trained on large image datasets without annotations using methods from the field of self-supervised learning.
arXiv Detail & Related papers (2023-08-12T11:31:01Z) - Transferring Learning Trajectories of Neural Networks [2.2299983745857896]
Training deep neural networks (DNNs) is computationally expensive.
We formulate the problem of "transferring" a given learning trajectory from one initial parameter to another one.
We empirically show that the transferred parameters achieve non-trivial accuracy before any direct training, and can be trained significantly faster than training from scratch.
arXiv Detail & Related papers (2023-05-23T14:46:32Z) - TWINS: A Fine-Tuning Framework for Improved Transferability of
Adversarial Robustness and Generalization [89.54947228958494]
This paper focuses on the fine-tuning of an adversarially pre-trained model in various classification tasks.
We propose a novel statistics-based approach, Two-WIng NormliSation (TWINS) fine-tuning framework.
TWINS is shown to be effective on a wide range of image classification datasets in terms of both generalization and robustness.
arXiv Detail & Related papers (2023-03-20T14:12:55Z) - Neural Maximum A Posteriori Estimation on Unpaired Data for Motion
Deblurring [87.97330195531029]
We propose a Neural Maximum A Posteriori (NeurMAP) estimation framework for training neural networks to recover blind motion information and sharp content from unpaired data.
The proposed NeurMAP is an approach to existing deblurring neural networks, and is the first framework that enables training image deblurring networks on unpaired datasets.
arXiv Detail & Related papers (2022-04-26T08:09:47Z) - Jigsaw Clustering for Unsupervised Visual Representation Learning [68.09280490213399]
We propose a new jigsaw clustering pretext task in this paper.
Our method makes use of information from both intra- and inter-images.
It is even comparable to the contrastive learning methods when only half of training batches are used.
arXiv Detail & Related papers (2021-04-01T08:09:26Z) - Self-Supervised Pretraining Improves Self-Supervised Pretraining [83.1423204498361]
Self-supervised pretraining requires expensive and lengthy computation, large amounts of data, and is sensitive to data augmentation.
This paper explores Hierarchical PreTraining (HPT), which decreases convergence time and improves accuracy by initializing the pretraining process with an existing pretrained model.
We show HPT converges up to 80x faster, improves accuracy across tasks, and improves the robustness of the self-supervised pretraining process to changes in the image augmentation policy or amount of pretraining data.
arXiv Detail & Related papers (2021-03-23T17:37:51Z) - Learning Neural Network Subspaces [74.44457651546728]
Recent observations have advanced our understanding of the neural network optimization landscape.
With a similar computational cost as training one model, we learn lines, curves, and simplexes of high-accuracy neural networks.
With a similar computational cost as training one model, we learn lines, curves, and simplexes of high-accuracy neural networks.
arXiv Detail & Related papers (2021-02-20T23:26:58Z) - Deep Learning in Diabetic Foot Ulcers Detection: A Comprehensive
Evaluation [14.227261503586599]
This paper summarises the results of DFUC 2020 by comparing the deep learning-based algorithms proposed by the winning teams.
The best performance was obtained from Deformable Convolution, a variant of Faster R-CNN, with a mean average precision (mAP) of 0.6940 and an F1-Score of 0.7434.
arXiv Detail & Related papers (2020-10-07T11:31:27Z) - Omni-supervised Facial Expression Recognition via Distilled Data [120.11782405714234]
We propose omni-supervised learning to exploit reliable samples in a large amount of unlabeled data for network training.
We experimentally verify that the new dataset can significantly improve the ability of the learned FER model.
To tackle this, we propose to apply a dataset distillation strategy to compress the created dataset into several informative class-wise images.
arXiv Detail & Related papers (2020-05-18T09:36:51Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.