Partly Supervised Multitask Learning
- URL: http://arxiv.org/abs/2005.02523v1
- Date: Tue, 5 May 2020 22:42:12 GMT
- Title: Partly Supervised Multitask Learning
- Authors: Abdullah-Al-Zubaer Imran, Chao Huang, Hui Tang, Wei Fan, Yuan Xiao,
Dingjun Hao, Zhen Qian, Demetri Terzopoulos
- Abstract summary: Experimental results on chest and spine X-ray datasets suggest that our S$4$MTL model significantly outperforms semi-supervised single task, semi/fully-supervised multitask, and fully-supervised single task models.
We hypothesize that our proposed model can be effective in tackling limited annotation problems for joint training, not only in medical imaging domains, but also for general-purpose vision tasks.
- Score: 19.64371980996412
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Semi-supervised learning has recently been attracting attention as an
alternative to fully supervised models that require large pools of labeled
data. Moreover, optimizing a model for multiple tasks can provide better
generalizability than single-task learning. Leveraging self-supervision and
adversarial training, we propose a novel general purpose semi-supervised,
multiple-task model---namely, self-supervised, semi-supervised, multitask
learning (S$^4$MTL)---for accomplishing two important tasks in medical imaging,
segmentation and diagnostic classification. Experimental results on chest and
spine X-ray datasets suggest that our S$^4$MTL model significantly outperforms
semi-supervised single task, semi/fully-supervised multitask, and
fully-supervised single task models, even with a 50\% reduction of class and
segmentation labels. We hypothesize that our proposed model can be effective in
tackling limited annotation problems for joint training, not only in medical
imaging domains, but also for general-purpose vision tasks.
Related papers
- Overcoming Data Scarcity in Biomedical Imaging with a Foundational
Multi-Task Model [2.5994154212235685]
Foundational models, pretrained on a large scale, have demonstrated substantial success across non-medical domains.
Here, we propose a multi-task learning strategy that decouples the number of training tasks from memory requirements.
arXiv Detail & Related papers (2023-11-16T12:20:25Z) - AdaMerging: Adaptive Model Merging for Multi-Task Learning [68.75885518081357]
This paper introduces an innovative technique called Adaptive Model Merging (AdaMerging)
It aims to autonomously learn the coefficients for model merging, either in a task-wise or layer-wise manner, without relying on the original training data.
Compared to the current state-of-the-art task arithmetic merging scheme, AdaMerging showcases a remarkable 11% improvement in performance.
arXiv Detail & Related papers (2023-10-04T04:26:33Z) - Many tasks make light work: Learning to localise medical anomalies from
multiple synthetic tasks [2.912977051718473]
A growing interest in single-class modelling and out-of-distribution detection.
Fully supervised machine learning models cannot reliably identify classes not included in their training.
We make use of multiple visually-distinct synthetic anomaly learning tasks for both training and validation.
arXiv Detail & Related papers (2023-07-03T09:52:54Z) - An Efficient General-Purpose Modular Vision Model via Multi-Task
Heterogeneous Training [79.78201886156513]
We present a model that can perform multiple vision tasks and can be adapted to other downstream tasks efficiently.
Our approach achieves comparable results to single-task state-of-the-art models and demonstrates strong generalization on downstream tasks.
arXiv Detail & Related papers (2023-06-29T17:59:57Z) - Composite Learning for Robust and Effective Dense Predictions [81.2055761433725]
Multi-task learning promises better model generalization on a target task by jointly optimizing it with an auxiliary task.
We find that jointly training a dense prediction (target) task with a self-supervised (auxiliary) task can consistently improve the performance of the target task, while eliminating the need for labeling auxiliary tasks.
arXiv Detail & Related papers (2022-10-13T17:59:16Z) - Effective Adaptation in Multi-Task Co-Training for Unified Autonomous
Driving [103.745551954983]
In this paper, we investigate the transfer performance of various types of self-supervised methods, including MoCo and SimCLR, on three downstream tasks.
We find that their performances are sub-optimal or even lag far behind the single-task baseline.
We propose a simple yet effective pretrain-adapt-finetune paradigm for general multi-task training.
arXiv Detail & Related papers (2022-09-19T12:15:31Z) - Explaining the Effectiveness of Multi-Task Learning for Efficient
Knowledge Extraction from Spine MRI Reports [2.5953185061765884]
We show that a single multi-tasking model can match the performance of task specific models.
We validate our observations on our internal radiologist-annotated datasets on the cervical and lumbar spine.
arXiv Detail & Related papers (2022-05-06T01:51:19Z) - SUPERB-SG: Enhanced Speech processing Universal PERformance Benchmark
for Semantic and Generative Capabilities [76.97949110580703]
We introduce SUPERB-SG, a new benchmark to evaluate pre-trained models across various speech tasks.
We use a lightweight methodology to test the robustness of representations learned by pre-trained models under shifts in data domain.
We also show that the task diversity of SUPERB-SG coupled with limited task supervision is an effective recipe for evaluating the generalizability of model representation.
arXiv Detail & Related papers (2022-03-14T04:26:40Z) - Multi-Task Self-Training for Learning General Representations [97.01728635294879]
Multi-task self-training (MuST) harnesses the knowledge in independent specialized teacher models to train a single general student model.
MuST is scalable with unlabeled or partially labeled datasets and outperforms both specialized supervised models and self-supervised models when training on large scale datasets.
arXiv Detail & Related papers (2021-08-25T17:20:50Z) - MultiMix: Sparingly Supervised, Extreme Multitask Learning From Medical
Images [13.690075845927606]
We propose a novel multitask learning model, namely MultiMix, which jointly learns disease classification and anatomical segmentation in a sparingly supervised manner.
Our experiments justify the effectiveness of our multitasking model for the classification of pneumonia and segmentation of lungs from chest X-ray images.
arXiv Detail & Related papers (2020-10-28T03:47:29Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.