Design Perspectives of Multitask Deep Learning Models and Applications

• URL: http://arxiv.org/abs/2209.13444v1
• Date: Tue, 27 Sep 2022 15:04:31 GMT
• Title: Design Perspectives of Multitask Deep Learning Models and Applications
• Authors: Yeshwant Singh, Anupam Biswas, Angshuman Bora, Debashish Malakar, Subham Chakraborty, Suman Bera
• Abstract summary: Multi-task learning has been able to generalize the models even better. We try to enhance the feature mapping of the multi-tasking models by sharing features among related tasks. Also, our interest is in learning the task relationships among various tasks for acquiring better benefits from multi-task learning.
• Score: 1.3701366534590496
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: In recent years, multi-task learning has turned out to be of great success in various applications. Though single model training has promised great results throughout these years, it ignores valuable information that might help us estimate a metric better. Under learning-related tasks, multi-task learning has been able to generalize the models even better. We try to enhance the feature mapping of the multi-tasking models by sharing features among related tasks and inductive transfer learning. Also, our interest is in learning the task relationships among various tasks for acquiring better benefits from multi-task learning. In this chapter, our objective is to visualize the existing multi-tasking models, compare their performances, the methods used to evaluate the performance of the multi-tasking models, discuss the problems faced during the design and implementation of these models in various domains, and the advantages and milestones achieved by them

Related papers

• LIMT: Language-Informed Multi-Task Visual World Models [6.128332310539627]
  Multi-task reinforcement learning can be very challenging due to the increased sample complexity and the potentially conflicting task objectives. We propose a method for learning multi-task visual world models, leveraging pre-trained language models to extract semantically meaningful task representations. Our results highlight the benefits of using language-driven task representations for world models and a clear advantage of model-based multi-task learning over the more common model-free paradigm.
  arXiv  Detail & Related papers  (2024-07-18T12:40:58Z)
• Sparsely Activated Mixture-of-Experts are Robust Multi-Task Learners [67.5865966762559]
  We study whether sparsely activated Mixture-of-Experts (MoE) improve multi-task learning. We devise task-aware gating functions to route examples from different tasks to specialized experts. This results in a sparsely activated multi-task model with a large number of parameters, but with the same computational cost as that of a dense model.
  arXiv  Detail & Related papers  (2022-04-16T00:56:12Z)
• Multi-Task Learning for Visual Scene Understanding [7.191593674138455]
  This thesis is concerned with multi-task learning in the context of computer vision. We propose several methods that tackle important aspects of multi-task learning. The results show several advances in the state-of-the-art of multi-task learning.
  arXiv  Detail & Related papers  (2022-03-28T16:57:58Z)
• Modular Adaptive Policy Selection for Multi-Task Imitation Learning through Task Division [60.232542918414985]
  Multi-task learning often suffers from negative transfer, sharing information that should be task-specific. This is done by using proto-policies as modules to divide the tasks into simple sub-behaviours that can be shared. We also demonstrate its ability to autonomously divide the tasks into both shared and task-specific sub-behaviours.
  arXiv  Detail & Related papers  (2022-03-28T15:53:17Z)
• The Effect of Diversity in Meta-Learning [79.56118674435844]
  Few-shot learning aims to learn representations that can tackle novel tasks given a small number of examples. Recent studies show that task distribution plays a vital role in the model's performance. We study different task distributions on a myriad of models and datasets to evaluate the effect of task diversity on meta-learning algorithms.
  arXiv  Detail & Related papers  (2022-01-27T19:39:07Z)
• Boosting a Model Zoo for Multi-Task and Continual Learning [15.110807414130923]
  "Model Zoo" is an algorithm that builds an ensemble of models, each of which is very small, and it is trained on a smaller set of tasks. Model Zoo achieves large gains in prediction accuracy compared to state-of-the-art methods in multi-task and continual learning.
  arXiv  Detail & Related papers  (2021-06-06T04:25:09Z)
• Small Towers Make Big Differences [59.243296878666285]
  Multi-task learning aims at solving multiple machine learning tasks at the same time. A good solution to a multi-task learning problem should be generalizable in addition to being Pareto optimal. We propose a method of under- parameterized self-auxiliaries for multi-task models to achieve the best of both worlds.
  arXiv  Detail & Related papers  (2020-08-13T10:45:31Z)
• Reparameterizing Convolutions for Incremental Multi-Task Learning without Task Interference [75.95287293847697]
  Two common challenges in developing multi-task models are often overlooked in literature. First, enabling the model to be inherently incremental, continuously incorporating information from new tasks without forgetting the previously learned ones (incremental learning) Second, eliminating adverse interactions amongst tasks, which has been shown to significantly degrade the single-task performance in a multi-task setup (task interference)
  arXiv  Detail & Related papers  (2020-07-24T14:44:46Z)
• Knowledge Distillation for Multi-task Learning [38.20005345733544]
  Multi-task learning (MTL) is to learn one single model that performs multiple tasks for achieving good performance on all tasks and lower cost on computation. Learning such a model requires to jointly optimize losses of a set of tasks with different difficulty levels, magnitudes, and characteristics. We propose a knowledge distillation based method in this work to address the imbalance problem in multi-task learning.
  arXiv  Detail & Related papers  (2020-07-14T08:02:42Z)
• Gradient Surgery for Multi-Task Learning [119.675492088251]
  Multi-task learning has emerged as a promising approach for sharing structure across multiple tasks. The reasons why multi-task learning is so challenging compared to single-task learning are not fully understood. We propose a form of gradient surgery that projects a task's gradient onto the normal plane of the gradient of any other task that has a conflicting gradient.
  arXiv  Detail & Related papers  (2020-01-19T06:33:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.