EMA-Net: Efficient Multitask Affinity Learning for Dense Scene
Predictions
- URL: http://arxiv.org/abs/2401.11124v1
- Date: Sat, 20 Jan 2024 05:31:47 GMT
- Title: EMA-Net: Efficient Multitask Affinity Learning for Dense Scene
Predictions
- Authors: Dimitrios Sinodinos, Narges Armanfard
- Abstract summary: We introduce the Efficient Multitask Affinity Learning Network (EMA-Net)
EMA-Net adeptly captures local, global, and cross-task interactions using our novel Cross-Task Affinity Learning (CTAL) module.
Our results show that we achieve state-of-the-art MTL performance for CNN-based decoder-focused models.
- Score: 7.01633634930651
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Multitask learning (MTL) has gained prominence for its ability to jointly
predict multiple tasks, achieving better per-task performance while using fewer
per-task model parameters than single-task learning. More recently,
decoder-focused architectures have considerably improved multitask performance
by refining task predictions using the features of other related tasks.
However, most of these refinement methods fail to simultaneously capture local
and global task-specific representations, as well as cross-task patterns in a
parameter-efficient manner. In this paper, we introduce the Efficient Multitask
Affinity Learning Network (EMA-Net), which is a lightweight framework that
enhances the task refinement capabilities of multitask networks. EMA-Net
adeptly captures local, global, and cross-task interactions using our novel
Cross-Task Affinity Learning (CTAL) module. The key innovation of CTAL lies in
its ability to manipulate task affinity matrices in a manner that is optimally
suited to apply parameter-efficient grouped convolutions without worrying about
information loss. Our results show that we achieve state-of-the-art MTL
performance for CNN-based decoder-focused models while using substantially
fewer model parameters. Our code is publicly available at
https://github.com/Armanfard-Lab/EMA-Net.
Related papers
- InterroGate: Learning to Share, Specialize, and Prune Representations
for Multi-task Learning [17.66308231838553]
We propose a novel multi-task learning (MTL) architecture designed to mitigate task interference while optimizing inference computational efficiency.
We employ a learnable gating mechanism to automatically balance the shared and task-specific representations while preserving the performance of all tasks.
arXiv Detail & Related papers (2024-02-26T18:59:52Z) - Sparsely Activated Mixture-of-Experts are Robust Multi-Task Learners [67.5865966762559]
We study whether sparsely activated Mixture-of-Experts (MoE) improve multi-task learning.
We devise task-aware gating functions to route examples from different tasks to specialized experts.
This results in a sparsely activated multi-task model with a large number of parameters, but with the same computational cost as that of a dense model.
arXiv Detail & Related papers (2022-04-16T00:56:12Z) - Medusa: Universal Feature Learning via Attentional Multitasking [65.94499390875046]
Recent approaches to multi-task learning have focused on modelling connections between tasks at the decoder level.
We argue that MTL is a stepping stone towards universal feature learning (UFL), which is the ability to learn generic features that can be applied to new tasks without retraining.
We show the effectiveness of Medusa in UFL (+13.18% improvement) while maintaining MTL performance and being 25% more efficient than previous approaches.
arXiv Detail & Related papers (2022-04-12T10:52:28Z) - Task Adaptive Parameter Sharing for Multi-Task Learning [114.80350786535952]
Adaptive Task Adapting Sharing (TAPS) is a method for tuning a base model to a new task by adaptively modifying a small, task-specific subset of layers.
Compared to other methods, TAPS retains high accuracy on downstream tasks while introducing few task-specific parameters.
We evaluate our method on a suite of fine-tuning tasks and architectures (ResNet, DenseNet, ViT) and show that it achieves state-of-the-art performance while being simple to implement.
arXiv Detail & Related papers (2022-03-30T23:16:07Z) - Attentive Task Interaction Network for Multi-Task Learning [4.1372815372396525]
ATI-Net employs knowledge distillation of the latent features for each task, then combines the feature maps to provide improved contextualized information to the decoder.
This novel approach to introducing knowledge distillation into an attention based multitask network outperforms state of the art MTL baselines.
arXiv Detail & Related papers (2022-01-25T22:03:20Z) - MultiTask-CenterNet (MCN): Efficient and Diverse Multitask Learning
using an Anchor Free Approach [0.13764085113103217]
Multitask learning is a common approach in machine learning.
In this paper we augment the CenterNet anchor-free approach for training multiple perception related tasks together.
arXiv Detail & Related papers (2021-08-11T06:57:04Z) - Reparameterizing Convolutions for Incremental Multi-Task Learning
without Task Interference [75.95287293847697]
Two common challenges in developing multi-task models are often overlooked in literature.
First, enabling the model to be inherently incremental, continuously incorporating information from new tasks without forgetting the previously learned ones (incremental learning)
Second, eliminating adverse interactions amongst tasks, which has been shown to significantly degrade the single-task performance in a multi-task setup (task interference)
arXiv Detail & Related papers (2020-07-24T14:44:46Z) - Knowledge Distillation for Multi-task Learning [38.20005345733544]
Multi-task learning (MTL) is to learn one single model that performs multiple tasks for achieving good performance on all tasks and lower cost on computation.
Learning such a model requires to jointly optimize losses of a set of tasks with different difficulty levels, magnitudes, and characteristics.
We propose a knowledge distillation based method in this work to address the imbalance problem in multi-task learning.
arXiv Detail & Related papers (2020-07-14T08:02:42Z) - Multi-Task Learning for Dense Prediction Tasks: A Survey [87.66280582034838]
Multi-task learning (MTL) techniques have shown promising results w.r.t. performance, computations and/or memory footprint.
We provide a well-rounded view on state-of-the-art deep learning approaches for MTL in computer vision.
arXiv Detail & Related papers (2020-04-28T09:15:50Z) - MTI-Net: Multi-Scale Task Interaction Networks for Multi-Task Learning [82.62433731378455]
We show that tasks with high affinity at a certain scale are not guaranteed to retain this behaviour at other scales.
We propose a novel architecture, namely MTI-Net, that builds upon this finding.
arXiv Detail & Related papers (2020-01-19T21:02:36Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.