Sequential Cross Attention Based Multi-task Learning
- URL: http://arxiv.org/abs/2209.02518v1
- Date: Tue, 6 Sep 2022 14:17:33 GMT
- Title: Sequential Cross Attention Based Multi-task Learning
- Authors: Sunkyung Kim, Hyesong Choi, Dongbo Min
- Abstract summary: We propose a novel architecture that effectively transfers informative features by applying the attention mechanism to the multi-scale features of the tasks.
Our method achieves state-of-the-art performance on the NYUD-v2 and PASCAL-Context dataset.
- Score: 22.430705836627148
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: In multi-task learning (MTL) for visual scene understanding, it is crucial to
transfer useful information between multiple tasks with minimal interferences.
In this paper, we propose a novel architecture that effectively transfers
informative features by applying the attention mechanism to the multi-scale
features of the tasks. Since applying the attention module directly to all
possible features in terms of scale and task requires a high complexity, we
propose to apply the attention module sequentially for the task and scale. The
cross-task attention module (CTAM) is first applied to facilitate the exchange
of relevant information between the multiple task features of the same scale.
The cross-scale attention module (CSAM) then aggregates useful information from
feature maps at different resolutions in the same task. Also, we attempt to
capture long range dependencies through the self-attention module in the
feature extraction network. Extensive experiments demonstrate that our method
achieves state-of-the-art performance on the NYUD-v2 and PASCAL-Context
dataset.
Related papers
- SGW-based Multi-Task Learning in Vision Tasks [8.459976488960269]
As the scale of datasets expands and the complexity of tasks increases, knowledge sharing becomes increasingly challenging.
We propose an information bottleneck knowledge extraction module (KEM)
This module aims to reduce inter-task interference by constraining the flow of information, thereby reducing computational complexity.
arXiv Detail & Related papers (2024-10-03T13:56:50Z) - Auxiliary Tasks Enhanced Dual-affinity Learning for Weakly Supervised
Semantic Segmentation [79.05949524349005]
We propose AuxSegNet+, a weakly supervised auxiliary learning framework to explore the rich information from saliency maps.
We also propose a cross-task affinity learning mechanism to learn pixel-level affinities from the saliency and segmentation feature maps.
arXiv Detail & Related papers (2024-03-02T10:03:21Z) - Distribution Matching for Multi-Task Learning of Classification Tasks: a
Large-Scale Study on Faces & Beyond [62.406687088097605]
Multi-Task Learning (MTL) is a framework, where multiple related tasks are learned jointly and benefit from a shared representation space.
We show that MTL can be successful with classification tasks with little, or non-overlapping annotations.
We propose a novel approach, where knowledge exchange is enabled between the tasks via distribution matching.
arXiv Detail & Related papers (2024-01-02T14:18:11Z) - Factorized Contrastive Learning: Going Beyond Multi-view Redundancy [116.25342513407173]
This paper proposes FactorCL, a new multimodal representation learning method to go beyond multi-view redundancy.
On large-scale real-world datasets, FactorCL captures both shared and unique information and achieves state-of-the-art results.
arXiv Detail & Related papers (2023-06-08T15:17:04Z) - MTLSegFormer: Multi-task Learning with Transformers for Semantic
Segmentation in Precision Agriculture [16.817025300716796]
We propose a semantic segmentation method, MTLSegFormer, which combines multi-task learning and attention mechanisms.
We tested the performance in two challenging problems with correlated tasks and observed a significant improvement in accuracy.
arXiv Detail & Related papers (2023-05-04T13:19:43Z) - Fast Inference and Transfer of Compositional Task Structures for
Few-shot Task Generalization [101.72755769194677]
We formulate it as a few-shot reinforcement learning problem where a task is characterized by a subtask graph.
Our multi-task subtask graph inferencer (MTSGI) first infers the common high-level task structure in terms of the subtask graph from the training tasks.
Our experiment results on 2D grid-world and complex web navigation domains show that the proposed method can learn and leverage the common underlying structure of the tasks for faster adaptation to the unseen tasks.
arXiv Detail & Related papers (2022-05-25T10:44:25Z) - Medusa: Universal Feature Learning via Attentional Multitasking [65.94499390875046]
Recent approaches to multi-task learning have focused on modelling connections between tasks at the decoder level.
We argue that MTL is a stepping stone towards universal feature learning (UFL), which is the ability to learn generic features that can be applied to new tasks without retraining.
We show the effectiveness of Medusa in UFL (+13.18% improvement) while maintaining MTL performance and being 25% more efficient than previous approaches.
arXiv Detail & Related papers (2022-04-12T10:52:28Z) - Attentive Task Interaction Network for Multi-Task Learning [4.1372815372396525]
ATI-Net employs knowledge distillation of the latent features for each task, then combines the feature maps to provide improved contextualized information to the decoder.
This novel approach to introducing knowledge distillation into an attention based multitask network outperforms state of the art MTL baselines.
arXiv Detail & Related papers (2022-01-25T22:03:20Z) - MTI-Net: Multi-Scale Task Interaction Networks for Multi-Task Learning [82.62433731378455]
We show that tasks with high affinity at a certain scale are not guaranteed to retain this behaviour at other scales.
We propose a novel architecture, namely MTI-Net, that builds upon this finding.
arXiv Detail & Related papers (2020-01-19T21:02:36Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.