Rethinking of Feature Interaction for Multi-task Learning on Dense
Prediction
- URL: http://arxiv.org/abs/2312.13514v1
- Date: Thu, 21 Dec 2023 01:30:44 GMT
- Title: Rethinking of Feature Interaction for Multi-task Learning on Dense
Prediction
- Authors: Jingdong Zhang, Jiayuan Fan, Peng Ye, Bo Zhang, Hancheng Ye, Baopu Li,
Yancheng Cai, Tao Chen
- Abstract summary: We observe that low-level representations with rich details and high-level representations with abundant task information are not both involved in the multi-task interaction process.
Low-quality and low-efficiency issues also exist in current multi-task learning architectures.
We propose a novel Bridge-Feature-Centirc Interaction (BRFI) method to learn a comprehensive intermediate feature globally from both task-generic and task-specific features.
- Score: 30.30105024946622
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Existing works generally adopt the encoder-decoder structure for Multi-task
Dense Prediction, where the encoder extracts the task-generic features, and
multiple decoders generate task-specific features for predictions. We observe
that low-level representations with rich details and high-level representations
with abundant task information are not both involved in the multi-task
interaction process. Additionally, low-quality and low-efficiency issues also
exist in current multi-task learning architectures. In this work, we propose to
learn a comprehensive intermediate feature globally from both task-generic and
task-specific features, we reveal an important fact that this intermediate
feature, namely the bridge feature, is a good solution to the above issues.
Based on this, we propose a novel Bridge-Feature-Centirc Interaction (BRFI)
method. A Bridge Feature Extractor (BFE) is designed for the generation of
strong bridge features and Task Pattern Propagation (TPP) is applied to ensure
high-quality task interaction participants. Then a Task-Feature Refiner (TFR)
is developed to refine final task predictions with the well-learned knowledge
from the bridge features. Extensive experiments are conducted on NYUD-v2 and
PASCAL Context benchmarks, and the superior performance shows the proposed
architecture is effective and powerful in promoting different dense prediction
tasks simultaneously.
Related papers
- A Multitask Deep Learning Model for Classification and Regression of Hyperspectral Images: Application to the large-scale dataset [44.94304541427113]
We propose a multitask deep learning model to perform multiple classification and regression tasks simultaneously on hyperspectral images.
We validated our approach on a large hyperspectral dataset called TAIGA.
A comprehensive qualitative and quantitative analysis of the results shows that the proposed method significantly outperforms other state-of-the-art methods.
arXiv Detail & Related papers (2024-07-23T11:14:54Z) - RepVF: A Unified Vector Fields Representation for Multi-task 3D Perception [64.80760846124858]
This paper proposes a novel unified representation, RepVF, which harmonizes the representation of various perception tasks.
RepVF characterizes the structure of different targets in the scene through a vector field, enabling a single-head, multi-task learning model.
Building upon RepVF, we introduce RFTR, a network designed to exploit the inherent connections between different tasks.
arXiv Detail & Related papers (2024-07-15T16:25:07Z) - Task Indicating Transformer for Task-conditional Dense Predictions [16.92067246179703]
We introduce a novel task-conditional framework called Task Indicating Transformer (TIT) to tackle this challenge.
Our approach designs a Mix Task Adapter module within the transformer block, which incorporates a Task Indicating Matrix through matrix decomposition.
We also propose a Task Gate Decoder module that harnesses a Task Indicating Vector and gating mechanism to facilitate adaptive multi-scale feature refinement.
arXiv Detail & Related papers (2024-03-01T07:06:57Z) - TaskExpert: Dynamically Assembling Multi-Task Representations with
Memorial Mixture-of-Experts [11.608682595506354]
Recent models consider directly decoding task-specific features from one shared task-generic feature.
As the input feature is fully shared and each task decoder also shares decoding parameters for different input samples, it leads to a static feature decoding process.
We propose TaskExpert, a novel multi-task mixture-of-experts model that enables learning multiple representative task-generic feature spaces.
arXiv Detail & Related papers (2023-07-28T06:00:57Z) - A Dynamic Feature Interaction Framework for Multi-task Visual Perception [100.98434079696268]
We devise an efficient unified framework to solve multiple common perception tasks.
These tasks include instance segmentation, semantic segmentation, monocular 3D detection, and depth estimation.
Our proposed framework, termed D2BNet, demonstrates a unique approach to parameter-efficient predictions for multi-task perception.
arXiv Detail & Related papers (2023-06-08T09:24:46Z) - Pre-training Multi-task Contrastive Learning Models for Scientific
Literature Understanding [52.723297744257536]
Pre-trained language models (LMs) have shown effectiveness in scientific literature understanding tasks.
We propose a multi-task contrastive learning framework, SciMult, to facilitate common knowledge sharing across different literature understanding tasks.
arXiv Detail & Related papers (2023-05-23T16:47:22Z) - Cross-task Attention Mechanism for Dense Multi-task Learning [16.040894192229043]
We jointly address 2D semantic segmentation, and two geometry-related tasks, namely dense depth and surface normal estimation.
We propose a novel multi-task learning architecture that exploits pair-wise cross-task exchange through correlation-guided attention and self-attention.
arXiv Detail & Related papers (2022-06-17T17:59:45Z) - Fast Inference and Transfer of Compositional Task Structures for
Few-shot Task Generalization [101.72755769194677]
We formulate it as a few-shot reinforcement learning problem where a task is characterized by a subtask graph.
Our multi-task subtask graph inferencer (MTSGI) first infers the common high-level task structure in terms of the subtask graph from the training tasks.
Our experiment results on 2D grid-world and complex web navigation domains show that the proposed method can learn and leverage the common underlying structure of the tasks for faster adaptation to the unseen tasks.
arXiv Detail & Related papers (2022-05-25T10:44:25Z) - Combining Modular Skills in Multitask Learning [149.8001096811708]
A modular design encourages neural models to disentangle and recombine different facets of knowledge to generalise more systematically to new tasks.
In this work, we assume each task is associated with a subset of latent discrete skills from a (potentially small) inventory.
We find that the modular design of a network significantly increases sample efficiency in reinforcement learning and few-shot generalisation in supervised learning.
arXiv Detail & Related papers (2022-02-28T16:07:19Z) - Exploring Relational Context for Multi-Task Dense Prediction [76.86090370115]
We consider a multi-task environment for dense prediction tasks, represented by a common backbone and independent task-specific heads.
We explore various attention-based contexts, such as global and local, in the multi-task setting.
We propose an Adaptive Task-Relational Context module, which samples the pool of all available contexts for each task pair.
arXiv Detail & Related papers (2021-04-28T16:45:56Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.