MTLSI-Net: A Linear Semantic Interaction Network for Parameter-Efficient Multi-Task Dense Prediction

• URL: http://arxiv.org/abs/2604.01995v1
• Date: Thu, 02 Apr 2026 13:02:48 GMT
• Title: MTLSI-Net: A Linear Semantic Interaction Network for Parameter-Efficient Multi-Task Dense Prediction
• Abstract summary: MTLSI-Net captures comprehensive cross-task interactions at linear complexity with reduced parameters.<n>Experiments on NYUDv2 and PASCAL-Context demonstrate that MTLSI-Net achieves state-of-the-art performance.
• Score: 50.992213877904305
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-task dense prediction aims to perform multiple pixel-level tasks simultaneously. However, capturing global cross-task interactions remains non-trivial due to the quadratic complexity of standard self-attention on high-resolution features. To address this limitation, we propose a Multi-Task Linear Semantic Interaction Network (MTLSI-Net), which facilitates cross-task interaction through linear attention. Specifically, MTLSI-Net incorporates three key components: a Multi-Task Multi-scale Query Linear Fusion Block, which captures cross-task dependencies across multiple scales with linear complexity using a shared global context matrix; a Semantic Token Distiller that compresses redundant features into compact semantic tokens, distilling essential cross-task knowledge; and a Cross-Window Integrated attention Block that injects global semantics into local features via a dual-branch architecture, preserving both global consistency and spatial precision. These components collectively enable the network to capture comprehensive cross-task interactions at linear complexity with reduced parameters. Extensive experiments on NYUDv2 and PASCAL-Context demonstrate that MTLSI-Net achieves state-of-the-art performance, validating its effectiveness and efficiency in multi-task learning.

Related papers

• T3S: Improving Multi-Task Reinforcement Learning with Task-Specific Feature Selector and Scheduler [51.05812650668468]
  Multi-task reinforcement learning (MTRL) is a technique to train multiple tasks simultaneously, where previous works usually train a single model to solve different tasks by sharing parameters across various tasks.<n>We propose a novel MTRL framework called Task-Specific feature Selector and Scheduler (T3S), which consists of two components: a feature selector and a task scheduler.<n>Specifically, the feature selectors employ hypernetworks to construct task-specific soft masks, which can be applied by globally shared representation to construct task-specific features.
  arXiv  Detail & Related papers  (2026-08-31T13:30:06Z)
• CoAction: Cross-task Correlation-aware Pareto Set Learning [9.837762929043649]
  We propose a cross-tAsk correlation-aware framework for multi-objective multitask optimization.<n>By assigning task-specific embedding vectors to individual tasks, the model effectively distinguishes between tasks while facilitating knowledge sharing.<n>The proposed approach is evaluated on comprehensive multitask test suites covering both benchmark problems and real-world applications.
  arXiv  Detail & Related papers  (2026-05-03T04:52:44Z)
• 3D-Aware Multi-Task Learning with Cross-View Correlations for Dense Scene Understanding [18.76513756741288]
  Current approaches mainly capture cross-task relations in the 2D image space, often leading to unstructured features lacking 3D-awareness.<n>We propose to address this problem by integrating correlations across views, i.e., cost volume, as geometric consistency in the MTL network.<n> Specifically, we introduce a lightweight Cross-view Module (CvM), shared across tasks, to exchange information across views and capture cross-view correlations.
  arXiv  Detail & Related papers  (2025-11-25T18:59:34Z)
• Tracking and Segmenting Anything in Any Modality [75.32774085793498]
  We propose a universal tracking and segmentation framework named SATA, which unifies a broad spectrum of tracking and segmentation subtasks with any modality input.<n> SATA demonstrates superior performance on 18 challenging tracking and segmentation benchmarks, offering a novel perspective for more generalizable video understanding.
  arXiv  Detail & Related papers  (2025-11-22T09:09:22Z)
• Enhancing Mamba Decoder with Bidirectional Interaction in Multi-Task Dense Prediction [37.625609555296364]
  Cross-task interaction is crucial for success in multi-task dense prediction.<n>Existing methods face the trade-off between interaction completeness and computational efficiency.<n>This work proposes a Bidirectional Interaction Mamba, which incorporates novel scanning mechanisms.
  arXiv  Detail & Related papers  (2025-08-28T02:50:19Z)
• TEM^3-Learning: Time-Efficient Multimodal Multi-Task Learning for Advanced Assistive Driving [22.22943635900334]
  TEM3-Learning is a novel framework that jointly optimize driver emotion recognition, driver behavior recognition, traffic context recognition, and vehicle behavior recognition.<n>It achieves state-of-the-art accuracy across all four tasks, maintaining a lightweight architecture with fewer than 6 million parameters and delivering an impressive 142.32 FPS inference speed.
  arXiv  Detail & Related papers  (2025-06-22T16:12:27Z)
• Pilot: Building the Federated Multimodal Instruction Tuning Framework [79.56362403673354]
  Our framework integrates two stages of "adapter on adapter" into the connector of the vision encoder and the LLM.<n>In stage 1, we extract task-specific features and client-specific features from visual information.<n>In stage 2, we build the cross-task Mixture-of-Adapters(CT-MoA) module to perform cross-task interaction.
  arXiv  Detail & Related papers  (2025-01-23T07:49:24Z)
• RepVF: A Unified Vector Fields Representation for Multi-task 3D Perception [64.80760846124858]
  This paper proposes a novel unified representation, RepVF, which harmonizes the representation of various perception tasks. RepVF characterizes the structure of different targets in the scene through a vector field, enabling a single-head, multi-task learning model. Building upon RepVF, we introduce RFTR, a network designed to exploit the inherent connections between different tasks.
  arXiv  Detail & Related papers  (2024-07-15T16:25:07Z)
• M$^3$ViT: Mixture-of-Experts Vision Transformer for Efficient Multi-task Learning with Model-Accelerator Co-design [95.41238363769892]
  Multi-task learning (MTL) encapsulates multiple learned tasks in a single model and often lets those tasks learn better jointly. Current MTL regimes have to activate nearly the entire model even to just execute a single task. We present a model-accelerator co-design framework to enable efficient on-device MTL.
  arXiv  Detail & Related papers  (2022-10-26T15:40:24Z)
• DenseMTL: Cross-task Attention Mechanism for Dense Multi-task Learning [18.745373058797714]
  We propose a novel multi-task learning architecture that leverages pairwise cross-task exchange through correlation-guided attention and self-attention. We conduct extensive experiments across three multi-task setups, showing the advantages of our approach compared to competitive baselines in both synthetic and real-world benchmarks.
  arXiv  Detail & Related papers  (2022-06-17T17:59:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.