One Model for ALL: Low-Level Task Interaction Is a Key to Task-Agnostic Image Fusion

• URL: http://arxiv.org/abs/2502.19854v2
• Date: Sun, 09 Mar 2025 06:49:43 GMT
• Title: One Model for ALL: Low-Level Task Interaction Is a Key to Task-Agnostic Image Fusion
• Authors: Chunyang Cheng, Tianyang Xu, Zhenhua Feng, Xiaojun Wu, ZhangyongTang, Hui Li, Zeyang Zhang, Sara Atito, Muhammad Awais, Josef Kittler,
• Abstract summary: We propose to leverage low-level vision tasks from digital photography fusion, allowing for effective feature interaction through pixel-level supervision.<n>The proposed GIFNet supports diverse fusion tasks, achieving high performance across both seen and unseen scenarios with a single model.
• Score: 38.16599550115468
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Advanced image fusion methods mostly prioritise high-level missions, where task interaction struggles with semantic gaps, requiring complex bridging mechanisms. In contrast, we propose to leverage low-level vision tasks from digital photography fusion, allowing for effective feature interaction through pixel-level supervision. This new paradigm provides strong guidance for unsupervised multimodal fusion without relying on abstract semantics, enhancing task-shared feature learning for broader applicability. Owning to the hybrid image features and enhanced universal representations, the proposed GIFNet supports diverse fusion tasks, achieving high performance across both seen and unseen scenarios with a single model. Uniquely, experimental results reveal that our framework also supports single-modality enhancement, offering superior flexibility for practical applications. Our code will be available at https://github.com/AWCXV/GIFNet.

Related papers

• Balancing Task-invariant Interaction and Task-specific Adaptation for Unified Image Fusion [82.74585945197231]
  Unified image fusion aims to integrate complementary information from multi-source images, enhancing image quality. Existing general image fusion methods incorporate explicit task identification to enable adaptation to different fusion tasks. We propose a novel unified image fusion framework named "TITA", which balances Task-invariant Interaction and Task-specific Adaptation.
  arXiv  Detail & Related papers  (2025-04-07T15:08:35Z)
• Task-Customized Mixture of Adapters for General Image Fusion [51.8742437521891]
  General image fusion aims at integrating important information from multi-source images. We propose a novel task-customized mixture of adapters (TC-MoA) for general image fusion, adaptively prompting various fusion tasks in a unified model.
  arXiv  Detail & Related papers  (2024-03-19T07:02:08Z)
• Multi-interactive Feature Learning and a Full-time Multi-modality Benchmark for Image Fusion and Segmentation [66.15246197473897]
  Multi-modality image fusion and segmentation play a vital role in autonomous driving and robotic operation. We propose a textbfMulti-textbfinteractive textbfFeature learning architecture for image fusion and textbfSegmentation.
  arXiv  Detail & Related papers  (2023-08-04T01:03:58Z)
• A Task-guided, Implicitly-searched and Meta-initialized Deep Model for Image Fusion [69.10255211811007]
  We present a Task-guided, Implicit-searched and Meta- generalizationd (TIM) deep model to address the image fusion problem in a challenging real-world scenario. Specifically, we propose a constrained strategy to incorporate information from downstream tasks to guide the unsupervised learning process of image fusion. Within this framework, we then design an implicit search scheme to automatically discover compact architectures for our fusion model with high efficiency.
  arXiv  Detail & Related papers  (2023-05-25T08:54:08Z)
• Bi-level Dynamic Learning for Jointly Multi-modality Image Fusion and Beyond [50.556961575275345]
  We build an image fusion module to fuse complementary characteristics and cascade dual task-related modules. We develop an efficient first-order approximation to compute corresponding gradients and present dynamic weighted aggregation to balance the gradients for fusion learning.
  arXiv  Detail & Related papers  (2023-05-11T10:55:34Z)
• Generative Modeling for Multi-task Visual Learning [40.96212750592383]
  We consider a novel problem of learning a shared generative model that is useful across various visual perception tasks. We propose a general multi-task oriented generative modeling framework, by coupling a discriminative multi-task network with a generative network. Our framework consistently outperforms state-of-the-art multi-task approaches.
  arXiv  Detail & Related papers  (2021-06-25T03:42:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.