Difference-Driven Gating: Adaptive Feature Fusion for U-Net Decoder

• URL: http://arxiv.org/abs/2607.11096v1
• Date: Mon, 13 Jul 2026 05:06:35 GMT
• Title: Difference-Driven Gating: Adaptive Feature Fusion for U-Net Decoder
• Abstract summary: A critical step in U-Net style models is to reconstruct the lower-level features using a top-down decoder.<n>Existing attention-based fusion methods derive attention weights from the top-down decoder features (global) alone.<n>We propose two difference-based gating approaches: Feature-difference gating and Entropy-difference gating.
• Score: 22.058661958215122
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The U-Net style models have been widely used in many applications. A critical step in these models is to reconstruct the lower-level features using a top-down decoder. This reconstruction requires precise fusion of high-level semantics and low-level details. Existing attention-based fusion methods typically derive attention weights from the top-down decoder features (global) alone or the correlation between the top-down decoder features and the bottom-up encoder features (local), then modulate the encoder features using these weights. In this work, we explore a different paradigm: deriving attention weights from the difference between the two feature streams. To this end, we propose two difference-based gating approaches: Feature-difference gating (FDG), which directly uses the absolute difference between global and local features to generate adaptive gating maps, and Entropy-difference gating (EDG), which measures the representational certainty of each stream via information entropy and uses their signed entropy difference to derive the attention weights. Both methods produce coupled gating maps that simultaneously modulate the global and local features. Experiments on different tasks including medical image segmentation, remote sensing image cloud removal and speech separation showed that both methods outperformed existing attention-based fusion methods, and EDG performed better. The results suggested a new paradigm for multi-scale feature fusion in the U-Net style structures.

Related papers

• Dual-Perspective United Transformer for Object Segmentation in Optical Remote Sensing Images [38.942152581251165]
  We propose a novel Dual-Perspective United Transformer (DPU-Former) with a unique structure designed to simultaneously integrate long-range dependencies and spatial details.<n>In particular, we design the global-local mixed attention, which captures diverse information through two perspectives.<n>We present a gated linear feed-forward network to increase the expressive ability.
  arXiv  Detail & Related papers  (2025-06-27T02:40:48Z)
• Local-to-Global Cross-Modal Attention-Aware Fusion for HSI-X Semantic Segmentation [19.461033552684576]
  We propose a Local-to-Global Cross-modal Attention-aware Fusion (LoGoCAF) framework for HSI-X classification. LoGoCAF adopts a pixel-to-pixel two-branch semantic segmentation architecture to learn information from HSI and X modalities.
  arXiv  Detail & Related papers  (2024-06-25T16:12:20Z)
• Object Segmentation by Mining Cross-Modal Semantics [68.88086621181628]
  We propose a novel approach by mining the Cross-Modal Semantics to guide the fusion and decoding of multimodal features. Specifically, we propose a novel network, termed XMSNet, consisting of (1) all-round attentive fusion (AF), (2) coarse-to-fine decoder (CFD), and (3) cross-layer self-supervision.
  arXiv  Detail & Related papers  (2023-05-17T14:30:11Z)
• CEDNet: A Cascade Encoder-Decoder Network for Dense Prediction [23.142342401794302]
  Multi-scale features are essential for dense prediction tasks, such as object detection, instance segmentation, and semantic segmentation. We propose a streamlined cascade encoder-decoder network, dubbed CEDNet, tailored for dense mboxprediction tasks. All stages in CEDNet share the same encoder-decoder structure and perform multi-scale feature fusion within the decoder.
  arXiv  Detail & Related papers  (2023-02-13T02:03:55Z)
• PSNet: Parallel Symmetric Network for Video Salient Object Detection [85.94443548452729]
  We propose a VSOD network with up and down parallel symmetry, named PSNet. Two parallel branches with different dominant modalities are set to achieve complete video saliency decoding.
  arXiv  Detail & Related papers  (2022-10-12T04:11:48Z)
• Joint Spatial-Temporal and Appearance Modeling with Transformer for Multiple Object Tracking [59.79252390626194]
  We propose a novel solution named TransSTAM, which leverages Transformer to model both the appearance features of each object and the spatial-temporal relationships among objects. The proposed method is evaluated on multiple public benchmarks including MOT16, MOT17, and MOT20, and it achieves a clear performance improvement in both IDF1 and HOTA.
  arXiv  Detail & Related papers  (2022-05-31T01:19:18Z)
• Attention guided global enhancement and local refinement network for semantic segmentation [5.881350024099048]
  A lightweight semantic segmentation network is developed using the encoder-decoder architecture. A Global Enhancement Method is proposed to aggregate global information from high-level feature maps. A Local Refinement Module is developed by utilizing the decoder features as the semantic guidance. The two methods are integrated into a Context Fusion Block, and based on that, a novel Attention guided Global enhancement and Local refinement Network (AGLN) is elaborately designed.
  arXiv  Detail & Related papers  (2022-04-09T02:32:24Z)
• Light Field Saliency Detection with Dual Local Graph Learning andReciprocative Guidance [148.9832328803202]
  We model the infor-mation fusion within focal stack via graph networks. We build a novel dual graph modelto guide the focal stack fusion process using all-focus pat-terns.
  arXiv  Detail & Related papers  (2021-10-02T00:54:39Z)
• Attentional Feature Fusion [4.265244011052538]
  We propose a uniform and general scheme, namely attentional feature fusion. We show that our models outperform state-of-the-art networks on both CIFAR-100 and ImageNet datasets.
  arXiv  Detail & Related papers  (2020-09-29T15:10:18Z)
• Multi-scale Interactive Network for Salient Object Detection [91.43066633305662]
  We propose the aggregate interaction modules to integrate the features from adjacent levels. To obtain more efficient multi-scale features, the self-interaction modules are embedded in each decoder unit. Experimental results on five benchmark datasets demonstrate that the proposed method without any post-processing performs favorably against 23 state-of-the-art approaches.
  arXiv  Detail & Related papers  (2020-07-17T15:41:37Z)
• Suppress and Balance: A Simple Gated Network for Salient Object Detection [89.88222217065858]
  We propose a simple gated network (GateNet) to solve both issues at once. With the help of multilevel gate units, the valuable context information from the encoder can be optimally transmitted to the decoder. In addition, we adopt the atrous spatial pyramid pooling based on the proposed "Fold" operation (Fold-ASPP) to accurately localize salient objects of various scales.
  arXiv  Detail & Related papers  (2020-07-16T02:00:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.