Ultra-High Resolution Segmentation via Boundary-Enhanced Patch-Merging Transformer

• URL: http://arxiv.org/abs/2412.10181v2
• Date: Sat, 21 Dec 2024 07:14:47 GMT
• Title: Ultra-High Resolution Segmentation via Boundary-Enhanced Patch-Merging Transformer
• Authors: Haopeng Sun, Yingwei Zhang, Lumin Xu, Sheng Jin, Yiqiang Chen,
• Abstract summary: We propose a novel UHR segmentation method called Boundary-enhanced Patch-merging Transformer (BPT) BPT consists of two key components: (1) Patch-Merging Transformer (PMT) for dynamically allocating tokens to informative regions, and (2) Boundary-Enhanced Module (BEM) that leverages boundary information to enrich fine details. Our BPT outperforms previous state-of-the-art methods without introducing extra computational overhead.
• Score: 14.252237735930953
• License:
• Abstract: Segmentation of ultra-high resolution (UHR) images is a critical task with numerous applications, yet it poses significant challenges due to high spatial resolution and rich fine details. Recent approaches adopt a dual-branch architecture, where a global branch learns long-range contextual information and a local branch captures fine details. However, they struggle to handle the conflict between global and local information while adding significant extra computational cost. Inspired by the human visual system's ability to rapidly orient attention to important areas with fine details and filter out irrelevant information, we propose a novel UHR segmentation method called Boundary-enhanced Patch-merging Transformer (BPT). BPT consists of two key components: (1) Patch-Merging Transformer (PMT) for dynamically allocating tokens to informative regions to acquire global and local representations, and (2) Boundary-Enhanced Module (BEM) that leverages boundary information to enrich fine details. Extensive experiments on multiple UHR image segmentation benchmarks demonstrate that our BPT outperforms previous state-of-the-art methods without introducing extra computational overhead. Codes will be released to facilitate research.

Related papers

• Decoupling Fine Detail and Global Geometry for Compressed Depth Map Super-Resolution [55.9977636042469]
  Bit-depth compression produces a uniform depth representation in regions with subtle variations, hindering the recovery of detailed information. densely distributed random noise reduces the accuracy of estimating the global geometric structure of the scene. We propose a novel framework, termed geometry-decoupled network (GDNet), for compressed depth map super-resolution.
  arXiv  Detail & Related papers  (2024-11-05T16:37:30Z)
• Remote Sensing Image Segmentation Using Vision Mamba and Multi-Scale Multi-Frequency Feature Fusion [9.098711843118629]
  This paper introduces state space model (SSM) and proposes a novel hybrid semantic segmentation network based on vision Mamba (CVMH-UNet) This method designs a cross-scanning visual state space block (CVSSBlock) that uses cross 2D scanning (CS2D) to fully capture global information from multiple directions. By incorporating convolutional neural network branches to overcome the constraints of Vision Mamba (VMamba) in acquiring local information, this approach facilitates a comprehensive analysis of both global and local features.
  arXiv  Detail & Related papers  (2024-10-08T02:17:38Z)
• MICDrop: Masking Image and Depth Features via Complementary Dropout for Domain-Adaptive Semantic Segmentation [155.0797148367653]
  Unsupervised Domain Adaptation (UDA) is the task of bridging the domain gap between a labeled source domain and an unlabeled target domain. We propose to leverage geometric information, i.e., depth predictions, as depth discontinuities often coincide with segmentation boundaries. We show that our method can be plugged into various recent UDA methods and consistently improve results across standard UDA benchmarks.
  arXiv  Detail & Related papers  (2024-08-29T12:15:10Z)
• Mixture-of-Noises Enhanced Forgery-Aware Predictor for Multi-Face Manipulation Detection and Localization [52.87635234206178]
  This paper proposes a new framework, namely MoNFAP, specifically tailored for multi-face manipulation detection and localization. The framework incorporates two novel modules: the Forgery-aware Unified Predictor (FUP) Module and the Mixture-of-Noises Module (MNM)
  arXiv  Detail & Related papers  (2024-08-05T08:35:59Z)
• Learning Accurate and Enriched Features for Stereo Image Super-Resolution [0.0]
  Stereo image super-resolution (stereoSR) aims to enhance the quality of super-resolution results by incorporating complementary information from an alternative view. We propose a mixed-scale selective fusion network (MSSFNet) to preserve precise spatial details and incorporate abundant contextual information. MSSFNet achieves significant improvements over state-of-the-art approaches on both quantitative and qualitative evaluations.
  arXiv  Detail & Related papers  (2024-06-23T03:34:17Z)
• DETR Doesn't Need Multi-Scale or Locality Design [69.56292005230185]
  This paper presents an improved DETR detector that maintains a "plain" nature. It uses a single-scale feature map and global cross-attention calculations without specific locality constraints. We show that two simple technologies are surprisingly effective within a plain design to compensate for the lack of multi-scale feature maps and locality constraints.
  arXiv  Detail & Related papers  (2023-08-03T17:59:04Z)
• DuAT: Dual-Aggregation Transformer Network for Medical Image Segmentation [21.717520350930705]
  Transformer-based models have been widely demonstrated to be successful in computer vision tasks. However, they are often dominated by features of large patterns leading to the loss of local details. We propose a Dual-Aggregation Transformer Network called DuAT, which is characterized by two innovative designs. Our proposed model outperforms state-of-the-art methods in the segmentation of skin lesion images, and polyps in colonoscopy images.
  arXiv  Detail & Related papers  (2022-12-21T07:54:02Z)
• Remote Sensing Cross-Modal Text-Image Retrieval Based on Global and Local Information [15.32353270625554]
  Cross-modal remote sensing text-image retrieval (RSCTIR) has recently become an urgent research hotspot due to its ability of enabling fast and flexible information extraction on remote sensing (RS) images. We first propose a novel RSCTIR framework based on global and local information (GaLR), and design a multi-level information dynamic fusion (MIDF) module to efficaciously integrate features of different levels. Experiments on public datasets strongly demonstrate the state-of-the-art performance of GaLR methods on the RSCTIR task.
  arXiv  Detail & Related papers  (2022-04-21T03:18:09Z)
• DRBANET: A Lightweight Dual-Resolution Network for Semantic Segmentation with Boundary Auxiliary [15.729067807920236]
  This paper introduces a lightweight dual-resolution network, called DRBANet, aiming to refine semantic segmentation results with the aid of boundary information. DRBANet adopts dual parallel architecture, including: high resolution branch (HRB) and low resolution branch (LRB) Experiments on Cityscapes and CamVid datasets demonstrate that our method achieves promising trade-off between segmentation accuracy and running efficiency.
  arXiv  Detail & Related papers  (2021-10-31T14:20:02Z)
• Boosting Few-shot Semantic Segmentation with Transformers [81.43459055197435]
  TRansformer-based Few-shot Semantic segmentation method (TRFS) Our model consists of two modules: Global Enhancement Module (GEM) and Local Enhancement Module (LEM)
  arXiv  Detail & Related papers  (2021-08-04T20:09:21Z)
• High-resolution Depth Maps Imaging via Attention-based Hierarchical Multi-modal Fusion [84.24973877109181]
  We propose a novel attention-based hierarchical multi-modal fusion network for guided DSR. We show that our approach outperforms state-of-the-art methods in terms of reconstruction accuracy, running speed and memory efficiency.
  arXiv  Detail & Related papers  (2021-04-04T03:28:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.