Region Rebalance for Long-Tailed Semantic Segmentation

• URL: http://arxiv.org/abs/2204.01969v1
• Date: Tue, 5 Apr 2022 03:47:47 GMT
• Title: Region Rebalance for Long-Tailed Semantic Segmentation
• Authors: Jiequan Cui, Yuhui Yuan, Zhisheng Zhong, Zhuotao Tian, Han Hu, Stephen Lin, Jiaya Jia
• Abstract summary: We first investigate and identify the main challenges of addressing this issue through pixel rebalance. Then a simple and yet effective region rebalance scheme is derived based on our analysis. With the proposed region rebalance scheme, state-of-the-art BEiT receives +0.7% gain in terms of mIoU on the ADE20K val set.
• Score: 89.84860341946283
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: In this paper, we study the problem of class imbalance in semantic segmentation. We first investigate and identify the main challenges of addressing this issue through pixel rebalance. Then a simple and yet effective region rebalance scheme is derived based on our analysis. In our solution, pixel features belonging to the same class are grouped into region features, and a rebalanced region classifier is applied via an auxiliary region rebalance branch during training. To verify the flexibility and effectiveness of our method, we apply the region rebalance module into various semantic segmentation methods, such as Deeplabv3+, OCRNet, and Swin. Our strategy achieves consistent improvement on the challenging ADE20K and COCO-Stuff benchmark. In particular, with the proposed region rebalance scheme, state-of-the-art BEiT receives +0.7% gain in terms of mIoU on the ADE20K val set.

Related papers

• Covariance-corrected Whitening Alleviates Network Degeneration on Imbalanced Classification [6.197116272789107]
  Class imbalance is a critical issue in image classification that significantly affects the performance of deep recognition models. We propose a novel framework called Whitening-Net to mitigate the degenerate solutions. In scenarios with extreme class imbalance, the batch covariance statistic exhibits significant fluctuations, impeding the convergence of the whitening operation.
  arXiv  Detail & Related papers  (2024-08-30T10:49:33Z)
• Class and Region-Adaptive Constraints for Network Calibration [17.583536041845402]
  We present a novel approach to calibrate segmentation networks that considers the inherent challenges posed by different categories and object regions. Finding the optimal penalty weights manually might be unfeasible, and potentially hinder the optimization process. We propose an approach based on Class and Region-Adaptive constraints (CRaC), which allows to learn the class and region-wise penalty weights during training.
  arXiv  Detail & Related papers  (2024-03-19T02:19:57Z)
• Local-consistent Transformation Learning for Rotation-invariant Point Cloud Analysis [61.04787144322498]
  We propose a Local-consistent Transformation (LocoTrans) learning strategy for point shape analysis. We first construct the local-consistent reference frame (LCRF) by considering the symmetry of the two axes in LRF. As the consistency only exists in local regions, the relative pose information is still lost in the intermediate layers of the network.
  arXiv  Detail & Related papers  (2024-03-17T06:40:50Z)
• Less is More: Fewer Interpretable Region via Submodular Subset Selection [54.07758302264416]
  This paper re-models the above image attribution problem as a submodular subset selection problem. We construct a novel submodular function to discover more accurate small interpretation regions. For correctly predicted samples, the proposed method improves the Deletion and Insertion scores with an average of 4.9% and 2.5% gain relative to HSIC-Attribution.
  arXiv  Detail & Related papers  (2024-02-14T13:30:02Z)
• Coupling Global Context and Local Contents for Weakly-Supervised Semantic Segmentation [54.419401869108846]
  We propose a single-stage WeaklySupervised Semantic (WSSS) model with only the image-level class label supervisions. A flexible context aggregation module is proposed to capture the global object context in different granular spaces. A semantically consistent feature fusion module is proposed in a bottom-up parameter-learnable fashion to aggregate the fine-grained local contents.
  arXiv  Detail & Related papers  (2023-04-18T15:29:23Z)
• Bidirectional Domain Mixup for Domain Adaptive Semantic Segmentation [73.3083304858763]
  This paper systematically studies the impact of mixup under the domain adaptaive semantic segmentation task. In specific, we achieve domain mixup in two-step: cut and paste. We provide extensive ablation experiments to empirically verify our main components of the framework.
  arXiv  Detail & Related papers  (2023-03-17T05:22:44Z)
• Fully Stochastic Trust-Region Sequential Quadratic Programming for Equality-Constrained Optimization Problems [62.83783246648714]
  We propose a sequential quadratic programming algorithm (TR-StoSQP) to solve nonlinear optimization problems with objectives and deterministic equality constraints. The algorithm adaptively selects the trust-region radius and, compared to the existing line-search StoSQP schemes, allows us to utilize indefinite Hessian matrices.
  arXiv  Detail & Related papers  (2022-11-29T05:52:17Z)
• Unsupervised Cross-Modality Domain Adaptation for Vestibular Schwannoma Segmentation and Koos Grade Prediction based on Semi-Supervised Contrastive Learning [1.5953825926551457]
  unsupervised domain adaptation framework for cross-modality vestibular schwannoma (VS) and cochlea segmentation and Koos grade prediction. nnU-Net model is utilized for VS and cochlea segmentation, while a semi-supervised contrastive learning pre-train approach is employed to improve the model performance. Our method received rank 4 in task1 with a mean Dice score of 0.8394 and rank 2 in task2 with Macro-Average Mean Square Error of 0.3941.
  arXiv  Detail & Related papers  (2022-10-09T13:12:20Z)
• Region-level Contrastive and Consistency Learning for Semi-Supervised Semantic Segmentation [30.1884540364192]
  We propose a novel region-level contrastive and consistency learning framework (RC2L) for semi-supervised semantic segmentation. Specifically, we first propose a Region Mask Contrastive (RMC) loss and a Region Feature Contrastive (RFC) loss to accomplish region-level contrastive property. Based on the proposed region-level contrastive and consistency regularization, we develop a region-level contrastive and consistency learning framework (RC2L) for semi-supervised semantic segmentation.
  arXiv  Detail & Related papers  (2022-04-28T07:22:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.