Towards the Uncharted: Density-Descending Feature Perturbation for Semi-supervised Semantic Segmentation

• URL: http://arxiv.org/abs/2403.06462v2
• Date: Thu, 14 Mar 2024 08:25:49 GMT
• Title: Towards the Uncharted: Density-Descending Feature Perturbation for Semi-supervised Semantic Segmentation
• Authors: Xiaoyang Wang, Huihui Bai, Limin Yu, Yao Zhao, Jimin Xiao,
• Abstract summary: We propose a novel feature-level consistency learning framework named Density-Descending Feature Perturbation (DDFP) Inspired by the low-density separation assumption in semi-supervised learning, our key insight is that feature density can shed a light on the most promising direction for the segmentation classifier to explore. The proposed DDFP outperforms other designs on feature-level perturbations and shows state of the art performances on both Pascal VOC and Cityscapes dataset.
• Score: 51.66997548477913
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Semi-supervised semantic segmentation allows model to mine effective supervision from unlabeled data to complement label-guided training. Recent research has primarily focused on consistency regularization techniques, exploring perturbation-invariant training at both the image and feature levels. In this work, we proposed a novel feature-level consistency learning framework named Density-Descending Feature Perturbation (DDFP). Inspired by the low-density separation assumption in semi-supervised learning, our key insight is that feature density can shed a light on the most promising direction for the segmentation classifier to explore, which is the regions with lower density. We propose to shift features with confident predictions towards lower-density regions by perturbation injection. The perturbed features are then supervised by the predictions on the original features, thereby compelling the classifier to explore less dense regions to effectively regularize the decision boundary. Central to our method is the estimation of feature density. To this end, we introduce a lightweight density estimator based on normalizing flow, allowing for efficient capture of the feature density distribution in an online manner. By extracting gradients from the density estimator, we can determine the direction towards less dense regions for each feature. The proposed DDFP outperforms other designs on feature-level perturbations and shows state of the art performances on both Pascal VOC and Cityscapes dataset under various partition protocols. The project is available at https://github.com/Gavinwxy/DDFP.

Related papers

• Density Matters: Improved Core-set for Active Domain Adaptive Segmentation [35.58476357071544]
  Active domain adaptation has emerged as a solution to balance the expensive annotation cost and the performance of trained models in semantic segmentation. In this work, we revisit the theoretical bound of the classical Core-set method and identify that the performance is closely related to the local sample distribution around selected samples. We introduce a local proxy estimator with Dynamic Masked Convolution and develop a Density-aware Greedy algorithm to optimize the bound.
  arXiv  Detail & Related papers  (2023-12-15T08:22:36Z)
• Semi-supervised Crowd Counting via Density Agency [57.3635501421658]
  We build a learnable auxiliary structure, namely the density agency to bring the recognized foreground regional features close to corresponding density sub-classes. Second, we propose a density-guided contrastive learning loss to consolidate the backbone feature extractor. Third, we build a regression head by using a transformer structure to refine the foreground features further.
  arXiv  Detail & Related papers  (2022-09-07T06:34:00Z)
• Semi-Supervised Building Footprint Generation with Feature and Output Consistency Training [17.6179873429447]
  State-of-the-art semi-supervised semantic segmentation networks with consistency training can help to deal with this issue. We propose to integrate the consistency of both features and outputs in the end-to-end network training of unlabeled samples. Experimental results show that the proposed approach can well extract more complete building structures.
  arXiv  Detail & Related papers  (2022-05-17T14:55:13Z)
• Density-Based Clustering with Kernel Diffusion [59.4179549482505]
  A naive density corresponding to the indicator function of a unit $d$-dimensional Euclidean ball is commonly used in density-based clustering algorithms. We propose a new kernel diffusion density function, which is adaptive to data of varying local distributional characteristics and smoothness.
  arXiv  Detail & Related papers  (2021-10-11T09:00:33Z)
• Featurized Density Ratio Estimation [82.40706152910292]
  In our work, we propose to leverage an invertible generative model to map the two distributions into a common feature space prior to estimation. This featurization brings the densities closer together in latent space, sidestepping pathological scenarios where the learned density ratios in input space can be arbitrarily inaccurate. At the same time, the invertibility of our feature map guarantees that the ratios computed in feature space are equivalent to those in input space.
  arXiv  Detail & Related papers  (2021-07-05T18:30:26Z)
• Deep Semi-supervised Knowledge Distillation for Overlapping Cervical Cell Instance Segmentation [54.49894381464853]
  We propose to leverage both labeled and unlabeled data for instance segmentation with improved accuracy by knowledge distillation. We propose a novel Mask-guided Mean Teacher framework with Perturbation-sensitive Sample Mining. Experiments show that the proposed method improves the performance significantly compared with the supervised method learned from labeled data only.
  arXiv  Detail & Related papers  (2020-07-21T13:27:09Z)
• Density-Aware Graph for Deep Semi-Supervised Visual Recognition [102.9484812869054]
  Semi-supervised learning (SSL) has been extensively studied to improve the generalization ability of deep neural networks for visual recognition. This paper proposes to solve the SSL problem by building a novel density-aware graph, based on which the neighborhood information can be easily leveraged.
  arXiv  Detail & Related papers  (2020-03-30T02:52:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.