SmooSeg: Smoothness Prior for Unsupervised Semantic Segmentation

• URL: http://arxiv.org/abs/2310.17874v1
• Date: Fri, 27 Oct 2023 03:29:25 GMT
• Title: SmooSeg: Smoothness Prior for Unsupervised Semantic Segmentation
• Authors: Mengcheng Lan, Xinjiang Wang, Yiping Ke, Jiaxing Xu, Litong Feng, Wayne Zhang
• Abstract summary: Unsupervised semantic segmentation is a challenging task that segments images into semantic groups without manual annotation. We propose a novel approach called SmooSeg that harnesses self-supervised learning methods to model the closeness relationships among observations as smoothness signals. Our SmooSeg significantly outperforms STEGO in terms of pixel accuracy on three datasets.
• Score: 27.367986520072147
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Unsupervised semantic segmentation is a challenging task that segments images into semantic groups without manual annotation. Prior works have primarily focused on leveraging prior knowledge of semantic consistency or priori concepts from self-supervised learning methods, which often overlook the coherence property of image segments. In this paper, we demonstrate that the smoothness prior, asserting that close features in a metric space share the same semantics, can significantly simplify segmentation by casting unsupervised semantic segmentation as an energy minimization problem. Under this paradigm, we propose a novel approach called SmooSeg that harnesses self-supervised learning methods to model the closeness relationships among observations as smoothness signals. To effectively discover coherent semantic segments, we introduce a novel smoothness loss that promotes piecewise smoothness within segments while preserving discontinuities across different segments. Additionally, to further enhance segmentation quality, we design an asymmetric teacher-student style predictor that generates smoothly updated pseudo labels, facilitating an optimal fit between observations and labeling outputs. Thanks to the rich supervision cues of the smoothness prior, our SmooSeg significantly outperforms STEGO in terms of pixel accuracy on three datasets: COCOStuff (+14.9%), Cityscapes (+13.0%), and Potsdam-3 (+5.7%).

Related papers

• SemSim: Revisiting Weak-to-Strong Consistency from a Semantic Similarity Perspective for Semi-supervised Medical Image Segmentation [18.223854197580145]
  Semi-supervised learning (SSL) for medical image segmentation is a challenging yet highly practical task. We propose a novel framework based on FixMatch, named SemSim, powered by two appealing designs from semantic similarity perspective. We show that SemSim yields consistent improvements over the state-of-the-art methods across three public segmentation benchmarks.
  arXiv  Detail & Related papers  (2024-10-17T12:31:37Z)
• Auxiliary Tasks Enhanced Dual-affinity Learning for Weakly Supervised Semantic Segmentation [79.05949524349005]
  We propose AuxSegNet+, a weakly supervised auxiliary learning framework to explore the rich information from saliency maps. We also propose a cross-task affinity learning mechanism to learn pixel-level affinities from the saliency and segmentation feature maps.
  arXiv  Detail & Related papers  (2024-03-02T10:03:21Z)
• Scribble Hides Class: Promoting Scribble-Based Weakly-Supervised Semantic Segmentation with Its Class Label [16.745019028033518]
  We propose a class-driven scribble promotion network, which utilizes both scribble annotations and pseudo-labels informed by image-level classes and global semantics for supervision. Experiments on the ScribbleSup dataset with different qualities of scribble annotations outperform all the previous methods, demonstrating the superiority and robustness of our method.
  arXiv  Detail & Related papers  (2024-02-27T14:51:56Z)
• MetaSeg: Content-Aware Meta-Net for Omni-Supervised Semantic Segmentation [17.59676962334776]
  Noisy labels, inevitably existing in pseudo segmentation labels generated from weak object-level annotations, severely hampers model optimization for semantic segmentation. Inspired by recent advances in meta learning, we argue that rather than struggling to tolerate noise hidden behind clean labels passively, a more feasible solution would be to find out the noisy regions actively. We present a novel meta learning based semantic segmentation method, MetaSeg, that comprises a primary content-aware meta-net (CAM-Net) to sever as a noise indicator for an arbitrary segmentation model counterpart.
  arXiv  Detail & Related papers  (2024-01-22T07:31:52Z)
• Semi-supervised Semantic Segmentation Meets Masked Modeling:Fine-grained Locality Learning Matters in Consistency Regularization [31.333862320143968]
  Semi-supervised semantic segmentation aims to utilize limited labeled images and abundant unlabeled images to achieve label-efficient learning. We propose a novel framework called textttMaskMatch, which enables fine-grained locality learning to achieve better dense segmentation.
  arXiv  Detail & Related papers  (2023-12-14T03:28:53Z)
• LESS: Label-Efficient Semantic Segmentation for LiDAR Point Clouds [62.49198183539889]
  We propose a label-efficient semantic segmentation pipeline for outdoor scenes with LiDAR point clouds. Our method co-designs an efficient labeling process with semi/weakly supervised learning. Our proposed method is even highly competitive compared to the fully supervised counterpart with 100% labels.
  arXiv  Detail & Related papers  (2022-10-14T19:13:36Z)
• A Survey on Label-efficient Deep Segmentation: Bridging the Gap between Weak Supervision and Dense Prediction [115.9169213834476]
  This paper offers a comprehensive review on label-efficient segmentation methods. We first develop a taxonomy to organize these methods according to the supervision provided by different types of weak labels. Next, we summarize the existing label-efficient segmentation methods from a unified perspective.
  arXiv  Detail & Related papers  (2022-07-04T06:21:01Z)
• Improving Semi-Supervised and Domain-Adaptive Semantic Segmentation with Self-Supervised Depth Estimation [94.16816278191477]
  We present a framework for semi-adaptive and domain-supervised semantic segmentation. It is enhanced by self-supervised monocular depth estimation trained only on unlabeled image sequences. We validate the proposed model on the Cityscapes dataset.
  arXiv  Detail & Related papers  (2021-08-28T01:33:38Z)
• Leveraging Auxiliary Tasks with Affinity Learning for Weakly Supervised Semantic Segmentation [88.49669148290306]
  We propose a novel weakly supervised multi-task framework called AuxSegNet to leverage saliency detection and multi-label image classification as auxiliary tasks. Inspired by their similar structured semantics, we also propose to learn a cross-task global pixel-level affinity map from the saliency and segmentation representations. The learned cross-task affinity can be used to refine saliency predictions and propagate CAM maps to provide improved pseudo labels for both tasks.
  arXiv  Detail & Related papers  (2021-07-25T11:39:58Z)
• A Simple Baseline for Semi-supervised Semantic Segmentation with Strong Data Augmentation [74.8791451327354]
  We propose a simple yet effective semi-supervised learning framework for semantic segmentation. A set of simple design and training techniques can collectively improve the performance of semi-supervised semantic segmentation significantly. Our method achieves state-of-the-art results in the semi-supervised settings on the Cityscapes and Pascal VOC datasets.
  arXiv  Detail & Related papers  (2021-04-15T06:01:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.