ModuSeg: Decoupling Object Discovery and Semantic Retrieval for Training-Free Weakly Supervised Segmentation

• URL: http://arxiv.org/abs/2604.07021v2
• Date: Thu, 16 Apr 2026 05:22:23 GMT
• Title: ModuSeg: Decoupling Object Discovery and Semantic Retrieval for Training-Free Weakly Supervised Segmentation
• Abstract summary: Weakly supervised semantic segmentation aims to achieve pixel-level predictions using image-level labels.<n>We present ModuSeg, a training-free weakly supervised semantic segmentation framework centered on explicitly decoupling object discovery and semantic assignment.
• Score: 8.671473711194741
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Weakly supervised semantic segmentation aims to achieve pixel-level predictions using image-level labels. Existing methods typically entangle semantic recognition and object localization, which often leads models to focus exclusively on sparse discriminative regions. Although foundation models show immense potential, many approaches still follow the tightly coupled optimization paradigm, struggling to effectively alleviate pseudo-label noise and often relying on time-consuming multi-stage retraining or unstable end-to-end joint optimization. To address the above challenges, we present ModuSeg, a training-free weakly supervised semantic segmentation framework centered on explicitly decoupling object discovery and semantic assignment. Specifically, we integrate a general mask proposer to extract geometric proposals with reliable boundaries, while leveraging semantic foundation models to construct an offline feature bank, transforming segmentation into a non-parametric feature retrieval process. Furthermore, we propose semantic boundary purification and soft-masked feature aggregation strategies to effectively mitigate boundary ambiguity and quantization errors, thereby extracting high-quality category prototypes. Extensive experiments demonstrate that the proposed decoupled architecture better preserves fine boundaries without parameter fine-tuning and achieves highly competitive performance on standard benchmark datasets. Code is available at https://github.com/Autumnair007/ModuSeg.

Related papers

• Optimizing against Infeasible Inclusions from Data for Semantic Segmentation through Morphology [58.17907376475596]
  State-of-the-art semantic segmentation models are typically optimized in a data-driven fashion.<n>InSeIn extracts explicit inclusion constraints that govern spatial class relations from the semantic segmentation training set at hand.<n>It then enforces a morphological yet differentiable loss that penalizes violations of these constraints during training to promote prediction feasibility.
  arXiv  Detail & Related papers  (2024-08-26T22:39:08Z)
• MetaSeg: Content-Aware Meta-Net for Omni-Supervised Semantic Segmentation [17.59676962334776]
  Noisy labels, inevitably existing in pseudo segmentation labels generated from weak object-level annotations, severely hampers model optimization for semantic segmentation. Inspired by recent advances in meta learning, we argue that rather than struggling to tolerate noise hidden behind clean labels passively, a more feasible solution would be to find out the noisy regions actively. We present a novel meta learning based semantic segmentation method, MetaSeg, that comprises a primary content-aware meta-net (CAM-Net) to sever as a noise indicator for an arbitrary segmentation model counterpart.
  arXiv  Detail & Related papers  (2024-01-22T07:31:52Z)
• Semantic Connectivity-Driven Pseudo-labeling for Cross-domain Segmentation [89.41179071022121]
  Self-training is a prevailing approach in cross-domain semantic segmentation. We propose a novel approach called Semantic Connectivity-driven pseudo-labeling. This approach formulates pseudo-labels at the connectivity level and thus can facilitate learning structured and low-noise semantics.
  arXiv  Detail & Related papers  (2023-12-11T12:29:51Z)
• Label-efficient Segmentation via Affinity Propagation [27.016747627689288]
  Weakly-supervised segmentation with label-efficient sparse annotations has attracted increasing research attention to reduce the cost of laborious pixel-wise labeling process. We formulate the affinity modeling as an affinity propagation process, and propose a local and a global pairwise affinity terms to generate accurate soft pseudo labels. An efficient algorithm is also developed to reduce significantly the computational cost.
  arXiv  Detail & Related papers  (2023-10-16T15:54:09Z)
• Semantics-Aware Dynamic Localization and Refinement for Referring Image Segmentation [102.25240608024063]
  Referring image segments an image from a language expression. We develop an algorithm that shifts from being localization-centric to segmentation-language. Compared to its counterparts, our method is more versatile yet effective.
  arXiv  Detail & Related papers  (2023-03-11T08:42:40Z)
• Semantic Diffusion Network for Semantic Segmentation [1.933681537640272]
  We introduce an operator-level approach to enhance semantic boundary awareness. We propose a novel learnable approach called semantic diffusion network (SDN) Our SDN aims to construct a differentiable mapping from the original feature to the inter-class boundary-enhanced feature.
  arXiv  Detail & Related papers  (2023-02-04T01:39:16Z)
• BoundarySqueeze: Image Segmentation as Boundary Squeezing [104.43159799559464]
  We propose a novel method for fine-grained high-quality image segmentation of both objects and scenes. Inspired by dilation and erosion from morphological image processing techniques, we treat the pixel level segmentation problems as squeezing object boundary. Our method yields large gains on COCO, Cityscapes, for both instance and semantic segmentation and outperforms previous state-of-the-art PointRend in both accuracy and speed under the same setting.
  arXiv  Detail & Related papers  (2021-05-25T04:58:51Z)
• Attentional Prototype Inference for Few-Shot Segmentation [128.45753577331422]
  We propose attentional prototype inference (API), a probabilistic latent variable framework for few-shot segmentation. We define a global latent variable to represent the prototype of each object category, which we model as a probabilistic distribution. We conduct extensive experiments on four benchmarks, where our proposal obtains at least competitive and often better performance than state-of-the-art prototype-based methods.
  arXiv  Detail & Related papers  (2021-05-14T06:58:44Z)
• Mixup-CAM: Weakly-supervised Semantic Segmentation via Uncertainty Regularization [73.03956876752868]
  We propose a principled and end-to-end train-able framework to allow the network to pay attention to other parts of the object. Specifically, we introduce the mixup data augmentation scheme into the classification network and design two uncertainty regularization terms to better interact with the mixup strategy.
  arXiv  Detail & Related papers  (2020-08-03T21:19:08Z)
• Improving Semantic Segmentation via Decoupled Body and Edge Supervision [89.57847958016981]
  Existing semantic segmentation approaches either aim to improve the object's inner consistency by modeling the global context, or refine objects detail along their boundaries by multi-scale feature fusion. In this paper, a new paradigm for semantic segmentation is proposed. Our insight is that appealing performance of semantic segmentation requires textitexplicitly modeling the object textitbody and textitedge, which correspond to the high and low frequency of the image. We show that the proposed framework with various baselines or backbone networks leads to better object inner consistency and object boundaries.
  arXiv  Detail & Related papers  (2020-07-20T12:11:22Z)
• Towards Bounding-Box Free Panoptic Segmentation [16.4548904544277]
  We introduce a new Bounding-Box Free Network (BBFNet) for panoptic segmentation. BBFNet predicts coarse watershed levels and uses them to detect large instance candidates where boundaries are well defined. For smaller instances, whose boundaries are less reliable, BBFNet also predicts instance centers by means of Hough voting followed by mean-shift to reliably detect small objects.
  arXiv  Detail & Related papers  (2020-02-18T16:34:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.