Dense Point-to-Mask Optimization with Reinforced Point Selection for Crowd Instance Segmentation

• URL: http://arxiv.org/abs/2604.01742v1
• Date: Thu, 02 Apr 2026 08:02:49 GMT
• Title: Dense Point-to-Mask Optimization with Reinforced Point Selection for Crowd Instance Segmentation
• Abstract summary: We propose Point-to-Mask Optimization (DPMO) to generate dense instance segmentation from point annotations.<n>Then, to predict instance segmentation in dense crowds, we propose a Reinforced Point Selection (RPS) framework.<n>We achieve state-of-the-art crowd instance segmentation performance on ShanghaiTech, UCF-QNRF, JHU-CROWD++, and NWPU-Crowd datasets.
• Score: 45.48207887391471
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Crowd instance segmentation is a crucial task with a wide range of applications, including surveillance and transportation. Currently, point labels are common in crowd datasets, while region labels (e.g., boxes) are rare and inaccurate. The masks obtained through segmentation help to improve the accuracy of region labels and resolve the correspondence between individual location coordinates and crowd density maps. However, directly applying currently popular large foundation models such as SAM does not yield ideal results in dense crowds. To this end, we first propose Dense Point-to-Mask Optimization (DPMO), which integrates SAM with the Nearest Neighbor Exclusive Circle (NNEC) constraint to generate dense instance segmentation from point annotations. With DPMO and manual correction, we obtain mask annotations from the existing point annotations for traditional crowd datasets. Then, to predict instance segmentation in dense crowds, we propose a Reinforced Point Selection (RPS) framework trained with Group Relative Policy Optimization (GRPO), which selects the best predicted point from a sampling of the initial point prediction. Through extensive experiments, we achieve state-of-the-art crowd instance segmentation performance on ShanghaiTech, UCF-QNRF, JHU-CROWD++, and NWPU-Crowd datasets. Furthermore, we design new loss functions supervised by masks that boost counting performance across different models, demonstrating the significant role of mask annotations in enhancing counting accuracy.

Related papers

• Exclusivity-Guided Mask Learning for Semi-Supervised Crowd Instance Segmentation and Counting [46.06620846193129]
  We propose an Exclusion-Constrained Dual-Prompt SAM (EDP-SAM) to generate mask supervision for current datasets.<n>We then propose Exclusivity-Guided Mask Learning (XMask), which enforces spatial separation through a discriminative mask objective.<n>Building on XMask, we present a semi-supervised crowd counting framework that uses instance mask priors as pseudo-labels.
  arXiv  Detail & Related papers  (2026-03-17T08:25:57Z)
• Bridge the Points: Graph-based Few-shot Segment Anything Semantically [79.1519244940518]
  Recent advancements in pre-training techniques have enhanced the capabilities of vision foundation models. Recent studies extend the SAM to Few-shot Semantic segmentation (FSS) We propose a simple yet effective approach based on graph analysis.
  arXiv  Detail & Related papers  (2024-10-09T15:02:28Z)
• Robust Zero-Shot Crowd Counting and Localization With Adaptive Resolution SAM [55.93697196726016]
  We propose a simple yet effective crowd counting method by utilizing the Segment-Everything-Everywhere Model (SEEM) We show that SEEM's performance in dense crowd scenes is limited, primarily due to the omission of many persons in high-density areas. Our proposed method achieves the best unsupervised performance in crowd counting, while also being comparable to some supervised methods.
  arXiv  Detail & Related papers  (2024-02-27T13:55:17Z)
• PWISeg: Point-based Weakly-supervised Instance Segmentation for Surgical Instruments [27.89003436883652]
  We propose a weakly-supervised surgical instrument segmentation approach, named Point-based Weakly-supervised Instance (PWISeg) PWISeg adopts an FCN-based architecture with point-to-box and point-to-mask branches to model the relationships between feature points and bounding boxes. Based on this, we propose a key pixel association loss and a key pixel distribution loss, driving the point-to-mask branch to generate more accurate segmentation predictions.
  arXiv  Detail & Related papers  (2023-11-16T11:48:29Z)
• Semantic Attention and Scale Complementary Network for Instance Segmentation in Remote Sensing Images [54.08240004593062]
  We propose an end-to-end multi-category instance segmentation model, which consists of a Semantic Attention (SEA) module and a Scale Complementary Mask Branch (SCMB) SEA module contains a simple fully convolutional semantic segmentation branch with extra supervision to strengthen the activation of interest instances on the feature map. SCMB extends the original single mask branch to trident mask branches and introduces complementary mask supervision at different scales.
  arXiv  Detail & Related papers  (2021-07-25T08:53:59Z)
• Pointly-Supervised Instance Segmentation [81.34136519194602]
  We propose point-based instance-level annotation, a new form of weak supervision for instance segmentation. It combines the standard bounding box annotation with labeled points that are uniformly sampled inside each bounding box. In our experiments, Mask R-CNN models trained on COCO, PASCAL VOC, Cityscapes, and LVIS with only 10 annotated points per object achieve 94%--98% of their fully-supervised performance.
  arXiv  Detail & Related papers  (2021-04-13T17:59:40Z)
• Mask-guided sample selection for Semi-Supervised Instance Segmentation [13.091166009687058]
  We propose a sample selection approach to decide which samples to annotate for semi-supervised instance segmentation. Our method consists in first predicting pseudo-masks for the unlabeled pool of samples, together with a score predicting the quality of the mask. We study which samples are better to annotate given the quality score, and show how our approach outperforms a random selection.
  arXiv  Detail & Related papers  (2020-08-25T14:44:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.