Towards Instance Segmentation with Polygon Detection Transformers

• URL: http://arxiv.org/abs/2603.09245v1
• Date: Tue, 10 Mar 2026 06:18:33 GMT
• Title: Towards Instance Segmentation with Polygon Detection Transformers
• Abstract summary: We present a Polygon Detection Transformer (Poly-DETR) to reformulate instance segmentation as sparse regression via Polar Representation.<n>Considering the box-to-polygon reference shift in Detection Transformers, we propose Polar Deformable Attention and Position-Aware Training Scheme.<n>Compared with state-of-the-art polar-based methods, Poly-DETR achieves a 4.7 mAP improvement on MS COCO test-dev.
• Score: 22.568845137284978
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: One of the bottlenecks for instance segmentation today lies in the conflicting requirements of high-resolution inputs and lightweight, real-time inference. To address this bottleneck, we present a Polygon Detection Transformer (Poly-DETR) to reformulate instance segmentation as sparse vertex regression via Polar Representation, thereby eliminating the reliance on dense pixel-wise mask prediction. Considering the box-to-polygon reference shift in Detection Transformers, we propose Polar Deformable Attention and Position-Aware Training Scheme to dynamically update supervision and focus attention on boundary cues. Compared with state-of-the-art polar-based methods, Poly-DETR achieves a 4.7 mAP improvement on MS COCO test-dev. Moreover, we construct a parallel mask-based counterpart to support a systematic comparison between polar and mask representations. Experimental results show that Poly-DETR is more lightweight in high-resolution scenarios, reducing memory consumption by almost half on Cityscapes dataset. Notably, on PanNuke (cell segmentation) and SpaceNet (building footprints) datasets, Poly-DETR surpasses its mask-based counterpart on all metrics, which validates its advantage on regular-shaped instances in domain-specific settings.

Related papers

• Global Collinearity-aware Polygonizer for Polygonal Building Mapping in Remote Sensing [18.151134198549574]
  This paper addresses the challenge of mapping polygonal buildings from remote sensing images.<n>It introduces a novel algorithm, the Global Collinearity-aware Polygonizer (GCP)
  arXiv  Detail & Related papers  (2025-05-02T16:49:07Z)
• Boosting Cross-Domain Point Classification via Distilling Relational Priors from 2D Transformers [59.0181939916084]
  Traditional 3D networks mainly focus on local geometric details and ignore the topological structure between local geometries. We propose a novel Priors Distillation (RPD) method to extract priors from the well-trained transformers on massive images. Experiments on the PointDA-10 and the Sim-to-Real datasets verify that the proposed method consistently achieves the state-of-the-art performance of UDA for point cloud classification.
  arXiv  Detail & Related papers  (2024-07-26T06:29:09Z)
• Semi-Supervised Coupled Thin-Plate Spline Model for Rotation Correction and Beyond [84.56978780892783]
  We propose CoupledTPS, which iteratively couples multiple TPS with limited control points into a more flexible and powerful transformation. In light of the laborious annotation cost, we develop a semi-supervised learning scheme to improve warping quality by exploiting unlabeled data. Experiments demonstrate the superiority and universality of CoupledTPS over the existing state-of-the-art solutions for rotation correction.
  arXiv  Detail & Related papers  (2024-01-24T13:03:28Z)
• Real-time instance segmentation with polygons using an Intersection-over-Union loss [13.020122353444497]
  We improve over CenterPoly by enhancing the classical regression L1 loss with a novel region-based loss and a novel order loss. Experiments show that using a combination of a regression loss and a region-based loss allows significant improvements on the Cityscapes and IDD test set.
  arXiv  Detail & Related papers  (2023-05-09T14:43:38Z)
• DPPD: Deformable Polar Polygon Object Detection [3.9236649268347765]
  We develop a novel Deformable Polar Polygon Object Detection method (DPPD) to detect objects in polygon shapes. DPPD has been demonstrated successfully in various object detection tasks for autonomous driving.
  arXiv  Detail & Related papers  (2023-04-05T06:43:41Z)
• PolyFormer: Referring Image Segmentation as Sequential Polygon Generation [20.55281741205142]
  Instead of directly predicting pixel-level segmentation masks, the problem of referring image segmentation is formulated as sequential polygon generation. This is enabled by a new sequence-to-sequence framework, Polygon Transformer (PolyFormer), which takes a sequence of image patches and text query tokens as input. For more accurate geometric localization, we propose a regression-based decoder, which predicts the precise floating-point coordinates directly.
  arXiv  Detail & Related papers  (2023-02-14T23:00:25Z)
• Lesion-aware Dynamic Kernel for Polyp Segmentation [49.63274623103663]
  We propose a lesion-aware dynamic network (LDNet) for polyp segmentation. It is a traditional u-shape encoder-decoder structure incorporated with a dynamic kernel generation and updating scheme. This simple but effective scheme endows our model with powerful segmentation performance and generalization capability.
  arXiv  Detail & Related papers  (2023-01-12T09:53:57Z)
• PnP-DETR: Towards Efficient Visual Analysis with Transformers [146.55679348493587]
  Recently, DETR pioneered the solution vision tasks with transformers, it directly translates the image feature map into the object result. Recent transformer-based image recognition model andTT show consistent efficiency gain.
  arXiv  Detail & Related papers  (2021-09-15T01:10:30Z)
• Polyp-PVT: Polyp Segmentation with Pyramid Vision Transformers [124.01928050651466]
  We propose a new type of polyp segmentation method, named Polyp-PVT. The proposed model, named Polyp-PVT, effectively suppresses noises in the features and significantly improves their expressive capabilities.
  arXiv  Detail & Related papers  (2021-08-16T07:09:06Z)
• PolarMask++: Enhanced Polar Representation for Single-Shot Instance Segmentation and Beyond [47.518550130850755]
  PolarMask reformulates the instance segmentation problem as predicting the contours of objects in the polar coordinate. Two modules are carefully designed (i.e. soft polar centerness and polar IoU loss) to sample high-quality center examples. PolarMask is fully convolutional and can be easily embedded into most off-the-shelf detection methods.
  arXiv  Detail & Related papers  (2021-05-05T16:55:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.