End-to-End Segmentation via Patch-wise Polygons Prediction

• URL: http://arxiv.org/abs/2112.02535v1
• Date: Sun, 5 Dec 2021 10:42:40 GMT
• Title: End-to-End Segmentation via Patch-wise Polygons Prediction
• Authors: Tal Shaharabany and Lior Wolf
• Abstract summary: The leading segmentation methods represent the output map as a pixel grid. We study an alternative representation in which the object edges are modeled, per image patch, as a polygon with $k$ vertices that is coupled with per-patch label probabilities.
• Score: 93.91375268580806
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The leading segmentation methods represent the output map as a pixel grid. We study an alternative representation in which the object edges are modeled, per image patch, as a polygon with $k$ vertices that is coupled with per-patch label probabilities. The vertices are optimized by employing a differentiable neural renderer to create a raster image. The delineated region is then compared with the ground truth segmentation. Our method obtains multiple state-of-the-art results: 76.26\% mIoU on the Cityscapes validation, 90.92\% IoU on the Vaihingen building segmentation benchmark, 66.82\% IoU for the MoNU microscopy dataset, and 90.91\% for the bird benchmark CUB. Our code for training and reproducing these results is attached as supplementary.

Related papers

• View-Consistent Hierarchical 3D Segmentation Using Ultrametric Feature Fields [52.08335264414515]
  We learn a novel feature field within a Neural Radiance Field (NeRF) representing a 3D scene. Our method takes view-inconsistent multi-granularity 2D segmentations as input and produces a hierarchy of 3D-consistent segmentations as output. We evaluate our method and several baselines on synthetic datasets with multi-view images and multi-granular segmentation, showcasing improved accuracy and viewpoint-consistency.
  arXiv  Detail & Related papers  (2024-05-30T04:14:58Z)
• CorrMatch: Label Propagation via Correlation Matching for Semi-Supervised Semantic Segmentation [73.89509052503222]
  This paper presents a simple but performant semi-supervised semantic segmentation approach, called CorrMatch. We observe that the correlation maps not only enable clustering pixels of the same category easily but also contain good shape information. We propose to conduct pixel propagation by modeling the pairwise similarities of pixels to spread the high-confidence pixels and dig out more. Then, we perform region propagation to enhance the pseudo labels with accurate class-agnostic masks extracted from the correlation maps.
  arXiv  Detail & Related papers  (2023-06-07T10:02:29Z)
• Random Edge Coding: One-Shot Bits-Back Coding of Large Labeled Graphs [24.761152163389735]
  We present a one-shot method for compressing large labeled graphs called Random Edge Coding. Experiments indicate Random Edge Coding can achieve competitive compression performance on real-world network datasets.
  arXiv  Detail & Related papers  (2023-05-16T12:23:18Z)
• PolyFormer: Referring Image Segmentation as Sequential Polygon Generation [20.55281741205142]
  Instead of directly predicting pixel-level segmentation masks, the problem of referring image segmentation is formulated as sequential polygon generation. This is enabled by a new sequence-to-sequence framework, Polygon Transformer (PolyFormer), which takes a sequence of image patches and text query tokens as input. For more accurate geometric localization, we propose a regression-based decoder, which predicts the precise floating-point coordinates directly.
  arXiv  Detail & Related papers  (2023-02-14T23:00:25Z)
• VoGE: A Differentiable Volume Renderer using Gaussian Ellipsoids for Analysis-by-Synthesis [62.47221232706105]
  We propose VoGE, which utilizes the Gaussian reconstruction kernels as volumetric primitives. To efficiently render via VoGE, we propose an approximate closeform solution for the volume density aggregation and a coarse-to-fine rendering strategy. VoGE outperforms SoTA when applied to various vision tasks, e.g., object pose estimation, shape/texture fitting, and reasoning.
  arXiv  Detail & Related papers  (2022-05-30T19:52:11Z)
• SegDiff: Image Segmentation with Diffusion Probabilistic Models [81.16986859755038]
  Diffusion Probabilistic Methods are employed for state-of-the-art image generation. We present a method for extending such models for performing image segmentation. The method learns end-to-end, without relying on a pre-trained backbone.
  arXiv  Detail & Related papers  (2021-12-01T10:17:25Z)
• PolyWorld: Polygonal Building Extraction with Graph Neural Networks in Satellite Images [10.661430927191205]
  This paper introduces PolyWorld, a neural network that directly extracts building vertices from an image and connects them correctly to create precise polygons. PolyWorld significantly outperforms the state-of-the-art in building polygonization.
  arXiv  Detail & Related papers  (2021-11-30T15:23:17Z)
• Edge-aware Graph Representation Learning and Reasoning for Face Parsing [61.5045850197694]
  Face parsing infers a pixel-wise label to each facial component, which has drawn much attention recently. Previous methods have shown their efficiency in face parsing, which however overlook the correlation among different face regions. We propose to model and reason the region-wise relations by learning graph representations.
  arXiv  Detail & Related papers  (2020-07-22T07:46:34Z)
• ContourRend: A Segmentation Method for Improving Contours by Rendering [10.13129256609938]
  Mask-based segmentation can not handle contour features well on a coarse prediction grid. We propose Contourend which adopts convolution contour to refine segmentation contours. Our method reaches 72.41% mean intersection over union (IoU) and surpasses baseline Polygon-GCN by 1.22%.
  arXiv  Detail & Related papers  (2020-07-15T02:16:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.