Instance Semantic Segmentation Benefits from Generative Adversarial Networks

• URL: http://arxiv.org/abs/2010.13757v2
• Date: Sat, 4 Dec 2021 05:00:33 GMT
• Title: Instance Semantic Segmentation Benefits from Generative Adversarial Networks
• Authors: Quang H. Le, Kamal Youcef-Toumi, Dzmitry Tsetserukou, Ali Jahanian
• Abstract summary: We define the problem of predicting masks as a GANs game framework. A segmentation network generates the masks, and a discriminator network decides on the quality of the masks. We report on cellphone recycling, autonomous driving, large-scale object detection, and medical glands.
• Score: 13.295723883560122
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In design of instance segmentation networks that reconstruct masks, segmentation is often taken as its literal definition -- assigning each pixel a label. This has led to thinking the problem as a template matching one with the goal of minimizing the loss between the reconstructed and the ground truth pixels. Rethinking reconstruction networks as a generator, we define the problem of predicting masks as a GANs game framework: A segmentation network generates the masks, and a discriminator network decides on the quality of the masks. To demonstrate this game, we show effective modifications on the general segmentation framework in Mask R-CNN. We find that playing the game in feature space is more effective than the pixel space leading to stable training between the discriminator and the generator, predicting object coordinates should be replaced by predicting contextual regions for objects, and overall the adversarial loss helps the performance and removes the need for any custom settings per different data domain. We test our framework in various domains and report on cellphone recycling, autonomous driving, large-scale object detection, and medical glands. We observe in general GANs yield masks that account for crispier boundaries, clutter, small objects, and details, being in domain of regular shapes or heterogeneous and coalescing shapes. Our code for reproducing the results is available publicly.

Related papers

• LAC-Net: Linear-Fusion Attention-Guided Convolutional Network for Accurate Robotic Grasping Under the Occlusion [79.22197702626542]
  This paper introduces a framework that explores amodal segmentation for robotic grasping in cluttered scenes. We propose a Linear-fusion Attention-guided Convolutional Network (LAC-Net) The results on different datasets show that our method achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-08-06T14:50:48Z)
• MaskCD: A Remote Sensing Change Detection Network Based on Mask Classification [29.15203530375882]
  Change (CD) from remote sensing (RS) images using deep learning has been widely investigated in the literature. We propose MaskCD to detect changed areas by adaptively generating categorized masks from input image pairs. It reconstructs the desired changed objects by decoding the pixel-wise representations into learnable mask proposals.
  arXiv  Detail & Related papers  (2024-04-18T11:05:15Z)
• Mask2Anomaly: Mask Transformer for Universal Open-set Segmentation [29.43462426812185]
  We propose a paradigm change by shifting from a per-pixel classification to a mask classification. Our mask-based method, Mask2Anomaly, demonstrates the feasibility of integrating a mask-classification architecture. By comprehensive qualitative and qualitative evaluation, we show Mask2Anomaly achieves new state-of-the-art results.
  arXiv  Detail & Related papers  (2023-09-08T20:07:18Z)
• Towards Robust Video Object Segmentation with Adaptive Object Calibration [18.094698623128146]
  Video object segmentation (VOS) aims at segmenting objects in all target frames of a video, given annotated object masks of reference frames. We propose a new deep network, which can adaptively construct object representations and calibrate object masks to achieve stronger robustness. Our model achieves the state-of-the-art performance among existing published works, and also exhibits superior robustness against perturbations.
  arXiv  Detail & Related papers  (2022-07-02T17:51:29Z)
• Discovering Object Masks with Transformers for Unsupervised Semantic Segmentation [75.00151934315967]
  MaskDistill is a novel framework for unsupervised semantic segmentation. Our framework does not latch onto low-level image cues and is not limited to object-centric datasets.
  arXiv  Detail & Related papers  (2022-06-13T17:59:43Z)
• Differentiable Soft-Masked Attention [115.5770357189209]
  "Differentiable Soft-Masked Attention" is used for the task of WeaklySupervised Video Object. We develop a transformer-based network for training, but can also benefit from cycle consistency training on a video with just one annotated frame.
  arXiv  Detail & Related papers  (2022-06-01T02:05:13Z)
• A Unified Architecture of Semantic Segmentation and Hierarchical Generative Adversarial Networks for Expression Manipulation [52.911307452212256]
  We develop a unified architecture of semantic segmentation and hierarchical GANs. A unique advantage of our framework is that on forward pass the semantic segmentation network conditions the generative model. We evaluate our method on two challenging facial expression translation benchmarks, AffectNet and RaFD, and a semantic segmentation benchmark, CelebAMask-HQ.
  arXiv  Detail & Related papers  (2021-12-08T22:06:31Z)
• Context-Aware Mixup for Domain Adaptive Semantic Segmentation [52.1935168534351]
  Unsupervised domain adaptation (UDA) aims to adapt a model of the labeled source domain to an unlabeled target domain. We propose end-to-end Context-Aware Mixup (CAMix) for domain adaptive semantic segmentation. Experimental results show that the proposed method outperforms the state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2021-08-08T03:00:22Z)
• Spatial Attention Pyramid Network for Unsupervised Domain Adaptation [66.75008386980869]
  Unsupervised domain adaptation is critical in various computer vision tasks. We design a new spatial attention pyramid network for unsupervised domain adaptation. Our method performs favorably against the state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2020-03-29T09:03:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.