Semantic Editing On Segmentation Map Via Multi-Expansion Loss

• URL: http://arxiv.org/abs/2010.08128v1
• Date: Fri, 16 Oct 2020 03:12:26 GMT
• Title: Semantic Editing On Segmentation Map Via Multi-Expansion Loss
• Authors: Jianfeng He, Xuchao Zhang, Shuo Lei, Shuhui Wang, Qingming Huang, Chang-Tien Lu, Bei Xiao
• Abstract summary: This paper aims to improve quality of edited segmentation map conditioned on semantic inputs. We propose MExGAN for semantic editing on segmentation map, which uses a novel Multi-Expansion (MEx) loss. Experiments on semantic editing on segmentation map and natural image inpainting show competitive results on four datasets.
• Score: 98.1131339357174
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Semantic editing on segmentation map has been proposed as an intermediate interface for image generation, because it provides flexible and strong assistance in various image generation tasks. This paper aims to improve quality of edited segmentation map conditioned on semantic inputs. Even though recent studies apply global and local adversarial losses extensively to generate images for higher image quality, we find that they suffer from the misalignment of the boundary area in the mask area. To address this, we propose MExGAN for semantic editing on segmentation map, which uses a novel Multi-Expansion (MEx) loss implemented by adversarial losses on MEx areas. Each MEx area has the mask area of the generation as the majority and the boundary of original context as the minority. To boost convenience and stability of MEx loss, we further propose an Approximated MEx (A-MEx) loss. Besides, in contrast to previous model that builds training data for semantic editing on segmentation map with part of the whole image, which leads to model performance degradation, MExGAN applies the whole image to build the training data. Extensive experiments on semantic editing on segmentation map and natural image inpainting show competitive results on four datasets.

Related papers

• MiSuRe is all you need to explain your image segmentation [3.363736106489207]
  We propose MiSuRe as an algorithm to generate saliency maps for image segmentation. The goal of the saliency maps generated by MiSuRe is to get rid of irrelevant regions, and only highlight those regions in the input image which are crucial to the image segmentation decision.
  arXiv  Detail & Related papers  (2024-06-18T00:45:54Z)
• MixReorg: Cross-Modal Mixed Patch Reorganization is a Good Mask Learner for Open-World Semantic Segmentation [110.09800389100599]
  We propose MixReorg, a novel and straightforward pre-training paradigm for semantic segmentation. Our approach involves generating fine-grained patch-text pairs data by mixing image patches while preserving the correspondence between patches and text. With MixReorg as a mask learner, conventional text-supervised semantic segmentation models can achieve highly generalizable pixel-semantic alignment ability.
  arXiv  Detail & Related papers  (2023-08-09T09:35:16Z)
• Self-Supervised Correction Learning for Semi-Supervised Biomedical Image Segmentation [84.58210297703714]
  We propose a self-supervised correction learning paradigm for semi-supervised biomedical image segmentation. We design a dual-task network, including a shared encoder and two independent decoders for segmentation and lesion region inpainting. Experiments on three medical image segmentation datasets for different tasks demonstrate the outstanding performance of our method.
  arXiv  Detail & Related papers  (2023-01-12T08:19:46Z)
• High-Quality Entity Segmentation [110.55724145851725]
  CropFormer is designed to tackle the intractability of instance-level segmentation on high-resolution images. It improves mask prediction by fusing high-res image crops that provide more fine-grained image details and the full image. With CropFormer, we achieve a significant AP gain of $1.9$ on the challenging entity segmentation task.
  arXiv  Detail & Related papers  (2022-11-10T18:58:22Z)
• A Generalist Framework for Panoptic Segmentation of Images and Videos [61.61453194912186]
  We formulate panoptic segmentation as a discrete data generation problem, without relying on inductive bias of the task. A diffusion model is proposed to model panoptic masks, with a simple architecture and generic loss function. Our method is capable of modeling video (in a streaming setting) and thereby learns to track object instances automatically.
  arXiv  Detail & Related papers  (2022-10-12T16:18:25Z)
• Weakly-supervised segmentation of referring expressions [81.73850439141374]
  Text grounded semantic SEGmentation learns segmentation masks directly from image-level referring expressions without pixel-level annotations. Our approach demonstrates promising results for weakly-supervised referring expression segmentation on the PhraseCut and RefCOCO datasets.
  arXiv  Detail & Related papers  (2022-05-10T07:52:24Z)
• PNM: Pixel Null Model for General Image Segmentation [17.971090313814447]
  We present a prior model that weights each pixel according to its probability of being correctly classified by a random segmenter. Experiments on semantic, instance, and panoptic segmentation tasks over three datasets confirm that PNM consistently improves the segmentation quality. We propose a new metric, textitPNM IoU, which perceives the boundary sharpness and better reflects the model segmentation performance in error-prone regions.
  arXiv  Detail & Related papers  (2022-03-13T15:17:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.