Adversarial Segmentation Loss for Sketch Colorization
- URL: http://arxiv.org/abs/2102.06192v1
- Date: Thu, 11 Feb 2021 18:54:56 GMT
- Title: Adversarial Segmentation Loss for Sketch Colorization
- Authors: Samet Hicsonmez, Nermin Samet, Emre Akbas, Pinar Duygulu
- Abstract summary: We introduce a new method for generating color images from sketches or edge maps.
We argue that segmentation information could provide valuable guidance for sketch colorization.
Our model improves its baseline up to 35 points on the FID metric.
- Score: 14.681690787310103
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We introduce a new method for generating color images from sketches or edge
maps. Current methods either require some form of additional user-guidance or
are limited to the "paired" translation approach. We argue that segmentation
information could provide valuable guidance for sketch colorization. To this
end, we propose to leverage semantic image segmentation, as provided by a
general purpose panoptic segmentation network, to create an additional
adversarial loss function. Our loss function can be integrated to any baseline
GAN model. Our method is not limited to datasets that contain segmentation
labels, and it can be trained for "unpaired" translation tasks. We show the
effectiveness of our method on four different datasets spanning scene level
indoor, outdoor, and children book illustration images using qualitative,
quantitative and user study analysis. Our model improves its baseline up to 35
points on the FID metric. Our code and pretrained models can be found at
https://github.com/giddyyupp/AdvSegLoss.
Related papers
- Joint Dense-Point Representation for Contour-Aware Graph Segmentation [2.138299283227551]
We present a novel methodology that combines graph and dense segmentation techniques by jointly learning both point and pixel contour representations.
This addresses deficiencies in typical graph segmentation methods where misaligned objectives restrict the network from learning discriminative and contour features.
Our approach is validated on several Chest X-ray datasets, demonstrating clear improvements in segmentation stability and accuracy.
arXiv Detail & Related papers (2023-06-21T10:07:17Z) - Improving Sketch Colorization using Adversarial Segmentation Consistency [12.55972766570669]
We propose a new method for producing color images from sketches.
We leverage semantic image segmentation from a general-purpose panoptic segmentation network to generate an additional adversarial loss function.
Our method is not restricted to datasets with segmentation labels and can be applied to unpaired translation tasks as well.
arXiv Detail & Related papers (2023-01-20T14:07:30Z) - From colouring-in to pointillism: revisiting semantic segmentation
supervision [48.637031591058175]
We propose a pointillist approach for semantic segmentation annotation, where only point-wise yes/no questions are answered.
We collected and released 22.6M point labels over 4,171 classes on the Open Images dataset.
arXiv Detail & Related papers (2022-10-25T16:42:03Z) - Distilling Ensemble of Explanations for Weakly-Supervised Pre-Training
of Image Segmentation Models [54.49581189337848]
We propose a method to enable the end-to-end pre-training for image segmentation models based on classification datasets.
The proposed method leverages a weighted segmentation learning procedure to pre-train the segmentation network en masse.
Experiment results show that, with ImageNet accompanied by PSSL as the source dataset, the proposed end-to-end pre-training strategy successfully boosts the performance of various segmentation models.
arXiv Detail & Related papers (2022-07-04T13:02:32Z) - One Sketch for All: One-Shot Personalized Sketch Segmentation [84.45203849671003]
We present the first one-shot personalized sketch segmentation method.
We aim to segment all sketches belonging to the same category with a single sketch with a given part annotation.
We preserve the parts semantics embedded in the exemplar, and we are robust to input style and abstraction.
arXiv Detail & Related papers (2021-12-20T20:10:44Z) - Sparse Object-level Supervision for Instance Segmentation with Pixel
Embeddings [4.038011160363972]
Most state-of-the-art instance segmentation methods have to be trained on densely annotated images.
We propose a proposal-free segmentation approach based on non-spatial embeddings.
We evaluate the proposed method on challenging 2D and 3D segmentation problems in different microscopy modalities.
arXiv Detail & Related papers (2021-03-26T16:36:56Z) - Group-Wise Semantic Mining for Weakly Supervised Semantic Segmentation [49.90178055521207]
This work addresses weakly supervised semantic segmentation (WSSS), with the goal of bridging the gap between image-level annotations and pixel-level segmentation.
We formulate WSSS as a novel group-wise learning task that explicitly models semantic dependencies in a group of images to estimate more reliable pseudo ground-truths.
In particular, we devise a graph neural network (GNN) for group-wise semantic mining, wherein input images are represented as graph nodes.
arXiv Detail & Related papers (2020-12-09T12:40:13Z) - A Few Guidelines for Incremental Few-Shot Segmentation [57.34237650765928]
Given a pretrained segmentation model and few images containing novel classes, our goal is to learn to segment novel classes while retaining the ability to segment previously seen ones.
We show how the main problems of end-to-end training in this scenario are.
i) the drift of the batch-normalization statistics toward novel classes that we can fix with batch renormalization and.
ii) the forgetting of old classes, that we can fix with regularization strategies.
arXiv Detail & Related papers (2020-11-30T20:45:56Z) - Contrastive Rendering for Ultrasound Image Segmentation [59.23915581079123]
The lack of sharp boundaries in US images remains an inherent challenge for segmentation.
We propose a novel and effective framework to improve boundary estimation in US images.
Our proposed method outperforms state-of-the-art methods and has the potential to be used in clinical practice.
arXiv Detail & Related papers (2020-10-10T07:14:03Z) - Learning to Segment from Scribbles using Multi-scale Adversarial
Attention Gates [16.28285034098361]
Weakly-supervised learning can train models by relying on weaker forms of annotation, such as scribbles.
We train a multi-scale GAN to generate realistic segmentation masks at multiple resolutions, while we use scribbles to learn their correct position in the image.
Central to the model's success is a novel attention gating mechanism, which we condition with adversarial signals to act as a shape prior.
arXiv Detail & Related papers (2020-07-02T14:39:08Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.