Semi-supervised reference-based sketch extraction using a contrastive learning framework

• URL: http://arxiv.org/abs/2407.14026v1
• Date: Fri, 19 Jul 2024 04:51:34 GMT
• Title: Semi-supervised reference-based sketch extraction using a contrastive learning framework
• Authors: Chang Wook Seo, Amirsaman Ashtari, Junyong Noh,
• Abstract summary: We propose a novel multi-modal sketch extraction method that can imitate the style of a given reference sketch with unpaired data training. Our method outperforms state-of-the-art sketch extraction methods and unpaired image translation methods in both quantitative and qualitative evaluations.
• Score: 6.20476217797034
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Sketches reflect the drawing style of individual artists; therefore, it is important to consider their unique styles when extracting sketches from color images for various applications. Unfortunately, most existing sketch extraction methods are designed to extract sketches of a single style. Although there have been some attempts to generate various style sketches, the methods generally suffer from two limitations: low quality results and difficulty in training the model due to the requirement of a paired dataset. In this paper, we propose a novel multi-modal sketch extraction method that can imitate the style of a given reference sketch with unpaired data training in a semi-supervised manner. Our method outperforms state-of-the-art sketch extraction methods and unpaired image translation methods in both quantitative and qualitative evaluations.

Related papers

• Stylized Face Sketch Extraction via Generative Prior with Limited Data [6.727433982111717]
  StyleSketch is a method for extracting high-resolution stylized sketches from a face image. Using the rich semantics of the deep features from a pretrained StyleGAN, we are able to train a sketch generator with 16 pairs of face and the corresponding sketch images.
  arXiv  Detail & Related papers  (2024-03-17T16:25:25Z)
• How to Handle Sketch-Abstraction in Sketch-Based Image Retrieval? [120.49126407479717]
  We propose a sketch-based image retrieval framework capable of handling sketch abstraction at varied levels. For granularity-level abstraction understanding, we dictate that the retrieval model should not treat all abstraction-levels equally. Our Acc.@q loss uniquely allows a sketch to narrow/broaden its focus in terms of how stringent the evaluation should be.
  arXiv  Detail & Related papers  (2024-03-11T23:08:29Z)
• Representative Feature Extraction During Diffusion Process for Sketch Extraction with One Example [6.520083224801834]
  We introduce DiffSketch, a method for generating a variety of stylized sketches from images. Our approach focuses on selecting representative features from the rich semantics of deep features within a pretrained diffusion model.
  arXiv  Detail & Related papers  (2024-01-09T05:22:15Z)
• DiffSketching: Sketch Control Image Synthesis with Diffusion Models [10.172753521953386]
  Deep learning models for sketch-to-image synthesis need to overcome the distorted input sketch without visual details. Our model matches sketches through the cross domain constraints, and uses a classifier to guide the image synthesis more accurately. Our model can beat GAN-based method in terms of generation quality and human evaluation, and does not rely on massive sketch-image datasets.
  arXiv  Detail & Related papers  (2023-05-30T07:59:23Z)
• I Know What You Draw: Learning Grasp Detection Conditioned on a Few Freehand Sketches [74.63313641583602]
  We propose a method to generate a potential grasp configuration relevant to the sketch-depicted objects. Our model is trained and tested in an end-to-end manner which is easy to be implemented in real-world applications.
  arXiv  Detail & Related papers  (2022-05-09T04:23:36Z)
• XCI-Sketch: Extraction of Color Information from Images for Generation of Colored Outlines and Sketches [0.0]
  We propose two methods to mimic human-drawn colored sketches. The first method renders colored outline sketches by applying image processing techniques aided by k-means color clustering. The second method uses a generative adversarial network to develop a model that can generate colored sketches from previously unobserved images.
  arXiv  Detail & Related papers  (2021-08-26T02:27:55Z)
• On Learning Semantic Representations for Million-Scale Free-Hand Sketches [146.52892067335128]
  We study learning semantic representations for million-scale free-hand sketches. We propose a dual-branch CNNRNN network architecture to represent sketches. We explore learning the sketch-oriented semantic representations in hashing retrieval and zero-shot recognition.
  arXiv  Detail & Related papers  (2020-07-07T15:23:22Z)
• Deep Self-Supervised Representation Learning for Free-Hand Sketch [51.101565480583304]
  We tackle the problem of self-supervised representation learning for free-hand sketches. Key for the success of our self-supervised learning paradigm lies with our sketch-specific designs. We show that the proposed approach outperforms the state-of-the-art unsupervised representation learning methods.
  arXiv  Detail & Related papers  (2020-02-03T16:28:29Z)
• SketchDesc: Learning Local Sketch Descriptors for Multi-view Correspondence [68.63311821718416]
  We study the problem of multi-view sketch correspondence, where we take as input multiple freehand sketches with different views of the same object. This problem is challenging since the visual features of corresponding points at different views can be very different. We take a deep learning approach and learn a novel local sketch descriptor from data.
  arXiv  Detail & Related papers  (2020-01-16T11:31:21Z)
• Deep Plastic Surgery: Robust and Controllable Image Editing with Human-Drawn Sketches [133.01690754567252]
  Sketch-based image editing aims to synthesize and modify photos based on the structural information provided by the human-drawn sketches. Deep Plastic Surgery is a novel, robust and controllable image editing framework that allows users to interactively edit images using hand-drawn sketch inputs.
  arXiv  Detail & Related papers  (2020-01-09T08:57:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.