Sketch Less Face Image Retrieval: A New Challenge
- URL: http://arxiv.org/abs/2302.05576v1
- Date: Sat, 11 Feb 2023 02:36:00 GMT
- Title: Sketch Less Face Image Retrieval: A New Challenge
- Authors: Dawei Dai, Yutang Li, Liang Wang, Shiyu Fu, Shuyin Xia, Guoyin Wang
- Abstract summary: Drawing a complete face sketch often needs skills and takes time, which hinders its widespread applicability in the practice.
In this study, we proposed sketch less face image retrieval (SLFIR), in which the retrieval was carried out at each stroke and aim to retrieve the target face photo using a partial sketch with as few strokes as possible.
Experiments indicate that the new framework can finish the retrieval using a partial or pool drawing sketch.
- Score: 9.703239229149261
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In some specific scenarios, face sketch was used to identify a person.
However, drawing a complete face sketch often needs skills and takes time,
which hinder its widespread applicability in the practice. In this study, we
proposed a new task named sketch less face image retrieval (SLFIR), in which
the retrieval was carried out at each stroke and aim to retrieve the target
face photo using a partial sketch with as few strokes as possible (see Fig.1).
Firstly, we developed a method to generate the data of sketch with drawing
process, and opened such dataset; Secondly, we proposed a two-stage method as
the baseline for SLFIR that (1) A triplet network, was first adopt to learn the
joint embedding space shared between the complete sketch and its target face
photo; (2) Regarding the sketch drawing episode as a sequence, we designed a
LSTM module to optimize the representation of the incomplete face sketch.
Experiments indicate that the new framework can finish the retrieval using a
partial or pool drawing sketch.
Related papers
- Stylized Face Sketch Extraction via Generative Prior with Limited Data [6.727433982111717]
StyleSketch is a method for extracting high-resolution stylized sketches from a face image.
Using the rich semantics of the deep features from a pretrained StyleGAN, we are able to train a sketch generator with 16 pairs of face and the corresponding sketch images.
arXiv Detail & Related papers (2024-03-17T16:25:25Z) - Multi-Granularity Representation Learning for Sketch-based Dynamic Face
Image Retrieval [13.287197931943961]
In specific scenarios, face sketch can be used to identify a person.
Sketch less face image retrieval (SLFIR) attempts to overcome the barriers by providing a means for humans and machines to interact during the drawing process.
In this study, we propose a multigranularity (MG) representation learning (MGRL) method to address the SLFIR problem.
arXiv Detail & Related papers (2023-12-31T02:17:14Z) - Picture that Sketch: Photorealistic Image Generation from Abstract
Sketches [109.69076457732632]
Given an abstract, deformed, ordinary sketch from untrained amateurs like you and me, this paper turns it into a photorealistic image.
We do not dictate an edgemap-like sketch to start with, but aim to work with abstract free-hand human sketches.
In doing so, we essentially democratise the sketch-to-photo pipeline, "picturing" a sketch regardless of how good you sketch.
arXiv Detail & Related papers (2023-03-20T14:49:03Z) - FS-COCO: Towards Understanding of Freehand Sketches of Common Objects in
Context [112.07988211268612]
We advance sketch research to scenes with the first dataset of freehand scene sketches, FS-COCO.
Our dataset comprises 10,000 freehand scene vector sketches with per point space-time information by 100 non-expert individuals.
We study for the first time the problem of the fine-grained image retrieval from freehand scene sketches and sketch captions.
arXiv Detail & Related papers (2022-03-04T03:00:51Z) - Multi-granularity Association Learning Framework for on-the-fly
Fine-Grained Sketch-based Image Retrieval [7.797006835701767]
Fine-grained sketch-based image retrieval (FG-SBIR) addresses the problem of retrieving a particular photo in a given query sketch.
In this study, we aim to retrieve the target photo with the least number of strokes possible (incomplete sketch)
We propose a multi-granularity association learning framework that further optimize the embedding space of all incomplete sketches.
arXiv Detail & Related papers (2022-01-13T14:38:50Z) - DeepFacePencil: Creating Face Images from Freehand Sketches [77.00929179469559]
Existing image-to-image translation methods require a large-scale dataset of paired sketches and images for supervision.
We propose DeepFacePencil, an effective tool that is able to generate photo-realistic face images from hand-drawn sketches.
arXiv Detail & Related papers (2020-08-31T03:35:21Z) - Sketch-BERT: Learning Sketch Bidirectional Encoder Representation from
Transformers by Self-supervised Learning of Sketch Gestalt [125.17887147597567]
We present a model of learning Sketch BiBERT Representation from Transformer (Sketch-BERT)
We generalize BERT to sketch domain, with the novel proposed components and pre-training algorithms.
We show that the learned representation of Sketch-BERT can help and improve the performance of the downstream tasks of sketch recognition, sketch retrieval, and sketch gestalt.
arXiv Detail & Related papers (2020-05-19T01:35:44Z) - Sketch Less for More: On-the-Fly Fine-Grained Sketch Based Image
Retrieval [203.2520862597357]
Fine-grained sketch-based image retrieval (FG-SBIR) addresses the problem of retrieving a particular photo instance given a user's query sketch.
We reformulate the conventional FG-SBIR framework to tackle these challenges.
We propose an on-the-fly design that starts retrieving as soon as the user starts drawing.
arXiv Detail & Related papers (2020-02-24T15:36:02Z) - Deep Self-Supervised Representation Learning for Free-Hand Sketch [51.101565480583304]
We tackle the problem of self-supervised representation learning for free-hand sketches.
Key for the success of our self-supervised learning paradigm lies with our sketch-specific designs.
We show that the proposed approach outperforms the state-of-the-art unsupervised representation learning methods.
arXiv Detail & Related papers (2020-02-03T16:28:29Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.