Making Robots Draw A Vivid Portrait In Two Minutes
- URL: http://arxiv.org/abs/2005.05526v3
- Date: Tue, 21 Jul 2020 07:20:32 GMT
- Title: Making Robots Draw A Vivid Portrait In Two Minutes
- Authors: Fei Gao, Jingjie Zhu, Zeyuan Yu, Peng Li, Tao Wang
- Abstract summary: We present a drawing robot, which can automatically transfer a facial picture to a vivid portrait, and then draw it on paper within two minutes averagely.
At the heart of our system is a novel portrait synthesis algorithm based on deep learning.
The whole portrait drawing robotic system is named AiSketcher.
- Score: 11.148458054454407
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Significant progress has been made with artistic robots. However, existing
robots fail to produce high-quality portraits in a short time. In this work, we
present a drawing robot, which can automatically transfer a facial picture to a
vivid portrait, and then draw it on paper within two minutes averagely. At the
heart of our system is a novel portrait synthesis algorithm based on deep
learning. Innovatively, we employ a self-consistency loss, which makes the
algorithm capable of generating continuous and smooth brush-strokes. Besides,
we propose a componential sparsity constraint to reduce the number of
brush-strokes over insignificant areas. We also implement a local sketch
synthesis algorithm, and several pre- and post-processing techniques to deal
with the background and details. The portrait produced by our algorithm
successfully captures individual characteristics by using a sparse set of
continuous brush-strokes. Finally, the portrait is converted to a sequence of
trajectories and reproduced by a 3-degree-of-freedom robotic arm. The whole
portrait drawing robotic system is named AiSketcher. Extensive experiments show
that AiSketcher can produce considerably high-quality sketches for a wide range
of pictures, including faces in-the-wild and universal images of arbitrary
content. To our best knowledge, AiSketcher is the first portrait drawing robot
that uses neural style transfer techniques. AiSketcher has attended a quite
number of exhibitions and shown remarkable performance under diverse
circumstances.
Related papers
- Stylized Face Sketch Extraction via Generative Prior with Limited Data [6.727433982111717]
StyleSketch is a method for extracting high-resolution stylized sketches from a face image.
Using the rich semantics of the deep features from a pretrained StyleGAN, we are able to train a sketch generator with 16 pairs of face and the corresponding sketch images.
arXiv Detail & Related papers (2024-03-17T16:25:25Z) - AniFaceDrawing: Anime Portrait Exploration during Your Sketching [9.933240729830151]
This paper focuses on how artificial intelligence can be used to assist users in the creation of anime portraits.
The input is a sequence of incomplete freehand sketches that are gradually refined stroke by stroke.
The output is a sequence of high-quality anime portraits that correspond to the input sketches as guidance.
arXiv Detail & Related papers (2023-06-13T00:43:47Z) - Semantic-aware Generation of Multi-view Portrait Drawings [16.854527555637063]
We propose a Semantic-Aware GEnerator (SAGE) for synthesizing multi-view portrait drawings.
Our motivation is that facial semantic labels are view-consistent and correlate with drawing techniques.
SAGE achieves significantly superior or highly competitive performance, compared to existing 3D-aware image synthesis methods.
arXiv Detail & Related papers (2023-05-04T07:48:27Z) - Picture that Sketch: Photorealistic Image Generation from Abstract
Sketches [109.69076457732632]
Given an abstract, deformed, ordinary sketch from untrained amateurs like you and me, this paper turns it into a photorealistic image.
We do not dictate an edgemap-like sketch to start with, but aim to work with abstract free-hand human sketches.
In doing so, we essentially democratise the sketch-to-photo pipeline, "picturing" a sketch regardless of how good you sketch.
arXiv Detail & Related papers (2023-03-20T14:49:03Z) - Make Your Brief Stroke Real and Stereoscopic: 3D-Aware Simplified Sketch
to Portrait Generation [51.64832538714455]
Existing studies only generate portraits in the 2D plane with fixed views, making the results less vivid.
In this paper, we present Stereoscopic Simplified Sketch-to-Portrait (SSSP), which explores the possibility of creating Stereoscopic 3D-aware portraits.
Our key insight is to design sketch-aware constraints that can fully exploit the prior knowledge of a tri-plane-based 3D-aware generative model.
arXiv Detail & Related papers (2023-02-14T06:28:42Z) - From Scratch to Sketch: Deep Decoupled Hierarchical Reinforcement
Learning for Robotic Sketching Agent [20.406075470956065]
We formulate the robotic sketching problem as a deep decoupled hierarchical reinforcement learning.
Two policies for stroke-based rendering and motor control are learned independently to achieve sub-tasks for drawing.
Our experimental results show that the two policies successfully learned the sub-tasks and collaborated to sketch the target images.
arXiv Detail & Related papers (2022-08-09T15:18:55Z) - CtlGAN: Few-shot Artistic Portraits Generation with Contrastive Transfer
Learning [77.27821665339492]
CtlGAN is a new few-shot artistic portraits generation model with a novel contrastive transfer learning strategy.
We adapt a pretrained StyleGAN in the source domain to a target artistic domain with no more than 10 artistic faces.
We propose a new encoder which embeds real faces into Z+ space and proposes a dual-path training strategy to better cope with the adapted decoder.
arXiv Detail & Related papers (2022-03-16T13:28:17Z) - Quality Metric Guided Portrait Line Drawing Generation from Unpaired
Training Data [88.78171717494688]
We propose a novel method to automatically transform face photos to portrait drawings using unpaired training data.
Our method can (1) learn to generate high quality portrait drawings in multiple styles using a single network and (2) generate portrait drawings in a "new style" unseen in the training data.
arXiv Detail & Related papers (2022-02-08T06:49:57Z) - DeepFacePencil: Creating Face Images from Freehand Sketches [77.00929179469559]
Existing image-to-image translation methods require a large-scale dataset of paired sketches and images for supervision.
We propose DeepFacePencil, an effective tool that is able to generate photo-realistic face images from hand-drawn sketches.
arXiv Detail & Related papers (2020-08-31T03:35:21Z) - B\'ezierSketch: A generative model for scalable vector sketches [132.5223191478268]
We present B'ezierSketch, a novel generative model for fully vector sketches that are automatically scalable and high-resolution.
We first introduce a novel inverse graphics approach to stroke embedding that trains an encoder to embed each stroke to its best fit B'ezier curve.
This enables us to treat sketches as short sequences of paramaterized strokes and thus train a recurrent sketch generator with greater capacity for longer sketches.
arXiv Detail & Related papers (2020-07-04T21:30:52Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.