Hierarchical Vectorization for Portrait Images
- URL: http://arxiv.org/abs/2205.11880v1
- Date: Tue, 24 May 2022 07:58:41 GMT
- Title: Hierarchical Vectorization for Portrait Images
- Authors: Qian Fu, Linlin Liu, Fei Hou, Ying He
- Abstract summary: We propose a novel vectorization method that can automatically convert images into a 3-tier hierarchical representation.
The base layer consists of a set of sparse diffusion curves which characterize salient geometric features and low-frequency colors.
The middle level encodes specular highlights and shadows to large and editable Poisson regions (PR) and allows the user to directly adjust illumination.
The top level contains two types of pixel-sized PRs for high-frequency residuals and fine details such as pimples and pigmentation.
- Score: 12.32304366243904
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Aiming at developing intuitive and easy-to-use portrait editing tools, we
propose a novel vectorization method that can automatically convert raster
images into a 3-tier hierarchical representation. The base layer consists of a
set of sparse diffusion curves (DC) which characterize salient geometric
features and low-frequency colors and provide means for semantic color transfer
and facial expression editing. The middle level encodes specular highlights and
shadows to large and editable Poisson regions (PR) and allows the user to
directly adjust illumination via tuning the strength and/or changing shape of
PR. The top level contains two types of pixel-sized PRs for high-frequency
residuals and fine details such as pimples and pigmentation. We also train a
deep generative model that can produce high-frequency residuals automatically.
Thanks to the meaningful organization of vector primitives, editing portraits
becomes easy and intuitive. In particular, our method supports color transfer,
facial expression editing, highlight and shadow editing and automatic
retouching. Thanks to the linearity of the Laplace operator, we introduce alpha
blending, linear dodge and linear burn to vector editing and show that they are
effective in editing highlights and shadows. To quantitatively evaluate the
results, we extend the commonly used FLIP metric (which measures differences
between two images) by considering illumination. The new metric, called
illumination-sensitive FLIP or IS-FLIP, can effectively capture the salient
changes in color transfer results, and is more consistent with human perception
than FLIP and other quality measures on portrait images. We evaluate our method
on the FFHQR dataset and show that our method is effective for common portrait
editing tasks, such as retouching, light editing, color transfer and expression
editing. We will make the code and trained models publicly available.
Related papers
- COMPOSE: Comprehensive Portrait Shadow Editing [25.727386174616868]
COMPOSE is a novel shadow editing pipeline for human portraits.
It offers precise control over shadow attributes such as shape, intensity, and position.
We have trained models to: (1) predict this light source representation from images, and (2) generate realistic shadows using this representation.
arXiv Detail & Related papers (2024-08-25T19:18:18Z) - LightningDrag: Lightning Fast and Accurate Drag-based Image Editing Emerging from Videos [101.59710862476041]
We present LightningDrag, a rapid approach enabling high quality drag-based image editing in 1 second.
Unlike most previous methods, we redefine drag-based editing as a conditional generation task.
Our approach can significantly outperform previous methods in terms of accuracy and consistency.
arXiv Detail & Related papers (2024-05-22T15:14:00Z) - DeferredGS: Decoupled and Editable Gaussian Splatting with Deferred Shading [50.331929164207324]
We introduce DeferredGS, a method for decoupling and editing the Gaussian splatting representation using deferred shading.
Both qualitative and quantitative experiments demonstrate the superior performance of DeferredGS in novel view and editing tasks.
arXiv Detail & Related papers (2024-04-15T01:58:54Z) - DiffEditor: Boosting Accuracy and Flexibility on Diffusion-based Image
Editing [66.43179841884098]
Large-scale Text-to-Image (T2I) diffusion models have revolutionized image generation over the last few years.
We propose DiffEditor to rectify two weaknesses in existing diffusion-based image editing.
Our method can efficiently achieve state-of-the-art performance on various fine-grained image editing tasks.
arXiv Detail & Related papers (2024-02-04T18:50:29Z) - LightPainter: Interactive Portrait Relighting with Freehand Scribble [79.95574780974103]
We introduce LightPainter, a scribble-based relighting system that allows users to interactively manipulate portrait lighting effect with ease.
To train the relighting module, we propose a novel scribble simulation procedure to mimic real user scribbles.
We demonstrate high-quality and flexible portrait lighting editing capability with both quantitative and qualitative experiments.
arXiv Detail & Related papers (2023-03-22T23:17:11Z) - RecolorNeRF: Layer Decomposed Radiance Fields for Efficient Color
Editing of 3D Scenes [21.284044381058575]
We present RecolorNeRF, a novel user-friendly color editing approach for neural radiance fields.
Our key idea is to decompose the scene into a set of pure-colored layers, forming a palette.
To support efficient palette-based editing, the color of each layer needs to be as representative as possible.
arXiv Detail & Related papers (2023-01-19T09:18:06Z) - PaletteNeRF: Palette-based Appearance Editing of Neural Radiance Fields [60.66412075837952]
We present PaletteNeRF, a novel method for appearance editing of neural radiance fields (NeRF) based on 3D color decomposition.
Our method decomposes the appearance of each 3D point into a linear combination of palette-based bases.
We extend our framework with compressed semantic features for semantic-aware appearance editing.
arXiv Detail & Related papers (2022-12-21T00:20:01Z) - Physically-Based Editing of Indoor Scene Lighting from a Single Image [106.60252793395104]
We present a method to edit complex indoor lighting from a single image with its predicted depth and light source segmentation masks.
We tackle this problem using two novel components: 1) a holistic scene reconstruction method that estimates scene reflectance and parametric 3D lighting, and 2) a neural rendering framework that re-renders the scene from our predictions.
arXiv Detail & Related papers (2022-05-19T06:44:37Z) - Flexible Portrait Image Editing with Fine-Grained Control [12.32304366243904]
We develop a new method for portrait image editing, which supports fine-grained editing of geometries, colors, lights and shadows using a single neural network model.
We adopt a novel asymmetric conditional GAN architecture: the generators take the transformed conditional inputs, such as edge maps, color palette, sliders and masks, that can be directly edited by the user.
We demonstrate the effectiveness of our method by evaluating it on the CelebAMask-HQ dataset with a wide range of tasks, including geometry/color/shadow/light editing, hand-drawn sketch to image translation, and color transfer.
arXiv Detail & Related papers (2022-04-04T08:39:37Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.