A Simple Baseline for StyleGAN Inversion
- URL: http://arxiv.org/abs/2104.07661v1
- Date: Thu, 15 Apr 2021 17:59:49 GMT
- Title: A Simple Baseline for StyleGAN Inversion
- Authors: Tianyi Wei and Dongdong Chen and Wenbo Zhou and Jing Liao and Weiming
Zhang and Lu Yuan and Gang Hua and Nenghai Yu
- Abstract summary: StyleGAN inversion plays an essential role in enabling the pretrained StyleGAN to be used for real facial image editing tasks.
Existing optimization-based methods can produce high quality results, but the optimization often takes a long time.
We present a new feed-forward network for StyleGAN inversion, with significant improvement in terms of efficiency and quality.
- Score: 133.5868210969111
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: This paper studies the problem of StyleGAN inversion, which plays an
essential role in enabling the pretrained StyleGAN to be used for real facial
image editing tasks. This problem has the high demand for quality and
efficiency. Existing optimization-based methods can produce high quality
results, but the optimization often takes a long time. On the contrary,
forward-based methods are usually faster but the quality of their results is
inferior. In this paper, we present a new feed-forward network for StyleGAN
inversion, with significant improvement in terms of efficiency and quality. In
our inversion network, we introduce: 1) a shallower backbone with multiple
efficient heads across scales; 2) multi-layer identity loss and multi-layer
face parsing loss to the loss function; and 3) multi-stage refinement.
Combining these designs together forms a simple and efficient baseline method
which exploits all benefits of optimization-based and forward-based methods.
Quantitative and qualitative results show that our method performs better than
existing forward-based methods and comparably to state-of-the-art
optimization-based methods, while maintaining the high efficiency as well as
forward-based methods. Moreover, a number of real image editing applications
demonstrate the efficacy of our method. Our project page is
~\url{https://wty-ustc.github.io/inversion}.
Related papers
- Coupling Fairness and Pruning in a Single Run: a Bi-level Optimization
Perspective [17.394732703591462]
We propose a framework to jointly optimize the pruning mask and weight update processes with fairness constraints.
This framework is engineered to compress models that maintain performance while ensuring fairness in a single execution.
Our empirical analysis contrasts our framework with several mainstream pruning strategies, emphasizing our method's superiority in maintaining model fairness, performance, and efficiency.
arXiv Detail & Related papers (2023-12-15T20:08:53Z) - Adaptive Image Registration: A Hybrid Approach Integrating Deep Learning
and Optimization Functions for Enhanced Precision [13.242184146186974]
We propose a single framework for image registration based on deep neural networks and optimization.
We show improvements of up to 1.6% in test data, while maintaining the same inference time, and a substantial 1.0% points performance gain in deformation field smoothness.
arXiv Detail & Related papers (2023-11-27T02:48:06Z) - HomOpt: A Homotopy-Based Hyperparameter Optimization Method [10.11271414863925]
We propose HomOpt, a data-driven approach based on a generalized additive model (GAM) surrogate combined with homotopy optimization.
We show how HomOpt can boost the performance and effectiveness of any given method with faster convergence to the optimum on continuous discrete, and categorical domain spaces.
arXiv Detail & Related papers (2023-08-07T06:01:50Z) - Sample Less, Learn More: Efficient Action Recognition via Frame Feature
Restoration [59.6021678234829]
We propose a novel method to restore the intermediate features for two sparsely sampled and adjacent video frames.
With the integration of our method, the efficiency of three commonly used baselines has been improved by over 50%, with a mere 0.5% reduction in recognition accuracy.
arXiv Detail & Related papers (2023-07-27T13:52:42Z) - StyleSwap: Style-Based Generator Empowers Robust Face Swapping [90.05775519962303]
We introduce a concise and effective framework named StyleSwap.
Our core idea is to leverage a style-based generator to empower high-fidelity and robust face swapping.
We identify that with only minimal modifications, a StyleGAN2 architecture can successfully handle the desired information from both source and target.
arXiv Detail & Related papers (2022-09-27T16:35:16Z) - Efficient Few-Shot Object Detection via Knowledge Inheritance [62.36414544915032]
Few-shot object detection (FSOD) aims at learning a generic detector that can adapt to unseen tasks with scarce training samples.
We present an efficient pretrain-transfer framework (PTF) baseline with no computational increment.
We also propose an adaptive length re-scaling (ALR) strategy to alleviate the vector length inconsistency between the predicted novel weights and the pretrained base weights.
arXiv Detail & Related papers (2022-03-23T06:24:31Z) - Resource-Efficient Invariant Networks: Exponential Gains by Unrolled
Optimization [8.37077056358265]
We propose a new computational primitive for building invariant networks based instead on optimization.
We provide empirical and theoretical corroboration of the efficiency gains and soundness of our proposed method.
We demonstrate its utility in constructing an efficient invariant network for a simple hierarchical object detection task.
arXiv Detail & Related papers (2022-03-09T19:04:08Z) - Low-light Image Enhancement by Retinex Based Algorithm Unrolling and
Adjustment [50.13230641857892]
We propose a new deep learning framework for the low-light image enhancement (LIE) problem.
The proposed framework contains a decomposition network inspired by algorithm unrolling, and adjustment networks considering both global brightness and local brightness sensitivity.
Experiments on a series of typical LIE datasets demonstrated the effectiveness of the proposed method, both quantitatively and visually, as compared with existing methods.
arXiv Detail & Related papers (2022-02-12T03:59:38Z) - Joint inference and input optimization in equilibrium networks [68.63726855991052]
deep equilibrium model is a class of models that foregoes traditional network depth and instead computes the output of a network by finding the fixed point of a single nonlinear layer.
We show that there is a natural synergy between these two settings.
We demonstrate this strategy on various tasks such as training generative models while optimizing over latent codes, training models for inverse problems like denoising and inpainting, adversarial training and gradient based meta-learning.
arXiv Detail & Related papers (2021-11-25T19:59:33Z) - Quantum variational optimization: The role of entanglement and problem
hardness [0.0]
We study the role of entanglement, the structure of the variational quantum circuit, and the structure of the optimization problem.
Our numerical results indicate an advantage in adapting the distribution of entangling gates to the problem's topology.
We find evidence that applying conditional value at risk type cost functions improves the optimization, increasing the probability of overlap with the optimal solutions.
arXiv Detail & Related papers (2021-03-26T14:06:54Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.