Pose Guided Image Generation from Misaligned Sources via Residual Flow
Based Correction
- URL: http://arxiv.org/abs/2202.00843v1
- Date: Wed, 2 Feb 2022 01:30:15 GMT
- Title: Pose Guided Image Generation from Misaligned Sources via Residual Flow
Based Correction
- Authors: Jiawei Lu, He Wang, Tianjia Shao, Yin Yang, Kun Zhou
- Abstract summary: We propose a new general approach which models multiple types of variations among sources, such as view angles, poses, facial expressions, in a unified framework.
We verify our approach on a variety of data including humans bodies, faces, city scenes and 3D objects.
- Score: 31.39424991391106
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Generating new images with desired properties (e.g. new view/poses) from
source images has been enthusiastically pursued recently, due to its wide range
of potential applications. One way to ensure high-quality generation is to use
multiple sources with complementary information such as different views of the
same object. However, as source images are often misaligned due to the large
disparities among the camera settings, strong assumptions have been made in the
past with respect to the camera(s) or/and the object in interest, limiting the
application of such techniques. Therefore, we propose a new general approach
which models multiple types of variations among sources, such as view angles,
poses, facial expressions, in a unified framework, so that it can be employed
on datasets of vastly different nature. We verify our approach on a variety of
data including humans bodies, faces, city scenes and 3D objects. Both the
qualitative and quantitative results demonstrate the better performance of our
method than the state of the art.
Related papers
- Enhance Image Classification via Inter-Class Image Mixup with Diffusion Model [80.61157097223058]
A prevalent strategy to bolster image classification performance is through augmenting the training set with synthetic images generated by T2I models.
In this study, we scrutinize the shortcomings of both current generative and conventional data augmentation techniques.
We introduce an innovative inter-class data augmentation method known as Diff-Mix, which enriches the dataset by performing image translations between classes.
arXiv Detail & Related papers (2024-03-28T17:23:45Z) - UpFusion: Novel View Diffusion from Unposed Sparse View Observations [66.36092764694502]
UpFusion can perform novel view synthesis and infer 3D representations for an object given a sparse set of reference images.
We show that this mechanism allows generating high-fidelity novel views while improving the synthesis quality given additional (unposed) images.
arXiv Detail & Related papers (2023-12-11T18:59:55Z) - Retrieving Conditions from Reference Images for Diffusion Models [30.14303302029618]
We consider Subject-Driven Generation as a unified retrieval problem with diffusion models.
We introduce a novel diffusion model architecture, named RetriNet, designed to address and solve these problems.
We also propose a research and friendly dataset, RetriBooru, to study a more difficult problem, concept composition.
arXiv Detail & Related papers (2023-12-05T06:04:16Z) - Cross-domain Compositing with Pretrained Diffusion Models [34.98199766006208]
We employ a localized, iterative refinement scheme which infuses the injected objects with contextual information derived from the background scene.
Our method produces higher quality and realistic results without requiring any annotations or training.
arXiv Detail & Related papers (2023-02-20T18:54:04Z) - Re-Imagen: Retrieval-Augmented Text-to-Image Generator [58.60472701831404]
Retrieval-Augmented Text-to-Image Generator (Re-Imagen)
Retrieval-Augmented Text-to-Image Generator (Re-Imagen)
arXiv Detail & Related papers (2022-09-29T00:57:28Z) - Explicitly Controllable 3D-Aware Portrait Generation [42.30481422714532]
We propose a 3D portrait generation network that produces consistent portraits according to semantic parameters regarding pose, identity, expression and lighting.
Our method outperforms prior arts in extensive experiments, producing realistic portraits with vivid expression in natural lighting when viewed in free viewpoint.
arXiv Detail & Related papers (2022-09-12T17:40:08Z) - InvGAN: Invertible GANs [88.58338626299837]
InvGAN, short for Invertible GAN, successfully embeds real images to the latent space of a high quality generative model.
This allows us to perform image inpainting, merging, and online data augmentation.
arXiv Detail & Related papers (2021-12-08T21:39:00Z) - Wide-angle Image Rectification: A Survey [86.36118799330802]
wide-angle images contain distortions that violate the assumptions underlying pinhole camera models.
Image rectification, which aims to correct these distortions, can solve these problems.
We present a detailed description and discussion of the camera models used in different approaches.
Next, we review both traditional geometry-based image rectification methods and deep learning-based methods.
arXiv Detail & Related papers (2020-10-30T17:28:40Z) - Single View Metrology in the Wild [94.7005246862618]
We present a novel approach to single view metrology that can recover the absolute scale of a scene represented by 3D heights of objects or camera height above the ground.
Our method relies on data-driven priors learned by a deep network specifically designed to imbibe weakly supervised constraints from the interplay of the unknown camera with 3D entities such as object heights.
We demonstrate state-of-the-art qualitative and quantitative results on several datasets as well as applications including virtual object insertion.
arXiv Detail & Related papers (2020-07-18T22:31:33Z) - Generating Annotated High-Fidelity Images Containing Multiple Coherent
Objects [10.783993190686132]
We propose a multi-object generation framework that can synthesize images with multiple objects without explicitly requiring contextual information.
We demonstrate how coherency and fidelity are preserved with our method through experiments on the Multi-MNIST and CLEVR datasets.
arXiv Detail & Related papers (2020-06-22T11:33:55Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.