Trajectory-aware Principal Manifold Framework for Data Augmentation and
Image Generation
- URL: http://arxiv.org/abs/2310.07801v1
- Date: Sun, 30 Jul 2023 07:31:38 GMT
- Title: Trajectory-aware Principal Manifold Framework for Data Augmentation and
Image Generation
- Authors: Elvis Han Cui, Bingbin Li, Yanan Li, Weng Kee Wong, Donghui Wang
- Abstract summary: Many existing methods generate new samples from a parametric distribution, like the Gaussian, with little attention to generate samples along the data manifold in either the input or feature space.
We propose a novel trajectory-aware principal manifold framework to restore the manifold backbone and generate samples along a specific trajectory.
We show that the novel framework is able to extract more compact manifold representation, improve classification accuracy and generate smooth transformation among few samples.
- Score: 5.31812036803692
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Data augmentation for deep learning benefits model training, image
transformation, medical imaging analysis and many other fields. Many existing
methods generate new samples from a parametric distribution, like the Gaussian,
with little attention to generate samples along the data manifold in either the
input or feature space. In this paper, we verify that there are theoretical and
practical advantages of using the principal manifold hidden in the feature
space than the Gaussian distribution. We then propose a novel trajectory-aware
principal manifold framework to restore the manifold backbone and generate
samples along a specific trajectory. On top of the autoencoder architecture, we
further introduce an intrinsic dimension regularization term to make the
manifold more compact and enable few-shot image generation. Experimental
results show that the novel framework is able to extract more compact manifold
representation, improve classification accuracy and generate smooth
transformation among few samples.
Related papers
- Varying Manifolds in Diffusion: From Time-varying Geometries to Visual Saliency [25.632973225129728]
We study the geometric properties of the diffusion model, whose forward diffusion process and reverse generation process construct a series of distributions on a manifold.
We show that the generation rate is highly correlated with intuitive visual properties, such as visual saliency, of the image component.
We propose an efficient and differentiable scheme to estimate the generation rate for a given image component over time, giving rise to a generation curve.
arXiv Detail & Related papers (2024-06-07T07:32:41Z) - Deep Generative Sampling in the Dual Divergence Space: A Data-efficient & Interpretative Approach for Generative AI [29.13807697733638]
We build on the remarkable achievements in generative sampling of natural images.
We propose an innovative challenge, potentially overly ambitious, which involves generating samples that resemble images.
The statistical challenge lies in the small sample size, sometimes consisting of a few hundred subjects.
arXiv Detail & Related papers (2024-04-10T22:35:06Z) - SatSynth: Augmenting Image-Mask Pairs through Diffusion Models for Aerial Semantic Segmentation [69.42764583465508]
We explore the potential of generative image diffusion to address the scarcity of annotated data in earth observation tasks.
To the best of our knowledge, we are the first to generate both images and corresponding masks for satellite segmentation.
arXiv Detail & Related papers (2024-03-25T10:30:22Z) - Hierarchical Integration Diffusion Model for Realistic Image Deblurring [71.76410266003917]
Diffusion models (DMs) have been introduced in image deblurring and exhibited promising performance.
We propose the Hierarchical Integration Diffusion Model (HI-Diff), for realistic image deblurring.
Experiments on synthetic and real-world blur datasets demonstrate that our HI-Diff outperforms state-of-the-art methods.
arXiv Detail & Related papers (2023-05-22T12:18:20Z) - VTAE: Variational Transformer Autoencoder with Manifolds Learning [144.0546653941249]
Deep generative models have demonstrated successful applications in learning non-linear data distributions through a number of latent variables.
The nonlinearity of the generator implies that the latent space shows an unsatisfactory projection of the data space, which results in poor representation learning.
We show that geodesics and accurate computation can substantially improve the performance of deep generative models.
arXiv Detail & Related papers (2023-04-03T13:13:19Z) - A Geometric Perspective on Variational Autoencoders [0.0]
This paper introduces a new interpretation of the Variational Autoencoder framework by taking a fully geometric point of view.
We show that using this scheme can make a vanilla VAE competitive and even better than more advanced versions on several benchmark datasets.
arXiv Detail & Related papers (2022-09-15T15:32:43Z) - Explicit Use of Fourier Spectrum in Generative Adversarial Networks [0.0]
We show that there is a dissimilarity between the spectrum of authentic images and fake ones.
We propose a new model to reduce the discrepancies between the spectrum of the actual and fake images.
We experimentally show promising improvements in the quality of the generated images.
arXiv Detail & Related papers (2022-08-02T06:26:44Z) - FewGAN: Generating from the Joint Distribution of a Few Images [95.6635227371479]
We introduce FewGAN, a generative model for generating novel, high-quality and diverse images.
FewGAN is a hierarchical patch-GAN that applies quantization at the first coarse scale, followed by a pyramid of residual fully convolutional GANs at finer scales.
In an extensive set of experiments, it is shown that FewGAN outperforms baselines both quantitatively and qualitatively.
arXiv Detail & Related papers (2022-07-18T07:11:28Z) - Manifold Topology Divergence: a Framework for Comparing Data Manifolds [109.0784952256104]
We develop a framework for comparing data manifold, aimed at the evaluation of deep generative models.
Based on the Cross-Barcode, we introduce the Manifold Topology Divergence score (MTop-Divergence)
We demonstrate that the MTop-Divergence accurately detects various degrees of mode-dropping, intra-mode collapse, mode invention, and image disturbance.
arXiv Detail & Related papers (2021-06-08T00:30:43Z) - Flow-based Generative Models for Learning Manifold to Manifold Mappings [39.60406116984869]
We introduce three kinds of invertible layers for manifold-valued data, which are analogous to their functionality in flow-based generative models.
We show promising results where we can reliably and accurately reconstruct brain images of a field of orientation distribution functions.
arXiv Detail & Related papers (2020-12-18T02:19:18Z) - Unsupervised Discovery of Disentangled Manifolds in GANs [74.24771216154105]
Interpretable generation process is beneficial to various image editing applications.
We propose a framework to discover interpretable directions in the latent space given arbitrary pre-trained generative adversarial networks.
arXiv Detail & Related papers (2020-11-24T02:18:08Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.