NeurInt : Learning to Interpolate through Neural ODEs
- URL: http://arxiv.org/abs/2111.04123v1
- Date: Sun, 7 Nov 2021 16:31:18 GMT
- Title: NeurInt : Learning to Interpolate through Neural ODEs
- Authors: Avinandan Bose, Aniket Das, Yatin Dandi, Piyush Rai
- Abstract summary: We propose a novel generative model that learns a distribution of trajectories between two images.
We demonstrate our approach's effectiveness in generating images improved quality as well as its ability to learn a diverse distribution over smooth trajectories for any pair of real source and target images.
- Score: 18.104328632453676
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: A wide range of applications require learning image generation models whose
latent space effectively captures the high-level factors of variation present
in the data distribution. The extent to which a model represents such
variations through its latent space can be judged by its ability to interpolate
between images smoothly. However, most generative models mapping a fixed prior
to the generated images lead to interpolation trajectories lacking smoothness
and containing images of reduced quality. In this work, we propose a novel
generative model that learns a flexible non-parametric prior over interpolation
trajectories, conditioned on a pair of source and target images. Instead of
relying on deterministic interpolation methods (such as linear or spherical
interpolation in latent space), we devise a framework that learns a
distribution of trajectories between two given images using Latent Second-Order
Neural Ordinary Differential Equations. Through a hybrid combination of
reconstruction and adversarial losses, the generator is trained to map the
sampled points from these trajectories to sequences of realistic images that
smoothly transition from the source to the target image. Through comprehensive
qualitative and quantitative experiments, we demonstrate our approach's
effectiveness in generating images of improved quality as well as its ability
to learn a diverse distribution over smooth interpolation trajectories for any
pair of real source and target images.
Related papers
- Oscillation Inversion: Understand the structure of Large Flow Model through the Lens of Inversion Method [60.88467353578118]
We show that a fixed-point-inspired iterative approach to invert real-world images does not achieve convergence, instead oscillating between distinct clusters.
We introduce a simple and fast distribution transfer technique that facilitates image enhancement, stroke-based recoloring, as well as visual prompt-guided image editing.
arXiv Detail & Related papers (2024-11-17T17:45:37Z) - Self-Supervised Multi-Scale Network for Blind Image Deblurring via Alternating Optimization [12.082424048578753]
We present a self-supervised multi-scale blind image deblurring method to jointly estimate the latent image and the blur kernel.
Thanks to the collaborative estimation across multiple scales, our method avoids the computationally intensive coarse-to-fine propagation and additional image deblurring processes.
arXiv Detail & Related papers (2024-09-02T07:08:17Z) - Varying Manifolds in Diffusion: From Time-varying Geometries to Visual Saliency [25.632973225129728]
We study the geometric properties of the diffusion model, whose forward diffusion process and reverse generation process construct a series of distributions on a manifold.
We show that the generation rate is highly correlated with intuitive visual properties, such as visual saliency, of the image component.
We propose an efficient and differentiable scheme to estimate the generation rate for a given image component over time, giving rise to a generation curve.
arXiv Detail & Related papers (2024-06-07T07:32:41Z) - IMPUS: Image Morphing with Perceptually-Uniform Sampling Using Diffusion Models [24.382275473592046]
We present a diffusion-based image morphing approach with perceptually-uniform sampling (IMPUS)
IMPUS produces smooth, direct and realistic adaptations given an image pair.
arXiv Detail & Related papers (2023-11-12T10:03:32Z) - Interpolating between Images with Diffusion Models [2.6027967363792865]
Interpolating between two input images is a task missing from image generation pipelines.
We propose a method for zero-shot using latent diffusion models.
For greater consistency, or to specify additional criteria, we can generate several candidates and use CLIP to select the highest quality image.
arXiv Detail & Related papers (2023-07-24T07:03:22Z) - Real-World Image Variation by Aligning Diffusion Inversion Chain [53.772004619296794]
A domain gap exists between generated images and real-world images, which poses a challenge in generating high-quality variations of real-world images.
We propose a novel inference pipeline called Real-world Image Variation by ALignment (RIVAL)
Our pipeline enhances the generation quality of image variations by aligning the image generation process to the source image's inversion chain.
arXiv Detail & Related papers (2023-05-30T04:09:47Z) - VTAE: Variational Transformer Autoencoder with Manifolds Learning [144.0546653941249]
Deep generative models have demonstrated successful applications in learning non-linear data distributions through a number of latent variables.
The nonlinearity of the generator implies that the latent space shows an unsatisfactory projection of the data space, which results in poor representation learning.
We show that geodesics and accurate computation can substantially improve the performance of deep generative models.
arXiv Detail & Related papers (2023-04-03T13:13:19Z) - Auto-regressive Image Synthesis with Integrated Quantization [55.51231796778219]
This paper presents a versatile framework for conditional image generation.
It incorporates the inductive bias of CNNs and powerful sequence modeling of auto-regression.
Our method achieves superior diverse image generation performance as compared with the state-of-the-art.
arXiv Detail & Related papers (2022-07-21T22:19:17Z) - Meta Internal Learning [88.68276505511922]
Internal learning for single-image generation is a framework, where a generator is trained to produce novel images based on a single image.
We propose a meta-learning approach that enables training over a collection of images, in order to model the internal statistics of the sample image more effectively.
Our results show that the models obtained are as suitable as single-image GANs for many common image applications.
arXiv Detail & Related papers (2021-10-06T16:27:38Z) - Unsupervised Discovery of Disentangled Manifolds in GANs [74.24771216154105]
Interpretable generation process is beneficial to various image editing applications.
We propose a framework to discover interpretable directions in the latent space given arbitrary pre-trained generative adversarial networks.
arXiv Detail & Related papers (2020-11-24T02:18:08Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.