Quantized Conditional COT-GAN for Video Prediction

• URL: http://arxiv.org/abs/2106.05658v1
• Date: Thu, 10 Jun 2021 11:10:53 GMT
• Title: Quantized Conditional COT-GAN for Video Prediction
• Authors: Tianlin Xu and Beatrice Acciaio
• Abstract summary: Causal Optimal Transport (COT) results from imposing a temporal causality constraint on classic optimal transport problems. We develop a conditional version of COT-GAN suitable for sequence prediction. The resulting quantized conditional COT-GAN algorithm is illustrated with an application for video prediction.
• Score: 3.8073142980733
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Causal Optimal Transport (COT) results from imposing a temporal causality constraint on classic optimal transport problems, which naturally generates a new concept of distances between distributions on path spaces. The first application of the COT theory for sequential learning was given in Xu et al. (2020), where COT-GAN was introduced as an adversarial algorithm to train implicit generative models optimized for producing sequential data. Relying on Xu et al. (2020), the contribution of the present paper is twofold. First, we develop a conditional version of COT-GAN suitable for sequence prediction. This means that the dataset is now used in order to learn how a sequence will evolve given the observation of its past evolution. Second, we improve on the convergence results by working with modifications of the empirical measures via a specific type of quantization due to Backhoff et al. (2020). The resulting quantized conditional COT-GAN algorithm is illustrated with an application for video prediction.

Related papers

• Model Ensembling for Constrained Optimization [7.4351710906830375]
  We consider a setting in which we wish to ensemble models for multidimensional output predictions that are in turn used for downstream optimization. More precisely, we imagine we are given a number of models mapping a state space to multidimensional real-valued predictions. These predictions form the coefficients of a linear objective that we would like to optimize under specified constraints. We apply multicalibration techniques that lead to two provably efficient and convergent algorithms.
  arXiv  Detail & Related papers  (2024-05-27T01:48:07Z)
• Exploiting Diffusion Prior for Generalizable Dense Prediction [85.4563592053464]
  Recent advanced Text-to-Image (T2I) diffusion models are sometimes too imaginative for existing off-the-shelf dense predictors to estimate. We introduce DMP, a pipeline utilizing pre-trained T2I models as a prior for dense prediction tasks. Despite limited-domain training data, the approach yields faithful estimations for arbitrary images, surpassing existing state-of-the-art algorithms.
  arXiv  Detail & Related papers  (2023-11-30T18:59:44Z)
• Variational Laplace Autoencoders [53.08170674326728]
  Variational autoencoders employ an amortized inference model to approximate the posterior of latent variables. We present a novel approach that addresses the limited posterior expressiveness of fully-factorized Gaussian assumption. We also present a general framework named Variational Laplace Autoencoders (VLAEs) for training deep generative models.
  arXiv  Detail & Related papers  (2022-11-30T18:59:27Z)
• Towards Out-of-Distribution Sequential Event Prediction: A Causal Treatment [72.50906475214457]
  The goal of sequential event prediction is to estimate the next event based on a sequence of historical events. In practice, the next-event prediction models are trained with sequential data collected at one time. We propose a framework with hierarchical branching structures for learning context-specific representations.
  arXiv  Detail & Related papers  (2022-10-24T07:54:13Z)
• Stochastic Trajectory Prediction via Motion Indeterminacy Diffusion [88.45326906116165]
  We present a new framework to formulate the trajectory prediction task as a reverse process of motion indeterminacy diffusion (MID) We encode the history behavior information and the social interactions as a state embedding and devise a Transformer-based diffusion model to capture the temporal dependencies of trajectories. Experiments on the human trajectory prediction benchmarks including the Stanford Drone and ETH/UCY datasets demonstrate the superiority of our method.
  arXiv  Detail & Related papers  (2022-03-25T16:59:08Z)
• Sliding Sequential CVAE with Time Variant Socially-aware Rethinking for Trajectory Prediction [13.105275905781632]
  Pedestrian trajectory prediction is a key technology in many applications such as video surveillance, social robot navigation, and autonomous driving. This work proposes a novel trajectory prediction method called CSR, which consists of a cascaded conditional autoencoder (CVAE) module and a socially-aware regression module. Experiments results demonstrate that the proposed method exhibits improvements over state-of-the-art method on the Stanford Drone dataset.
  arXiv  Detail & Related papers  (2021-10-28T10:56:21Z)
• On the Existence of Optimal Transport Gradient for Learning Generative Models [8.602553195689513]
  Training of Wasserstein Generative Adversarial Networks (WGAN) relies on the calculation of the gradient of the optimal transport cost. We first demonstrate that such gradient may not be defined, which can result in numerical instabilities during gradient-based optimization. By exploiting the discrete nature of empirical data, we formulate the gradient in a semi-discrete setting and propose an algorithm for the optimization of the generative model parameters.
  arXiv  Detail & Related papers  (2021-02-10T16:28:20Z)
• Conditional Versus Adversarial Euler-based Generators For Time Series [2.2344764434954256]
  We introduce new generative models for time series based on Euler discretization. Tests show how the Euler discretization and the use of Wasserstein distance allow the proposed GANs and (more considerably) CEGEN to outperform state-of-the-art Time Series GAN generation.
  arXiv  Detail & Related papers  (2021-02-10T08:18:35Z)
• Comparing Probability Distributions with Conditional Transport [63.11403041984197]
  We propose conditional transport (CT) as a new divergence and approximate it with the amortized CT (ACT) cost. ACT amortizes the computation of its conditional transport plans and comes with unbiased sample gradients that are straightforward to compute. On a wide variety of benchmark datasets generative modeling, substituting the default statistical distance of an existing generative adversarial network with ACT is shown to consistently improve the performance.
  arXiv  Detail & Related papers  (2020-12-28T05:14:22Z)
• Autoregressive Score Matching [113.4502004812927]
  We propose autoregressive conditional score models (AR-CSM) where we parameterize the joint distribution in terms of the derivatives of univariable log-conditionals (scores) For AR-CSM models, this divergence between data and model distributions can be computed and optimized efficiently, requiring no expensive sampling or adversarial training. We show with extensive experimental results that it can be applied to density estimation on synthetic data, image generation, image denoising, and training latent variable models with implicit encoders.
  arXiv  Detail & Related papers  (2020-10-24T07:01:24Z)
• COT-GAN: Generating Sequential Data via Causal Optimal Transport [4.588028371034406]
  We introduce COT-GAN, an adversarial algorithm to train implicit generative models for producing sequential data. The success of the algorithm also relies on a new, improved version of the Sinkhorn divergence which demonstrates less bias in learning.
  arXiv  Detail & Related papers  (2020-06-15T17:37:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.