Reconstructing Spatiotemporal Data with C-VAEs

• URL: http://arxiv.org/abs/2307.06243v2
• Date: Mon, 28 Aug 2023 16:18:30 GMT
• Title: Reconstructing Spatiotemporal Data with C-VAEs
• Authors: Tiago F. R. Ribeiro, Fernando Silva, Rog\'erio Lu\'is de C. Costa
• Abstract summary: Conditional continuous representation of moving regions is commonly used. In this work, we explore the capabilities of Conditional Varitemporal Autoencoder (C-VAE) models to generate realistic representations of regions' evolution.
• Score: 49.1574468325115
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The continuous representation of spatiotemporal data commonly relies on using abstract data types, such as \textit{moving regions}, to represent entities whose shape and position continuously change over time. Creating this representation from discrete snapshots of real-world entities requires using interpolation methods to compute in-between data representations and estimate the position and shape of the object of interest at arbitrary temporal points. Existing region interpolation methods often fail to generate smooth and realistic representations of a region's evolution. However, recent advancements in deep learning techniques have revealed the potential of deep models trained on discrete observations to capture spatiotemporal dependencies through implicit feature learning. In this work, we explore the capabilities of Conditional Variational Autoencoder (C-VAE) models to generate smooth and realistic representations of the spatiotemporal evolution of moving regions. We evaluate our proposed approach on a sparsely annotated dataset on the burnt area of a forest fire. We apply compression operations to sample from the dataset and use the C-VAE model and other commonly used interpolation algorithms to generate in-between region representations. To evaluate the performance of the methods, we compare their interpolation results with manually annotated data and regions generated by a U-Net model. We also assess the quality of generated data considering temporal consistency metrics. The proposed C-VAE-based approach demonstrates competitive results in geometric similarity metrics. It also exhibits superior temporal consistency, suggesting that C-VAE models may be a viable alternative to modelling the spatiotemporal evolution of 2D moving regions.

Related papers

• How to Bridge Spatial and Temporal Heterogeneity in Link Prediction? A Contrastive Method [11.719027225797037]
  We propose a novel textbfContrastive Learning-based textbfLink textbfPrediction model, textbfCLP. Our mymodel consistently outperforms the state-of-the-art models, demonstrating an average improvement of 10.10%, 13.44% in terms of AUC and AP.
  arXiv  Detail & Related papers  (2024-11-01T14:20:53Z)
• Spatially-Aware Diffusion Models with Cross-Attention for Global Field Reconstruction with Sparse Observations [1.371691382573869]
  We develop and enhance score-based diffusion models in field reconstruction tasks. We introduce a condition encoding approach to construct a tractable mapping mapping between observed and unobserved regions. We demonstrate the ability of the model to capture possible reconstructions and improve the accuracy of fused results.
  arXiv  Detail & Related papers  (2024-08-30T19:46:23Z)
• VTAE: Variational Transformer Autoencoder with Manifolds Learning [144.0546653941249]
  Deep generative models have demonstrated successful applications in learning non-linear data distributions through a number of latent variables. The nonlinearity of the generator implies that the latent space shows an unsatisfactory projection of the data space, which results in poor representation learning. We show that geodesics and accurate computation can substantially improve the performance of deep generative models.
  arXiv  Detail & Related papers  (2023-04-03T13:13:19Z)
• Distributed Bayesian Learning of Dynamic States [65.7870637855531]
  The proposed algorithm is a distributed Bayesian filtering task for finite-state hidden Markov models. It can be used for sequential state estimation, as well as for modeling opinion formation over social networks under dynamic environments.
  arXiv  Detail & Related papers  (2022-12-05T19:40:17Z)
• Geo-Adaptive Deep Spatio-Temporal predictive modeling for human mobility [5.864710987890994]
  Deep GA-vLS assumes data to be of fixed and regular shaped tensor shaped and face challenges of handling irregular data. We present a novel geo-aware enabled learning operation based on a novel data structure for dependencies while maintaining the recurrent mechanism.
  arXiv  Detail & Related papers  (2022-11-27T16:51:28Z)
• Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learning [112.69497636932955]
  Federated learning aims to train models across different clients without the sharing of data for privacy considerations. We study how data heterogeneity affects the representations of the globally aggregated models. We propose sc FedDecorr, a novel method that can effectively mitigate dimensional collapse in federated learning.
  arXiv  Detail & Related papers  (2022-10-01T09:04:17Z)
• Learning from Temporal Spatial Cubism for Cross-Dataset Skeleton-based Action Recognition [88.34182299496074]
  Action labels are only available on a source dataset, but unavailable on a target dataset in the training stage. We utilize a self-supervision scheme to reduce the domain shift between two skeleton-based action datasets. By segmenting and permuting temporal segments or human body parts, we design two self-supervised learning classification tasks.
  arXiv  Detail & Related papers  (2022-07-17T07:05:39Z)
• Contrastive Neighborhood Alignment [81.65103777329874]
  We present Contrastive Neighborhood Alignment (CNA), a manifold learning approach to maintain the topology of learned features. The target model aims to mimic the local structure of the source representation space using a contrastive loss. CNA is illustrated in three scenarios: manifold learning, where the model maintains the local topology of the original data in a dimension-reduced space; model distillation, where a small student model is trained to mimic a larger teacher; and legacy model update, where an older model is replaced by a more powerful one.
  arXiv  Detail & Related papers  (2022-01-06T04:58:31Z)
• Conditional Local Filters with Explainers for Spatio-Temporal Forecasting [27.62110162024104]
  A novel graph-based directed convolution is proposed to capture the spatial dependency. The filter is embedded in a Recurrent Neural Network (RNN) architecture for modeling the temporal dynamics. The methods are evaluated on real-world datasets including road network traffic flow, earth surface temperature & wind flows and disease spread datasets.
  arXiv  Detail & Related papers  (2021-01-04T14:22:11Z)
• Localising In Complex Scenes Using Balanced Adversarial Adaptation [19.160686658569507]
  Domain adaptation and generative modelling have collectively mitigated the expensive nature of data collection and labelling. We study the performance gap that exists between representations optimised for localisation on simulation environments and the application of such representations in a real-world setting.
  arXiv  Detail & Related papers  (2020-11-09T00:40:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.