TimewarpVAE: Simultaneous Time-Warping and Representation Learning of Trajectories

• URL: http://arxiv.org/abs/2310.16027v2
• Date: Fri, 7 Jun 2024 00:47:21 GMT
• Title: TimewarpVAE: Simultaneous Time-Warping and Representation Learning of Trajectories
• Authors: Travers Rhodes, Daniel D. Lee,
• Abstract summary: TimewarpVAE is a manifold-learning algorithm that simultaneously learns timing variations and latent factors of spatial variation. We show how the algorithm learns appropriate time alignments and meaningful representations of spatial variations in handwriting and fork manipulation datasets.
• Score: 15.28090738928877
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Human demonstrations of trajectories are an important source of training data for many machine learning problems. However, the difficulty of collecting human demonstration data for complex tasks makes learning efficient representations of those trajectories challenging. For many problems, such as for dexterous manipulation, the exact timings of the trajectories should be factored from their spatial path characteristics. In this work, we propose TimewarpVAE, a fully differentiable manifold-learning algorithm that incorporates Dynamic Time Warping (DTW) to simultaneously learn both timing variations and latent factors of spatial variation. We show how the TimewarpVAE algorithm learns appropriate time alignments and meaningful representations of spatial variations in handwriting and fork manipulation datasets. Our results have lower spatial reconstruction test error than baseline approaches and the learned low-dimensional representations can be used to efficiently generate semantically meaningful novel trajectories. We demonstrate the utility of our algorithm to generate novel high-speed trajectories for a robotic arm.

Related papers

• Linear Attention is Enough in Spatial-Temporal Forecasting [0.0]
  We propose treating nodes in road networks at different time steps as independent spatial-temporal tokens. We then feed them into a vanilla Transformer to learn complex spatial-temporal patterns. Our code achieves state-of-the-art performance at an affordable computational cost.
  arXiv  Detail & Related papers  (2024-08-17T10:06:50Z)
• Temporal Embeddings: Scalable Self-Supervised Temporal Representation Learning from Spatiotemporal Data for Multimodal Computer Vision [1.4127889233510498]
  A novel approach is proposed to stratify landscape based on mobility activity time series. The pixel-wise embeddings are converted to image-like channels that can be used for task-based, multimodal modeling.
  arXiv  Detail & Related papers  (2023-10-16T02:53:29Z)
• Spatio-Temporal Branching for Motion Prediction using Motion Increments [55.68088298632865]
  Human motion prediction (HMP) has emerged as a popular research topic due to its diverse applications. Traditional methods rely on hand-crafted features and machine learning techniques. We propose a noveltemporal-temporal branching network using incremental information for HMP.
  arXiv  Detail & Related papers  (2023-08-02T12:04:28Z)
• Self-supervised Trajectory Representation Learning with Temporal Regularities and Travel Semantics [30.9735101687326]
  Trajectory Representation Learning (TRL) is a powerful tool for spatial-temporal data analysis and management. Existing TRL works usually treat trajectories as ordinary sequence data, while some important spatial-temporal characteristics, such as temporal regularities and travel semantics, are not fully exploited. We propose a novel Self-supervised trajectory representation learning framework with TemporAl Regularities and Travel semantics, namely START.
  arXiv  Detail & Related papers  (2022-11-17T13:14:47Z)
• Semi-supervised Learning of Partial Differential Operators and Dynamical Flows [68.77595310155365]
  We present a novel method that combines a hyper-network solver with a Fourier Neural Operator architecture. We test our method on various time evolution PDEs, including nonlinear fluid flows in one, two, and three spatial dimensions. The results show that the new method improves the learning accuracy at the time point of supervision point, and is able to interpolate and the solutions to any intermediate time.
  arXiv  Detail & Related papers  (2022-07-28T19:59:14Z)
• Weakly-supervised Temporal Path Representation Learning with Contrastive Curriculum Learning -- Extended Version [35.86394282979721]
  A temporal path(TP) that includes temporal information, e.g., departure time, into the path is of fundamental to enable such applications. Existing methods fail to achieve the goal since (i) supervised methods require large amounts of task-specific labels when training and thus fail to generalize the obtained TPRs to other tasks. We propose a Weakly-Supervised Contrastive (WSC) learning model that encodes both the spatial and temporal information of a temporal path into a TPR.
  arXiv  Detail & Related papers  (2022-03-30T07:36:20Z)
• Averaging Spatio-temporal Signals using Optimal Transport and Soft Alignments [110.79706180350507]
  We show that our proposed loss can be used to define temporal-temporal baryechecenters as Fr'teche means duality. Experiments on handwritten letters and brain imaging data confirm our theoretical findings.
  arXiv  Detail & Related papers  (2022-03-11T09:46:22Z)
• Adaptive Machine Learning for Time-Varying Systems: Low Dimensional Latent Space Tuning [91.3755431537592]
  We present a recently developed method of adaptive machine learning for time-varying systems. Our approach is to map very high (N>100k) dimensional inputs into the low dimensional (N2) latent space at the output of the encoder section of an encoder-decoder CNN. This method allows us to learn correlations within and to track their evolution in real time based on feedback without interrupts.
  arXiv  Detail & Related papers  (2021-07-13T16:05:28Z)
• Adaptive Latent Space Tuning for Non-Stationary Distributions [62.997667081978825]
  We present a method for adaptive tuning of the low-dimensional latent space of deep encoder-decoder style CNNs. We demonstrate our approach for predicting the properties of a time-varying charged particle beam in a particle accelerator.
  arXiv  Detail & Related papers  (2021-05-08T03:50:45Z)
• Deep Cellular Recurrent Network for Efficient Analysis of Time-Series Data with Spatial Information [52.635997570873194]
  This work proposes a novel deep cellular recurrent neural network (DCRNN) architecture to process complex multi-dimensional time series data with spatial information. The proposed architecture achieves state-of-the-art performance while utilizing substantially less trainable parameters when compared to comparable methods in the literature.
  arXiv  Detail & Related papers  (2021-01-12T20:08:18Z)
• Fast Modeling and Understanding Fluid Dynamics Systems with Encoder-Decoder Networks [0.0]
  We show that an accurate deep-learning-based proxy model can be taught efficiently by a finite-volume-based simulator. Compared to traditional simulation, the proposed deep learning approach enables much faster forward computation. We quantify the sensitivity of the deep learning model to key physical parameters and hence demonstrate that the inversion problems can be solved with great acceleration.
  arXiv  Detail & Related papers  (2020-06-09T17:14:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.