Efficient learning of nonlinear prediction models with time-series privileged information

• URL: http://arxiv.org/abs/2209.07067v4
• Date: Mon, 20 Nov 2023 13:49:25 GMT
• Title: Efficient learning of nonlinear prediction models with time-series privileged information
• Authors: Bastian Jung and Fredrik D Johansson
• Abstract summary: We show that for prediction in linear-Gaussian dynamical systems, a LuPI learner with access to intermediate time series data is never worse than any unbiased classical learner. We propose algorithms based on random features and representation learning for the case when this map is unknown.
• Score: 11.679648862014655
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In domains where sample sizes are limited, efficient learning algorithms are critical. Learning using privileged information (LuPI) offers increased sample efficiency by allowing prediction models access to auxiliary information at training time which is unavailable when the models are used. In recent work, it was shown that for prediction in linear-Gaussian dynamical systems, a LuPI learner with access to intermediate time series data is never worse and often better in expectation than any unbiased classical learner. We provide new insights into this analysis and generalize it to nonlinear prediction tasks in latent dynamical systems, extending theoretical guarantees to the case where the map connecting latent variables and observations is known up to a linear transform. In addition, we propose algorithms based on random features and representation learning for the case when this map is unknown. A suite of empirical results confirm theoretical findings and show the potential of using privileged time-series information in nonlinear prediction.

Related papers

• Learning Latent Graph Structures and their Uncertainty [63.95971478893842]
  Graph Neural Networks (GNNs) use relational information as an inductive bias to enhance the model's accuracy. As task-relevant relations might be unknown, graph structure learning approaches have been proposed to learn them while solving the downstream prediction task.
  arXiv  Detail & Related papers  (2024-05-30T10:49:22Z)
• Informed Spectral Normalized Gaussian Processes for Trajectory Prediction [0.0]
  We propose a novel regularization-based continual learning method for SNGPs. Our proposal builds upon well-established methods and requires no rehearsal memory or parameter expansion. We apply our informed SNGP model to the trajectory prediction problem in autonomous driving by integrating prior drivability knowledge.
  arXiv  Detail & Related papers  (2024-03-18T17:05:24Z)
• Graph-enabled Reinforcement Learning for Time Series Forecasting with Adaptive Intelligence [11.249626785206003]
  We propose a novel approach for predicting time-series data using Graphical neural network (GNN) and monitoring with Reinforcement Learning (RL) GNNs are able to explicitly incorporate the graph structure of the data into the model, allowing them to capture temporal dependencies in a more natural way. This approach allows for more accurate predictions in complex temporal structures, such as those found in healthcare, traffic and weather forecasting.
  arXiv  Detail & Related papers  (2023-09-18T22:25:12Z)
• Inexact iterative numerical linear algebra for neural network-based spectral estimation and rare-event prediction [0.0]
  Leading eigenfunctions of the transition operator are useful for visualization. We develop inexact iterative linear algebra methods for computing these eigenfunctions.
  arXiv  Detail & Related papers  (2023-03-22T13:07:03Z)
• Deep networks for system identification: a Survey [56.34005280792013]
  System identification learns mathematical descriptions of dynamic systems from input-output data. Main aim of the identified model is to predict new data from previous observations. We discuss architectures commonly adopted in the literature, like feedforward, convolutional, and recurrent networks.
  arXiv  Detail & Related papers  (2023-01-30T12:38:31Z)
• Using Time-Series Privileged Information for Provably Efficient Learning of Prediction Models [6.7015527471908625]
  We study prediction of future outcomes with supervised models that use privileged information during learning. privileged information comprises samples of time series observed between the baseline time of prediction and the future outcome. We show that our approach is generally preferable to classical learning, particularly when data is scarce.
  arXiv  Detail & Related papers  (2021-10-28T10:07:29Z)
• Imputation-Free Learning from Incomplete Observations [73.15386629370111]
  We introduce the importance of guided gradient descent (IGSGD) method to train inference from inputs containing missing values without imputation. We employ reinforcement learning (RL) to adjust the gradients used to train the models via back-propagation. Our imputation-free predictions outperform the traditional two-step imputation-based predictions using state-of-the-art imputation methods.
  arXiv  Detail & Related papers  (2021-07-05T12:44:39Z)
• Meta-Learning for Koopman Spectral Analysis with Short Time-series [49.41640137945938]
  Existing methods require long time-series for training neural networks. We propose a meta-learning method for estimating embedding functions from unseen short time-series. We experimentally demonstrate that the proposed method achieves better performance in terms of eigenvalue estimation and future prediction.
  arXiv  Detail & Related papers  (2021-02-09T07:19:19Z)
• Learning summary features of time series for likelihood free inference [93.08098361687722]
  We present a data-driven strategy for automatically learning summary features from time series data. Our results indicate that learning summary features from data can compete and even outperform LFI methods based on hand-crafted values.
  arXiv  Detail & Related papers  (2020-12-04T19:21:37Z)
• Representation Learning for Sequence Data with Deep Autoencoding Predictive Components [96.42805872177067]
  We propose a self-supervised representation learning method for sequence data, based on the intuition that useful representations of sequence data should exhibit a simple structure in the latent space. We encourage this latent structure by maximizing an estimate of predictive information of latent feature sequences, which is the mutual information between past and future windows at each time step. We demonstrate that our method recovers the latent space of noisy dynamical systems, extracts predictive features for forecasting tasks, and improves automatic speech recognition when used to pretrain the encoder on large amounts of unlabeled data.
  arXiv  Detail & Related papers  (2020-10-07T03:34:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.