Scalable Signature-Based Distribution Regression via Reference Sets

• URL: http://arxiv.org/abs/2410.09196v1
• Date: Fri, 11 Oct 2024 18:58:28 GMT
• Title: Scalable Signature-Based Distribution Regression via Reference Sets
• Authors: Andrew Alden, Carmine Ventre, Blanka Horvath,
• Abstract summary: Path signatures are used to leverage the information encoded in paths via signature-based features. Current state of the art DR solutions are memory intensive and incur a high cost. This computational bottleneck limits the application to small sample sizes. We present a methodology for addressing the above issues; resolving estimation uncertainties. We also propose a pipeline that enables us to use DR for a wide variety of learning tasks.
• Score: 1.8980236415886387
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Distribution Regression (DR) on stochastic processes describes the learning task of regression on collections of time series. Path signatures, a technique prevalent in stochastic analysis, have been used to solve the DR problem. Recent works have demonstrated the ability of such solutions to leverage the information encoded in paths via signature-based features. However, current state of the art DR solutions are memory intensive and incur a high computation cost. This leads to a trade-off between path length and the number of paths considered. This computational bottleneck limits the application to small sample sizes which consequently introduces estimation uncertainty. In this paper, we present a methodology for addressing the above issues; resolving estimation uncertainties whilst also proposing a pipeline that enables us to use DR for a wide variety of learning tasks. Integral to our approach is our novel distance approximator. This allows us to seamlessly apply our methodology across different application domains, sampling rates, and stochastic process dimensions. We show that our model performs well in applications related to estimation theory, quantitative finance, and physical sciences. We demonstrate that our model generalises well, not only to unseen data within a given distribution, but also under unseen regimes (unseen classes of stochastic models).

Related papers

• Truncating Trajectories in Monte Carlo Policy Evaluation: an Adaptive Approach [51.76826149868971]
  Policy evaluation via Monte Carlo simulation is at the core of many MC Reinforcement Learning (RL) algorithms. We propose as a quality index a surrogate of the mean squared error of a return estimator that uses trajectories of different lengths. We present an adaptive algorithm called Robust and Iterative Data collection strategy Optimization (RIDO)
  arXiv  Detail & Related papers  (2024-10-17T11:47:56Z)
• DistPred: A Distribution-Free Probabilistic Inference Method for Regression and Forecasting [14.390842560217743]
  We propose a novel approach called DistPred for regression and forecasting tasks. We transform proper scoring rules that measure the discrepancy between the predicted distribution and the target distribution into a differentiable discrete form. This allows the model to sample numerous samples in a single forward pass to estimate the potential distribution of the response variable.
  arXiv  Detail & Related papers  (2024-06-17T10:33:00Z)
• Resampling Stochastic Gradient Descent Cheaply for Efficient Uncertainty Quantification [6.832893024035705]
  We investigate two computationally cheap resampling-based methods to construct confidence intervals for SGD solutions. One uses multiple, but few, SGDs in parallel via resampling with replacement from the data, and another operates this in an online fashion.
  arXiv  Detail & Related papers  (2023-10-17T08:18:10Z)
• Amortizing intractable inference in large language models [56.92471123778389]
  We use amortized Bayesian inference to sample from intractable posterior distributions. We empirically demonstrate that this distribution-matching paradigm of LLM fine-tuning can serve as an effective alternative to maximum-likelihood training. As an important application, we interpret chain-of-thought reasoning as a latent variable modeling problem.
  arXiv  Detail & Related papers  (2023-10-06T16:36:08Z)
• DBsurf: A Discrepancy Based Method for Discrete Stochastic Gradient Estimation [2.89784213091656]
  We introduce DBsurf, a reinforce-based estimator for discrete distributions. It uses a novel sampling procedure to reduce the discrepancy between the samples and the actual distribution. DBsurf attains the lowest variance in a least squares problem commonly used in the literature for benchmarking.
  arXiv  Detail & Related papers  (2023-09-07T19:15:40Z)
• A deep learning approach to the probabilistic numerical solution of path-dependent partial differential equations [1.09650651784511]
  PPDE solutions can be approximated by a probabilistic representation, implemented in the literature by the estimation of conditional expectations using regression. We overcome this limitation by the use of deep learning methods, and we show that this setting allows for the derivation of the bounds of error on the approximation of conditional expectations. In comparison with other deep learning approaches, our algorithm appears to be more accurate, especially in large dimensions.
  arXiv  Detail & Related papers  (2022-09-28T14:34:58Z)
• Deep Equilibrium Optical Flow Estimation [80.80992684796566]
  Recent state-of-the-art (SOTA) optical flow models use finite-step recurrent update operations to emulate traditional algorithms. These RNNs impose large computation and memory overheads, and are not directly trained to model such stable estimation. We propose deep equilibrium (DEQ) flow estimators, an approach that directly solves for the flow as the infinite-level fixed point of an implicit layer.
  arXiv  Detail & Related papers  (2022-04-18T17:53:44Z)
• Scaling Structured Inference with Randomization [64.18063627155128]
  We propose a family of dynamic programming (RDP) randomized for scaling structured models to tens of thousands of latent states. Our method is widely applicable to classical DP-based inference. It is also compatible with automatic differentiation so can be integrated with neural networks seamlessly.
  arXiv  Detail & Related papers  (2021-12-07T11:26:41Z)
• Attentional-Biased Stochastic Gradient Descent [74.49926199036481]
  We present a provable method (named ABSGD) for addressing the data imbalance or label noise problem in deep learning. Our method is a simple modification to momentum SGD where we assign an individual importance weight to each sample in the mini-batch. ABSGD is flexible enough to combine with other robust losses without any additional cost.
  arXiv  Detail & Related papers  (2020-12-13T03:41:52Z)
• Modern Multiple Imputation with Functional Data [6.624726878647541]
  This work considers the problem of fitting functional models with sparsely and irregularly sampled functional data. It overcomes the limitations of the state-of-the-art methods, which face major challenges in the fitting of more complex non-linear models.
  arXiv  Detail & Related papers  (2020-11-25T04:22:30Z)
• A Kernel-Based Approach to Non-Stationary Reinforcement Learning in Metric Spaces [53.47210316424326]
  KeRNS is an algorithm for episodic reinforcement learning in non-stationary Markov Decision Processes. We prove a regret bound that scales with the covering dimension of the state-action space and the total variation of the MDP with time.
  arXiv  Detail & Related papers  (2020-07-09T21:37:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.