Feature-aligned N-BEATS with Sinkhorn divergence

• URL: http://arxiv.org/abs/2305.15196v3
• Date: Sun, 25 Feb 2024 11:37:19 GMT
• Title: Feature-aligned N-BEATS with Sinkhorn divergence
• Abstract summary: We propose Feature-aligned N-BEATS as a domain-generalized time series forecasting model. It is a nontrivial extension of N-BEATS with doubly residual stacking principle into a representation learning framework.
• Score: 8.209488690262882
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose Feature-aligned N-BEATS as a domain-generalized time series forecasting model. It is a nontrivial extension of N-BEATS with doubly residual stacking principle (Oreshkin et al. [45]) into a representation learning framework. In particular, it revolves around marginal feature probability measures induced by the intricate composition of residual and feature extracting operators of N-BEATS in each stack and aligns them stack-wise via an approximate of an optimal transport distance referred to as the Sinkhorn divergence. The training loss consists of an empirical risk minimization from multiple source domains, i.e., forecasting loss, and an alignment loss calculated with the Sinkhorn divergence, which allows the model to learn invariant features stack-wise across multiple source data sequences while retaining N-BEATS's interpretable design and forecasting power. Comprehensive experimental evaluations with ablation studies are provided and the corresponding results demonstrate the proposed model's forecasting and generalization capabilities.

Related papers

• The role of parameter Jacobians in the stability of network outputs [2.000834478067184]
  We show that the corresponding linearized dynamics leads naturally to a semigroup formulation.<n>We prove norm-bounds on the corresponding semigroup perturbations.<n>We extend the comparison to nonautonomous NTK evolutions through piecewise-frozen approximations.
  arXiv  Detail & Related papers  (2026-08-27T22:21:10Z)
• Beyond NNGP: Large Deviations and Feature Learning in Bayesian Neural Networks [1.1470070927586018]
  Large-deviation theory provides explicit variational objectives-rate functions-on predictors.<n>We show that the posterior output rate function is obtained by a joint optimization over predictors and internal kernels.
  arXiv  Detail & Related papers  (2026-02-26T12:15:11Z)
• SHAP values through General Fourier Representations: Theory and Applications [0.0]
  This article establishes a rigorous framework for the mathematical analysis of SHAP values.<n>We show that any predictive model defined on a discrete or multi-valued input space admits a generalized Fourier expansion with respect to an orthonormalisation tensor-product basis constructed under a product probability measure.<n>In the deterministic regime, we derive quantitative stability estimates for SHAP values under Fourier truncation, showing that the attribution map is Lipschitz continuous with respect to the distance between predictors.<n>In the probabilistic regime, we consider neural networks in their infinite-width limit and prove convergence of SHAP values toward those induced by the corresponding Gaussian process
  arXiv  Detail & Related papers  (2025-10-31T18:41:36Z)
• Neural Optimal Transport Meets Multivariate Conformal Prediction [58.43397908730771]
  We propose a framework for conditional vectorile regression (CVQR)<n>CVQR combines neural optimal transport with quantized optimization, and apply it to predictions.
  arXiv  Detail & Related papers  (2025-09-29T19:50:19Z)
• A Simple Approximate Bayesian Inference Neural Surrogate for Stochastic Petri Net Models [0.0]
  We introduce a neural-network-based approximation of the posterior distribution framework.<n>Our model employs a lightweight 1D Convolutional Residual Network trained end-to-end on Gillespie-simulated SPN realizations.<n>On synthetic SPNs with 20% missing events, our surrogate recovers rate-function coefficients with an RMSE = 0.108 and substantially runs faster than traditional Bayesian approaches.
  arXiv  Detail & Related papers  (2025-07-14T18:31:19Z)
• Repulsive Ensembles for Bayesian Inference in Physics-informed Neural Networks [10.075911116030621]
  We consider the inverse problem and employ repulsive ensembles of PINNs for obtaining uncertainty estimates.<n>The repulsive ensemble produces significantly more accurate uncertainty estimates and exhibits higher sample diversity.
  arXiv  Detail & Related papers  (2025-05-22T21:58:40Z)
• Unveiling the Statistical Foundations of Chain-of-Thought Prompting Methods [59.779795063072655]
  Chain-of-Thought (CoT) prompting and its variants have gained popularity as effective methods for solving multi-step reasoning problems. We analyze CoT prompting from a statistical estimation perspective, providing a comprehensive characterization of its sample complexity.
  arXiv  Detail & Related papers  (2024-08-25T04:07:18Z)
• Rethinking Clustered Federated Learning in NOMA Enhanced Wireless Networks [60.09912912343705]
  This study explores the benefits of integrating the novel clustered federated learning (CFL) approach with non-independent and identically distributed (non-IID) datasets. A detailed theoretical analysis of the generalization gap that measures the degree of non-IID in the data distribution is presented. Solutions to address the challenges posed by non-IID conditions are proposed with the analysis of the properties.
  arXiv  Detail & Related papers  (2024-03-05T17:49:09Z)
• Variational excess risk bound for general state space models [0.0]
  We consider variational autoencoders (VAE) for general state space models. We consider a backward factorization of the variational distributions to analyze the excess risk associated with VAE.
  arXiv  Detail & Related papers  (2023-12-15T08:41:07Z)
• A probabilistic, data-driven closure model for RANS simulations with aleatoric, model uncertainty [1.8416014644193066]
  We propose a data-driven, closure model for Reynolds-averaged Navier-Stokes (RANS) simulations that incorporates aleatoric, model uncertainty. A fully Bayesian formulation is proposed, combined with a sparsity-inducing prior in order to identify regions in the problem domain where the parametric closure is insufficient.
  arXiv  Detail & Related papers  (2023-07-05T16:53:31Z)
• Reinforcement Learning from Partial Observation: Linear Function Approximation with Provable Sample Efficiency [111.83670279016599]
  We study reinforcement learning for partially observed decision processes (POMDPs) with infinite observation and state spaces. We make the first attempt at partial observability and function approximation for a class of POMDPs with a linear structure.
  arXiv  Detail & Related papers  (2022-04-20T21:15:38Z)
• Optimal variance-reduced stochastic approximation in Banach spaces [114.8734960258221]
  We study the problem of estimating the fixed point of a contractive operator defined on a separable Banach space. We establish non-asymptotic bounds for both the operator defect and the estimation error.
  arXiv  Detail & Related papers  (2022-01-21T02:46:57Z)
• Posterior-Aided Regularization for Likelihood-Free Inference [23.708122045184698]
  Posterior-Aided Regularization (PAR) is applicable to learning the density estimator, regardless of the model structure. We provide a unified estimation method of PAR to estimate both reverse KL term and mutual information term with a single neural network.
  arXiv  Detail & Related papers  (2021-02-15T16:59:30Z)
• CASTLE: Regularization via Auxiliary Causal Graph Discovery [89.74800176981842]
  We introduce Causal Structure Learning (CASTLE) regularization and propose to regularize a neural network by jointly learning the causal relationships between variables. CASTLE efficiently reconstructs only the features in the causal DAG that have a causal neighbor, whereas reconstruction-based regularizers suboptimally reconstruct all input features.
  arXiv  Detail & Related papers  (2020-09-28T09:49:38Z)
• The Heavy-Tail Phenomenon in SGD [7.366405857677226]
  We show that depending on the structure of the Hessian of the loss at the minimum, the SGD iterates will converge to a emphheavy-tailed stationary distribution. We translate our results into insights about the behavior of SGD in deep learning.
  arXiv  Detail & Related papers  (2020-06-08T16:43:56Z)
• An Optimal Statistical and Computational Framework for Generalized Tensor Estimation [10.899518267165666]
  This paper describes a flexible framework for low-rank tensor estimation problems. It includes many important instances from applications in computational imaging, genomics, and network analysis.
  arXiv  Detail & Related papers  (2020-02-26T01:54:35Z)
• Supervised Learning for Non-Sequential Data: A Canonical Polyadic Decomposition Approach [85.12934750565971]
  Efficient modelling of feature interactions underpins supervised learning for non-sequential tasks. To alleviate this issue, it has been proposed to implicitly represent the model parameters as a tensor. For enhanced expressiveness, we generalize the framework to allow feature mapping to arbitrarily high-dimensional feature vectors.
  arXiv  Detail & Related papers  (2020-01-27T22:38:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.