Robust Time Series Forecasting with Non-Heavy-Tailed Gaussian Loss-Weighted Sampler

• URL: http://arxiv.org/abs/2406.13871v1
• Date: Wed, 19 Jun 2024 22:28:18 GMT
• Title: Robust Time Series Forecasting with Non-Heavy-Tailed Gaussian Loss-Weighted Sampler
• Authors: Jiang You, Arben Cela, René Natowicz, Jacob Ouanounou, Patrick Siarry,
• Abstract summary: Recent resampling methods aim to increase training efficiency by reweighting samples based on their running losses. We introduce a novel approach: a Gaussian loss-weighted sampler that multiplies their running losses with a Gaussian distribution weight. It reduces the probability of selecting samples with very low or very high losses while favoring those close to average losses.
• Score: 1.8816077341295625
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Forecasting multivariate time series is a computationally intensive task challenged by extreme or redundant samples. Recent resampling methods aim to increase training efficiency by reweighting samples based on their running losses. However, these methods do not solve the problems caused by heavy-tailed distribution losses, such as overfitting to outliers. To tackle these issues, we introduce a novel approach: a Gaussian loss-weighted sampler that multiplies their running losses with a Gaussian distribution weight. It reduces the probability of selecting samples with very low or very high losses while favoring those close to average losses. As it creates a weighted loss distribution that is not heavy-tailed theoretically, there are several advantages to highlight compared to existing methods: 1) it relieves the inefficiency in learning redundant easy samples and overfitting to outliers, 2) It improves training efficiency by preferentially learning samples close to the average loss. Application on real-world time series forecasting datasets demonstrate improvements in prediction quality for 1%-4% using mean square error measurements in channel-independent settings. The code will be available online after 1 the review.

Related papers

• Double Correction Framework for Denoising Recommendation [45.98207284259792]
  In implicit feedback, noisy samples can affect precise user preference learning. A popular solution is based on dropping noisy samples in the model training phase. We propose a Double Correction Framework for Denoising Recommendation.
  arXiv  Detail & Related papers  (2024-05-18T12:15:10Z)
• Probabilistic Contrastive Learning for Long-Tailed Visual Recognition [78.70453964041718]
  Longtailed distributions frequently emerge in real-world data, where a large number of minority categories contain a limited number of samples. Recent investigations have revealed that supervised contrastive learning exhibits promising potential in alleviating the data imbalance. We propose a novel probabilistic contrastive (ProCo) learning algorithm that estimates the data distribution of the samples from each class in the feature space.
  arXiv  Detail & Related papers  (2024-03-11T13:44:49Z)
• Efficient Hybrid Oversampling and Intelligent Undersampling for Imbalanced Big Data Classification [1.03590082373586]
  We present a novel resampling method called SMOTENN that combines intelligent undersampling and oversampling using a MapReduce framework. Our experimental results show the virtues of this approach, outperforming alternative resampling techniques for small- and medium-sized datasets.
  arXiv  Detail & Related papers  (2023-10-09T15:22:13Z)
• Learning to Re-weight Examples with Optimal Transport for Imbalanced Classification [74.62203971625173]
  Imbalanced data pose challenges for deep learning based classification models. One of the most widely-used approaches for tackling imbalanced data is re-weighting. We propose a novel re-weighting method based on optimal transport (OT) from a distributional point of view.
  arXiv  Detail & Related papers  (2022-08-05T01:23:54Z)
• Delving into Sample Loss Curve to Embrace Noisy and Imbalanced Data [17.7825114228313]
  Corrupted labels and class imbalance are commonly encountered in practically collected training data. Existing approaches alleviate these issues by adopting a sample re-weighting strategy. However, biased samples with corrupted labels and of tailed classes commonly co-exist in training data.
  arXiv  Detail & Related papers  (2021-12-30T09:20:07Z)
• Unrolling Particles: Unsupervised Learning of Sampling Distributions [102.72972137287728]
  Particle filtering is used to compute good nonlinear estimates of complex systems. We show in simulations that the resulting particle filter yields good estimates in a wide range of scenarios.
  arXiv  Detail & Related papers  (2021-10-06T16:58:34Z)
• Machine Learning's Dropout Training is Distributionally Robust Optimal [10.937094979510212]
  This paper shows that dropout training in Generalized Linear Models provides out-of-sample expected loss guarantees. It also provides a novel, parallelizable, Unbiased Multi-Level Monte Carlo algorithm to speed-up the implementation of dropout training.
  arXiv  Detail & Related papers  (2020-09-13T23:13:28Z)
• Learning a Unified Sample Weighting Network for Object Detection [113.98404690619982]
  Region sampling or weighting is significantly important to the success of modern region-based object detectors. We argue that sample weighting should be data-dependent and task-dependent. We propose a unified sample weighting network to predict a sample's task weights.
  arXiv  Detail & Related papers  (2020-06-11T16:19:16Z)
• Bandit Samplers for Training Graph Neural Networks [63.17765191700203]
  Several sampling algorithms with variance reduction have been proposed for accelerating the training of Graph Convolution Networks (GCNs) These sampling algorithms are not applicable to more general graph neural networks (GNNs) where the message aggregator contains learned weights rather than fixed weights, such as Graph Attention Networks (GAT)
  arXiv  Detail & Related papers  (2020-06-10T12:48:37Z)
• Robust and On-the-fly Dataset Denoising for Image Classification [72.10311040730815]
  On-the-fly Data Denoising (ODD) is robust to mislabeled examples, while introducing almost zero computational overhead compared to standard training. ODD is able to achieve state-of-the-art results on a wide range of datasets including real-world ones such as WebVision and Clothing1M.
  arXiv  Detail & Related papers  (2020-03-24T03:59:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.