The DONUT Approach to EnsembleCombination Forecasting
- URL: http://arxiv.org/abs/2201.00426v1
- Date: Sun, 2 Jan 2022 22:19:26 GMT
- Title: The DONUT Approach to EnsembleCombination Forecasting
- Authors: Lars Lien Ankile, Kjartan Krange
- Abstract summary: This paper presents an ensemble forecasting method that shows strong results on the M4Competition dataset.
Our assumption reductions, consisting mainly of auto-generated features and a more diverse model pool, significantly outperforms the statistical-feature-based ensemble method FFORMA.
We also present a formal ex-post-facto analysis of optimal combination and selection for ensembles, quantifying differences through linear optimization on the M4 dataset.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: This paper presents an ensemble forecasting method that shows strong results
on the M4Competition dataset by decreasing feature and model selection
assumptions, termed DONUT(DO Not UTilize human assumptions). Our assumption
reductions, consisting mainly of auto-generated features and a more diverse
model pool for the ensemble, significantly outperforms the
statistical-feature-based ensemble method FFORMA by Montero-Manso et al.
(2020). Furthermore, we investigate feature extraction with a Long short-term
memory Network(LSTM) Autoencoder and find that such features contain crucial
information not captured by traditional statistical feature approaches. The
ensemble weighting model uses both LSTM features and statistical features to
combine the models accurately. Analysis of feature importance and interaction
show a slight superiority for LSTM features over the statistical ones alone.
Clustering analysis shows that different essential LSTM features are different
from most statistical features and each other. We also find that increasing the
solution space of the weighting model by augmenting the ensemble with new
models is something the weighting model learns to use, explaining part of the
accuracy gains. Lastly, we present a formal ex-post-facto analysis of optimal
combination and selection for ensembles, quantifying differences through linear
optimization on the M4 dataset. We also include a short proof that model
combination is superior to model selection, a posteriori.
Related papers
- Model-GLUE: Democratized LLM Scaling for A Large Model Zoo in the Wild [84.57103623507082]
This paper introduces Model-GLUE, a holistic Large Language Models scaling guideline.
Our work starts with a benchmarking of existing LLM scaling techniques, especially selective merging, and variants of mixture.
Our methodology involves the clustering of mergeable models and optimal merging strategy selection, and the integration of clusters through a model mixture.
arXiv Detail & Related papers (2024-10-07T15:55:55Z) - EMR-Merging: Tuning-Free High-Performance Model Merging [55.03509900949149]
We show that Elect, Mask & Rescale-Merging (EMR-Merging) shows outstanding performance compared to existing merging methods.
EMR-Merging is tuning-free, thus requiring no data availability or any additional training while showing impressive performance.
arXiv Detail & Related papers (2024-05-23T05:25:45Z) - Latent Semantic Consensus For Deterministic Geometric Model Fitting [109.44565542031384]
We propose an effective method called Latent Semantic Consensus (LSC)
LSC formulates the model fitting problem into two latent semantic spaces based on data points and model hypotheses.
LSC is able to provide consistent and reliable solutions within only a few milliseconds for general multi-structural model fitting.
arXiv Detail & Related papers (2024-03-11T05:35:38Z) - Binary Feature Mask Optimization for Feature Selection [0.0]
We introduce a novel framework that selects features considering the predictions of the model.
Our framework innovates by using a novel feature masking approach to eliminate the features during the selection process.
We demonstrate significant performance improvements on the real-life datasets using LightGBM and Multi-Layer Perceptron as our ML models.
arXiv Detail & Related papers (2024-01-23T10:54:13Z) - Embedded feature selection in LSTM networks with multi-objective
evolutionary ensemble learning for time series forecasting [49.1574468325115]
We present a novel feature selection method embedded in Long Short-Term Memory networks.
Our approach optimize the weights and biases of the LSTM in a partitioned manner.
Experimental evaluations on air quality time series data from Italy and southeast Spain demonstrate that our method substantially improves the ability generalization of conventional LSTMs.
arXiv Detail & Related papers (2023-12-29T08:42:10Z) - A Bayesian Framework on Asymmetric Mixture of Factor Analyser [0.0]
This paper introduces an MFA model with a rich and flexible class of skew normal (unrestricted) generalized hyperbolic (called SUNGH) distributions.
The SUNGH family provides considerable flexibility to model skewness in different directions as well as allowing for heavy tailed data.
Considering factor analysis models, the SUNGH family also allows for skewness and heavy tails for both the error component and factor scores.
arXiv Detail & Related papers (2022-11-01T20:19:52Z) - Model ensemble instead of prompt fusion: a sample-specific knowledge
transfer method for few-shot prompt tuning [85.55727213502402]
We focus on improving the few-shot performance of prompt tuning by transferring knowledge from soft prompts of source tasks.
We propose Sample-specific Ensemble of Source Models (SESoM)
SESoM learns to adjust the contribution of each source model for each target sample separately when ensembling source model outputs.
arXiv Detail & Related papers (2022-10-23T01:33:16Z) - Sparse MoEs meet Efficient Ensembles [49.313497379189315]
We study the interplay of two popular classes of such models: ensembles of neural networks and sparse mixture of experts (sparse MoEs)
We present Efficient Ensemble of Experts (E$3$), a scalable and simple ensemble of sparse MoEs that takes the best of both classes of models, while using up to 45% fewer FLOPs than a deep ensemble.
arXiv Detail & Related papers (2021-10-07T11:58:35Z) - A Data-driven feature selection and machine-learning model benchmark for
the prediction of longitudinal dispersion coefficient [29.58577229101903]
An accurate prediction on Longitudinal Dispersion(LD) coefficient can produce a performance leap in related simulation.
In this study, a global optimal feature set was proposed through numerical comparison of the distilled local optimums in performance with representative ML models.
Results show that the support vector machine has significantly better performance than other models.
arXiv Detail & Related papers (2021-07-16T09:50:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.