A Self-organizing Interval Type-2 Fuzzy Neural Network for Multi-Step Time Series Prediction

• URL: http://arxiv.org/abs/2407.08010v2
• Date: Mon, 28 Apr 2025 20:35:20 GMT
• Title: A Self-organizing Interval Type-2 Fuzzy Neural Network for Multi-Step Time Series Prediction
• Authors: Fulong Yao, Wanqing Zhao, Matthew Forshaw, Yang Song,
• Abstract summary: Interval type 2 fuzzy neural network (IT2FNN) has shown exceptional performance in uncertainty modelling for single-step prediction tasks.<n>This paper proposes a new selforganizing interval type-2 fuzzy neural network with multiple outputs (SOIT2FNN-MO)<n> Experimental results on chaotic and microgrid prediction problems demonstrate that SOIT2FNN-MO outperforms state-of-the-art methods.
• Score: 9.546043411729206
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Data uncertainty is inherent in many real-world applications and poses significant challenges for accurate time series predictions. The interval type 2 fuzzy neural network (IT2FNN) has shown exceptional performance in uncertainty modelling for single-step prediction tasks. However, extending it for multi-step ahead predictions introduces further issues in uncertainty handling as well as model interpretability and accuracy. To address these issues, this paper proposes a new selforganizing interval type-2 fuzzy neural network with multiple outputs (SOIT2FNN-MO). Differing from the traditional six-layer IT2FNN, a nine-layer network architecture is developed. First, a new co-antecedent layer and a modified consequent layer are devised to improve the interpretability of the fuzzy model for multi-step time series prediction problems. Second, a new link layer is created to improve the accuracy by building temporal connections between multi-step predictions. Third, a new transformation layer is designed to address the problem of the vanishing rule strength caused by high-dimensional inputs. Furthermore, a two-stage, self-organizing learning mechanism is developed to automatically extract fuzzy rules from data and optimize network parameters. Experimental results on chaotic and microgrid prediction problems demonstrate that SOIT2FNN-MO outperforms state-of-the-art methods, by achieving a better accuracy ranging from 1.6% to 30% depending on the level of noises in data. Additionally, the proposed model is more interpretable, offering deeper insights into the prediction process.

Related papers

• Differential Machine Learning for Time Series Prediction [1.3812010983144802]
  We propose a novel approach that enhances neural network predictions through differential learning. We develop a differential long short-term memory (Diff-LSTM) network that uses a shared LSTM cell to simultaneously process both data streams.
  arXiv  Detail & Related papers  (2025-03-05T09:36:57Z)
• Neural Conformal Control for Time Series Forecasting [54.96087475179419]
  We introduce a neural network conformal prediction method for time series that enhances adaptivity in non-stationary environments. Our approach acts as a neural controller designed to achieve desired target coverage, leveraging auxiliary multi-view data with neural network encoders. We empirically demonstrate significant improvements in coverage and probabilistic accuracy, and find that our method is the only one that combines good calibration with consistency in prediction intervals.
  arXiv  Detail & Related papers  (2024-12-24T03:56:25Z)
• Jointly-Learned Exit and Inference for a Dynamic Neural Network : JEI-DNN [20.380620709345898]
  Early-exiting dynamic neural networks (EDNN) allow a model to make some of its predictions from intermediate layers (i.e., early-exit) Training an EDNN architecture is challenging as it consists of two intertwined components: the gating mechanism (GM) that controls early-exiting decisions and the intermediate inference modules (IMs) that perform inference from intermediate representations. We propose a novel architecture that connects these two modules. This leads to significant performance improvements on classification datasets and enables better uncertainty characterization capabilities.
  arXiv  Detail & Related papers  (2023-10-13T14:56:38Z)
• In-Context Convergence of Transformers [63.04956160537308]
  We study the learning dynamics of a one-layer transformer with softmax attention trained via gradient descent. For data with imbalanced features, we show that the learning dynamics take a stage-wise convergence process.
  arXiv  Detail & Related papers  (2023-10-08T17:55:33Z)
• TACTiS-2: Better, Faster, Simpler Attentional Copulas for Multivariate Time Series [57.4208255711412]
  Building on copula theory, we propose a simplified objective for the recently-introduced transformer-based attentional copulas (TACTiS) We show that the resulting model has significantly better training dynamics and achieves state-of-the-art performance across diverse real-world forecasting tasks.
  arXiv  Detail & Related papers  (2023-10-02T16:45:19Z)
• Deep Neural Networks Tend To Extrapolate Predictably [51.303814412294514]
  neural network predictions tend to be unpredictable and overconfident when faced with out-of-distribution (OOD) inputs. We observe that neural network predictions often tend towards a constant value as input data becomes increasingly OOD. We show how one can leverage our insights in practice to enable risk-sensitive decision-making in the presence of OOD inputs.
  arXiv  Detail & Related papers  (2023-10-02T03:25:32Z)
• Variational Density Propagation Continual Learning [0.0]
  Deep Neural Networks (DNNs) deployed to the real world are regularly subject to out-of-distribution (OoD) data. This paper proposes a framework for adapting to data distribution drift modeled by benchmark Continual Learning datasets.
  arXiv  Detail & Related papers  (2023-08-22T21:51:39Z)
• Probabilistic MIMO U-Net: Efficient and Accurate Uncertainty Estimation for Pixel-wise Regression [1.4528189330418977]
  Uncertainty estimation in machine learning is paramount for enhancing the reliability and interpretability of predictive models. We present an adaptation of the Multiple-Input Multiple-Output (MIMO) framework for pixel-wise regression tasks.
  arXiv  Detail & Related papers  (2023-08-14T22:08:28Z)
• Contextually Enhanced ES-dRNN with Dynamic Attention for Short-Term Load Forecasting [1.1602089225841632]
  The proposed model is composed of two simultaneously trained tracks: the context track and the main track. The RNN architecture consists of multiple recurrent layers stacked with hierarchical dilations and equipped with recently proposed attentive recurrent cells. The model produces both point forecasts and predictive intervals.
  arXiv  Detail & Related papers  (2022-12-18T07:42:48Z)
• An Adaptive and Stability-Promoting Layerwise Training Approach for Sparse Deep Neural Network Architecture [0.0]
  This work presents a two-stage adaptive framework for developing deep neural network (DNN) architectures that generalize well for a given training data set. In the first stage, a layerwise training approach is adopted where a new layer is added each time and trained independently by freezing parameters in the previous layers. We introduce a epsilon-delta stability-promoting concept as a desirable property for a learning algorithm and show that employing manifold regularization yields a epsilon-delta stability-promoting algorithm.
  arXiv  Detail & Related papers  (2022-11-13T09:51:16Z)
• Confidence-Nets: A Step Towards better Prediction Intervals for regression Neural Networks on small datasets [0.0]
  We propose an ensemble method that attempts to estimate the uncertainty of predictions, increase their accuracy and provide an interval for the expected variation. The proposed method is tested on various datasets, and a significant improvement in the performance of the neural network model is seen.
  arXiv  Detail & Related papers  (2022-10-31T06:38:40Z)
• Scalable computation of prediction intervals for neural networks via matrix sketching [79.44177623781043]
  Existing algorithms for uncertainty estimation require modifying the model architecture and training procedure. This work proposes a new algorithm that can be applied to a given trained neural network and produces approximate prediction intervals.
  arXiv  Detail & Related papers  (2022-05-06T13:18:31Z)
• Probabilistic AutoRegressive Neural Networks for Accurate Long-range Forecasting [6.295157260756792]
  We introduce the Probabilistic AutoRegressive Neural Networks (PARNN) PARNN is capable of handling complex time series data exhibiting non-stationarity, nonlinearity, non-seasonality, long-range dependence, and chaotic patterns. We evaluate the performance of PARNN against standard statistical, machine learning, and deep learning models, including Transformers, NBeats, and DeepAR.
  arXiv  Detail & Related papers  (2022-04-01T17:57:36Z)
• Spatial-Temporal-Fusion BNN: Variational Bayesian Feature Layer [77.78479877473899]
  We design a spatial-temporal-fusion BNN for efficiently scaling BNNs to large models. Compared to vanilla BNNs, our approach can greatly reduce the training time and the number of parameters, which contributes to scale BNNs efficiently.
  arXiv  Detail & Related papers  (2021-12-12T17:13:14Z)
• Semi-supervised Impedance Inversion by Bayesian Neural Network Based on 2-d CNN Pre-training [0.966840768820136]
  We improve the semi-supervised learning from two aspects. First, by replacing 1-d convolutional neural network layers in deep learning structure with 2-d CNN layers and 2-d maxpooling layers, the prediction accuracy is improved. Second, prediction uncertainty can also be estimated by embedding the network into a Bayesian inference framework.
  arXiv  Detail & Related papers  (2021-11-20T14:12:05Z)
• Bilinear Input Normalization for Neural Networks in Financial Forecasting [101.89872650510074]
  We propose a novel data-driven normalization method for deep neural networks that handle high-frequency financial time-series. The proposed normalization scheme takes into account the bimodal characteristic of financial time-series. Our experiments, conducted with state-of-the-arts neural networks and high-frequency data, show significant improvements over other normalization techniques.
  arXiv  Detail & Related papers  (2021-09-01T07:52:03Z)
• LocalDrop: A Hybrid Regularization for Deep Neural Networks [98.30782118441158]
  We propose a new approach for the regularization of neural networks by the local Rademacher complexity called LocalDrop. A new regularization function for both fully-connected networks (FCNs) and convolutional neural networks (CNNs) has been developed based on the proposed upper bound of the local Rademacher complexity.
  arXiv  Detail & Related papers  (2021-03-01T03:10:11Z)
• Improving Uncertainty Calibration via Prior Augmented Data [56.88185136509654]
  Neural networks have proven successful at learning from complex data distributions by acting as universal function approximators. They are often overconfident in their predictions, which leads to inaccurate and miscalibrated probabilistic predictions. We propose a solution by seeking out regions of feature space where the model is unjustifiably overconfident, and conditionally raising the entropy of those predictions towards that of the prior distribution of the labels.
  arXiv  Detail & Related papers  (2021-02-22T07:02:37Z)
• Predicting Temporal Sets with Deep Neural Networks [50.53727580527024]
  We propose an integrated solution based on the deep neural networks for temporal sets prediction. A unique perspective is to learn element relationship by constructing set-level co-occurrence graph. We design an attention-based module to adaptively learn the temporal dependency of elements and sets.
  arXiv  Detail & Related papers  (2020-06-20T03:29:02Z)
• Link Prediction for Temporally Consistent Networks [6.981204218036187]
  Link prediction estimates the next relationship in dynamic networks. The use of adjacency matrix to represent dynamically evolving networks limits the ability to analytically learn from heterogeneous, sparse, or forming networks. We propose a new method of canonically representing heterogeneous time-evolving activities as a temporally parameterized network model.
  arXiv  Detail & Related papers  (2020-06-06T07:28:03Z)
• Diversity inducing Information Bottleneck in Model Ensembles [73.80615604822435]
  In this paper, we target the problem of generating effective ensembles of neural networks by encouraging diversity in prediction. We explicitly optimize a diversity inducing adversarial loss for learning latent variables and thereby obtain diversity in the output predictions necessary for modeling multi-modal data. Compared to the most competitive baselines, we show significant improvements in classification accuracy, under a shift in the data distribution.
  arXiv  Detail & Related papers  (2020-03-10T03:10:41Z)
• Learn to Predict Sets Using Feed-Forward Neural Networks [63.91494644881925]
  This paper addresses the task of set prediction using deep feed-forward neural networks. We present a novel approach for learning to predict sets with unknown permutation and cardinality. We demonstrate the validity of our set formulations on relevant vision problems.
  arXiv  Detail & Related papers  (2020-01-30T01:52:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.