Multi-Source Knowledge-Based Hybrid Neural Framework for Time Series Representation Learning

• URL: http://arxiv.org/abs/2408.12409v1
• Date: Thu, 22 Aug 2024 13:58:55 GMT
• Title: Multi-Source Knowledge-Based Hybrid Neural Framework for Time Series Representation Learning
• Authors: Sagar Srinivas Sakhinana, Krishna Sai Sudhir Aripirala, Shivam Gupta, Venkataramana Runkana,
• Abstract summary: The proposed hybrid architecture addresses limitations by combining both domain-specific knowledge and implicit knowledge of the relational structure underlying the MTS data. The architecture shows promising results on multiple benchmark datasets, outperforming state-of-the-art forecasting methods.
• Score: 2.368662284133926
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Accurately predicting the behavior of complex dynamical systems, characterized by high-dimensional multivariate time series(MTS) in interconnected sensor networks, is crucial for informed decision-making in various applications to minimize risk. While graph forecasting networks(GFNs) are ideal for forecasting MTS data that exhibit spatio-temporal dependencies, prior works rely solely on the domain-specific knowledge of time-series variables inter-relationships to model the nonlinear dynamics, neglecting inherent relational structural dependencies among the variables within the MTS data. In contrast, contemporary works infer relational structures from MTS data but neglect domain-specific knowledge. The proposed hybrid architecture addresses these limitations by combining both domain-specific knowledge and implicit knowledge of the relational structure underlying the MTS data using Knowledge-Based Compositional Generalization. The hybrid architecture shows promising results on multiple benchmark datasets, outperforming state-of-the-art forecasting methods. Additionally, the architecture models the time varying uncertainty of multi-horizon forecasts.

Related papers

• A Survey of Model Architectures in Information Retrieval [64.75808744228067]
  We focus on two key aspects: backbone models for feature extraction and end-to-end system architectures for relevance estimation. We trace the development from traditional term-based methods to modern neural approaches, particularly highlighting the impact of transformer-based models and subsequent large language models (LLMs) We conclude by discussing emerging challenges and future directions, including architectural optimizations for performance and scalability, handling of multimodal, multilingual data, and adaptation to novel application domains beyond traditional search paradigms.
  arXiv  Detail & Related papers  (2025-02-20T18:42:58Z)
• Generalized Factor Neural Network Model for High-dimensional Regression [50.554377879576066]
  We tackle the challenges of modeling high-dimensional data sets with latent low-dimensional structures hidden within complex, non-linear, and noisy relationships. Our approach enables a seamless integration of concepts from non-parametric regression, factor models, and neural networks for high-dimensional regression.
  arXiv  Detail & Related papers  (2025-02-16T23:13:55Z)
• Online Multi-modal Root Cause Analysis [61.94987309148539]
  Root Cause Analysis (RCA) is essential for pinpointing the root causes of failures in microservice systems. Existing online RCA methods handle only single-modal data overlooking, complex interactions in multi-modal systems. We introduce OCEAN, a novel online multi-modal causal structure learning method for root cause localization.
  arXiv  Detail & Related papers  (2024-10-13T21:47:36Z)
• Multi-Knowledge Fusion Network for Time Series Representation Learning [2.368662284133926]
  We propose a hybrid architecture that combines prior knowledge with implicit knowledge of the relational structure within the MTS data. The proposed architecture has shown promising results on multiple benchmark datasets and outperforms state-of-the-art forecasting methods by a significant margin.
  arXiv  Detail & Related papers  (2024-08-22T14:18:16Z)
• Deep Coupling Network For Multivariate Time Series Forecasting [24.01637416183444]
  We propose a novel deep coupling network for MTS forecasting, named DeepCN. Our proposed DeepCN achieves superior performance compared with the state-of-the-art baselines.
  arXiv  Detail & Related papers  (2024-02-23T06:38:08Z)
• Structural Knowledge Informed Continual Multivariate Time Series Forecasting [23.18105409644709]
  We propose a novel Structural Knowledge Informed Continual Learning (SKI-CL) framework to perform MTS forecasting within a continual learning paradigm. Specifically, we develop a forecasting model based on graph structure learning, where a consistency regularization scheme is imposed between the learned variable dependencies and the structural knowledge. We develop a representation-matching memory replay scheme that maximizes the temporal coverage of MTS data to efficiently preserve the underlying temporal dynamics and dependency structures of each regime.
  arXiv  Detail & Related papers  (2024-02-20T05:11:20Z)
• Disentangling Structured Components: Towards Adaptive, Interpretable and Scalable Time Series Forecasting [52.47493322446537]
  We develop a adaptive, interpretable and scalable forecasting framework, which seeks to individually model each component of the spatial-temporal patterns. SCNN works with a pre-defined generative process of MTS, which arithmetically characterizes the latent structure of the spatial-temporal patterns. Extensive experiments are conducted to demonstrate that SCNN can achieve superior performance over state-of-the-art models on three real-world datasets.
  arXiv  Detail & Related papers  (2023-05-22T13:39:44Z)
• Leveraging the structure of dynamical systems for data-driven modeling [111.45324708884813]
  We consider the impact of the training set and its structure on the quality of the long-term prediction. We show how an informed design of the training set, based on invariants of the system and the structure of the underlying attractor, significantly improves the resulting models.
  arXiv  Detail & Related papers  (2021-12-15T20:09:20Z)
• LAVARNET: Neural Network Modeling of Causal Variable Relationships for Multivariate Time Series Forecasting [18.89688469820947]
  A novel neural network-based architecture is proposed, termed LAgged VAriable NETwork. It intrinsically estimates the importance of latent lagged variables and combines high dimensional representations of them to predict future values time series. Our model is compared with other baseline and state of the art neural network architectures on one simulated data set and four real data sets from meteorology, music, solar activity, finance areas.
  arXiv  Detail & Related papers  (2020-09-02T10:57:28Z)
• MTHetGNN: A Heterogeneous Graph Embedding Framework for Multivariate Time Series Forecasting [4.8274015390665195]
  We propose a novel end-to-end deep learning model, termed Multivariate Time Series Forecasting via Heterogeneous Graph Neural Networks (MTHetGNN) To characterize complex relations among variables, a relation embedding module is designed in MTHetGNN, where each variable is regarded as a graph node. A temporal embedding module is introduced for time series features extraction, where involving convolutional neural network (CNN) filters with different perception scales.
  arXiv  Detail & Related papers  (2020-08-19T18:21:22Z)
• Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks [91.65637773358347]
  We propose a general graph neural network framework designed specifically for multivariate time series data. Our approach automatically extracts the uni-directed relations among variables through a graph learning module. Our proposed model outperforms the state-of-the-art baseline methods on 3 of 4 benchmark datasets.
  arXiv  Detail & Related papers  (2020-05-24T04:02:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.