Kernel-U-Net: Symmetric and Hierarchical Architecture for Multivariate
Time Series Forecasting
- URL: http://arxiv.org/abs/2401.01479v2
- Date: Mon, 12 Feb 2024 03:50:55 GMT
- Title: Kernel-U-Net: Symmetric and Hierarchical Architecture for Multivariate
Time Series Forecasting
- Authors: Jiang You, Ren\'e Natowicz, Arben Cela, Jacob Ouanounou, Patrick
Siarry
- Abstract summary: Kernel-U-Net is a symmetric and hierarchical U-shape neural network architecture.
Our method offers two primary advantages: 1) Flexibility in kernel customization to adapt to specific datasets; 2) Enhanced computational efficiency, with the complexity of the Transformer layer reduced to linear.
The source code for Kernel-U-Net will be made publicly available for further research and application.
- Score: 2.0186752447895993
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Time series forecasting task predicts future trends based on historical
information. Transformer-based U-Net architectures, despite their success in
medical image segmentation, have limitations in both expressiveness and
computation efficiency in time series forecasting as evidenced in YFormer. To
tackle these challenges, we introduce Kernel-U-Net, a symmetric and
hierarchical U-shape neural network architecture. The kernel-U-Net encoder
compresses gradually input series into latent vectors, and its symmetric
decoder subsequently expands these vectors into output series. Specifically,
Kernel-U-Net separates the procedure of partitioning input time series into
patches from kernel manipulation, thereby providing the convenience of
executing customized kernels. Our method offers two primary advantages: 1)
Flexibility in kernel customization to adapt to specific datasets; 2) Enhanced
computational efficiency, with the complexity of the Transformer layer reduced
to linear. Experiments on seven real-world datasets, considering both
multivariate and univariate settings, demonstrate that Kernel-U-Net's
performance either exceeds or meets that of the existing state-of-the-art model
PatchTST in the majority of cases and outperforms Yformer. The source code for
Kernel-U-Net will be made publicly available for further research and
application.
Related papers
- TCCT-Net: Two-Stream Network Architecture for Fast and Efficient Engagement Estimation via Behavioral Feature Signals [58.865901821451295]
We present a novel two-stream feature fusion "Tensor-Convolution and Convolution-Transformer Network" (TCCT-Net) architecture.
To better learn the meaningful patterns in the temporal-spatial domain, we design a "CT" stream that integrates a hybrid convolutional-transformer.
In parallel, to efficiently extract rich patterns from the temporal-frequency domain, we introduce a "TC" stream that uses Continuous Wavelet Transform (CWT) to represent information in a 2D tensor form.
arXiv Detail & Related papers (2024-04-15T06:01:48Z) - ConvTimeNet: A Deep Hierarchical Fully Convolutional Model for
Multivariate Time Series Analysis [8.560776357590088]
ConvTimeNet is a novel deep hierarchical fully convolutional network designed to serve as a general-purpose model for time series analysis.
The results consistently outperformed strong baselines in most situations in terms of effectiveness.
arXiv Detail & Related papers (2024-03-03T12:05:49Z) - WinNet: Make Only One Convolutional Layer Effective for Time Series Forecasting [11.232780368635416]
We present a highly accurate and simply structured CNN-based model with only one convolutional layer, called WinNet.
Results demonstrate that WinNet can achieve SOTA performance and lower complexity over CNN-based methods.
arXiv Detail & Related papers (2023-11-01T01:23:59Z) - Tensor Slicing and Optimization for Multicore NPUs [2.670309629218727]
This paper proposes a compiler optimization pass for Multicore NPUs, called Slicing Optimization (TSO)
TSO identifies the best tensor slicing that minimizes execution time for a set of CNN models.
Results show that TSO is capable of identifying the best tensor slicing that minimizes execution time for a set of CNN models.
arXiv Detail & Related papers (2023-04-06T12:03:03Z) - An Adaptive Device-Edge Co-Inference Framework Based on Soft
Actor-Critic [72.35307086274912]
High-dimension parameter model and large-scale mathematical calculation restrict execution efficiency, especially for Internet of Things (IoT) devices.
We propose a new Deep Reinforcement Learning (DRL)-Soft Actor Critic for discrete (SAC-d), which generates the emphexit point, emphexit point, and emphcompressing bits by soft policy iterations.
Based on the latency and accuracy aware reward design, such an computation can well adapt to the complex environment like dynamic wireless channel and arbitrary processing, and is capable of supporting the 5G URL
arXiv Detail & Related papers (2022-01-09T09:31:50Z) - Kernel Identification Through Transformers [54.3795894579111]
Kernel selection plays a central role in determining the performance of Gaussian Process (GP) models.
This work addresses the challenge of constructing custom kernel functions for high-dimensional GP regression models.
We introduce a novel approach named KITT: Kernel Identification Through Transformers.
arXiv Detail & Related papers (2021-06-15T14:32:38Z) - Random Features for the Neural Tangent Kernel [57.132634274795066]
We propose an efficient feature map construction of the Neural Tangent Kernel (NTK) of fully-connected ReLU network.
We show that dimension of the resulting features is much smaller than other baseline feature map constructions to achieve comparable error bounds both in theory and practice.
arXiv Detail & Related papers (2021-04-03T09:08:12Z) - Einsum Networks: Fast and Scalable Learning of Tractable Probabilistic
Circuits [99.59941892183454]
We propose Einsum Networks (EiNets), a novel implementation design for PCs.
At their core, EiNets combine a large number of arithmetic operations in a single monolithic einsum-operation.
We show that the implementation of Expectation-Maximization (EM) can be simplified for PCs, by leveraging automatic differentiation.
arXiv Detail & Related papers (2020-04-13T23:09:15Z) - PolyScientist: Automatic Loop Transformations Combined with Microkernels
for Optimization of Deep Learning Primitives [55.79741270235602]
We develop a hybrid solution to the development of deep learning kernels.
We use the advanced polyhedral technology to automatically tune the outer loops for performance.
arXiv Detail & Related papers (2020-02-06T08:02:34Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.