Multi-Scale Convolution with Optimal Transport Attention Effect on Multivariate Time Series

• URL: http://arxiv.org/abs/2607.10740v1
• Date: Sun, 12 Jul 2026 12:47:50 GMT
• Title: Multi-Scale Convolution with Optimal Transport Attention Effect on Multivariate Time Series
• Abstract summary: Multi-Scale Convolution with Optimal Transport Attention (MSC-OT) is proposed in this paper.<n>MSC-OT combines multi-scale convolution with Sinkhorn optimal transport method based on inverted embedding.<n>Experiments on widely-used datasets, including ETT, Electricity, Traffic, Solar-Energy, and Exchange-Rate, show that MSC-OT achieves well performance in both short-term and long-term forecasting tasks.
• Score: 2.161642470654705
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The analysis of Multivariate Time Series (MTS) plays an important role in a lot of real-world practical applications, but it still remains some challenging problem about capturing multi-granularity structural patterns and suppressing noise appropriately. Multi-Scale Convolution with Optimal Transport Attention (MSC-OT) is proposed in this paper. MSC-OT is a useful architecture to optimize the attention mechanism. It combines multi-scale convolution with Sinkhorn optimal transport method based on inverted embedding. The inverted embedding approach embeds each variable as a token and allows the model to capture cross-variate relationships better. MSC-OT consists of two part: (1) Multi-Scale Convolution Enhancement, that applies multi-scale convolutions to attention score matrices based on inverted embedding, capturing local structural patterns in the variate-interaction space induced by compressed temporal representations; (2) Sinkhorn Optimal Transport Regularization, that formulates attention computation as an optimal transport problem and employs iterative matrix scaling to ensure balanced information flow across variates. Adaptive Fusion Strategy utilizes softmax-normalized learnable weights to dynamically combine base attention, convolution-enhanced, and OT-regularized scores. Experiments on widely-used datasets, including ETT, Electricity, Traffic, Solar-Energy, and Exchange-Rate, show that MSC-OT achieves well performance in both short-term and long-term forecasting tasks. Ablation experiments further validate the effectiveness of each proposed component and their synergistic contributions to improving prediction accuracy for multivariate time series forecasting.

Related papers

• FreSH: Frequency-Segmented Hierarchical Multi-Expert Framework for Multivariate Time Series Classification [15.396551583872009]
  We present FreSH, a Frequency-Segmented Hierarchical Multi-Expert Framework designed to address these challenges.<n>FreSH introduces a new perspective for MTSC by enabling adaptive, multi-scale analysis of temporal signals.<n>It consistently outperforms state-of-the-art methods in classification accuracy, while substantially reducing model size and efficiency.
  arXiv  Detail & Related papers  (2026-08-08T16:05:17Z)
• Multiscale Supervised Unbalanced Optimal Transport Flow Matching [2.421459418045937]
  Unbalanced optimal transport (UOT) provides a principled framework for modeling single-cell transitions and birth-death dynamics.<n>We introduce Multiscale Supervised Unbalanced Optimal Transport Flow Matching (MUST-FM), a simulation-free framework that scales UOT by leveraging hierarchical data structure.
  arXiv  Detail & Related papers  (2026-05-15T18:26:50Z)
• Multimodal Forecasting for Commodity Prices Using Spectrogram-Based and Time Series Representations [6.409220743870009]
  This paper presents Spectrogram-Enhanced Multimodal Fusion (SEMF)<n>SEMF combines spectral and temporal representations for more accurate and robust forecasting.<n>Results demonstrate the effectiveness of multimodal fusion and spectrogram-based encoding in capturing multi-scale patterns.
  arXiv  Detail & Related papers  (2026-03-28T15:57:42Z)
• AlignMamba-2: Enhancing Multimodal Fusion and Sentiment Analysis with Modality-Aware Mamba [56.52470564147458]
  We propose textbfAlignMamba-2, an effective and efficient framework for multimodal fusion and sentiment analysis.<n>Our approach introduces a dual alignment strategy that regularizes the model using both Optimal Transport distance and Maximum Mean Discrepancy.<n>More importantly, we design a Modality-Aware Mamba layer, which employs a Mixture-of-Experts architecture with modality-specific and modality-shared experts.
  arXiv  Detail & Related papers  (2026-03-19T03:47:21Z)
• FAIM: Frequency-Aware Interactive Mamba for Time Series Classification [87.84511960413715]
  Time series classification (TSC) is crucial in numerous real-world applications, such as environmental monitoring, medical diagnosis, and posture recognition.<n>We propose FAIM, a lightweight Frequency-Aware Interactive Mamba model.<n>We show that FAIM consistently outperforms existing state-of-the-art (SOTA) methods, achieving a superior trade-off between accuracy and efficiency.
  arXiv  Detail & Related papers  (2025-11-26T08:36:33Z)
• ESPFormer: Doubly-Stochastic Attention with Expected Sliced Transport Plans [13.695885742446027]
  Self-attention can lead to over-concentration on a few tokens during training, resulting in suboptimal information flow.<n>We introduce a novel, fully parallelizable doubly-stochastic attention mechanism based on sliced optimal transport.<n>Our method enforces doublyity without iterative Sinkhorn normalization, significantly enhancing efficiency.
  arXiv  Detail & Related papers  (2025-02-11T21:20:48Z)
• Sparser is Faster and Less is More: Efficient Sparse Attention for Long-Range Transformers [58.5711048151424]
  We introduce SPARSEK Attention, a novel sparse attention mechanism designed to overcome computational and memory obstacles. Our approach integrates a scoring network and a differentiable top-k mask operator, SPARSEK, to select a constant number of KV pairs for each query. Experimental results reveal that SPARSEK Attention outperforms previous sparse attention methods.
  arXiv  Detail & Related papers  (2024-06-24T15:55:59Z)
• MGCP: A Multi-Grained Correlation based Prediction Network for Multivariate Time Series [54.91026286579748]
  We propose a Multi-Grained Correlations-based Prediction Network. It simultaneously considers correlations at three levels to enhance prediction performance. It employs adversarial training with an attention mechanism-based predictor and conditional discriminator to optimize prediction results at coarse-grained level.
  arXiv  Detail & Related papers  (2024-05-30T03:32:44Z)
• Correlated Attention in Transformers for Multivariate Time Series [22.542109523780333]
  We propose a novel correlated attention mechanism, which efficiently captures feature-wise dependencies, and can be seamlessly integrated within the encoder blocks of existing Transformers. In particular, correlated attention operates across feature channels to compute cross-covariance matrices between queries and keys with different lag values, and selectively aggregate representations at the sub-series level. This architecture facilitates automated discovery and representation learning of not only instantaneous but also lagged cross-correlations, while inherently capturing time series auto-correlation.
  arXiv  Detail & Related papers  (2023-11-20T17:35:44Z)
• IPCC-TP: Utilizing Incremental Pearson Correlation Coefficient for Joint Multi-Agent Trajectory Prediction [73.25645602768158]
  IPCC-TP is a novel relevance-aware module based on Incremental Pearson Correlation Coefficient to improve multi-agent interaction modeling. Our module can be conveniently embedded into existing multi-agent prediction methods to extend original motion distribution decoders.
  arXiv  Detail & Related papers  (2023-03-01T15:16:56Z)
• Enhancing Transformer Efficiency for Multivariate Time Series Classification [12.128991867050487]
  We propose a methodology to investigate the relationship between model efficiency and accuracy, as well as its complexity. Comprehensive experiments on benchmark MTS datasets illustrate the effectiveness of our method.
  arXiv  Detail & Related papers  (2022-03-28T03:25:19Z)
• Proximal Policy Optimization-based Transmit Beamforming and Phase-shift Design in an IRS-aided ISAC System for the THz Band [90.45915557253385]
  IRS-aided integrated sensing and communications (ISAC) system operating in the terahertz (THz) band is proposed to maximize the system capacity. Transmit beamforming and phase-shift design are transformed into a universal optimization problem with ergodic constraints.
  arXiv  Detail & Related papers  (2022-03-21T09:15:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.