FAConvLSTM: Factorized-Attention ConvLSTM for Efficient Feature Extraction in Multivariate Climate Data

• URL: http://arxiv.org/abs/2601.10914v1
• Date: Fri, 16 Jan 2026 00:07:46 GMT
• Title: FAConvLSTM: Factorized-Attention ConvLSTM for Efficient Feature Extraction in Multivariate Climate Data
• Abstract summary: FAConvLSTM is a Factorized-Attention ConvLSTM layer designed as a drop-in replacement for ConvLSTM2D.<n>We show that FAConvLSTM yields more stable, interpretable, and robust latent representations, while significantly reducing computational overhead.
• Score: 0.193544284424478
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Learning physically meaningful spatiotemporal representations from high-resolution multivariate Earth observation data is challenging due to strong local dynamics, long-range teleconnections, multi-scale interactions, and nonstationarity. While ConvLSTM2D is a commonly used baseline, its dense convolutional gating incurs high computational cost and its strictly local receptive fields limit the modeling of long-range spatial structure and disentangled climate dynamics. To address these limitations, we propose FAConvLSTM, a Factorized-Attention ConvLSTM layer designed as a drop-in replacement for ConvLSTM2D that simultaneously improves efficiency, spatial expressiveness, and physical interpretability. FAConvLSTM factorizes recurrent gate computations using lightweight [1 times 1] bottlenecks and shared depthwise spatial mixing, substantially reducing channel complexity while preserving recurrent dynamics. Multi-scale dilated depthwise branches and squeeze-and-excitation recalibration enable efficient modeling of interacting physical processes across spatial scales, while peephole connections enhance temporal precision. To capture teleconnection-scale dependencies without incurring global attention cost, FAConvLSTM incorporates a lightweight axial spatial attention mechanism applied sparsely in time. A dedicated subspace head further produces compact per timestep embeddings refined through temporal self-attention with fixed seasonal positional encoding. Experiments on multivariate spatiotemporal climate data shows superiority demonstrating that FAConvLSTM yields more stable, interpretable, and robust latent representations than standard ConvLSTM, while significantly reducing computational overhead.

Related papers

• Learning Spatio-Temporal Foundation Models from Pure Synthetic Data [85.23980441738811]
  NeoST is a scalable synthetic-training corpus to real-world bias, a latent-space reasoning architecture that generates and iteratively refines multiple future trajectories, and latent-space objectives that emphasize structural dynamics and enable inference-time correction under distribution shifts.<n>Extensive experiments across diverse real-world benchmarks show that NeoST consistently outperforms existing STFMs in diverse real-worldtemporal systems.
  arXiv  Detail & Related papers  (2026-06-27T02:56:46Z)
• PIMSM: Physics-Informed Multi-Scale Mamba for Stable Neural Representations under Distribution Shift [16.230225739605316]
  We argue that failure to preserve scientific temporal structure contributes to brittleness under distribution shift.<n>We propose Physics-Informed Multi-Scale Mamba, a state-space architecture that maps spectrum-estimated transition points between frequency regimes.<n>On Human Connectome Project fMRI, PIMSM improves robustness and representation stability under severe temporal-context truncation.
  arXiv  Detail & Related papers  (2026-05-08T04:58:05Z)
• FoSS: Modeling Long Range Dependencies and Multimodal Uncertainty in Trajectory Prediction via Fourier State Space Integration [21.39395366378851]
  We present FoSS, a dual-branch framework that unifies frequency-domain reasoning with linear-time sequence modeling.<n>Experiments on Argoverse 1 and Argoverse 2 benchmarks demonstrate that FoSS achieves state-of-the-art accuracy while reducing computation by 22.5% and parameters by over 40%.
  arXiv  Detail & Related papers  (2026-03-01T21:38:59Z)
• Contextual and Seasonal LSTMs for Time Series Anomaly Detection [49.50689313712684]
  We propose a novel prediction-based framework named Contextual and Seasonal LSTMs (CS-LSTMs)<n>CS-LSTMs are built upon a noise decomposition strategy and jointly leverage contextual dependencies and seasonal patterns.<n>They consistently outperform state-of-the-art methods, highlighting their effectiveness and practical value in robust time series anomaly detection.
  arXiv  Detail & Related papers  (2026-02-10T11:46:15Z)
• Scalable Spatio-Temporal SE(3) Diffusion for Long-Horizon Protein Dynamics [51.85385061275941]
  Molecular dynamics (MD) simulations remain the gold standard for studying protein dynamics.<n>Recent generative models have shown promise in accelerating simulations, yet they struggle with long-horizon generation.<n>We present STAR-MD, a scalable diffusion model that generates physically plausible protein trajectories over micro-scale timescales.
  arXiv  Detail & Related papers  (2026-02-02T14:13:28Z)
• OptiVote: Non-Coherent FSO Over-the-Air Majority Vote for Communication-Efficient Distributed Federated Learning in Space Data Centers [68.73273027298625]
  megaconstellations are driving the long-term vision of space data centers (SDCs)<n>AirComp is an in-network aggregation framework for learning free-space (FSO)<n>AirVote integrates sign gradient (SGD) with a majority-signposition modulation (PPM), where each satellite conveys local gradient by activating PPM time slots.<n>OptiVote mitigates phase-sensitive field superposition into phase-agnostic optical intensity combining.
  arXiv  Detail & Related papers  (2025-12-30T16:40:02Z)
• SSCM: A Spatial-Semantic Consistent Model for Multi-Contrast MRI Super-Resolution [11.194678655584788]
  MC-MRI SR aims to enhance low-resolution (LR) contrasts leveraging high-resolution (HR) references.<n>Main challenge lies in maintaining spatial-semantic consistency.
  arXiv  Detail & Related papers  (2025-09-23T03:24:32Z)
• ST-LINK: Spatially-Aware Large Language Models for Spatio-Temporal Forecasting [7.853736939635847]
  We introduce ST-LINK, a novel framework that enhances the capability of Large Language Models to capture sequential-temporal dependencies.<n>Its key components are spatially-Enhanced Attention (SE-Attention) and the Memory Retrieval Feed-Forward Network (MRFFN)
  arXiv  Detail & Related papers  (2025-09-17T07:11:45Z)
• B-TGAT: A Bi-directional Temporal Graph Attention Transformer for Clustering Multivariate Spatiotemporal Data [0.15763762230817438]
  We present a time-distributed hybrid U-Net autocoder that integrates a Bi-multidimensional Graph Attention Transformer (B-TGAT) to guide efficient temporal clustering.<n>The encoder and decoder are equipped with ConvLSTM2D modules that extract joint spatial-temporal features.<n> experiments show superior cluster separability, temporal stability, and alignment with known climate transitions compared to state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-09-16T16:08:21Z)
• StoxLSTM: A Stochastic Extended Long Short-Term Memory Network for Time Series Forecasting [20.120876019697445]
  Extended Long Short-Term Memory (xLSTM) network has attracted widespread research interest due to its enhanced capability to model complex temporal dependencies in diverse time series applications.<n>We propose a xLSTM, termed StoxLSTM, that improves the original architecture into a state space modeling framework by incorporating latent variables within xLSTM.<n>Experiments on publicly available benchmark datasets from multiple research communities demonstrate that StoxLSTM consistently outperforms state-of-the-art baselines with better robustness and stronger generalization ability.
  arXiv  Detail & Related papers  (2025-09-01T07:11:05Z)
• Learning to Dissipate Energy in Oscillatory State-Space Models [51.98491034847041]
  State-space models (SSMs) are a class of networks for sequence learning.<n>We show that D-LinOSS consistently outperforms previous LinOSS methods on long-range learning tasks.
  arXiv  Detail & Related papers  (2025-05-17T23:15:17Z)
• Efficient High-Resolution Visual Representation Learning with State Space Model for Human Pose Estimation [60.80423207808076]
  Capturing long-range dependencies while preserving high-resolution visual representations is crucial for dense prediction tasks such as human pose estimation.<n>We propose the Dynamic Visual State Space (DVSS) block, which augments visual state space models with multi-scale convolutional operations.<n>We build HRVMamba, a novel model for efficient high-resolution representation learning.
  arXiv  Detail & Related papers  (2024-10-04T06:19:29Z)
• Cross-Scan Mamba with Masked Training for Robust Spectral Imaging [51.557804095896174]
  We propose the Cross-Scanning Mamba, named CS-Mamba, that employs a Spatial-Spectral SSM for global-local balanced context encoding.<n>Experiment results show that our CS-Mamba achieves state-of-the-art performance and the masked training method can better reconstruct smooth features to improve the visual quality.
  arXiv  Detail & Related papers  (2024-08-01T15:14:10Z)
• SpectralMamba: Efficient Mamba for Hyperspectral Image Classification [39.18999103115206]
  Recurrent neural networks and Transformers have dominated most applications in hyperspectral (HS) imaging. We propose SpectralMamba -- a novel state space model incorporated efficient deep learning framework for HS image classification. We show that SpectralMamba surprisingly creates promising win-wins from both performance and efficiency perspectives.
  arXiv  Detail & Related papers  (2024-04-12T14:12:03Z)
• Attractor Memory for Long-Term Time Series Forecasting: A Chaos Perspective [63.60312929416228]
  textbftextitAttraos incorporates chaos theory into long-term time series forecasting. We show that Attraos outperforms various LTSF methods on mainstream datasets and chaotic datasets with only one-twelfth of the parameters compared to PatchTST.
  arXiv  Detail & Related papers  (2024-02-18T05:35:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.