Lightweight PCGAE-Net: Parallel CrossGate Attention and Bottleneck AutoEncoder for Efficient 5G Channel Prediction

• URL: http://arxiv.org/abs/2606.25401v1
• Date: Wed, 24 Jun 2026 04:55:25 GMT
• Title: Lightweight PCGAE-Net: Parallel CrossGate Attention and Bottleneck AutoEncoder for Efficient 5G Channel Prediction
• Abstract summary: This paper introduces Lightweight PCGAE-Net, which addresses the efficiency problem not by post-hoc compression but by correcting two architectural flaws in the current state of the art.<n>It produces a model with just 8.54,M parameters -- 58% fewer than the CS3T-UNet baseline.<n>It outperforms it by up to 3.26,dB at 5,km/h and 6.0,dB at 9,km/h in single-step prediction on QuaDriGa dataset.
• Score: 1.1083289076967893
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accurate channel state information (CSI) prediction is essential for proactive beamforming and resource management in 5G massive MIMO systems, yet the deployment of high-accuracy transformer-based predictors on base-station hardware remains challenging because the most capable models carry upwards of 30\,M parameters. This paper introduces Lightweight PCGAE-Net, which addresses the efficiency problem not by post-hoc compression but by correcting two architectural flaws in the current state of the art. The first is a sequential attention ordering bias: in CS3T-UNet, group-wise temporal attention (GTA) always operates on features that have already been transformed by cross-shaped spatial attention (CSA), distorting what temporal information GTA can capture. We remove this dependency by routing both attention modules to the same layer-normalized input and combining their independent outputs through a learned per-channel sigmoid CrossGate. The second flaw is an uncompressed bottleneck: applying full self-attention at the deepest encoder stage, where channel depth reaches $4C$, is quadratically expensive and carries redundant features. A Bottleneck AutoEncoder (BAE) with $1\times1$ convolutions halves this depth and uses an auxiliary reconstruction loss to prevent information collapse. Wrapping these components inside a shallower encoder-decoder with frequency-domain dimensionality reduction ($N_f\!=\!32$, $C\!=\!48$) produces a model with just 8.54\,M parameters -- 58\% fewer than the CS3T-UNet baseline -- that outperforms it by up to 3.26\,dB at 5\,km/h and 6.0\,dB at 9\,km/h in single-step prediction on QuaDriGa dataset.

Related papers

• KV Cache Compression Through the Lens of Transform Coding [9.884224071643606]
  We introduce Attention-Aware Transform Coding (AATC), which allocates bits over a calibration set to minimize attention-aware distortion.<n>Our method achieves near-lossless accuracy at approximately $5.8times$ compression, whereas each baseline degrades in at least some settings.
  arXiv  Detail & Related papers  (2026-08-14T11:08:01Z)
• Kaleido: Algorithm-Hardware Co-Design for Video Diffusion Transformers by Exploiting Latent Space Correlations [47.69636045010864]
  Video diffusion transformers (vDiTs) generate high quality video but introduce extremely high compute cost.<n>Existing acceleration approaches largely inherit attention techniques from sparse models which fail to consider unique video data.<n>This paper presents Kale Kale, an algorithm hardware codesign that accelerates all operations in vDiTs by exploiting channel-wisetemporal correlations in latent space.<n> Evaluations across three mainstream vDiT models show that Kale Kale achieves 5.9x speedup and 16.0x energy savings over state of the art accelerators.
  arXiv  Detail & Related papers  (2026-07-15T12:31:19Z)
• HEPTv2: End-to-End Efficient Point Transformer for Charged Particle Reconstruction [9.951462226612865]
  HEPTv2 is an end-to-end point-transformer architecture that reconstructs tracks from detector hits in one trainable pipeline.<n>On TrackML, HEPTv2 achieves 98.6% double-majority tracking efficiency at a 0.8% fake rate.<n>These results show end-to-end transformers can deliver the accuracy and efficiency required for real-time particle reconstruction at the HL-LHC.
  arXiv  Detail & Related papers  (2026-06-18T16:18:10Z)
• DYNA-PRUNER: Input-Adaptive Data-Model Co-Pruning for Efficient and Scalable Spatio-Temporal Media Prediction [24.16400471463046]
  Dyna-Pruner is an end-to-end framework for input-dependent co-pruning of data and model structure.<n>A shared-importance synchronization mechanism generates coupled masks that prune redundant regions.<n>Experiments on WeatherBench, SEVIR, and TaxiBJ show seamless integration with CNN, RNN, and Transformer backbones.
  arXiv  Detail & Related papers  (2026-06-13T15:14:05Z)
• Scaling Parallel Sequence Models to Foundation-Scale Vision Encoders [123.58723804218151]
  Vision foundation models are bottlenecked by the quadratic cost of self-attention.<n>We present C-GSPN based on 2D spatial propagation propagation.
  arXiv  Detail & Related papers  (2026-05-30T14:29:43Z)
• Linearizing Vision Transformer with Test-Time Training [54.52616866374482]
  Inheriting weights from pretrained Transformers provides an appealing shortcut, yet the fundamental representational gap between Softmax and linear attention prevents effective weight transfer.<n>We identify Test-Time Training (TTT) as a linear-complexity architecture whose two-layer dynamic formulation is structurally aligned with Softmax attention.<n>We validate our approach by linearizing Stable Diffusion 3.5 and introduce SD3.5-T$5$ (Transformer To Test Time Training)
  arXiv  Detail & Related papers  (2026-05-04T16:16:26Z)
• BWTA: Accurate and Efficient Binarized Transformer by Algorithm-Hardware Co-design [71.97035034203275]
  We analyze zero-point distortion in binarization and propose a Binary Weights & Ternary Activations (BWTA) quantization scheme.<n>For training, we propose Smooth Multi-Stage Quantization, combining a Levelwise Degradation Strategy and a MagnitudeAlignment Projection Factor.<n>Experiments show that BWTA approaches full-precision performance for Transformer-based models, with an average 3.5% drop on GLUE and less than 2% drop on five tasks.
  arXiv  Detail & Related papers  (2026-04-05T04:25:07Z)
• Scaling Attention via Feature Sparsity [50.64995497733461]
  Scaling Transformers to ultra-long contexts is bottlenecked by the $O(n2 d)$ cost of self-attention.<n>We propose Sparse Feature Attention, where queries and keys are represented as $k$-sparse codes that preserve high-dimensional expressivity.<n>Across GPT-2 and Qwen3 pretraining, SFA matches dense baselines while improving speed by up to $2.5times$ and reducing FLOPs and KV-cache by nearly 50%.
  arXiv  Detail & Related papers  (2026-03-17T08:41:50Z)
• Quantized Visual Geometry Grounded Transformer [67.15451442018258]
  This paper proposes the first Quantization framework for VGGTs, namely QuantVGGT.<n>We introduce Dual-Smoothed Fine-Grained Quantization, which integrates pre-global Hadamard rotation and post-local channel smoothing.<n>We also design Noise-Filtered Diverse Sampling, which filters outliers via deep-layer statistics.
  arXiv  Detail & Related papers  (2025-09-25T15:17:11Z)
• Head-Free Lightweight Semantic Segmentation with Linear Transformer [21.38163906180886]
  We propose a head-free lightweight architecture specifically for semantic segmentation, named Adaptive Frequency Transformer. It adopts a parallel architecture to leverage prototype representations as specific learnable local descriptions which replaces the decoder. Although removing the decoder compresses most of the computation, the accuracy of the parallel structure is still limited by low computational resources.
  arXiv  Detail & Related papers  (2023-01-11T18:59:46Z)
• TCCT: Tightly-Coupled Convolutional Transformer on Time Series Forecasting [6.393659160890665]
  We propose the concept of tightly-coupled convolutional Transformer(TCCT) and three TCCT architectures. Our experiments on real-world datasets show that our TCCT architectures could greatly improve the performance of existing state-of-art Transformer models.
  arXiv  Detail & Related papers  (2021-08-29T08:49:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.