WALoMA: A Multitask Wireless Foundation Model via Adaptive Low-Rank Masked Autoencoders

• URL: http://arxiv.org/abs/2607.25763v1
• Date: Tue, 28 Jul 2026 14:18:22 GMT
• Title: WALoMA: A Multitask Wireless Foundation Model via Adaptive Low-Rank Masked Autoencoders
• Abstract summary: This paper proposes a multitask wireless foundation model via adaptive low-rank masked autoencoders (WALoMA)<n>It addresses the limitations of specialized, task-specific deep learning models and the practical challenge of scarce labeled wireless datasets.
• Score: 23.225139930889526
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper proposes a multitask wireless foundation model via adaptive low-rank masked autoencoders (WALoMA), a unified multi-task foundation model for sixth-generation (6G) wireless physical layer architectures, to address the limitations of specialized, task-specific deep learning models and the practical challenge of scarce labeled wireless datasets. By leveraging concepts inspired by foundation models, the proposed framework adopts a masked autoencoder (MAE) paradigm to learn from unlabeled channel data, to significantly reduce reliance on extensive annotations. The model treats wireless channel state information (CSI) as a universal modality and learns transferable representations through self-supervised channel reconstruction. Key architectural novelties include the use of 2D positional encoding (PE) to explicitly preserve the spatial-frequency relationships between antennas and subcarriers, and low-rank adaptation (LoRA) for parameter-efficient fine-tuning. The framework's efficacy is demonstrated across five downstream tasks, achieving individual scores of 96.47\% for LoS/NLoS classification, 80.45\% for beam prediction, 85.78\% for channel interpolation, 99.12\% for channel estimation, and 77.18\% for channel charting. Consequently, numerical results show that the proposed model achieves a composite score of 87.80\%, significantly outperforming the 59.90\% achieved by the large wireless model (LWM) baseline while training an average of only 14.68\% of total parameters, and maintaining strong performance even under extremely limited labeled data conditions.

Related papers

• Modality-Decoupled Federated Learning for Privacy-Preserving Embodied Intelligence in 6G [90.42669277637026]
  Sixth-generation (6G) wireless networks are expected to provide a key infrastructure for large-scale embodied intelligence.<n>This article proposes FedMVLA, a modality-decoupled FL framework for privacy-preserving embodied intelligence in 6G networks.
  arXiv  Detail & Related papers  (2026-09-09T01:36:44Z)
• A VAE-Driven Multi-Task Satellite-Aided Semantic Communication Framework for 6G-Enabled Connected Autonomous Vehicles [51.449502491030756]
  We present a multi-task semantic communication framework for satellite-assisted autonomous driving.<n>The framework is trained end-to-end to jointly optimize both tasks.<n>Results show that the proposed approach achieves significant bandwidth reduction of up to 87.23% to 98.17%.
  arXiv  Detail & Related papers  (2026-07-15T06:49:41Z)
• Constrained Decoding for Diffusion Language Models via Efficient Inference over Finite Automata [57.27430779838529]
  We present an exact and tractable algorithm for sampling from the constrained mean-field posterior under any constraint expressible as a finite automaton.<n>The approach guarantees constraint satisfaction by construction, supports both greedy and sampling-based decoding, and is compatible with parallel and block-wise decoding.<n> Empirical evaluations on Dream-7B and LLaDA-8 show substantial accuracy gains across various tasks.
  arXiv  Detail & Related papers  (2026-07-08T05:48:57Z)
• Towards CSI-Native Foundation Models: A Channel-Adaptive Roadmap for 6G [14.956130211673894]
  Wireless foundation models offer a path toward reusable channel state information (CSI) intelligence for sixth-generation (6G) systems.<n>Existing generic-backbone adaptation and CSI pretraining methods often treat CSI as task tensors rather than propagation-conditioned channel responses.<n>This paper proposes a unified framework that aligns pretraining, positional modeling, and attention control with three channel requirements.
  arXiv  Detail & Related papers  (2026-06-12T11:01:12Z)
• RF-LSCM: Pushing Radiance Fields to Multi-Domain Localized Statistical Channel Modeling for Cellular Network Optimization [26.457933013666253]
  We propose RF-LSCM, a novel framework that models the channel APS by jointly representing large-scale signal attenuation and multipath components within a radiance field.<n>RF-LSCM significantly outperforms state-of-the-art methods, achieving up to a 30% reduction in mean absolute error (MAE) for coverage prediction and a 22% MAE improvement by effectively fusing multi-frequency data.
  arXiv  Detail & Related papers  (2025-09-17T04:31:18Z)
• A Foundation Model for DAS Signal Recognition and Visual Prompt Tuning of the Pre-trained Model for Downstream Tasks [6.14430079610632]
  This study proposes a foundational model for DAS signal recognition based on a Masked Autocoder, named MAEPD.<n>The model is pretrained on a dataset of 635860 samples, encompassing DAS gait signals, 2temporal GASF images for perimeter security, 2D time-frequency images for pipeline leakage, and open-dataset signals including whale vocalizations and seismic activities.<n>The VPT-Deep approach achieves a classification accuracy of 96.94% with just 0.322% of parameters fine-tuned, surpassing the traditional Full Fine Tuning (FFT) method by 0.61% and reducing training time by
  arXiv  Detail & Related papers  (2025-08-06T11:02:25Z)
• A Lightweight Deep Learning Model for Automatic Modulation Classification using Dual Path Deep Residual Shrinkage Network [0.0]
  Automatic Modulation Classification (AMC) plays a key role in enhancing spectrum efficiency.<n>There is a pressing need for lightweight AMC models that balance low complexity with high classification accuracy.<n>This paper proposes a low-complexity, lightweight deep learning (DL) AMC model optimized for resource-constrained edge devices.
  arXiv  Detail & Related papers  (2025-07-07T00:37:54Z)
• IQFM A Wireless Foundational Model for I/Q Streams in AI-Native 6G [7.531126877550286]
  This paper presents IQFM, the first I/Q signal foundational model for wireless communications.<n> IQFM supports diverse tasks: modulation classification, angle-of-arrival (AoA), beam prediction, and RF fingerprinting.<n>We also introduce a task-aware augmentation strategy that categorizes transformations into core augmentations.
  arXiv  Detail & Related papers  (2025-06-07T09:01:38Z)
• QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models [44.515165695546614]
  Quantization-Aware Training (QAT) offers a solution, but its extensive training costs make Post-Training Quantization (PTQ) a more practical approach for Large Language Models (LLMs) We propose QLLM, an accurate and efficient low-bitwidth PTQ method designed for LLMs.
  arXiv  Detail & Related papers  (2023-10-12T05:25:49Z)
• Prompt Tuning for Parameter-efficient Medical Image Segmentation [79.09285179181225]
  We propose and investigate several contributions to achieve a parameter-efficient but effective adaptation for semantic segmentation on two medical imaging datasets. We pre-train this architecture with a dedicated dense self-supervision scheme based on assignments to online generated prototypes. We demonstrate that the resulting neural network model is able to attenuate the gap between fully fine-tuned and parameter-efficiently adapted models.
  arXiv  Detail & Related papers  (2022-11-16T21:55:05Z)
• Non-Parametric Adaptive Network Pruning [125.4414216272874]
  We introduce non-parametric modeling to simplify the algorithm design. Inspired by the face recognition community, we use a message passing algorithm to obtain an adaptive number of exemplars. EPruner breaks the dependency on the training data in determining the "important" filters.
  arXiv  Detail & Related papers  (2021-01-20T06:18:38Z)
• Harnessing Wireless Channels for Scalable and Privacy-Preserving Federated Learning [56.94644428312295]
  Wireless connectivity is instrumental in enabling federated learning (FL) Channel randomnessperturbs each worker inversions model update while multiple workers updates incur significant interference on bandwidth. In A-FADMM, all workers upload their model updates to the parameter server using a single channel via analog transmissions. This not only saves communication bandwidth, but also hides each worker's exact model update trajectory from any eavesdropper.
  arXiv  Detail & Related papers  (2020-07-03T16:31:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.