PReD: An LLM-based Foundation Multimodal Model for Electromagnetic Perception, Recognition, and Decision

• URL: http://arxiv.org/abs/2603.28183v2
• Date: Wed, 01 Apr 2026 02:48:58 GMT
• Title: PReD: An LLM-based Foundation Multimodal Model for Electromagnetic Perception, Recognition, and Decision
• Abstract summary: PReD is the first foundation model for the electromagnetic (EM) domain that covers the intelligent closed-loop of "perception, recognition, decision-making"<n>It supports a range of core tasks, including signal detection, modulation recognition, parameter estimation, protocol recognition, radio frequency fingerprint recognition, and anti-jamming decision-making.<n>It achieves closed-loop optimization from end-to-end signal understanding to language-driven reasoning and decision-making.
• Score: 13.529896365435286
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal Large Language Models have demonstrated powerful cross-modal understanding and reasoning capabilities in general domains. However, in the electromagnetic (EM) domain, they still face challenges such as data scarcity and insufficient integration of domain knowledge. This paper proposes PReD, the first foundation model for the EM domain that covers the intelligent closed-loop of "perception, recognition, decision-making." We constructed a high-quality multitask EM dataset, PReD-1.3M, and an evaluation benchmark, PReD-Bench. The dataset encompasses multi-perspective representations such as raw time-domain waveform, frequency-domain spectrograms, and constellation diagrams, covering typical features of communication and radar signals. It supports a range of core tasks, including signal detection, modulation recognition, parameter estimation, protocol recognition, radio frequency fingerprint recognition, and anti-jamming decision-making. PReD adopts a multi-stage training strategy that unifies multiple tasks for EM signals. It achieves closed-loop optimization from end-to-end signal understanding to language-driven reasoning and decision-making, significantly enhancing EM domain expertise while maintaining general multimodal capabilities. Experimental results show that PReD achieves state-of-the-art performance on PReD-Bench constructed from both open-source and self-collected signal datasets. These results collectively validate the feasibility and potential of vision-aligned foundation models in advancing the understanding and reasoning of EM signals.

Related papers

• DKDNet: Dual Knowledge and Data-Driven Network for Cross-Domain Automatic Modulation Classification [45.220843998540545]
  We consider signal prior knowledge, grounded in communication protocols and physical principles, as a potential way to enhance cross-domain representation learning.<n>This paper first analyzes five commonly adopted signal representations that instantiate different signal priors.<n>Based on that, a dual knowledge and data-driven network (DKDNet) is proposed for cross-domain AMC.
  arXiv  Detail & Related papers  (2026-07-09T01:14:42Z)
• WaveMind: Towards a Conversational EEG Foundation Model Aligned to Textual and Visual Modalities [55.00677513249723]
  EEG signals simultaneously encode both cognitive processes and intrinsic neural states.<n>We map EEG signals and their corresponding modalities into a unified semantic space to achieve generalized interpretation.<n>The resulting model demonstrates robust classification accuracy while supporting flexible, open-ended conversations.
  arXiv  Detail & Related papers  (2025-09-26T06:21:51Z)
• RadioDiff-Loc: Diffusion Model Enhanced Scattering Congnition for NLoS Localization with Sparse Radio Map Estimation [24.71314779247058]
  This paper proposes a novel generative inference framework for NLoS localization based on conditional diffusion models.<n>By leveraging the physical insight that diffracted electromagnetic energy concentrates near building edges, we develop a sampling strategy.<n>We normalize all sampled RSS values relative to the maximum observed intensity, enabling the construction of a power-invariant radio map.
  arXiv  Detail & Related papers  (2025-09-02T01:43:23Z)
• EMind: A Foundation Model for Multi-task Electromagnetic Signals Understanding [13.118523730875383]
  EMind is an electromagnetic signals foundation model that bridges large scale pretraining and the unique nature of this modality.<n>We build the first unified and largest standardized electromagnetic signal dataset covering multiple signal types and tasks.<n>EMind achieves strong performance and broad generalization across many downstream tasks, moving decisively from task specific models to a unified framework for electromagnetic intelligence.
  arXiv  Detail & Related papers  (2025-08-26T08:11:57Z)
• RadioDiff-$k^2$: Helmholtz Equation Informed Generative Diffusion Model for Multi-Path Aware Radio Map Construction [76.24833675757033]
  We propose a physics-informed generative learning approach, named RadioDiff-$k2$, for accurate and efficient multipath-aware radio map (RM) construction.<n>We show that the proposed RadioDiff-$k2$ framework achieves state-of-the-art (SOTA) performance in both image-level RM construction and localization tasks.
  arXiv  Detail & Related papers  (2025-04-22T06:28:13Z)
• Resource-Efficient Beam Prediction in mmWave Communications with Multimodal Realistic Simulation Framework [57.994965436344195]
  Beamforming is a key technology in millimeter-wave (mmWave) communications that improves signal transmission by optimizing directionality and intensity.<n> multimodal sensing-aided beam prediction has gained significant attention, using various sensing data to predict user locations or network conditions.<n>Despite its promising potential, the adoption of multimodal sensing-aided beam prediction is hindered by high computational complexity, high costs, and limited datasets.
  arXiv  Detail & Related papers  (2025-04-07T15:38:25Z)
• MultiSensor-Home: A Wide-area Multi-modal Multi-view Dataset for Action Recognition and Transformer-based Sensor Fusion [2.7745600113170994]
  We introduce the MultiSensor-Home dataset, a novel benchmark for comprehensive action recognition in home environments.<n>We also propose the Multi-modal Multi-view Transformer-based Sensor Fusion (MultiTSF) method.
  arXiv  Detail & Related papers  (2025-04-03T05:23:08Z)
• MMA-DFER: MultiModal Adaptation of unimodal models for Dynamic Facial Expression Recognition in-the-wild [81.32127423981426]
  Multimodal emotion recognition based on audio and video data is important for real-world applications. Recent methods have focused on exploiting advances of self-supervised learning (SSL) for pre-training of strong multimodal encoders. We propose a different perspective on the problem and investigate the advancement of multimodal DFER performance by adapting SSL-pre-trained disjoint unimodal encoders.
  arXiv  Detail & Related papers  (2024-04-13T13:39:26Z)
• Multi-task Learning for Radar Signal Characterisation [48.265859815346985]
  This paper presents an approach for tackling radar signal classification and characterisation as a multi-task learning (MTL) problem. We propose the IQ Signal Transformer (IQST) among several reference architectures that allow for simultaneous optimisation of multiple regression and classification tasks. We demonstrate the performance of our proposed MTL model on a synthetic radar dataset, while also providing a first-of-its-kind benchmark for radar signal characterisation.
  arXiv  Detail & Related papers  (2023-06-19T12:01:28Z)
• On Neural Architectures for Deep Learning-based Source Separation of Co-Channel OFDM Signals [104.11663769306566]
  We study the single-channel source separation problem involving frequency-division multiplexing (OFDM) signals. We propose critical domain-informed modifications to the network parameterization, based on insights from OFDM structures.
  arXiv  Detail & Related papers  (2023-03-11T16:29:13Z)
• Multi-task Learning Approach for Modulation and Wireless Signal Classification for 5G and Beyond: Edge Deployment via Model Compression [1.218340575383456]
  Future communication networks must address the scarce spectrum to accommodate growth of heterogeneous wireless devices. We exploit the potential of deep neural networks based multi-task learning framework to simultaneously learn modulation and signal classification tasks. We provide a comprehensive heterogeneous wireless signals dataset for public use.
  arXiv  Detail & Related papers  (2022-02-26T14:51:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.