Knowledge Distillation for mmWave Beam Prediction Using Sub-6 GHz Channels

• URL: http://arxiv.org/abs/2602.04703v1
• Date: Wed, 04 Feb 2026 16:15:32 GMT
• Title: Knowledge Distillation for mmWave Beam Prediction Using Sub-6 GHz Channels
• Authors: Sina Tavakolian, Nhan Thanh Nguyen, Ahmed Alkhateeb, Markku Juntti,
• Abstract summary: We propose a framework for sub-6 GHz channel-mmWave beam mapping based on the knowledge distillation (KD) technique.<n>We show that the proposed student models achieve the teacher's beam prediction accuracy and spectral efficiency while reducing trainable parameters and computational complexity by 99%.
• Score: 18.712418156283437
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Beamforming in millimeter-wave (mmWave) high-mobility environments typically incurs substantial training overhead. While prior studies suggest that sub-6 GHz channels can be exploited to predict optimal mmWave beams, existing methods depend on large deep learning (DL) models with prohibitive computational and memory requirements. In this paper, we propose a computationally efficient framework for sub-6 GHz channel-mmWave beam mapping based on the knowledge distillation (KD) technique. We develop two compact student DL architectures based on individual and relational distillation strategies, which retain only a few hidden layers yet closely mimic the performance of large teacher DL models. Extensive simulations demonstrate that the proposed student models achieve the teacher's beam prediction accuracy and spectral efficiency while reducing trainable parameters and computational complexity by 99%.

Related papers

• Every Attention Matters: An Efficient Hybrid Architecture for Long-Context Reasoning [73.10669391954801]
  We present the Ring-linear model series, specifically including Ring-mini-linear-2.0 and Ring-flash-linear-2.0.<n>Both models adopt a hybrid architecture that effectively integrates linear attention and softmax attention.<n>Compared to a 32 billion parameter dense model, this series reduces inference cost to 1/10, and compared to the original Ring series, the cost is also reduced by over 50%.
  arXiv  Detail & Related papers  (2025-10-22T07:59:38Z)
• A Lightweight Deep Learning Model for Automatic Modulation Classification using Dual Path Deep Residual Shrinkage Network [0.0]
  Automatic Modulation Classification (AMC) plays a key role in enhancing spectrum efficiency.<n>There is a pressing need for lightweight AMC models that balance low complexity with high classification accuracy.<n>This paper proposes a low-complexity, lightweight deep learning (DL) AMC model optimized for resource-constrained edge devices.
  arXiv  Detail & Related papers  (2025-07-07T00:37:54Z)
• EfficientLLM: Efficiency in Large Language Models [64.3537131208038]
  Large Language Models (LLMs) have driven significant progress, yet their growing counts and context windows incur prohibitive compute, energy, and monetary costs.<n>We introduce EfficientLLM, a novel benchmark and the first comprehensive empirical study evaluating efficiency techniques for LLMs at scale.
  arXiv  Detail & Related papers  (2025-05-20T02:27:08Z)
• Resource-Efficient Beam Prediction in mmWave Communications with Multimodal Realistic Simulation Framework [57.994965436344195]
  Beamforming is a key technology in millimeter-wave (mmWave) communications that improves signal transmission by optimizing directionality and intensity.<n> multimodal sensing-aided beam prediction has gained significant attention, using various sensing data to predict user locations or network conditions.<n>Despite its promising potential, the adoption of multimodal sensing-aided beam prediction is hindered by high computational complexity, high costs, and limited datasets.
  arXiv  Detail & Related papers  (2025-04-07T15:38:25Z)
• Efficient Gravitational Wave Parameter Estimation via Knowledge Distillation: A ResNet1D-IAF Approach [2.4184866684341473]
  This study presents a novel approach using knowledge distillation techniques to enhance computational efficiency in gravitational wave analysis.<n>We develop a framework combining ResNet1D and Inverse Autoregressive Flow (IAF) architectures, where knowledge from a complex teacher model is transferred to a lighter student model.<n>Our experimental results show that the student model achieves a validation loss of 3.70 with optimal configuration (40,100,0.75), compared to the teacher model's 4.09, while reducing the number of parameters by 43%.
  arXiv  Detail & Related papers  (2024-12-11T03:56:46Z)
• BOOT: Data-free Distillation of Denoising Diffusion Models with Bootstrapping [64.54271680071373]
  Diffusion models have demonstrated excellent potential for generating diverse images. Knowledge distillation has been recently proposed as a remedy that can reduce the number of inference steps to one or a few. We present a novel technique called BOOT, that overcomes limitations with an efficient data-free distillation algorithm.
  arXiv  Detail & Related papers  (2023-06-08T20:30:55Z)
• Deep Learning and Image Super-Resolution-Guided Beam and Power Allocation for mmWave Networks [80.37827344656048]
  We develop a deep learning (DL)-guided hybrid beam and power allocation approach for millimeter-wave (mmWave) networks. We exploit the synergy of supervised learning and super-resolution technology to enable low-overhead beam- and power allocation.
  arXiv  Detail & Related papers  (2023-05-08T05:40:54Z)
• Deep Learning Framework for the Design of Orbital Angular Momentum Generators Enabled by Leaky-wave Holograms [0.6999740786886535]
  We present a novel approach for the design of leaky-wave holographic antennas that generates OAM-carrying electromagnetic waves by combining Flat Optics (FO) and machine learning (ML) techniques. To improve the performance of our system, we use a machine learning technique to discover a mathematical function that can effectively control the entire radiation pattern. We can determine the optimal values for each parameter, resulting in the desired radiation pattern, using a total of 77,000 generated datasets.
  arXiv  Detail & Related papers  (2023-04-25T10:01:04Z)
• Learning to Estimate RIS-Aided mmWave Channels [50.15279409856091]
  We focus on uplink cascaded channel estimation, where known and fixed base station combining and RIS phase control matrices are considered for collecting observations. To boost the estimation performance and reduce the training overhead, the inherent channel sparsity of mmWave channels is leveraged in the deep unfolding method. It is verified that the proposed deep unfolding network architecture can outperform the least squares (LS) method with a relatively smaller training overhead and online computational complexity.
  arXiv  Detail & Related papers  (2021-07-27T06:57:56Z)
• Learning and Adaptation in Millimeter-Wave: a Dual Timescale Variational Framework [4.162663632560141]
  Millimeter-wave vehicular networks incur enormous beam-training overhead to enable narrow-beam communications. This paper proposes a learning and adaptation framework in which the dynamics of the communication beams are learned and then exploited to design adaptive beam-training with low overhead.
  arXiv  Detail & Related papers  (2021-06-27T19:04:18Z)
• Learning Based Hybrid Beamforming Design for Full-Duplex Millimeter Wave Systems [22.478350298755892]
  We propose two learning schemes to design HBF for FD mmWave systems, i.e., extreme learning machine based HBF and convolutional neural networks based HBF. Results show that both learning based schemes can provide more robust HBF performance and achieve at least 22.1% higher spectral efficiency.
  arXiv  Detail & Related papers  (2020-04-16T15:48:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.