Cross-Attention Message-Passing Transformers for Code-Agnostic Decoding in 6G Networks

• URL: http://arxiv.org/abs/2507.01038v1
• Date: Sun, 22 Jun 2025 16:08:42 GMT
• Title: Cross-Attention Message-Passing Transformers for Code-Agnostic Decoding in 6G Networks
• Authors: Seong-Joon Park, Hee-Youl Kwak, Sang-Hyo Kim, Yongjune Kim, Jong-Seon No,
• Abstract summary: Channel coding for 6G networks is expected to support a wide range of requirements arising from heterogeneous communication scenarios.<n>Traditional code-specific decoders lack the flexibility and scalability required for next-generation systems.<n>We propose an AI-native foundation model for unified and code-agnostic decoding based on the transformer architecture.
• Score: 14.631435001491514
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Channel coding for 6G networks is expected to support a wide range of requirements arising from heterogeneous communication scenarios. These demands challenge traditional code-specific decoders, which lack the flexibility and scalability required for next-generation systems. To tackle this problem, we propose an AI-native foundation model for unified and code-agnostic decoding based on the transformer architecture. We first introduce a cross-attention message-passing transformer (CrossMPT). CrossMPT employs two masked cross-attention blocks that iteratively update two distinct input representations-magnitude and syndrome vectors-allowing the model to effectively learn the decoding problem. Notably, our CrossMPT has achieved state-of-the-art decoding performance among single neural decoders. Building on this, we develop foundation CrossMPT (FCrossMPT) by making the architecture invariant to code length, rate, and class, allowing a single trained model to decode a broad range of codes without retraining. To further enhance decoding performance, particularly for short blocklength codes, we propose CrossMPT ensemble decoder (CrossED), an ensemble decoder composed of multiple parallel CrossMPT blocks employing different parity-check matrices. This architecture can also serve as a foundation model, showing strong generalization across diverse code types. Overall, the proposed AI-native code-agnostic decoder offers flexibility, scalability, and high performance, presenting a promising direction to channel coding for 6G networks.

Related papers

• QTSeg: A Query Token-Based Dual-Mix Attention Framework with Multi-Level Feature Distribution for Medical Image Segmentation [13.359001333361272]
  Medical image segmentation plays a crucial role in assisting healthcare professionals with accurate diagnoses and enabling automated diagnostic processes.<n>Traditional convolutional neural networks (CNNs) often struggle with capturing long-range dependencies, while transformer-based architectures come with increased computational complexity.<n>Recent efforts have focused on combining CNNs and transformers to balance performance and efficiency, but existing approaches still face challenges in achieving high segmentation accuracy while maintaining low computational costs.<n>We propose QTSeg, a novel architecture for medical image segmentation that effectively integrates local and global information.
  arXiv  Detail & Related papers  (2024-12-23T03:22:44Z)
• GNN-based Auto-Encoder for Short Linear Block Codes: A DRL Approach [43.17241175857862]
  It integrates deep reinforcement learning (DRL) and graph neural networks (GNN) in code design.<n>An edge-weighted GNN (EW-GNN) decoder is proposed, which operates on the Tanner graph with an iterative message-passing structure.<n>An iterative joint training of the DRL-based code designer and the EW-GNN decoder is performed to optimize the end-end encoding and decoding process.
  arXiv  Detail & Related papers  (2024-12-03T00:25:14Z)
• On the Design and Performance of Machine Learning Based Error Correcting Decoders [3.8289109929360245]
  We first consider the so-called single-label neural network (SLNN) and the multi-label neural network (MLNN) decoders which have been reported to achieve near maximum likelihood (ML) performance. We then turn our attention to two transformer-based decoders: the error correction code transformer (ECCT) and the cross-attention message passing transformer (CrossMPT)
  arXiv  Detail & Related papers  (2024-10-21T11:23:23Z)
• Error Correction Code Transformer: From Non-Unified to Unified [20.902351179839282]
  Traditional decoders were typically designed as fixed hardware circuits tailored to specific decoding algorithms.<n>This paper proposes a unified, code-agnostic Transformer-based decoding architecture capable of handling multiple linear block codes.
  arXiv  Detail & Related papers  (2024-10-04T12:30:42Z)
• When Video Coding Meets Multimodal Large Language Models: A Unified Paradigm for Video Coding [118.72266141321647]
  Cross-Modality Video Coding (CMVC) is a pioneering approach to explore multimodality representation and video generative models in video coding.<n>During decoding, previously encoded components and video generation models are leveraged to create multiple encoding-decoding modes.<n>Experiments indicate that TT2V achieves effective semantic reconstruction, while IT2V exhibits competitive perceptual consistency.
  arXiv  Detail & Related papers  (2024-08-15T11:36:18Z)
• Learning Linear Block Error Correction Codes [62.25533750469467]
  We propose for the first time a unified encoder-decoder training of binary linear block codes. We also propose a novel Transformer model in which the self-attention masking is performed in a differentiable fashion for the efficient backpropagation of the code gradient.
  arXiv  Detail & Related papers  (2024-05-07T06:47:12Z)
• CrossMPT: Cross-attention Message-Passing Transformer for Error Correcting Codes [14.631435001491514]
  We propose a novel Cross-attention Message-Passing Transformer (CrossMPT) We show that CrossMPT significantly outperforms existing neural network-based decoders for various code classes. Notably, CrossMPT achieves this decoding performance improvement, while significantly reducing the memory usage, complexity, inference time, and training time.
  arXiv  Detail & Related papers  (2024-05-02T06:30:52Z)
• Coding for Gaussian Two-Way Channels: Linear and Learning-Based Approaches [27.356869653157677]
  We propose two different two-way coding strategies: linear coding and learning-based coding.<n>For learning-based coding, we introduce a novel recurrent neural network (RNN)-based coding architecture.<n>Our two-way coding methodologies outperform conventional channel coding schemes significantly in sum-error performance.
  arXiv  Detail & Related papers  (2023-12-31T12:40:18Z)
• Towards Diverse Binary Segmentation via A Simple yet General Gated Network [71.19503376629083]
  We propose a simple yet general gated network (GateNet) to tackle binary segmentation tasks. With the help of multi-level gate units, the valuable context information from the encoder can be selectively transmitted to the decoder. We introduce a "Fold" operation to improve the atrous convolution and form a novel folded atrous convolution.
  arXiv  Detail & Related papers  (2023-03-18T11:26:36Z)
• Error Correction Code Transformer [92.10654749898927]
  We propose to extend for the first time the Transformer architecture to the soft decoding of linear codes at arbitrary block lengths. We encode each channel's output dimension to high dimension for better representation of the bits information to be processed separately. The proposed approach demonstrates the extreme power and flexibility of Transformers and outperforms existing state-of-the-art neural decoders by large margins at a fraction of their time complexity.
  arXiv  Detail & Related papers  (2022-03-27T15:25:58Z)
• UniXcoder: Unified Cross-Modal Pre-training for Code Representation [65.6846553962117]
  We present UniXcoder, a unified cross-modal pre-trained model for programming language. We propose a one-to-one mapping method to transform AST in a sequence structure that retains all structural information from the tree. We evaluate UniXcoder on five code-related tasks over nine datasets.
  arXiv  Detail & Related papers  (2022-03-08T04:48:07Z)
• Crosslink-Net: Double-branch Encoder Segmentation Network via Fusing Vertical and Horizontal Convolutions [58.71117402626524]
  We present a novel double-branch encoder architecture for medical image segmentation. Our architecture is inspired by two observations: 1) Since the discrimination of features learned via square convolutional kernels needs to be further improved, we propose to utilize non-square vertical and horizontal convolutional kernels. The experiments validate the effectiveness of our model on four datasets.
  arXiv  Detail & Related papers  (2021-07-24T02:58:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.