Component Training of Turbo Autoencoders

• URL: http://arxiv.org/abs/2305.09216v1
• Date: Tue, 16 May 2023 06:48:40 GMT
• Title: Component Training of Turbo Autoencoders
• Abstract summary: Isolated training with Gaussian priors (TGP) of the component autoencoders of turbo-autoencoder enables faster, more consistent training. We propose fitting the components via extrinsic information transfer (EXIT) charts to a desired behavior which enables scaling to larger message lengths.
• Score: 8.692972779213932
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Isolated training with Gaussian priors (TGP) of the component autoencoders of turbo-autoencoder architectures enables faster, more consistent training and better generalization to arbitrary decoding iterations than training based on deep unfolding. We propose fitting the components via extrinsic information transfer (EXIT) charts to a desired behavior which enables scaling to larger message lengths ($k \approx 1000$) while retaining competitive performance. To the best of our knowledge, this is the first autoencoder that performs close to classical codes in this regime. Although the binary cross-entropy (BCE) loss function optimizes the bit error rate (BER) of the components, the design via EXIT charts enables to focus on the block error rate (BLER). In serially concatenated systems the component-wise TGP approach is well known for inner components with a fixed outer binary interface, e.g., a learned inner code or equalizer, with an outer binary error correcting code. In this paper we extend the component training to structures with an inner and outer autoencoder, where we propose a new 1-bit quantization strategy for the encoder outputs based on the underlying communication problem. Finally, we discuss the model complexity of the learned components during design time (training) and inference and show that the number of weights in the encoder can be reduced by 99.96 %.

Related papers

• MaskCode: Mask Transformer for Feedback-Assisted Coding With Linear Block Codes [27.89688326131855]
  MaskCode is a Transformer-based inner feedback code for ECCd coding systems.<n>It explicitly incorporates structural knowledge of the outer linear block code into the inner feedback encoder design.<n>It consistently outperforms all baselines, achieving up to 1.5 dB SNR gain.
  arXiv  Detail & Related papers  (2026-09-01T04:47:42Z)
• Coset Ensemble Decoder for Quantum Error Correction with Algorithm-Hardware Co-Design [16.143601808851162]
  quantum error correction (QEC) continuously extracts and decodes error syndromes in real time.<n>A critical component in QEC is the decoder, a classical subsystem that must simultaneously deliver high logical accuracy and ultra-low latency.<n>This paper presents a novel algorithm-hardware co-design that improves the accuracy-latency trade-off over existing approaches.
  arXiv  Detail & Related papers  (2026-06-09T16:37:13Z)
• AI-Enabled Decoding of Qubit Loss for Quantum Error-Correcting Codes [16.0178741156376]
  Qubit loss is a major source of error in quantum computation.<n>We develop an artificial-intelligence-enabled decoder based on a Graph Neural Network architecture.
  arXiv  Detail & Related papers  (2026-04-15T17:59:35Z)
• Generalization Bounds for Transformer Channel Decoders [61.55280736553095]
  This paper studies the generalization performance of ECCT from a learning-theoretic perspective.<n>To the best of our knowledge, this work provides the first theoretical generalization guarantees for this class of decoders.
  arXiv  Detail & Related papers  (2026-01-11T15:56:37Z)
• Learning Binary Autoencoder-Based Codes with Progressive Training [1.8620637029128544]
  Autoencoder (AE) based approaches have gained attention for the end-to-end design of communication systems.<n>These results indicate that compact AE architectures can effectively learn structured, algebraically optimal binary codes through stable and straightforward training.
  arXiv  Detail & Related papers  (2025-11-12T11:32:03Z)
• Neural Decoders for Universal Quantum Algorithms [0.43553942673960666]
  We introduce a modular attention-based neural decoder that learns gate-induced correlations.<n>Our decoders achieve fast inference and logical error rates comparable to most-likely-error decoders.<n>These results establish neural decoders as practical, versatile, and high-performance tools for quantum computing.
  arXiv  Detail & Related papers  (2025-09-14T17:51:46Z)
• Fast correlated decoding of transversal logical algorithms [67.01652927671279]
  Quantum error correction (QEC) is required for large-scale computation, but incurs a significant resource overhead.<n>Recent advances have shown that by jointly decoding logical qubits in algorithms composed of logical gates, the number of syndrome extraction rounds can be reduced.<n>Here, we reform the problem of decoding circuits by directly decoding relevant logical operator products as they propagate through the circuit.
  arXiv  Detail & Related papers  (2025-05-19T18:00:00Z)
• On the Design and Performance of Machine Learning Based Error Correcting Decoders [3.8289109929360245]
  We first consider the so-called single-label neural network (SLNN) and the multi-label neural network (MLNN) decoders which have been reported to achieve near maximum likelihood (ML) performance. We then turn our attention to two transformer-based decoders: the error correction code transformer (ECCT) and the cross-attention message passing transformer (CrossMPT)
  arXiv  Detail & Related papers  (2024-10-21T11:23:23Z)
• Accelerating Error Correction Code Transformers [56.75773430667148]
  We introduce a novel acceleration method for transformer-based decoders. We achieve a 90% compression ratio and reduce arithmetic operation energy consumption by at least 224 times on modern hardware.
  arXiv  Detail & Related papers  (2024-10-08T11:07:55Z)
• Learning Linear Block Error Correction Codes [62.25533750469467]
  We propose for the first time a unified encoder-decoder training of binary linear block codes. We also propose a novel Transformer model in which the self-attention masking is performed in a differentiable fashion for the efficient backpropagation of the code gradient.
  arXiv  Detail & Related papers  (2024-05-07T06:47:12Z)
• Efficient Encoder-Decoder Transformer Decoding for Decomposable Tasks [53.550782959908524]
  We introduce a new configuration for encoder-decoder models that improves efficiency on structured output and decomposable tasks. Our method, prompt-in-decoder (PiD), encodes the input once and decodes the output in parallel, boosting both training and inference efficiency.
  arXiv  Detail & Related papers  (2024-03-19T19:27:23Z)
• Coding for Gaussian Two-Way Channels: Linear and Learning-Based Approaches [28.98777190628006]
  We propose two different two-way coding strategies: linear coding and learning-based coding. For learning-based coding, we introduce a novel recurrent neural network (RNN)-based coding architecture. Our two-way coding methodologies outperform conventional channel coding schemes significantly in sum-error performance.
  arXiv  Detail & Related papers  (2023-12-31T12:40:18Z)
• Practical Conformer: Optimizing size, speed and flops of Conformer for on-Device and cloud ASR [67.63332492134332]
  We design an optimized conformer that is small enough to meet on-device restrictions and has fast inference on TPUs. Our proposed encoder can double as a strong standalone encoder in on device, and as the first part of a high-performance ASR pipeline.
  arXiv  Detail & Related papers  (2023-03-31T23:30:48Z)
• Graph Neural Networks for Channel Decoding [71.15576353630667]
  We showcase competitive decoding performance for various coding schemes, such as low-density parity-check (LDPC) and BCH codes. The idea is to let a neural network (NN) learn a generalized message passing algorithm over a given graph. We benchmark our proposed decoder against state-of-the-art in conventional channel decoding as well as against recent deep learning-based results.
  arXiv  Detail & Related papers  (2022-07-29T15:29:18Z)
• Error Correction Code Transformer [92.10654749898927]
  We propose to extend for the first time the Transformer architecture to the soft decoding of linear codes at arbitrary block lengths. We encode each channel's output dimension to high dimension for better representation of the bits information to be processed separately. The proposed approach demonstrates the extreme power and flexibility of Transformers and outperforms existing state-of-the-art neural decoders by large margins at a fraction of their time complexity.
  arXiv  Detail & Related papers  (2022-03-27T15:25:58Z)
• Capacity-Approaching Autoencoders for Communications [4.86067125387358]
  The current approach to train an autoencoder relies on the use of the cross-entropy loss function. We propose a methodology that computes an estimate of the channel capacity and constructs an optimal coded signal approaching it.
  arXiv  Detail & Related papers  (2020-09-11T08:19:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.