Optimizing Serially Concatenated Neural Codes with Classical Decoders
- URL: http://arxiv.org/abs/2212.10355v3
- Date: Wed, 3 May 2023 20:34:25 GMT
- Title: Optimizing Serially Concatenated Neural Codes with Classical Decoders
- Authors: Jannis Clausius, Marvin Geiselhart and Stephan ten Brink
- Abstract summary: We show that a classical decoding algorithm is applied to a non-trivial, real-valued neural code.
As the BCJR algorithm is fully differentiable, it is possible to train, or fine-tune, the neural encoder in an end-to-end fashion.
- Score: 8.692972779213932
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: For improving short-length codes, we demonstrate that classic decoders can
also be used with real-valued, neural encoders, i.e., deep-learning based
codeword sequence generators. Here, the classical decoder can be a valuable
tool to gain insights into these neural codes and shed light on weaknesses.
Specifically, the turbo-autoencoder is a recently developed channel coding
scheme where both encoder and decoder are replaced by neural networks. We first
show that the limited receptive field of convolutional neural network
(CNN)-based codes enables the application of the BCJR algorithm to optimally
decode them with feasible computational complexity. These maximum a posteriori
(MAP) component decoders then are used to form classical (iterative) turbo
decoders for parallel or serially concatenated CNN encoders, offering a
close-to-maximum likelihood (ML) decoding of the learned codes. To the best of
our knowledge, this is the first time that a classical decoding algorithm is
applied to a non-trivial, real-valued neural code. Furthermore, as the BCJR
algorithm is fully differentiable, it is possible to train, or fine-tune, the
neural encoder in an end-to-end fashion.
Related papers
- Learning Linear Block Error Correction Codes [62.25533750469467]
We propose for the first time a unified encoder-decoder training of binary linear block codes.
We also propose a novel Transformer model in which the self-attention masking is performed in a differentiable fashion for the efficient backpropagation of the code gradient.
arXiv Detail & Related papers (2024-05-07T06:47:12Z) - Convolutional neural network based decoders for surface codes [0.0]
This work reports a study of decoders based on convolutional neural networks, tested on different code distances and noise models.
The results show that decoders based on convolutional neural networks have good performance and can adapt to different noise models.
arXiv Detail & Related papers (2023-12-06T14:07:31Z) - Generalization Bounds for Neural Belief Propagation Decoders [10.96453955114324]
In this paper, we investigate the generalization capabilities of neural network based decoders.
Specifically, the generalization gap of a decoder is the difference between empirical and expected bit-error-rate(s)
Results are presented for both regular and irregular parity-check matrices.
arXiv Detail & Related papers (2023-05-17T19:56:04Z) - A Scalable Graph Neural Network Decoder for Short Block Codes [49.25571364253986]
We propose a novel decoding algorithm for short block codes based on an edge-weighted graph neural network (EW-GNN)
The EW-GNN decoder operates on the Tanner graph with an iterative message-passing structure.
We show that the EW-GNN decoder outperforms the BP and deep-learning-based BP methods in terms of the decoding error rate.
arXiv Detail & Related papers (2022-11-13T17:13:12Z) - Graph Neural Networks for Channel Decoding [71.15576353630667]
We showcase competitive decoding performance for various coding schemes, such as low-density parity-check (LDPC) and BCH codes.
The idea is to let a neural network (NN) learn a generalized message passing algorithm over a given graph.
We benchmark our proposed decoder against state-of-the-art in conventional channel decoding as well as against recent deep learning-based results.
arXiv Detail & Related papers (2022-07-29T15:29:18Z) - Adversarial Neural Networks for Error Correcting Codes [76.70040964453638]
We introduce a general framework to boost the performance and applicability of machine learning (ML) models.
We propose to combine ML decoders with a competing discriminator network that tries to distinguish between codewords and noisy words.
Our framework is game-theoretic, motivated by generative adversarial networks (GANs)
arXiv Detail & Related papers (2021-12-21T19:14:44Z) - ProductAE: Towards Training Larger Channel Codes based on Neural Product
Codes [9.118706387430885]
It is prohibitively complex to design and train relatively large neural channel codes via deep learning techniques.
In this paper, we construct ProductAEs, a computationally efficient family of deep-learning driven (encoder, decoder) pairs.
We show significant gains, over all ranges of signal-to-noise ratio (SNR), for a code of parameters $(100,225)$ and a moderate-length code of parameters $(196,441)$.
arXiv Detail & Related papers (2021-10-09T06:00:40Z) - KO codes: Inventing Nonlinear Encoding and Decoding for Reliable
Wireless Communication via Deep-learning [76.5589486928387]
Landmark codes underpin reliable physical layer communication, e.g., Reed-Muller, BCH, Convolution, Turbo, LDPC and Polar codes.
In this paper, we construct KO codes, a computationaly efficient family of deep-learning driven (encoder, decoder) pairs.
KO codes beat state-of-the-art Reed-Muller and Polar codes, under the low-complexity successive cancellation decoding.
arXiv Detail & Related papers (2021-08-29T21:08:30Z) - Cyclically Equivariant Neural Decoders for Cyclic Codes [33.63188063525036]
We propose a novel neural decoder for cyclic codes by exploiting their cyclically invariant property.
Our new decoder consistently outperforms previous neural decoders when decoding cyclic codes.
Finally, we propose a list decoding procedure that can significantly reduce the decoding error probability for BCH codes and punctured RM codes.
arXiv Detail & Related papers (2021-05-12T09:41:13Z) - Less is More: Pre-training a Strong Siamese Encoder Using a Weak Decoder [75.84152924972462]
Many real-world applications use Siamese networks to efficiently match text sequences at scale.
This paper pre-trains language models dedicated to sequence matching in Siamese architectures.
arXiv Detail & Related papers (2021-02-18T08:08:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.