Generalization Bounds for Neural Belief Propagation Decoders

• URL: http://arxiv.org/abs/2305.10540v2
• Date: Sat, 20 Apr 2024 23:01:56 GMT
• Title: Generalization Bounds for Neural Belief Propagation Decoders
• Authors: Sudarshan Adiga, Xin Xiao, Ravi Tandon, Bane Vasic, Tamal Bose,
• Abstract summary: In this paper, we investigate the generalization capabilities of neural network based decoders. Specifically, the generalization gap of a decoder is the difference between empirical and expected bit-error-rate(s) Results are presented for both regular and irregular parity-check matrices.
• Score: 10.96453955114324
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine learning based approaches are being increasingly used for designing decoders for next generation communication systems. One widely used framework is neural belief propagation (NBP), which unfolds the belief propagation (BP) iterations into a deep neural network and the parameters are trained in a data-driven manner. NBP decoders have been shown to improve upon classical decoding algorithms. In this paper, we investigate the generalization capabilities of NBP decoders. Specifically, the generalization gap of a decoder is the difference between empirical and expected bit-error-rate(s). We present new theoretical results which bound this gap and show the dependence on the decoder complexity, in terms of code parameters (blocklength, message length, variable/check node degrees), decoding iterations, and the training dataset size. Results are presented for both regular and irregular parity-check matrices. To the best of our knowledge, this is the first set of theoretical results on generalization performance of neural network based decoders. We present experimental results to show the dependence of generalization gap on the training dataset size, and decoding iterations for different codes.

Related papers

• Graph Neural Networks for Enhanced Decoding of Quantum LDPC Codes [6.175503577352742]
  We propose a differentiable iterative decoder for quantum low-density parity-check (LDPC) codes. The proposed algorithm is composed of classical belief propagation (BP) decoding stages and intermediate graph neural network (GNN) layers.
  arXiv  Detail & Related papers  (2023-10-26T19:56:25Z)
• Optimizing Serially Concatenated Neural Codes with Classical Decoders [8.692972779213932]
  We show that a classical decoding algorithm is applied to a non-trivial, real-valued neural code. As the BCJR algorithm is fully differentiable, it is possible to train, or fine-tune, the neural encoder in an end-to-end fashion.
  arXiv  Detail & Related papers  (2022-12-20T15:40:08Z)
• A Scalable Graph Neural Network Decoder for Short Block Codes [49.25571364253986]
  We propose a novel decoding algorithm for short block codes based on an edge-weighted graph neural network (EW-GNN) The EW-GNN decoder operates on the Tanner graph with an iterative message-passing structure. We show that the EW-GNN decoder outperforms the BP and deep-learning-based BP methods in terms of the decoding error rate.
  arXiv  Detail & Related papers  (2022-11-13T17:13:12Z)
• Towards Better Out-of-Distribution Generalization of Neural Algorithmic Reasoning Tasks [51.8723187709964]
  We study the OOD generalization of neural algorithmic reasoning tasks. The goal is to learn an algorithm from input-output pairs using deep neural networks.
  arXiv  Detail & Related papers  (2022-11-01T18:33:20Z)
• Graph Neural Networks for Channel Decoding [71.15576353630667]
  We showcase competitive decoding performance for various coding schemes, such as low-density parity-check (LDPC) and BCH codes. The idea is to let a neural network (NN) learn a generalized message passing algorithm over a given graph. We benchmark our proposed decoder against state-of-the-art in conventional channel decoding as well as against recent deep learning-based results.
  arXiv  Detail & Related papers  (2022-07-29T15:29:18Z)
• COIN++: Data Agnostic Neural Compression [55.27113889737545]
  COIN++ is a neural compression framework that seamlessly handles a wide range of data modalities. We demonstrate the effectiveness of our method by compressing various data modalities.
  arXiv  Detail & Related papers  (2022-01-30T20:12:04Z)
• Adversarial Neural Networks for Error Correcting Codes [76.70040964453638]
  We introduce a general framework to boost the performance and applicability of machine learning (ML) models. We propose to combine ML decoders with a competing discriminator network that tries to distinguish between codewords and noisy words. Our framework is game-theoretic, motivated by generative adversarial networks (GANs)
  arXiv  Detail & Related papers  (2021-12-21T19:14:44Z)
• Cyclically Equivariant Neural Decoders for Cyclic Codes [33.63188063525036]
  We propose a novel neural decoder for cyclic codes by exploiting their cyclically invariant property. Our new decoder consistently outperforms previous neural decoders when decoding cyclic codes. Finally, we propose a list decoding procedure that can significantly reduce the decoding error probability for BCH codes and punctured RM codes.
  arXiv  Detail & Related papers  (2021-05-12T09:41:13Z)
• Pruning Neural Belief Propagation Decoders [77.237958592189]
  We introduce a method to tailor an overcomplete parity-check matrix to (neural) BP decoding using machine learning. We achieve performance within 0.27 dB and 1.5 dB of the ML performance while reducing the complexity of the decoder.
  arXiv  Detail & Related papers  (2020-01-21T12:05:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.