LightCode: Light Analytical and Neural Codes for Channels with Feedback

• URL: http://arxiv.org/abs/2403.10751v3
• Date: Sat, 16 Nov 2024 19:55:18 GMT
• Title: LightCode: Light Analytical and Neural Codes for Channels with Feedback
• Authors: Sravan Kumar Ankireddy, Krishna Narayanan, Hyeji Kim,
• Abstract summary: We focus on designing low-complexity coding schemes that are interpretable and more suitable for communication systems. First, we demonstrate that PowerBlast, an analytical coding scheme inspired by Schalkwijk-Kailath (SK) and Gallager-Nakibouglu (GN) schemes, achieves notable reliability improvements over both SK and GN schemes. Next, to enhance reliability in low-SNR regions, we propose LightCode, a lightweight neural code that achieves state-of-the-art reliability while using a fraction of memory and compute compared to existing deeplearning-based codes.
• Score: 10.619569069690185
• License:
• Abstract: The design of reliable and efficient codes for channels with feedback remains a longstanding challenge in communication theory. While significant improvements have been achieved by leveraging deep learning techniques, neural codes often suffer from high computational costs, a lack of interpretability, and limited practicality in resource-constrained settings. We focus on designing low-complexity coding schemes that are interpretable and more suitable for communication systems. We advance both analytical and neural codes. First, we demonstrate that PowerBlast, an analytical coding scheme inspired by Schalkwijk-Kailath (SK) and Gallager-Nakibo\u{g}lu (GN) schemes, achieves notable reliability improvements over both SK and GN schemes, outperforming neural codes in high signal-to-noise ratio (SNR) regions. Next, to enhance reliability in low-SNR regions, we propose LightCode, a lightweight neural code that achieves state-of-the-art reliability while using a fraction of memory and compute compared to existing deeplearning-based codes. Finally, we systematically analyze the learned codes, establishing connections between LightCode and PowerBlast, identifying components crucial for performance, and providing interpretation aided by linear regression analysis.

Related papers

• Stochastic Spiking Neural Networks with First-to-Spike Coding [7.955633422160267]
  Spiking Neural Networks (SNNs) are known for their bio-plausibility and energy efficiency. In this work, we explore the merger of novel computing and information encoding schemes in SNN architectures. We investigate the tradeoffs of our proposal in terms of accuracy, inference latency, spiking sparsity, energy consumption, and datasets.
  arXiv  Detail & Related papers  (2024-04-26T22:52:23Z)
• The Role of Foundation Models in Neuro-Symbolic Learning and Reasoning [54.56905063752427]
  Neuro-Symbolic AI (NeSy) holds promise to ensure the safe deployment of AI systems. Existing pipelines that train the neural and symbolic components sequentially require extensive labelling. New architecture, NeSyGPT, fine-tunes a vision-language foundation model to extract symbolic features from raw data.
  arXiv  Detail & Related papers  (2024-02-02T20:33:14Z)
• Coding for Gaussian Two-Way Channels: Linear and Learning-Based Approaches [28.98777190628006]
  We propose two different two-way coding strategies: linear coding and learning-based coding. For learning-based coding, we introduce a novel recurrent neural network (RNN)-based coding architecture. Our two-way coding methodologies outperform conventional channel coding schemes significantly in sum-error performance.
  arXiv  Detail & Related papers  (2023-12-31T12:40:18Z)
• NeuralFastLAS: Fast Logic-Based Learning from Raw Data [54.938128496934695]
  Symbolic rule learners generate interpretable solutions, however they require the input to be encoded symbolically. Neuro-symbolic approaches overcome this issue by mapping raw data to latent symbolic concepts using a neural network. We introduce NeuralFastLAS, a scalable and fast end-to-end approach that trains a neural network jointly with a symbolic learner.
  arXiv  Detail & Related papers  (2023-10-08T12:33:42Z)
• A Hybrid Neural Coding Approach for Pattern Recognition with Spiking Neural Networks [53.31941519245432]
  Brain-inspired spiking neural networks (SNNs) have demonstrated promising capabilities in solving pattern recognition tasks. These SNNs are grounded on homogeneous neurons that utilize a uniform neural coding for information representation. In this study, we argue that SNN architectures should be holistically designed to incorporate heterogeneous coding schemes.
  arXiv  Detail & Related papers  (2023-05-26T02:52:12Z)
• Robust Non-Linear Feedback Coding via Power-Constrained Deep Learning [7.941112438865385]
  We develop a new family of non-linear feedback codes that greatly enhance robustness to channel noise. Our autoencoder-based architecture is designed to learn codes based on consecutive blocks of bits. We show that our scheme outperforms state-of-the-art feedback codes by wide margins over practical forward and feedback noise regimes.
  arXiv  Detail & Related papers  (2023-04-25T22:21:26Z)
• Learning to Precode for Integrated Sensing and Communications Systems [11.689567114100514]
  We present an unsupervised learning neural model to design transmit precoders for ISAC systems. We show that the proposed method outperforms traditional optimization-based methods in presence of channel estimation errors.
  arXiv  Detail & Related papers  (2023-03-11T11:24:18Z)
• Graph Neural Networks for Channel Decoding [71.15576353630667]
  We showcase competitive decoding performance for various coding schemes, such as low-density parity-check (LDPC) and BCH codes. The idea is to let a neural network (NN) learn a generalized message passing algorithm over a given graph. We benchmark our proposed decoder against state-of-the-art in conventional channel decoding as well as against recent deep learning-based results.
  arXiv  Detail & Related papers  (2022-07-29T15:29:18Z)
• Great Truths are Always Simple: A Rather Simple Knowledge Encoder for Enhancing the Commonsense Reasoning Capacity of Pre-Trained Models [89.98762327725112]
  Commonsense reasoning in natural language is a desired ability of artificial intelligent systems. For solving complex commonsense reasoning tasks, a typical solution is to enhance pre-trained language models(PTMs) with a knowledge-aware graph neural network(GNN) encoder. Despite the effectiveness, these approaches are built on heavy architectures, and can't clearly explain how external knowledge resources improve the reasoning capacity of PTMs.
  arXiv  Detail & Related papers  (2022-05-04T01:27:36Z)
• Credit Assignment in Neural Networks through Deep Feedback Control [59.14935871979047]
  Deep Feedback Control (DFC) is a new learning method that uses a feedback controller to drive a deep neural network to match a desired output target and whose control signal can be used for credit assignment. The resulting learning rule is fully local in space and time and approximates Gauss-Newton optimization for a wide range of connectivity patterns. To further underline its biological plausibility, we relate DFC to a multi-compartment model of cortical pyramidal neurons with a local voltage-dependent synaptic plasticity rule, consistent with recent theories of dendritic processing.
  arXiv  Detail & Related papers  (2021-06-15T05:30:17Z)
• Infomax Neural Joint Source-Channel Coding via Adversarial Bit Flip [41.28049430114734]
  We propose a novel regularization method called Infomax Adversarial-Bit-Flip (IABF) to improve the stability and robustness of the neural joint source-channel coding scheme. Our IABF can achieve state-of-the-art performances on both compression and error correction benchmarks and outperform the baselines by a significant margin.
  arXiv  Detail & Related papers  (2020-04-03T10:00:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.