Lost and Found in Translation: Variational Diagnostics for Neural Codebook Channels

• URL: http://arxiv.org/abs/2605.18846v1
• Date: Wed, 13 May 2026 06:52:21 GMT
• Title: Lost and Found in Translation: Variational Diagnostics for Neural Codebook Channels
• Abstract summary: Variational autoencoders (VAEs) train an encoder $q_$ and decoder $p_$ jointly, and practitioners treat the resulting latent space as a discrete code.<n>Standard VAE diagnostics certify only whether this code is used, never whether the decoder reads each latent under the encoder's code.<n>We develop a coupled encoder-decoder diagnostic whose off-diagonal mass is bounded by an architecture-free Bernoulli-KL certificate.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Classical communication systems fail not only through random noise but also when transmitter and receiver use incompatible operational codebooks. Variational autoencoders (VAEs) train an encoder $q_φ$ and decoder $p_θ$ jointly, and practitioners treat the resulting latent space as a discrete code -- for clustering, conditional generation, and mechanistic interpretability. Yet standard VAE diagnostics -- ELBO, active units, mutual information, and code histograms -- certify only whether this code is used, never whether the decoder reads each latent under the encoder's code. We close this gap with the neural codebook channel $K_{e\to d}(j\mid i)$, a coupled encoder-decoder diagnostic whose off-diagonal mass is bounded by an architecture-free Bernoulli-KL certificate $d_{\mathrm{bin}}(1-\mathcal{A} \,\|\, \barη_p) \le \barΔ$ controlled by the variational gap. The certificate is the operational specialization of the classical KL chain rule under disintegration to the encoder-decoder disagreement event, complemented by a constructive marginal-impossibility result: no combination of marginal histograms, entropies, active-code counts, or mutual information determines $K_{e\to d}$. We audit the certificate on four sklearn datasets (finite-grid exact, 5/5 seeds, 20/20 pairs satisfy the bound), a 2D model where the bound is non-vacuous at $2.71\times$ the observed disagreement and the four-term identity closes within $10^{-4}$, MNIST under importance-sampling control, and a VQ-VAE attaining the predicted limit $\hat{\mathcal{A}}=1.000$. The package $(K_{e\to d}, \mathcal{A}, R_{\mathrm{eff}}, R, \mathrm{AU})$ is an audit-ready reporting unit. More broadly, the framework makes mismatched decoding -- a failure mode classical communication theory named decades ago -- visible inside a single deep generative model.

Related papers

• High-threshold decoding of non-Pauli codes for 2D universality [0.15999407512883507]
  Topological codes have many desirable properties that allow fault-tolerant quantum computation with relatively low overhead.<n>A core challenge for these codes is to achieve a low-overhead universal gate set with limited connectivity.<n>We explore a non-Pauli stabilizer code that can be used to complete a universal gate set on topological toric and surface codes in strictly two dimensions.
  arXiv  Detail & Related papers  (2026-04-02T13:39:46Z)
• Exploiting Discriminative Codebook Prior for Autoregressive Image Generation [54.14166700058777]
  token-based autoregressive image generation systems first tokenize images into sequences of token indices with a codebook, and then model these sequences in an autoregressive paradigm.<n>While autoregressive generative models are trained only on index values, the prior encoded in the codebook, which contains rich token similarity information, is not exploited.<n>Recent studies have attempted to incorporate this prior by performing naive k-means clustering on the tokens, helping to facilitate the training of generative models with a reduced codebook.<n>We propose the Discriminative Codebook Prior Extractor (DCPE) as an alternative to k-means
  arXiv  Detail & Related papers  (2025-08-14T15:00:00Z)
• CODA: Repurposing Continuous VAEs for Discrete Tokenization [31.932323809073477]
  textbfCODA(textbfCOntinuous-to-textbfDiscrete textbfAdaptation) is a framework that decouples compression and discretization.<n>Our approach achieves a remarkable codebook utilization of 100% and notable reconstruction FID (rFID) of $mathbf0.43$ and $mathbf1.34$ for $8 times$ and $16 times$ compression on ImageNet 256$times$ 256 benchmark.
  arXiv  Detail & Related papers  (2025-03-22T12:59:00Z)
• Exact Certification of (Graph) Neural Networks Against Label Poisoning [50.87615167799367]
  We introduce an exact certification method for label flipping in Graph Neural Networks (GNNs)<n>We apply our method to certify a broad range of GNN architectures in node classification tasks.<n>Our work presents the first exact certificate to a poisoning attack ever derived for neural networks.
  arXiv  Detail & Related papers  (2024-11-30T17:05:12Z)
• Entanglement-assisted Quantum Error Correcting Code Saturating The Classical Singleton Bound [44.154181086513574]
  We introduce a construction for entanglement-assisted quantum error-correcting codes (EAQECCs) that saturates the classical Singleton bound with less shared entanglement than any known method for code rates below $ frackn = frac13 $. We demonstrate that any classical $[n,k,d]_q$ code can be transformed into an EAQECC with parameters $[n,k,d;2k]]_q$ using $2k$ pre-shared maximally entangled pairs.
  arXiv  Detail & Related papers  (2024-10-05T11:56:15Z)
• Far from Perfect: Quantum Error Correction with (Hyperinvariant) Evenbly Codes [38.729065908701585]
  We introduce a new class of qubit codes that we call Evenbly codes.<n>Our work indicates that Evenbly codes may show promise for practical quantum computing applications.
  arXiv  Detail & Related papers  (2024-07-16T17:18:13Z)
• Estimating the Decoding Failure Rate of Binary Regular Codes Using Iterative Decoding [84.0257274213152]
  We propose a new technique to provide accurate estimates of the DFR of a two-iterations (parallel) bit flipping decoder.<n>We validate our results, providing comparisons of the modeled and simulated weight of the syndrome, incorrectly-guessed error bit distribution at the end of the first iteration, and two-itcrypteration Decoding Failure Rates (DFR)
  arXiv  Detail & Related papers  (2024-01-30T11:40:24Z)
• Fault-tolerant hyperbolic Floquet quantum error correcting codes [0.4527817188861902]
  We introduce a family of dynamically generated quantum error correcting codes that we call "hyperbolic Floquet codes"<n>One of our hyperbolic Floquet codes uses 400 physical qubits to encode 52 logical qubits with a code distance of 8, i.e., it is a $[[400,52,8]]$ code.<n>At small error rates, comparable logical error suppression to this code requires 5x as many physical qubits (1924) when using the honeycomb Floquet code with the same noise model and decoder.
  arXiv  Detail & Related papers  (2023-09-18T18:00:02Z)
• Causal Recurrent Variational Autoencoder for Medical Time Series Generation [12.82521953179345]
  We propose causal recurrent variational autoencoder (CR-VAE), a novel generative model that learns a Granger causal graph from a time series x. Our model consistently outperforms state-of-the-art time series generative models both qualitatively and quantitatively.
  arXiv  Detail & Related papers  (2023-01-16T19:13:33Z)
• Local Probabilistic Decoding of a Quantum Code [0.0]
  flip is an extremely simple and maximally local classical decoder. Lowest-weight uncorrectable errors for this decoder are closer to correctable errors than to other uncorrectable errors. Introducing randomness into the decoder can allow it to correct these "uncorrectable" errors with finite probability.
  arXiv  Detail & Related papers  (2022-12-14T02:44:26Z)
• Neural Distributed Source Coding [59.630059301226474]
  We present a framework for lossy DSC that is agnostic to the correlation structure and can scale to high dimensions. We evaluate our method on multiple datasets and show that our method can handle complex correlations and state-of-the-art PSNR.
  arXiv  Detail & Related papers  (2021-06-05T04:50:43Z)
• Learning Autoencoders with Relational Regularization [89.53065887608088]
  A new framework is proposed for learning autoencoders of data distributions. We minimize the discrepancy between the model and target distributions, with a emphrelational regularization We implement the framework with two scalable algorithms, making it applicable for both probabilistic and deterministic autoencoders.
  arXiv  Detail & Related papers  (2020-02-07T17:27:30Z)
• Deep Q-learning decoder for depolarizing noise on the toric code [0.0]
  We present an AI-based decoding agent for quantum error correction of depolarizing noise on the toric code. The agent is trained using deep reinforcement learning (DRL), where an artificial neural network encodes the state-action Q-values of error-correcting $X$, $Y$, and $Z$ Pauli operations. We argue that the DRL-type decoder provides a promising framework for future practical error correction of topological codes.
  arXiv  Detail & Related papers  (2019-12-30T13:27:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.