Adapting Neural Audio Codecs to EEG

• URL: http://arxiv.org/abs/2511.23142v1
• Date: Fri, 28 Nov 2025 12:47:05 GMT
• Title: Adapting Neural Audio Codecs to EEG
• Authors: Ard Kastrati, Luca Lanzendörfer, Riccardo Rigoni, John Staib Matilla, Roger Wattenhofer,
• Abstract summary: We show that pretrained neural audio codecs can serve as effective starting points for EEG compression.<n>We propose DAC-MC, a multi-channel extension with attention-based cross-channel aggregation and channel-specific decoding.<n> Evaluations on the TUH Abnormal and Epilepsy datasets show that the adapted codecs preserve clinically relevant information.
• Score: 27.20793132729464
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: EEG and audio are inherently distinct modalities, differing in sampling rate, channel structure, and scale. Yet, we show that pretrained neural audio codecs can serve as effective starting points for EEG compression, provided that the data are preprocessed to be suitable to the codec's input constraints. Using DAC, a state-of-the-art neural audio codec as our base, we demonstrate that raw EEG can be mapped into the codec's stride-based framing, enabling direct reuse of the audio-pretrained encoder-decoder. Even without modification, this setup yields stable EEG reconstructions, and fine-tuning on EEG data further improves fidelity and generalization compared to training from scratch. We systematically explore compression-quality trade-offs by varying residual codebook depth, codebook (vocabulary) size, and input sampling rate. To capture spatial dependencies across electrodes, we propose DAC-MC, a multi-channel extension with attention-based cross-channel aggregation and channel-specific decoding, while retaining the audio-pretrained initialization. Evaluations on the TUH Abnormal and Epilepsy datasets show that the adapted codecs preserve clinically relevant information, as reflected in spectrogram-based reconstruction loss and downstream classification accuracy.

Related papers

• Latent-Mark: An Audio Watermark Robust to Neural Resynthesis [62.09761127079914]
  Latent-Mark is the first zero-bit audio watermarking framework designed to survive semantic compression.<n>Our key insight is that robustness to the encode-decode process requires embedding the watermark within the invariant latent space.<n>Our work inspires future research into universal watermarking frameworks capable of maintaining integrity across increasingly complex and diverse generative distortions.
  arXiv  Detail & Related papers  (2026-03-05T15:51:09Z)
• The Equalizer: Introducing Shape-Gain Decomposition in Neural Audio Codecs [20.468614667204093]
  We propose to introduce shape-gain decomposition, widely used in classical speech/audio coding, into the NAC framework.<n>The proposed Equalizer methodology is to decompose the input signal -- before the NAC encoder -- into gain and normalized shape vector on a short-term basis.<n>Our experiments conducted on speech signals show that this general methodology, easily applicable to any NAC, enables a substantial gain in-distortion performance.
  arXiv  Detail & Related papers  (2026-02-17T10:59:33Z)
• Perceptually Aligning Representations of Music via Noise-Augmented Autoencoders [13.596509137642103]
  We show that, after training an audio autoencoder in this manner, perceptually salient information is captured in coarser representation structures than with conventional training.<n>We show that such perceptual hierarchies improve latent diffusion decoding in the context of estimating surprisal in music pitches and predicting EEG-brain responses to music listening.
  arXiv  Detail & Related papers  (2025-11-07T15:44:12Z)
• Towards Generalized Source Tracing for Codec-Based Deepfake Speech [52.68106957822706]
  We introduce the Semantic-Acoustic Source Tracing Network (SASTNet), which jointly leverages Whisper for semantic feature encoding and Wav2vec2 with AudioMAE for acoustic feature encoding.<n>Our proposed SASTNet achieves state-of-the-art performance on the CoSG test set of the CodecFake+ dataset, demonstrating its effectiveness for reliable source tracing.
  arXiv  Detail & Related papers  (2025-06-08T21:36:10Z)
• Neural Spectral Band Generation for Audio Coding [14.466825532313795]
  We propose a deep neural network (DNN)-based generative approach for coding the high-frequency bands.<n>We show that the proposed method achieves a better perceptual quality than HE-AAC-v1 with much less side information.
  arXiv  Detail & Related papers  (2025-06-07T09:35:08Z)
• Variable Bitrate Residual Vector Quantization for Audio Coding [29.368893236587343]
  Recent neural audio compression models have progressively adopted residual vector quantization (RVQ)<n>These models employ a fixed number of codebooks per frame, which can be suboptimal in terms of rate-distortion tradeoffs.<n>We propose variable RVQ (VRVQ) for audio codecs, which allows for more efficient coding by adapting the number of codebooks used per frame.
  arXiv  Detail & Related papers  (2024-10-08T13:18:24Z)
• Epsilon-VAE: Denoising as Visual Decoding [61.29255979767292]
  We propose denoising as decoding, shifting from single-step reconstruction to iterative refinement.<n>Specifically, we replace the decoder with a diffusion process that iteratively refines noise to recover the original image.<n>By adopting iterative reconstruction through diffusion, our autoencoder, namely Epsilon-VAE, achieves high reconstruction quality.
  arXiv  Detail & Related papers  (2024-10-05T08:27:53Z)
• Learning Source Disentanglement in Neural Audio Codec [20.335701584949526]
  We introduce the Source-Disentangled Neural Audio Codec (SD-Codec), a novel approach that combines audio coding and source separation.<n>By jointly learning audio resynthesis and separation, SD-Codec explicitly assigns audio signals from different domains to distinct codebooks, sets of discrete representations.<n> Experimental results indicate that SD-Codec not only maintains competitive resynthesis quality but also, supported by the separation results, demonstrates successful disentanglement of different sources in the latent space.
  arXiv  Detail & Related papers  (2024-09-17T14:21:02Z)
• Streaming Audio-Visual Speech Recognition with Alignment Regularization [69.30185151873707]
  We propose a streaming AV-ASR system based on a hybrid connectionist temporal classification ( CTC)/attention neural network architecture. The proposed AV-ASR model achieves WERs of 2.0% and 2.6% on the Lip Reading Sentences 3 dataset in an offline and online setup.
  arXiv  Detail & Related papers  (2022-11-03T20:20:47Z)
• High Fidelity Neural Audio Compression [92.4812002532009]
  We introduce a state-of-the-art real-time, high-fidelity, audio leveraging neural networks. It consists in a streaming encoder-decoder architecture with quantized latent space trained in an end-to-end fashion. We simplify and speed-up the training by using a single multiscale spectrogram adversary.
  arXiv  Detail & Related papers  (2022-10-24T17:52:02Z)
• Denoising Diffusion Error Correction Codes [92.10654749898927]
  Recently, neural decoders have demonstrated their advantage over classical decoding techniques. Recent state-of-the-art neural decoders suffer from high complexity and lack the important iterative scheme characteristic of many legacy decoders. We propose to employ denoising diffusion models for the soft decoding of linear codes at arbitrary block lengths.
  arXiv  Detail & Related papers  (2022-09-16T11:00:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.