NeuroSonic: Conditional Flow Matching for EEG-to-Speech Reconstruction

• URL: http://arxiv.org/abs/2606.24087v1
• Date: Tue, 23 Jun 2026 03:04:53 GMT
• Title: NeuroSonic: Conditional Flow Matching for EEG-to-Speech Reconstruction
• Abstract summary: Reconstructing continuous speech from scalp electroencephalography (EEG) remains fundamentally challenging.<n>We introduce NeuroSonic, a conditional flow-matching framework for EEG-to-speech reconstruction.<n>NeuroSonic learns a deterministic probability-flow field that transports a noise-corrupted acoustic state toward clean speech under EEG conditioning.
• Score: 30.839290792276852
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reconstructing continuous speech from scalp electroencephalography (EEG) remains fundamentally challenging. EEG provides a weak, spatially diffuse, and highly variable measurement of distributed cortical activity, whereas speech is organized as a coherent acoustic trajectory with strong harmonic and temporal structure. The resulting mismatch makes waveform regression unstable and causes stochastic multi-step generation to be sensitive to artifact-dependent conditioning and subject variability. We introduce NeuroSonic, a conditional flow-matching framework for EEG-to-speech reconstruction. Instead of predicting waveforms directly or refining them through stochastic denoising, NeuroSonic learns a deterministic probability-flow velocity field that transports a noise-corrupted acoustic state toward clean speech under EEG conditioning. EEG and audio are embedded into a shared token space and processed by a time-conditioned gated Transformer that parameterizes the transport ordinary differential equation. This formulation models trajectory evolution explicitly while avoiding iterative stochastic sampling. We evaluate NeuroSonic on the CineBrain and EAV benchmarks under cross-subject evaluation. Across both datasets, the proposed method improves distributional realism, spectral fidelity, and perceptual quality over representative GAN-, diffusion-, and mean-flow baselines, with up to a 26.3\% gain in overall perceptual quality. The performance gap is most evident in artifact-heavy segments, where conditioning variability is strongest. These findings indicate that deterministic conditional transport provides a stable and effective formulation for EEG-driven speech reconstruction. Code is available at https://github.com/Y-Research-SBU/NeuroSonic/ .

Related papers

• DCRA: Diffusion-Conditioned Representation Alignment for Robust Time-Series Learning [7.328100870402178]
  We propose a training framework that repurposes the forward diffusion process as a structured corruption scheduler for representation learning.<n>DCRA produces more balanced and structured representations compared to baseline and diffusion-only models.
  arXiv  Detail & Related papers  (2026-09-09T22:52:37Z)
• ProgFormer: Hierarchical Voxel Diffusion Transformer for Longitudinal Brain MRI Prediction [81.07048249760953]
  longitudinal changes are often subtle and confined to specific anatomical regions.<n>Most subject-specific brain structure remains stable over time.<n>ProgFormer is a hierarchical voxel-space Diffusion Transformer for longitudinal brain MRI prediction.
  arXiv  Detail & Related papers  (2026-07-30T00:13:42Z)
• From Atoms to Entropy: Optimal Noise Allocation for Diffusion Training in the Convex Regime [52.459513910242485]
  We develop a general statistical framework for studying optimal noise-level allocation in diffusion training.<n>We show that the optimized training schedule admits an atomic minimizer, concentrated on finitely many noise levels.<n>We then evaluate the entropic schedule in larger-scale experiments, where full schedule optimization is currently intractable.
  arXiv  Detail & Related papers  (2026-07-10T20:59:06Z)
• BandRouteNet: An Adaptive Band Routing Neural Network for EEG Artifact Removal [0.0]
  BandRouteNet is an adaptive frequency-aware neural network for EEG denoising.<n>It exploits band-specific processing and full-band contextual modeling.<n>Extensive experiments on the EEGDenoiseNet benchmark dataset demonstrate that BandRouteNet outperforms other methods.
  arXiv  Detail & Related papers  (2026-04-27T12:54:31Z)
• DiffSOS: Acoustic Conditional Diffusion Model for Speed-of-Sound Reconstruction in Ultrasound Computed Tomography [2.6915545700357986]
  We propose DiffSOS, a conditional diffusion model that directly maps acoustic waveforms to SoS maps.<n>Our framework employs a specialized acoustic ControlNet to ground the denoising process in physical wave measurements.<n>We exploit the generative nature of our framework to estimate pixel-wise uncertainty, providing a measure of reliability.
  arXiv  Detail & Related papers  (2026-02-27T23:51:16Z)
• Self-induced stochastic resonance: A physics-informed machine learning approach [0.0]
  Self-induced resonance (SISR) is the emergence of coherent oscillations in excitable systems driven solely by noise.<n>This work presents a physics-informed machine learning framework for modeling and predicting SISR in the FitzHugh neuron.
  arXiv  Detail & Related papers  (2025-10-26T21:49:20Z)
• WaveNet's Precision in EEG Classification [1.0885910878567457]
  This study introduces a WaveNet-based deep learning model designed to automate the classification of EEG signals into physiological, pathological, artifact, and noise categories.<n>The model was trained, validated, and tested on 209,232 samples with a 70/20/10 percent split.<n>WaveNet's architecture, originally developed for raw audio synthesis, is well suited for EEG data due to its use of dilated causal convolutions and residual connections.
  arXiv  Detail & Related papers  (2025-10-10T09:21:21Z)
• MiSTR: Multi-Modal iEEG-to-Speech Synthesis with Transformer-Based Prosody Prediction and Neural Phase Reconstruction [7.233654849867492]
  We introduce MiSTR, a deep-learning framework that integrates temporal, spectral, and neurophysiological representations of iEEG signals.<n> evaluated on a public iEEG dataset, MiSTR achieves state-of-the-art speech intelligibility.
  arXiv  Detail & Related papers  (2025-08-05T07:12:52Z)
• Latent Diffusion Model Based Denoising Receiver for 6G Semantic Communication: From Stochastic Differential Theory to Application [11.385703484113552]
  We propose a novel semantic communication framework empowered by generative artificial intelligence (GAI)<n>A latent diffusion model (LDM)-based semantic communication framework is proposed that combines a variational autoencoder for semantic features extraction.<n>The proposed system is a training-free framework that supports zero-shot generalization, and achieves superior performance under low-SNR and out-of-distribution conditions.
  arXiv  Detail & Related papers  (2025-06-06T03:20:32Z)
• High-Fidelity Speech Synthesis with Minimal Supervision: All Using Diffusion Models [56.00939852727501]
  Minimally-supervised speech synthesis decouples TTS by combining two types of discrete speech representations. Non-autoregressive framework enhances controllability, and duration diffusion model enables diversified prosodic expression.
  arXiv  Detail & Related papers  (2023-09-27T09:27:03Z)
• Discretization and Re-synthesis: an alternative method to solve the Cocktail Party Problem [65.25725367771075]
  This study demonstrates, for the first time, that the synthesis-based approach can also perform well on this problem. Specifically, we propose a novel speech separation/enhancement model based on the recognition of discrete symbols. By utilizing the synthesis model with the input of discrete symbols, after the prediction of discrete symbol sequence, each target speech could be re-synthesized.
  arXiv  Detail & Related papers  (2021-12-17T08:35:40Z)
• A Study on Speech Enhancement Based on Diffusion Probabilistic Model [63.38586161802788]
  We propose a diffusion probabilistic model-based speech enhancement model (DiffuSE) model that aims to recover clean speech signals from noisy signals. The experimental results show that DiffuSE yields performance that is comparable to related audio generative models on the standardized Voice Bank corpus task.
  arXiv  Detail & Related papers  (2021-07-25T19:23:18Z)
• Simultaneous Denoising and Dereverberation Using Deep Embedding Features [64.58693911070228]
  We propose a joint training method for simultaneous speech denoising and dereverberation using deep embedding features. At the denoising stage, the DC network is leveraged to extract noise-free deep embedding features. At the dereverberation stage, instead of using the unsupervised K-means clustering algorithm, another neural network is utilized to estimate the anechoic speech.
  arXiv  Detail & Related papers  (2020-04-06T06:34:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.