Condition-Invariant fMRI Decoding of Speech Intelligibility with Deep State Space Model

• URL: http://arxiv.org/abs/2511.01868v1
• Date: Tue, 21 Oct 2025 09:04:09 GMT
• Title: Condition-Invariant fMRI Decoding of Speech Intelligibility with Deep State Space Model
• Abstract summary: Clarifying the neural basis of speech intelligibility is critical for computational neuroscience and digital speech processing.<n>Recent neuroimaging studies have shown that intelligibility modulates cortical activity beyond simple acoustics.<n>We propose a novel architecture for decoding intelligibility from fMRI signals, specifically tailored to their high-dimensional temporal structure.
• Score: 14.428950312699149
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Clarifying the neural basis of speech intelligibility is critical for computational neuroscience and digital speech processing. Recent neuroimaging studies have shown that intelligibility modulates cortical activity beyond simple acoustics, primarily in the superior temporal and inferior frontal gyri. However, previous studies have been largely confined to clean speech, leaving it unclear whether the brain employs condition-invariant neural codes across diverse listening environments. To address this gap, we propose a novel architecture built upon a deep state space model for decoding intelligibility from fMRI signals, specifically tailored to their high-dimensional temporal structure. We present the first attempt to decode intelligibility across acoustically distinct conditions, showing our method significantly outperforms classical approaches. Furthermore, region-wise analysis highlights contributions from auditory, frontal, and parietal regions, and cross-condition transfer indicates the presence of condition-invariant neural codes, thereby advancing understanding of abstract linguistic representations in the brain.

Related papers

• Cross-Subject Semantic Decoding with Shared-Space Alignment for Generalized Neural Representation Learning [80.18561652862725]
  Generalizing across subjects remains challenging in invasive neural recordings.<n>We propose a cross-subject semantic decoding framework that aligns neural responses to speech perception from multiple subjects into a shared latent space.
  arXiv  Detail & Related papers  (2026-07-03T03:59:16Z)
• BrainJanus: A Unified Model for Understanding and Generation across Brain, Vision, and Language [65.1211456110319]
  BrainJanus is the first unified brain model that integrates brain, vision, and language within a single framework.<n>Our framework exhibits zero-shot generalization preserves and interpretable biological topography, highlighting its potential as a general-purpose brain modeling paradigm.
  arXiv  Detail & Related papers  (2026-06-29T14:02:15Z)
• MindVoice: Reconstructing Intelligible Speech from Non-invasive Neural Signals with Pretrained Priors [32.688206260149386]
  We introduce MindVoice, a neuro-to-speech reconstruction framework that uses pretrained models to compensate for the incomplete semantic and acoustic information in neural recordings.<n> MindVoice disentangles reconstruction into two complementary pathways: one recovers high-level semantic content, while the other estimates fine-grained acoustic attributes.<n>These results show that pretrained priors provide a principled way to bridge the gap between noisy neural recordings and natural speech, highlighting a promising attempt for auditory neuroscience research and non-invasive speech brain-computer interfaces.
  arXiv  Detail & Related papers  (2026-05-29T11:38:38Z)
• Decoding Predictive Inference in Visual Language Processing via Spatiotemporal Neural Coherence [2.208251557767776]
  We present a machine learning framework for decoding neural responses to visual language stimuli in Deaf signers.<n>Our results reveal distributed left-hemispheric and low-frequency coherence as key features in language comprehension.<n>This work demonstrates a novel approach for probing experience-driven generative models of perception in the brain.
  arXiv  Detail & Related papers  (2025-12-24T04:19:20Z)
• Disentangling Neurodegeneration with Brain Age Gap Prediction Models: A Graph Signal Processing Perspective [89.99666725996975]
  The brain age gap prediction (BAGP) models estimate the difference between a person's predicted brain age from data and their chronological age.<n>This tutorial article provides an overview of BAGP and introduces a principled framework for this application based on recent advancements in graph signal processing (GSP)<n>VNNs offer strong theoretical grounding and operational interpretability, enabling robust estimation of brain age gap predictions.
  arXiv  Detail & Related papers  (2025-10-14T17:44:45Z)
• Neuro2Semantic: A Transfer Learning Framework for Semantic Reconstruction of Continuous Language from Human Intracranial EEG [11.531598524209969]
  We introduce Neuro2Semantic, a novel framework that reconstructs the semantic content of perceived speech from intracranial EEG (iEEG) recordings.<n>Our approach consists of two phases: first, an LSTM-based adapter aligns neural signals with pre-trained text embeddings; second, a corrector module generates continuous, natural text directly from these aligned embeddings.<n>Neuro2Semantic achieves strong performance with as little as 30 minutes of neural data, outperforming a recent state-of-the-art method in low-data settings.
  arXiv  Detail & Related papers  (2025-05-31T04:17:19Z)
• BrainStratify: Coarse-to-Fine Disentanglement of Intracranial Neural Dynamics [8.36470471250669]
  Decoding speech directly from neural activity is a central goal in brain-computer interface (BCI) research.<n>In recent years, exciting advances have been made through the growing use of intracranial field potential recordings, such as stereo-ElectroEncephaloGraphy (sEEG) and ElectroCorticoGraphy (ECoG)<n>These neural signals capture rich population-level activity but present key challenges: (i) task-relevant neural signals are sparsely distributed across sEEG electrodes, and (ii) they are often entangled with task-irrelevant neural signals in both sEEG and ECo
  arXiv  Detail & Related papers  (2025-05-26T19:36:39Z)
• Brain2Text Decoding Model Reveals the Neural Mechanisms of Visual Semantic Processing [0.17188280334580194]
  We present a novel framework that directly decodes fMRI signals into textual descriptions of viewed natural images.<n>Our novel deep learning model, trained without visual information, achieves state-of-the-art semantic decoding performance.
  arXiv  Detail & Related papers  (2025-03-15T07:28:02Z)
• Towards Unified Neural Decoding of Perceived, Spoken and Imagined Speech from EEG Signals [1.33134751838052]
  This research investigated the effectiveness of deep learning models for non-invasive neural signal decoding. It focused on distinguishing between different speech paradigms, including perceived, overt, whispered, and imagined speech.
  arXiv  Detail & Related papers  (2024-11-14T07:20:08Z)
• Large Language Model-based FMRI Encoding of Language Functions for Subjects with Neurocognitive Disorder [53.575426835313536]
  This paper explores language-related functional changes in older NCD adults using LLM-based fMRI encoding and brain scores. We analyze the correlation between brain scores and cognitive scores at both whole-brain and language-related ROI levels. Our findings reveal that higher cognitive abilities correspond to better brain scores, with correlations peaking in the middle temporal gyrus.
  arXiv  Detail & Related papers  (2024-07-15T01:09:08Z)
• Interpretable Spatio-Temporal Embedding for Brain Structural-Effective Network with Ordinary Differential Equation [56.34634121544929]
  In this study, we first construct the brain-effective network via the dynamic causal model. We then introduce an interpretable graph learning framework termed Spatio-Temporal Embedding ODE (STE-ODE) This framework incorporates specifically designed directed node embedding layers, aiming at capturing the dynamic interplay between structural and effective networks.
  arXiv  Detail & Related papers  (2024-05-21T20:37:07Z)
• Du-IN: Discrete units-guided mask modeling for decoding speech from Intracranial Neural signals [5.283718601431859]
  Invasive brain-computer interfaces with Electrocorticography (ECoG) have shown promise for high-performance speech decoding in medical applications. We developed the Du-IN model, which extracts contextual embeddings based on region-level tokens through discrete codex-guided mask modeling. Our model achieves state-of-the-art performance on the 61-word classification task, surpassing all baselines.
  arXiv  Detail & Related papers  (2024-05-19T06:00:36Z)
• Exploring neural oscillations during speech perception via surrogate gradient spiking neural networks [59.38765771221084]
  We present a physiologically inspired speech recognition architecture compatible and scalable with deep learning frameworks. We show end-to-end gradient descent training leads to the emergence of neural oscillations in the central spiking neural network. Our findings highlight the crucial inhibitory role of feedback mechanisms, such as spike frequency adaptation and recurrent connections, in regulating and synchronising neural activity to improve recognition performance.
  arXiv  Detail & Related papers  (2024-04-22T09:40:07Z)
• MindBridge: A Cross-Subject Brain Decoding Framework [60.58552697067837]
  Brain decoding aims to reconstruct stimuli from acquired brain signals. Currently, brain decoding is confined to a per-subject-per-model paradigm. We present MindBridge, that achieves cross-subject brain decoding by employing only one model.
  arXiv  Detail & Related papers  (2024-04-11T15:46:42Z)
• Self-supervised models of audio effectively explain human cortical responses to speech [71.57870452667369]
  We capitalize on the progress of self-supervised speech representation learning to create new state-of-the-art models of the human auditory system. We show that these results show that self-supervised models effectively capture the hierarchy of information relevant to different stages of speech processing in human cortex.
  arXiv  Detail & Related papers  (2022-05-27T22:04:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.