Cross-Subject Semantic Decoding with Shared-Space Alignment for Generalized Neural Representation Learning

• URL: http://arxiv.org/abs/2607.19394v1
• Date: Fri, 03 Jul 2026 03:59:16 GMT
• Title: Cross-Subject Semantic Decoding with Shared-Space Alignment for Generalized Neural Representation Learning
• Abstract summary: Generalizing across subjects remains challenging in invasive neural recordings.<n>We propose a cross-subject semantic decoding framework that aligns neural responses to speech perception from multiple subjects into a shared latent space.
• Score: 80.18561652862725
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generalizing across subjects remains challenging in invasive neural recordings because electrode configurations, anatomical structures, and neural signal patterns vary substantially across individuals. To investigate such inter-subject variability, we propose a cross-subject semantic decoding framework that aligns neural responses to speech perception from multiple subjects into a shared latent space and learns a mapping from the aligned neural representations to contextual embeddings. More specifically, using electrocorticography data collected during natural language comprehension, we estimate the shared space using the shared response model and train a decoder to predict contextual semantic embeddings from projected neural responses. For a held-out subject, we estimate a subject-specific projection into the predefined shared space, and directly apply the pretrained decoder without any retraining. Experimental results demonstrate that the proposed framework consistently outperforms baseline methods across evaluation settings and exhibits a reduced performance drop from source subject to held-out subject testing, indicating improved cross-subject generalization. These results suggest that aligning neural activity into a shared latent space, while decoding in a semantic embedding space, provides an effective strategy for improving cross-subject generalization by reducing subject-specific differences in neural responses while effectively capturing shared stimulus-related representations.

Related papers

• Probe-EM: Targeted Neuron Tracing via Training-Free Semantic Verification [13.282533076475383]
  We propose a training-free, targeted neuron tracing framework for processing terabyte- or petabyte-scale electron microscopy data.<n>Specifically, we introduce a skeleton-guided Heuristic Spatial Search paradigm that leverages geometric priors to iteratively reconstruct neuronal morphologies.<n>We further develop a Dimension-Aware Semantic Verification strategy built upon the foundation model NeuroSAM 2.
  arXiv  Detail & Related papers  (2026-07-06T05:56:26Z)
• Towards Voxel Spacing Consistency for Medical Image Segmentation [56.992050146446495]
  We study the correlation between image resampling and segmentation, and propose Consispace, a semantic-aware resampling framework.<n> Consispace achieves consistent voxel spacing in the axial direction while preserving anatomical and semantic consistency.
  arXiv  Detail & Related papers  (2026-06-30T15:42:23Z)
• Task-guided cross-subject latent alignment: a multi-encoder-decoder VAE [1.936594951038608]
  We introduce a Multi-Encoder-Decoder Variational Autoencoder (MED-VAE) that achieves cross-subject alignment without shared stimuli.<n>We show that MED-VAE creates common latent spaces with superior semantic organisation, achieving higher cross-subject alignment than common methods.<n>We also show that this superior alignment directly enables cross-subject neural prediction, as demonstrated via cross-subject image decoding.
  arXiv  Detail & Related papers  (2026-06-14T19:30:11Z)
• Flow Matching with In-Context Priors for Out-of-Distribution Brain Dynamics [0.0]
  We propose a per-timestep conditioned diffusion transformer for generating realistic fMRI brain dynamics during unseen cognitive tasks.<n>We evaluate across hundreds of held-out task conditions and characterize predictive performance in relation to the training manifold.<n>This is the first generative model of whole-cortex fMRI dynamics for unseen cognitive tasks.
  arXiv  Detail & Related papers  (2026-06-10T09:15:33Z)
• Uncovering Semantic Selectivity of Latent Groups in Higher Visual Cortex with Mutual Information-Guided Diffusion [19.983291706164923]
  We present MIG-Vis, a method to visualize and validate the visual-semantic attributes encoded in neural latent subspaces.<n>We validate MIG-Vis on multi-session neural spiking datasets from the inferior temporal (IT) cortex of two macaques.
  arXiv  Detail & Related papers  (2025-10-02T16:33:40Z)
• Discovering Chunks in Neural Embeddings for Interpretability [53.80157905839065]
  We propose leveraging the principle of chunking to interpret artificial neural population activities.<n>We first demonstrate this concept in recurrent neural networks (RNNs) trained on artificial sequences with imposed regularities.<n>We identify similar recurring embedding states corresponding to concepts in the input, with perturbations to these states activating or inhibiting the associated concepts.
  arXiv  Detail & Related papers  (2025-02-03T20:30:46Z)
• TractGeoNet: A geometric deep learning framework for pointwise analysis of tract microstructure to predict language assessment performance [66.43360974979386]
  We propose a geometric deep-learning-based framework, TractGeoNet, for performing regression using diffusion magnetic resonance imaging (dMRI) tractography. To improve regression performance, we propose a novel loss function, the Paired-Siamese Regression loss. We evaluate the effectiveness of the proposed method by predicting individual performance on two neuropsychological assessments of language.
  arXiv  Detail & Related papers  (2023-07-08T14:10:37Z)
• Contrastive-Signal-Dependent Plasticity: Self-Supervised Learning in Spiking Neural Circuits [61.94533459151743]
  This work addresses the challenge of designing neurobiologically-motivated schemes for adjusting the synapses of spiking networks. Our experimental simulations demonstrate a consistent advantage over other biologically-plausible approaches when training recurrent spiking networks.
  arXiv  Detail & Related papers  (2023-03-30T02:40:28Z)
• White Matter Tracts are Point Clouds: Neuropsychological Score Prediction and Critical Region Localization via Geometric Deep Learning [68.5548609642999]
  We propose a deep-learning-based framework for neuropsychological score prediction using white matter tract data. We represent the arcuate fasciculus (AF) as a point cloud with microstructure measurements at each point. We improve prediction performance with the proposed Paired-Siamese Loss that utilizes information about differences between continuous neuropsychological scores.
  arXiv  Detail & Related papers  (2022-07-06T02:03:28Z)
• Deep Neural Convolutive Matrix Factorization for Articulatory Representation Decomposition [48.56414496900755]
  This work uses a neural implementation of convolutive sparse matrix factorization to decompose the articulatory data into interpretable gestures and gestural scores. Phoneme recognition experiments were additionally performed to show that gestural scores indeed code phonological information successfully.
  arXiv  Detail & Related papers  (2022-04-01T14:25:19Z)
• A shared neural encoding model for the prediction of subject-specific fMRI response [17.020869686284165]
  We propose a shared convolutional neural encoding method that accounts for individual-level differences. Our method leverages multi-subject data to improve the prediction of subject-specific responses evoked by visual or auditory stimuli.
  arXiv  Detail & Related papers  (2020-06-29T04:10:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.