Domain Adaptation-Enhanced Searchlight: Enabling classification of brain states from visual perception to mental imagery

• URL: http://arxiv.org/abs/2408.01163v3
• Date: Mon, 27 Jan 2025 10:34:47 GMT
• Title: Domain Adaptation-Enhanced Searchlight: Enabling classification of brain states from visual perception to mental imagery
• Authors: Alexander Olza, David Soto, Roberto Santana,
• Abstract summary: Domain Adaptation (DA) can be used to enhance imagery prediction in fMRI scans.<n>We develop several models to improve imagery prediction, comparing different DA methods.<n>Our DA-enhanced searchlight predicts imagery contents in a highly distributed set of brain regions, including the visual cortex and the frontoparietal cortex.
• Score: 46.27328641616778
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: In cognitive neuroscience and brain-computer interface research, accurately predicting imagined stimuli is crucial. This study investigates the effectiveness of Domain Adaptation (DA) in enhancing imagery prediction using primarily visual data from fMRI scans of 18 subjects. Initially, we train a baseline model on visual stimuli to predict imagined stimuli, utilizing data from 14 brain regions. We then develop several models to improve imagery prediction, comparing different DA methods. Our results demonstrate that DA significantly enhances imagery prediction in binary classification on our dataset, as well as in multiclass classification on a publicly available dataset. We then conduct a DA-enhanced searchlight analysis, followed by permutation-based statistical tests to identify brain regions where imagery decoding is consistently above chance across subjects. Our DA-enhanced searchlight predicts imagery contents in a highly distributed set of brain regions, including the visual cortex and the frontoparietal cortex, thereby outperforming standard cross-domain classification methods. The complete code and data for this paper have been made openly available for the use of the scientific community.

Related papers

• Transformer brain encoders explain human high-level visual responses [0.0]
  We study how retinotopic visual features can be dynamically routed to category-selective areas in high-level visual processing.<n>We show that this computational motif is significantly more powerful than alternative methods in predicting brain activity during natural scene viewing.
  arXiv  Detail & Related papers  (2025-05-22T22:48:15Z)
• Decoding Visual Experience and Mapping Semantics through Whole-Brain Analysis Using fMRI Foundation Models [10.615012396285337]
  We develop algorithms to enhance our understanding of visual processes by incorporating whole-brain activation maps. We first compare our method with state-of-the-art approaches to decoding visual processing and show improved predictive semantic accuracy by 43%.
  arXiv  Detail & Related papers  (2024-11-11T16:51:17Z)
• Generalizability analysis of deep learning predictions of human brain responses to augmented and semantically novel visual stimuli [0.6259089590341245]
  The purpose of this work is to investigate the soundness and utility of a neural network-based approach as a framework for exploring the impact of image enhancement techniques on visual cortex activation. We prepare a set of state-of-the-art brain encoding models, selected among the top 10 methods that participated in The Algonauts Project 2023 Challenge. We analyze their ability to make valid predictions about the effects of various image enhancement techniques on neural responses.
  arXiv  Detail & Related papers  (2024-10-06T14:29:02Z)
• Visual Prompt Engineering for Vision Language Models in Radiology [0.17183214167143138]
  Contrastive Language-Image Pretraining (CLIPP) offers a solution by enabling zero-shot classification through large-scale pretraining. Visual markers improve AUROC2013$ by up to 0.185, highlighting their effectiveness enhancing classification performance. We release our code and preprocessing pipeline, providing a reference point for future work on localized classification in medical imaging.
  arXiv  Detail & Related papers  (2024-08-28T13:53:27Z)
• Multimodal Learning and Cognitive Processes in Radiology: MedGaze for Chest X-ray Scanpath Prediction [10.388541520456714]
  Our proposed system aims to predict eye gaze sequences from radiology reports and CXR images. Our model predicts fixation coordinates and durations critical for medical scanpath prediction, outperforming existing models in the computer vision community. Based on the radiologist's evaluation, MedGaze can generate human-like gaze sequences with a high focus on relevant regions.
  arXiv  Detail & Related papers  (2024-06-28T06:38:58Z)
• A Textbook Remedy for Domain Shifts: Knowledge Priors for Medical Image Analysis [48.84443450990355]
  Deep networks have achieved broad success in analyzing natural images, when applied to medical scans, they often fail in unexcepted situations. We investigate this challenge and focus on model sensitivity to domain shifts, such as data sampled from different hospitals or data confounded by demographic variables such as sex, race, etc, in the context of chest X-rays and skin lesion images. Taking inspiration from medical training, we propose giving deep networks a prior grounded in explicit medical knowledge communicated in natural language.
  arXiv  Detail & Related papers  (2024-05-23T17:55:02Z)
• BrainSCUBA: Fine-Grained Natural Language Captions of Visual Cortex Selectivity [6.285481522918523]
  We introduce a data-driven method that generates natural language descriptions for images predicted to maximally activate individual voxels of interest. We validate our method through fine-grained voxel-level captioning across higher-order visual regions. To demonstrate how our method enables scientific discovery, we perform exploratory investigations on the distribution of "person" representations in the brain.
  arXiv  Detail & Related papers  (2023-10-06T17:59:53Z)
• Performance of GAN-based augmentation for deep learning COVID-19 image classification [57.1795052451257]
  The biggest challenge in the application of deep learning to the medical domain is the availability of training data. Data augmentation is a typical methodology used in machine learning when confronted with a limited data set. In this work, a StyleGAN2-ADA model of Generative Adversarial Networks is trained on the limited COVID-19 chest X-ray image set.
  arXiv  Detail & Related papers  (2023-04-18T15:39:58Z)
• FAST-AID Brain: Fast and Accurate Segmentation Tool using Artificial Intelligence Developed for Brain [0.8376091455761259]
  A novel deep learning method is proposed for fast and accurate segmentation of the human brain into 132 regions. The proposed model uses an efficient U-Net-like network and benefits from the intersection points of different views and hierarchical relations. The proposed method can be applied to brain MRI data including skull or any other artifacts without preprocessing the images or a drop in performance.
  arXiv  Detail & Related papers  (2022-08-30T16:06:07Z)
• Feature visualization for convolutional neural network models trained on neuroimaging data [0.0]
  We show for the first time results using feature visualization of convolutional neural networks (CNNs) We have trained CNNs for different tasks including sex classification and artificial lesion classification based on structural magnetic resonance imaging (MRI) data. The resulting images reveal the learned concepts of the artificial lesions, including their shapes, but remain hard to interpret for abstract features in the sex classification task.
  arXiv  Detail & Related papers  (2022-03-24T15:24:38Z)
• Voxel-level Importance Maps for Interpretable Brain Age Estimation [70.5330922395729]
  We focus on the task of brain age regression from 3D brain Magnetic Resonance (MR) images using a Convolutional Neural Network, termed prediction model. We implement a noise model which aims to add as much noise as possible to the input without harming the performance of the prediction model. We test our method on 13,750 3D brain MR images from the UK Biobank, and our findings are consistent with the existing neuropathology literature.
  arXiv  Detail & Related papers  (2021-08-11T18:08:09Z)
• Classifying Eye-Tracking Data Using Saliency Maps [8.524684315458245]
  This paper proposes a visual saliency based novel feature extraction method for automatic and quantitative classification of eye-tracking data. Comparing the saliency amplitudes, similarity and dissimilarity of saliency maps with the corresponding eye fixations maps gives an extra dimension of information which is effectively utilized to generate discriminative features to classify the eye-tracking data.
  arXiv  Detail & Related papers  (2020-10-24T15:18:07Z)
• Fader Networks for domain adaptation on fMRI: ABIDE-II study [68.5481471934606]
  We use 3D convolutional autoencoders to build the domain irrelevant latent space image representation and demonstrate this method to outperform existing approaches on ABIDE data.
  arXiv  Detail & Related papers  (2020-10-14T16:50:50Z)
• Domain Shift in Computer Vision models for MRI data analysis: An Overview [64.69150970967524]
  Machine learning and computer vision methods are showing good performance in medical imagery analysis. Yet only a few applications are now in clinical use. Poor transferability of themodels to data from different sources or acquisition domains is one of the reasons for that.
  arXiv  Detail & Related papers  (2020-10-14T16:34:21Z)
• Modeling Shared Responses in Neuroimaging Studies through MultiView ICA [94.31804763196116]
  Group studies involving large cohorts of subjects are important to draw general conclusions about brain functional organization. We propose a novel MultiView Independent Component Analysis model for group studies, where data from each subject are modeled as a linear combination of shared independent sources plus noise. We demonstrate the usefulness of our approach first on fMRI data, where our model demonstrates improved sensitivity in identifying common sources among subjects.
  arXiv  Detail & Related papers  (2020-06-11T17:29:53Z)
• Pathological Retinal Region Segmentation From OCT Images Using Geometric Relation Based Augmentation [84.7571086566595]
  We propose improvements over previous GAN-based medical image synthesis methods by jointly encoding the intrinsic relationship of geometry and shape. The proposed method outperforms state-of-the-art segmentation methods on the public RETOUCH dataset having images captured from different acquisition procedures.
  arXiv  Detail & Related papers  (2020-03-31T11:50:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.