Multimodal Brain-Computer Interfaces: AI-powered Decoding Methodologies

• URL: http://arxiv.org/abs/2502.02830v1
• Date: Wed, 05 Feb 2025 02:14:08 GMT
• Title: Multimodal Brain-Computer Interfaces: AI-powered Decoding Methodologies
• Authors: Siyang Li, Hongbin Wang, Xiaoqing Chen, Dongrui Wu,
• Abstract summary: This review highlights the core decoding algorithms that enable multimodal BCIs. We emphasize algorithmic advancements in cross-modality mapping, sequential modeling, besides classic multi-modality fusion. The current literature of BCI applications on visual, speech, and affective decoding are comprehensively explored.
• Score: 21.599095979583414
• License:
• Abstract: Brain-computer interfaces (BCIs) enable direct communication between the brain and external devices. This review highlights the core decoding algorithms that enable multimodal BCIs, including a dissection of the elements, a unified view of diversified approaches, and a comprehensive analysis of the present state of the field. We emphasize algorithmic advancements in cross-modality mapping, sequential modeling, besides classic multi-modality fusion, illustrating how these novel AI approaches enhance decoding of brain data. The current literature of BCI applications on visual, speech, and affective decoding are comprehensively explored. Looking forward, we draw attention on the impact of emerging architectures like multimodal Transformers, and discuss challenges such as brain data heterogeneity and common errors. This review also serves as a bridge in this interdisciplinary field for experts with neuroscience background and experts that study AI, aiming to provide a comprehensive understanding for AI-powered multimodal BCIs.

Related papers

• A Survey on Bridging EEG Signals and Generative AI: From Image and Text to Beyond [4.720913027054481]
  Integration of Brain-Computer Interfaces (BCIs) and Generative Artificial Intelligence (GenAI) has opened new frontiers in brain signal decoding. Recent advances in deep learning, including Generative Adversarial Networks (GANs) and Transformer-based Large Language Models (LLMs), have significantly improved EEG-based generation of images, text, and speech.
  arXiv  Detail & Related papers  (2025-02-17T17:16:41Z)
• BrainMAP: Learning Multiple Activation Pathways in Brain Networks [77.15180533984947]
  We introduce a novel framework BrainMAP to learn Multiple Activation Pathways in Brain networks. Our framework enables explanatory analyses of crucial brain regions involved in tasks.
  arXiv  Detail & Related papers  (2024-12-23T09:13:35Z)
• Brain-Conditional Multimodal Synthesis: A Survey and Taxonomy [18.130004804879896]
  Key to multimodal synthesis technology is to establish the mapping relationship between different modalities. Brian-conditional multimodal synthesis refers to decoding brain signals back to perceptual experience. This survey comprehensively examines the emerging field of AIGC-based Brain-conditional Multimodal Synthesis, termed AIGC-Brain.
  arXiv  Detail & Related papers  (2023-12-31T09:00:40Z)
• Re-mine, Learn and Reason: Exploring the Cross-modal Semantic Correlations for Language-guided HOI detection [57.13665112065285]
  Human-Object Interaction (HOI) detection is a challenging computer vision task. We present a framework that enhances HOI detection by incorporating structured text knowledge.
  arXiv  Detail & Related papers  (2023-07-25T14:20:52Z)
• The evolution of AI approaches for motor imagery EEG-based BCIs [2.294014185517203]
  The Motor Imagery (MI) electroencephalography (EEG) based Brain Computer Interfaces (BCIs) allow the direct communication between humans and machines. These systems open the possibility of developing applications that could span from the medical field to the entertainment industry. Artificial Intelligence (AI) approaches become of fundamental importance especially when wanting to provide a correct and coherent feedback to BCI users.
  arXiv  Detail & Related papers  (2022-10-11T07:42:54Z)
• Foundations and Recent Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions [68.6358773622615]
  This paper provides an overview of the computational and theoretical foundations of multimodal machine learning. We propose a taxonomy of 6 core technical challenges: representation, alignment, reasoning, generation, transference, and quantification. Recent technical achievements will be presented through the lens of this taxonomy, allowing researchers to understand the similarities and differences across new approaches.
  arXiv  Detail & Related papers  (2022-09-07T19:21:19Z)
• Multimodal foundation models are better simulators of the human brain [65.10501322822881]
  We present a newly-designed multimodal foundation model pre-trained on 15 million image-text pairs. We find that both visual and lingual encoders trained multimodally are more brain-like compared with unimodal ones.
  arXiv  Detail & Related papers  (2022-08-17T12:36:26Z)
• DIME: Fine-grained Interpretations of Multimodal Models via Disentangled Local Explanations [119.1953397679783]
  We focus on advancing the state-of-the-art in interpreting multimodal models. Our proposed approach, DIME, enables accurate and fine-grained analysis of multimodal models.
  arXiv  Detail & Related papers  (2022-03-03T20:52:47Z)
• Siamese Neural Networks for EEG-based Brain-computer Interfaces [18.472950822801362]
  We propose a new EEG processing and feature extraction paradigm based on Siamese neural networks. Siamese architecture is developed based on Convolutional Neural Networks (CNN) and provides a binary output on the similarity of two inputs. The efficacy of this architecture is evaluated on a 4-class Motor Imagery dataset from Brain-computer Interfaces (BCI) Competition IV-2a.
  arXiv  Detail & Related papers  (2020-02-03T17:31:39Z)
• EEG-based Brain-Computer Interfaces (BCIs): A Survey of Recent Studies on Signal Sensing Technologies and Computational Intelligence Approaches and their Applications [65.32004302942218]
  Brain-Computer Interface (BCI) is a powerful communication tool between users and systems. Recent technological advances have increased interest in electroencephalographic (EEG) based BCI for translational and healthcare applications.
  arXiv  Detail & Related papers  (2020-01-28T10:36:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.