DEEPER: Dense Electroencephalography Passage Retrieval

• URL: http://arxiv.org/abs/2412.06695v2
• Date: Fri, 24 Jan 2025 15:37:53 GMT
• Title: DEEPER: Dense Electroencephalography Passage Retrieval
• Authors: Niall McGuire, Yashar Moshfeghi,
• Abstract summary: This paper introduces DEEPER Dense EEG Passage Retrieval, a novel framework that bypasses the need for explicit query translation. Our approach employs dense retrieval architectures to create a unified semantic space where both EEG signals and text passages can be effectively compared.
• Score: 6.084958172018792
• License:
• Abstract: A fundamental challenge in Information Retrieval (IR) is the cognitive burden of translating internal information needs into explicit textual queries. This translation barrier particularly affects users with undefined information needs or those who face physical constraints in traditional text input methods. While Brain-Machine Interfaces (BMIs) have emerged as a potential solution for direct neural query interpretation, existing approaches that attempt to convert brain signals into text queries have demonstrated limited success in capturing the complexity of neural semantic patterns. This paper introduces DEEPER Dense EEG Passage Retrieval, a novel framework that bypasses the need for explicit query translation by directly mapping electroencephalography (EEG) signals to relevant text passages. Our approach employs dense retrieval architectures to create a unified semantic space where both EEG signals and text passages can be effectively compared. Experimental evaluation on the ZuCo dataset shows that DEEPER substantially outperforms current EEG-to-text baselines, achieving nearly 5x improvement in retrieval precision while demonstrating robust performance across a diverse set of 30 participants. Through detailed ablation analysis, we identify key architectural components, including specialized neural encoders and strategic negative sampling techniques, that enable effective cross-modal semantic alignment. Our findings demonstrate the feasibility of direct EEG passage retrieval and suggest new possibilities for developing IR systems that can more naturally interface with users' cognitive processes.

Related papers

• CognitionCapturer: Decoding Visual Stimuli From Human EEG Signal With Multimodal Information [61.1904164368732]
  We propose CognitionCapturer, a unified framework that fully leverages multimodal data to represent EEG signals. Specifically, CognitionCapturer trains Modality Experts for each modality to extract cross-modal information from the EEG modality. The framework does not require any fine-tuning of the generative models and can be extended to incorporate more modalities.
  arXiv  Detail & Related papers  (2024-12-13T16:27:54Z)
• SEE: Semantically Aligned EEG-to-Text Translation [5.460650382586978]
  Decoding neurophysiological signals into language is of great research interest within brain-computer interface (BCI) applications. Current EEG-to-Text decoding approaches face challenges due to the huge domain gap between EEG recordings and raw texts. We propose SEE: Semantically Aligned EEG-to-Text Translation, a novel method aimed at improving EEG-to-Text decoding.
  arXiv  Detail & Related papers  (2024-09-14T05:37:15Z)
• Con-ReCall: Detecting Pre-training Data in LLMs via Contrastive Decoding [118.75567341513897]
  Existing methods typically analyze target text in isolation or solely with non-member contexts. We propose Con-ReCall, a novel approach that leverages the asymmetric distributional shifts induced by member and non-member contexts.
  arXiv  Detail & Related papers  (2024-09-05T09:10:38Z)
• Towards Linguistic Neural Representation Learning and Sentence Retrieval from Electroencephalogram Recordings [27.418738450536047]
  We propose a two-step pipeline for converting EEG signals into sentences. We first confirm that word-level semantic information can be learned from EEG data recorded during natural reading. We employ a training-free retrieval method to retrieve sentences based on the predictions from the EEG encoder.
  arXiv  Detail & Related papers  (2024-08-08T03:40:25Z)
• EEG decoding with conditional identification information [7.873458431535408]
  Decoding EEG signals is crucial for unraveling human brain and advancing brain-computer interfaces. Traditional machine learning algorithms have been hindered by the high noise levels and inherent inter-person variations in EEG signals. Recent advances in deep neural networks (DNNs) have shown promise, owing to their advanced nonlinear modeling capabilities.
  arXiv  Detail & Related papers  (2024-03-21T13:38:59Z)
• Re-mine, Learn and Reason: Exploring the Cross-modal Semantic Correlations for Language-guided HOI detection [57.13665112065285]
  Human-Object Interaction (HOI) detection is a challenging computer vision task. We present a framework that enhances HOI detection by incorporating structured text knowledge.
  arXiv  Detail & Related papers  (2023-07-25T14:20:52Z)
• fMRI from EEG is only Deep Learning away: the use of interpretable DL to unravel EEG-fMRI relationships [68.8204255655161]
  We present an interpretable domain grounded solution to recover the activity of several subcortical regions from multichannel EEG data. We recover individual spatial and time-frequency patterns of scalp EEG predictive of the hemodynamic signal in the subcortical nuclei.
  arXiv  Detail & Related papers  (2022-10-23T15:11:37Z)
• Multimodal Emotion Recognition using Transfer Learning from Speaker Recognition and BERT-based models [53.31917090073727]
  We propose a neural network-based emotion recognition framework that uses a late fusion of transfer-learned and fine-tuned models from speech and text modalities. We evaluate the effectiveness of our proposed multimodal approach on the interactive emotional dyadic motion capture dataset.
  arXiv  Detail & Related papers  (2022-02-16T00:23:42Z)
• Open Vocabulary Electroencephalography-To-Text Decoding and Zero-shot Sentiment Classification [78.120927891455]
  State-of-the-art brain-to-text systems have achieved great success in decoding language directly from brain signals using neural networks. In this paper, we extend the problem to open vocabulary Electroencephalography(EEG)-To-Text Sequence-To-Sequence decoding and zero-shot sentence sentiment classification on natural reading tasks. Our model achieves a 40.1% BLEU-1 score on EEG-To-Text decoding and a 55.6% F1 score on zero-shot EEG-based ternary sentiment classification, which significantly outperforms supervised baselines.
  arXiv  Detail & Related papers  (2021-12-05T21:57:22Z)
• Subject Independent Emotion Recognition using EEG Signals Employing Attention Driven Neural Networks [2.76240219662896]
  A novel deep learning framework capable of doing subject-independent emotion recognition is presented. A convolutional neural network (CNN) with attention framework is presented for performing the task. The proposed approach has been validated using publicly available datasets.
  arXiv  Detail & Related papers  (2021-06-07T09:41:15Z)
• Leveraging Semantic Scene Characteristics and Multi-Stream Convolutional Architectures in a Contextual Approach for Video-Based Visual Emotion Recognition in the Wild [31.40575057347465]
  We tackle the task of video-based visual emotion recognition in the wild. Standard methodologies that rely solely on the extraction of bodily and facial features often fall short of accurate emotion prediction. We aspire to alleviate this problem by leveraging visual context in the form of scene characteristics and attributes.
  arXiv  Detail & Related papers  (2021-05-16T17:31:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.