EEG-Language Modeling for Pathology Detection

• URL: http://arxiv.org/abs/2409.07480v2
• Date: Wed, 2 Oct 2024 15:02:13 GMT
• Title: EEG-Language Modeling for Pathology Detection
• Authors: Sam Gijsen, Kerstin Ritter,
• Abstract summary: This study pioneers EEG-language models trained on clinical reports and 15000 EEGs. Our results indicate that models learn richer representations from being exposed to a variety of report segments. representations of EEG-language models can significantly improve pathology detection compared to those of EEG-only models.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal language modeling constitutes a recent breakthrough which leverages advances in large language models to pretrain capable multimodal models. The integration of natural language during pretraining has been shown to significantly improve learned representations, particularly in computer vision. However, the efficacy of multimodal language modeling in the realm of functional brain data, specifically for advancing pathology detection, remains unexplored. This study pioneers EEG-language models trained on clinical reports and 15000 EEGs. We extend methods for multimodal alignment to this novel domain and investigate which textual information in reports is useful for training EEG-language models. Our results indicate that models learn richer representations from being exposed to a variety of report segments, including the patient's clinical history, description of the EEG, and the physician's interpretation. Compared to models exposed to narrower clinical text information, we find such models to retrieve EEGs based on clinical reports (and vice versa) with substantially higher accuracy. Yet, this is only observed when using a contrastive learning approach. Particularly in regimes with few annotations, we observe that representations of EEG-language models can significantly improve pathology detection compared to those of EEG-only models, as demonstrated by both zero-shot classification and linear probes. In sum, these results highlight the potential of integrating brain activity data with clinical text, suggesting that EEG-language models represent significant progress for clinical applications.

Related papers

• A generative framework to bridge data-driven models and scientific theories in language neuroscience [84.76462599023802]
  We present generative explanation-mediated validation, a framework for generating concise explanations of language selectivity in the brain. We show that explanatory accuracy is closely related to the predictive power and stability of the underlying statistical models.
  arXiv  Detail & Related papers  (2024-10-01T15:57:48Z)
• TRRG: Towards Truthful Radiology Report Generation With Cross-modal Disease Clue Enhanced Large Language Model [22.305034251561835]
  We propose a truthful radiology report generation framework, namely TRRG, based on stage-wise training for cross-modal disease clue injection into large language models. Our proposed framework achieves state-of-the-art performance in radiology report generation on datasets such as IU-Xray and MIMIC-CXR.
  arXiv  Detail & Related papers  (2024-08-22T05:52:27Z)
• CXR-Agent: Vision-language models for chest X-ray interpretation with uncertainty aware radiology reporting [0.0]
  We evaluate the publicly available, state of the art, foundational vision-language models for chest X-ray interpretation. We find that vision-language models often hallucinate with confident language, which slows down clinical interpretation. We develop an agent-based vision-language approach for report generation using CheXagent's linear probes and BioViL-T's phrase grounding tools.
  arXiv  Detail & Related papers  (2024-07-11T18:39:19Z)
• Potential of Multimodal Large Language Models for Data Mining of Medical Images and Free-text Reports [51.45762396192655]
  Multimodal large language models (MLLMs) have recently transformed many domains, significantly affecting the medical field. Notably, Gemini-Vision-series (Gemini) and GPT-4-series (GPT-4) models have epitomized a paradigm shift in Artificial General Intelligence for computer vision. This study evaluated the performance of the Gemini, GPT-4, and 4 popular large models for an exhaustive evaluation across 14 medical imaging datasets.
  arXiv  Detail & Related papers  (2024-07-08T09:08:42Z)
• Language Representations Can be What Recommenders Need: Findings and Potentials [57.90679739598295]
  We show that item representations, when linearly mapped from advanced LM representations, yield superior recommendation performance. This outcome suggests the possible homomorphism between the advanced language representation space and an effective item representation space for recommendation. Our findings highlight the connection between language modeling and behavior modeling, which can inspire both natural language processing and recommender system communities.
  arXiv  Detail & Related papers  (2024-07-07T17:05:24Z)
• A Transformer-based representation-learning model with unified processing of multimodal input for clinical diagnostics [63.106382317917344]
  We report a Transformer-based representation-learning model as a clinical diagnostic aid that processes multimodal input in a unified manner. The unified model outperformed an image-only model and non-unified multimodal diagnosis models in the identification of pulmonary diseases.
  arXiv  Detail & Related papers  (2023-06-01T16:23:47Z)
• Language Models are Few-shot Learners for Prognostic Prediction [0.4254099382808599]
  We explore the use of transformers and language models in prognostic prediction for immunotherapy using real-world patients' clinical data and molecular profiles. The study benchmarks the efficacy of baselines and language models on prognostic prediction across multiple cancer types and investigates the impact of different pretrained language models under few-shot regimes.
  arXiv  Detail & Related papers  (2023-02-24T15:35:36Z)
• Making the Most of Text Semantics to Improve Biomedical Vision--Language Processing [17.96645738679543]
  We show that textual semantic modelling can substantially improve contrastive learning in self-supervised vision--language processing. We propose a self-supervised joint vision--language approach with a focus on better text modelling.
  arXiv  Detail & Related papers  (2022-04-21T00:04:35Z)
• CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark [51.38557174322772]
  We present the first Chinese Biomedical Language Understanding Evaluation benchmark. It is a collection of natural language understanding tasks including named entity recognition, information extraction, clinical diagnosis normalization, single-sentence/sentence-pair classification. We report empirical results with the current 11 pre-trained Chinese models, and experimental results show that state-of-the-art neural models perform by far worse than the human ceiling.
  arXiv  Detail & Related papers  (2021-06-15T12:25:30Z)
• Neural Language Models with Distant Supervision to Identify Major Depressive Disorder from Clinical Notes [2.1060613825447407]
  Major depressive disorder (MDD) is a prevalent psychiatric disorder that is associated with significant healthcare burden worldwide. Recent advancements in neural language models, such as Bidirectional Representations for Transformers (BERT) model, resulted in state-of-the-art neural language models. We propose to leverage the neural language models in a distant supervision paradigm to identify MDD phenotypes from clinical notes.
  arXiv  Detail & Related papers  (2021-04-19T21:11:41Z)
• Select-ProtoNet: Learning to Select for Few-Shot Disease Subtype Prediction [55.94378672172967]
  We focus on few-shot disease subtype prediction problem, identifying subgroups of similar patients. We introduce meta learning techniques to develop a new model, which can extract the common experience or knowledge from interrelated clinical tasks. Our new model is built upon a carefully designed meta-learner, called Prototypical Network, that is a simple yet effective meta learning machine for few-shot image classification.
  arXiv  Detail & Related papers  (2020-09-02T02:50:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.