Learning Contextualized Document Representations for Healthcare Answer Retrieval

• URL: http://arxiv.org/abs/2002.00835v1
• Date: Mon, 3 Feb 2020 15:47:19 GMT
• Title: Learning Contextualized Document Representations for Healthcare Answer Retrieval
• Authors: Sebastian Arnold, Betty van Aken, Paul Grundmann, Felix A. Gers, Alexander L\"oser
• Abstract summary: Contextual Discourse Vectors (CDV) is a distributed document representation for efficient answer retrieval from long documents. Our model leverages a dual encoder architecture with hierarchical LSTM layers and multi-task training to encode the position of clinical entities and aspects alongside the document discourse. We show that our generalized model significantly outperforms several state-of-the-art baselines for healthcare passage ranking.
• Score: 68.02029435111193
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present Contextual Discourse Vectors (CDV), a distributed document representation for efficient answer retrieval from long healthcare documents. Our approach is based on structured query tuples of entities and aspects from free text and medical taxonomies. Our model leverages a dual encoder architecture with hierarchical LSTM layers and multi-task training to encode the position of clinical entities and aspects alongside the document discourse. We use our continuous representations to resolve queries with short latency using approximate nearest neighbor search on sentence level. We apply the CDV model for retrieving coherent answer passages from nine English public health resources from the Web, addressing both patients and medical professionals. Because there is no end-to-end training data available for all application scenarios, we train our model with self-supervised data from Wikipedia. We show that our generalized model significantly outperforms several state-of-the-art baselines for healthcare passage ranking and is able to adapt to heterogeneous domains without additional fine-tuning.

Related papers

• Medical Vision-Language Pre-Training for Brain Abnormalities [96.1408455065347]
  We show how to automatically collect medical image-text aligned data for pretraining from public resources such as PubMed. In particular, we present a pipeline that streamlines the pre-training process by initially collecting a large brain image-text dataset. We also investigate the unique challenge of mapping subfigures to subcaptions in the medical domain.
  arXiv  Detail & Related papers  (2024-04-27T05:03:42Z)
• Hierarchical Text-to-Vision Self Supervised Alignment for Improved Histopathology Representation Learning [64.1316997189396]
  We present a novel language-tied self-supervised learning framework, Hierarchical Language-tied Self-Supervision (HLSS) for histopathology images. Our resulting model achieves state-of-the-art performance on two medical imaging benchmarks, OpenSRH and TCGA datasets.
  arXiv  Detail & Related papers  (2024-03-21T17:58:56Z)
• ViMQ: A Vietnamese Medical Question Dataset for Healthcare Dialogue System Development [1.4315915057750197]
  We publish a Vietnamese dataset of medical questions from patients with sentence-level and entity-level annotations. We propose a simple self-supervised training strategy with span-noise modelling that improves the performance.
  arXiv  Detail & Related papers  (2023-04-27T17:59:53Z)
• UnifieR: A Unified Retriever for Large-Scale Retrieval [84.61239936314597]
  Large-scale retrieval is to recall relevant documents from a huge collection given a query. Recent retrieval methods based on pre-trained language models (PLM) can be coarsely categorized into either dense-vector or lexicon-based paradigms. We propose a new learning framework, UnifieR which unifies dense-vector and lexicon-based retrieval in one model with a dual-representing capability.
  arXiv  Detail & Related papers  (2022-05-23T11:01:59Z)
• Self-supervised Answer Retrieval on Clinical Notes [68.87777592015402]
  We introduce CAPR, a rule-based self-supervision objective for training Transformer language models for domain-specific passage matching. We apply our objective in four Transformer-based architectures: Contextual Document Vectors, Bi-, Poly- and Cross-encoders. We report that CAPR outperforms strong baselines in the retrieval of domain-specific passages and effectively generalizes across rule-based and human-labeled passages.
  arXiv  Detail & Related papers  (2021-08-02T10:42:52Z)
• A Meta-embedding-based Ensemble Approach for ICD Coding Prediction [64.42386426730695]
  International Classification of Diseases (ICD) are the de facto codes used globally for clinical coding. These codes enable healthcare providers to claim reimbursement and facilitate efficient storage and retrieval of diagnostic information. Our proposed approach enhances the performance of neural models by effectively training word vectors using routine medical data as well as external knowledge from scientific articles.
  arXiv  Detail & Related papers  (2021-02-26T17:49:58Z)
• Towards an Automated SOAP Note: Classifying Utterances from Medical Conversations [0.6875312133832078]
  We bridge the gap for classifying utterances from medical conversations according to (i) the SOAP section and (ii) the speaker role. We present a systematic analysis in which we adapt an existing deep learning architecture to the two aforementioned tasks. The results suggest that modelling context in a hierarchical manner, which captures both word and utterance level context, yields substantial improvements on both classification tasks.
  arXiv  Detail & Related papers  (2020-07-17T04:19:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.