OSLAT: Open Set Label Attention Transformer for Medical Entity Span Extraction

• URL: http://arxiv.org/abs/2207.05817v1
• Date: Tue, 12 Jul 2022 20:22:55 GMT
• Title: OSLAT: Open Set Label Attention Transformer for Medical Entity Span Extraction
• Authors: Raymond Li, Ilya Valmianski, Li Deng, Xavier Amatriain, Anitha Kannan
• Abstract summary: We present a new transformer-based architecture called OSLAT, Open Set Label Attention Transformer. Our approach uses the label-attention mechanism to implicitly learn spans associated with entities of interest. These entities can be provided as free text, including entities not seen during OSLAT's training, and the model can extract spans even when they are disjoint.
• Score: 6.392638268995324
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Identifying spans in medical texts that correspond to medical entities is one of the core steps for many healthcare NLP tasks such as ICD coding, medical finding extraction, medical note contextualization, to name a few. Existing entity extraction methods rely on a fixed and limited vocabulary of medical entities and have difficulty with extracting entities represented by disjoint spans. In this paper, we present a new transformer-based architecture called OSLAT, Open Set Label Attention Transformer, that addresses many of the limitations of the previous methods. Our approach uses the label-attention mechanism to implicitly learn spans associated with entities of interest. These entities can be provided as free text, including entities not seen during OSLAT's training, and the model can extract spans even when they are disjoint. To test the generalizability of our method, we train two separate models on two different datasets, which have very low entity overlap: (1) a public discharge notes dataset from hNLP, and (2) a much more challenging proprietary patient text dataset "Reasons for Encounter" (RFE). We find that OSLAT models trained on either dataset outperform rule-based and fuzzy string matching baselines when applied to the RFE dataset as well as to the portion of hNLP dataset where entities are represented by disjoint spans. Our code can be found at https://github.com/curai/curai-research/tree/main/OSLAT.

Related papers

• Leveraging Semantic Type Dependencies for Clinical Named Entity Recognition [24.179910886684745]
  We exploit additional evidence by making use of domain-specific semantic type dependencies. In some cases NER effectiveness can be significantly improved by making use of domain-specific semantic type dependencies.
  arXiv  Detail & Related papers  (2025-03-07T12:29:21Z)
• Seed-Guided Fine-Grained Entity Typing in Science and Engineering Domains [51.02035914828596]
  We study the task of seed-guided fine-grained entity typing in science and engineering domains. We propose SEType which first enriches the weak supervision by finding more entities for each seen type from an unlabeled corpus. It then matches the enriched entities to unlabeled text to get pseudo-labeled samples and trains a textual entailment model that can make inferences for both seen and unseen types.
  arXiv  Detail & Related papers  (2024-01-23T22:36:03Z)
• From Alignment to Entailment: A Unified Textual Entailment Framework for Entity Alignment [17.70562397382911]
  Existing methods usually encode the triples of entities as embeddings and learn to align the embeddings. We transform both triples into unified textual sequences, and model the EA task as a bi-directional textual entailment task. Our approach captures the unified correlation pattern of two kinds of information between entities, and explicitly models the fine-grained interaction between original entity information.
  arXiv  Detail & Related papers  (2023-05-19T08:06:50Z)
• Slot Filling for Biomedical Information Extraction [0.5330240017302619]
  We present a slot filling approach to the task of biomedical IE. We follow the proposed paradigm of coupling a Tranformer-based bi-encoder, Dense Passage Retrieval, with a Transformer-based reader model.
  arXiv  Detail & Related papers  (2021-09-17T14:16:00Z)
• UniRE: A Unified Label Space for Entity Relation Extraction [67.53850477281058]
  Joint entity relation extraction models setup two separated label spaces for the two sub-tasks. We argue that this setting may hinder the information interaction between entities and relations. In this work, we propose to eliminate the different treatment on the two sub-tasks' label spaces.
  arXiv  Detail & Related papers  (2021-07-09T08:09:37Z)
• A Cascade Dual-Decoder Model for Joint Entity and Relation Extraction [18.66493402386152]
  We propose an effective cascade dual-decoder method to extract overlapping relational triples. Our approach is straightforward and it includes a text-specific relation decoder and a relation-corresponded entity decoder. We conducted experiments on a real-world open-pit mine dataset and two public datasets to verify the method's generalizability.
  arXiv  Detail & Related papers  (2021-06-27T07:42:05Z)
• PharmKE: Knowledge Extraction Platform for Pharmaceutical Texts using Transfer Learning [0.0]
  PharmKE is a text analysis platform that applies deep learning through several stages for thorough semantic analysis of pharmaceutical articles. The methodology is used to create accurately labeled training and test datasets, which are then used to train models for custom entity labeling tasks. The obtained results are compared to the fine-tuned BERT and BioBERT models trained on the same dataset.
  arXiv  Detail & Related papers  (2021-02-25T19:36:35Z)
• Local Additivity Based Data Augmentation for Semi-supervised NER [59.90773003737093]
  Named Entity Recognition (NER) is one of the first stages in deep language understanding. Current NER models heavily rely on human-annotated data. We propose a Local Additivity based Data Augmentation (LADA) method for semi-supervised NER.
  arXiv  Detail & Related papers  (2020-10-04T20:46:26Z)
• LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention [37.111204321059084]
  We propose new pretrained contextualized representations of words and entities based on the bidirectional transformer. Our model is trained using a new pretraining task based on the masked language model of BERT. We also propose an entity-aware self-attention mechanism that is an extension of the self-attention mechanism of the transformer.
  arXiv  Detail & Related papers  (2020-10-02T15:38:03Z)
• Autoregressive Entity Retrieval [55.38027440347138]
  Entities are at the center of how we represent and aggregate knowledge. The ability to retrieve such entities given a query is fundamental for knowledge-intensive tasks such as entity linking and open-domain question answering. We propose GENRE, the first system that retrieves entities by generating their unique names, left to right, token-by-token in an autoregressive fashion.
  arXiv  Detail & Related papers  (2020-10-02T10:13:31Z)
• DART: Open-Domain Structured Data Record to Text Generation [91.23798751437835]
  We present DART, an open domain structured DAta Record to Text generation dataset with over 82k instances (DARTs) We propose a procedure of extracting semantic triples from tables that encode their structures by exploiting the semantic dependencies among table headers and the table title. Our dataset construction framework effectively merged heterogeneous sources from open domain semantic parsing and dialogue-act-based meaning representation tasks.
  arXiv  Detail & Related papers  (2020-07-06T16:35:30Z)
• ATSO: Asynchronous Teacher-Student Optimization for Semi-Supervised Medical Image Segmentation [99.90263375737362]
  We propose ATSO, an asynchronous version of teacher-student optimization. ATSO partitions the unlabeled data into two subsets and alternately uses one subset to fine-tune the model and updates the label on the other subset. We evaluate ATSO on two popular medical image segmentation datasets and show its superior performance in various semi-supervised settings.
  arXiv  Detail & Related papers  (2020-06-24T04:05:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.