Task formulation for Extracting Social Determinants of Health from Clinical Narratives

• URL: http://arxiv.org/abs/2301.11386v1
• Date: Thu, 26 Jan 2023 20:00:54 GMT
• Title: Task formulation for Extracting Social Determinants of Health from Clinical Narratives
• Authors: Manabu Torii, Ian M. Finn, Son Doan, Paul Wang, Elly W. Yang, Daniel S. Zisook
• Abstract summary: The 2022 n2c2 NLP Challenge posed identification of social determinants of health in clinical narratives. We present three systems that we developed for the Challenge and discuss the distinctive task formulation used in each of the three systems.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Objective: The 2022 n2c2 NLP Challenge posed identification of social determinants of health (SDOH) in clinical narratives. We present three systems that we developed for the Challenge and discuss the distinctive task formulation used in each of the three systems. Materials and Methods: The first system identifies target pieces of information independently using machine learning classifiers. The second system uses a large language model (LLM) to extract complete structured outputs per document. The third system extracts candidate phrases using machine learning and identifies target relations with hand-crafted rules. Results: The three systems achieved F1 scores of 0.884, 0.831, and 0.663 in the Subtask A of the Challenge, which are ranked third, seventh, and eighth among the 15 participating teams. The review of the extraction results from our systems reveals characteristics of each approach and those of the SODH extraction task. Discussion: Phrases and relations annotated in the task is unique and diverse, not conforming to the conventional event extraction task. These annotations are difficult to model with limited training data. The system that extracts information independently, ignoring the annotated relations, achieves the highest F1 score. Meanwhile, LLM with its versatile capability achieves the high F1 score, while respecting the annotated relations. The rule-based system tackling relation extraction obtains the low F1 score, while it is the most explainable approach. Conclusion: The F1 scores of the three systems vary in this challenge setting, but each approach has advantages and disadvantages in a practical application. The selection of the approach depends not only on the F1 score but also on the requirements in the application.

Related papers

• Software Mention Recognition with a Three-Stage Framework Based on BERTology Models at SOMD 2024 [0.0]
  This paper describes our systems for the sub-task I in the Software Mention Detection in Scholarly Publications shared-task. Our best performing system addresses the named entity recognition problem through a three-stage framework. Our framework based on the XLM-R-based model achieves a weighted F1-score of 67.80%, delivering our team the 3rd rank in Sub-task I for the Software Mention Recognition task.
  arXiv  Detail & Related papers  (2024-04-23T17:06:24Z)
• Clarify When Necessary: Resolving Ambiguity Through Interaction with LMs [58.620269228776294]
  We propose a task-agnostic framework for resolving ambiguity by asking users clarifying questions. We evaluate systems across three NLP applications: question answering, machine translation and natural language inference. We find that intent-sim is robust, demonstrating improvements across a wide range of NLP tasks and LMs.
  arXiv  Detail & Related papers  (2023-11-16T00:18:50Z)
• GPT-FinRE: In-context Learning for Financial Relation Extraction using Large Language Models [1.9559144041082446]
  This paper describes our solution to relation extraction on one such dataset REFinD. In this paper, we employed OpenAI models under the framework of in-context learning (ICL) We were able to achieve 3rd rank overall. Our best F1-score is 0.718.
  arXiv  Detail & Related papers  (2023-06-30T10:12:30Z)
• Zero-Shot Learning for Requirements Classification: An Exploratory Study [6.855054517723465]
  Requirements Engineering (RE) researchers have been experimenting Machine Learning (ML) and Deep Learning (DL) approaches for a range of RE tasks. Most of today's ML-DL approaches are based on supervised learning techniques, meaning that they need to be trained using annotated datasets. This paper proposes an approach that employs the embedding-based unsupervised Zero-Shot Learning (ZSL) technique to perform requirements classification.
  arXiv  Detail & Related papers  (2023-02-09T16:05:01Z)
• The 2022 n2c2/UW Shared Task on Extracting Social Determinants of Health [0.9023847175654602]
  The n2c2/UW SDOH Challenge explores the extraction of social determinant of health (SDOH) information from clinical notes. This paper presents the shared task, data, participating teams, performance results, and considerations for future work.
  arXiv  Detail & Related papers  (2023-01-13T14:20:23Z)
• COLO: A Contrastive Learning based Re-ranking Framework for One-Stage Summarization [84.70895015194188]
  We propose a Contrastive Learning based re-ranking framework for one-stage summarization called COLO. COLO boosts the extractive and abstractive results of one-stage systems on CNN/DailyMail benchmark to 44.58 and 46.33 ROUGE-1 score.
  arXiv  Detail & Related papers  (2022-09-29T06:11:21Z)
• Distant finetuning with discourse relations for stance classification [55.131676584455306]
  We propose a new method to extract data with silver labels from raw text to finetune a model for stance classification. We also propose a 3-stage training framework where the noisy level in the data used for finetuning decreases over different stages. Our approach ranks 1st among 26 competing teams in the stance classification track of the NLPCC 2021 shared task Argumentative Text Understanding for AI Debater.
  arXiv  Detail & Related papers  (2022-04-27T04:24:35Z)
• AIFB-WebScience at SemEval-2022 Task 12: Relation Extraction First -- Using Relation Extraction to Identify Entities [0.0]
  We present an end-to-end joint entity and relation extraction approach based on transformer-based language models. In contrast to existing approaches, which perform entity and relation extraction in sequence, our system incorporates information from relation extraction into entity extraction.
  arXiv  Detail & Related papers  (2022-03-10T12:19:44Z)
• Exploring Neural Models for Query-Focused Summarization [74.41256438059256]
  We conduct a systematic exploration of neural approaches to query-focused summarization (QFS) We present two model extensions that achieve state-of-the-art performance on the QMSum dataset by a margin of up to 3.38 ROUGE-1, 3.72 ROUGE-2, and 3.28 ROUGE-L.
  arXiv  Detail & Related papers  (2021-12-14T18:33:29Z)
• Zero-Shot Information Extraction as a Unified Text-to-Triple Translation [56.01830747416606]
  We cast a suite of information extraction tasks into a text-to-triple translation framework. We formalize the task as a translation between task-specific input text and output triples. We study the zero-shot performance of this framework on open information extraction.
  arXiv  Detail & Related papers  (2021-09-23T06:54:19Z)
• CAiRE in DialDoc21: Data Augmentation for Information-Seeking Dialogue System [55.43871578056878]
  In DialDoc21 competition, our system achieved 74.95 F1 score and 60.74 Exact Match score in subtask 1, and 37.72 SacreBLEU score in subtask 2.
  arXiv  Detail & Related papers  (2021-06-07T11:40:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.