Sebis at SemEval-2023 Task 7: A Joint System for Natural Language Inference and Evidence Retrieval from Clinical Trial Reports

• URL: http://arxiv.org/abs/2304.13180v2
• Date: Tue, 2 May 2023 16:46:33 GMT
• Title: Sebis at SemEval-2023 Task 7: A Joint System for Natural Language Inference and Evidence Retrieval from Clinical Trial Reports
• Authors: Juraj Vladika, Florian Matthes
• Abstract summary: SemEval-2023 Task 7 was to develop an NLP system for two tasks: evidence retrieval and natural language inference from clinical trial data. Our system ranked 3rd out of 40 participants with a final submission.
• Score: 0.799536002595393
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With the increasing number of clinical trial reports generated every day, it is becoming hard to keep up with novel discoveries that inform evidence-based healthcare recommendations. To help automate this process and assist medical experts, NLP solutions are being developed. This motivated the SemEval-2023 Task 7, where the goal was to develop an NLP system for two tasks: evidence retrieval and natural language inference from clinical trial data. In this paper, we describe our two developed systems. The first one is a pipeline system that models the two tasks separately, while the second one is a joint system that learns the two tasks simultaneously with a shared representation and a multi-task learning approach. The final system combines their outputs in an ensemble system. We formalize the models, present their characteristics and challenges, and provide an analysis of achieved results. Our system ranked 3rd out of 40 participants with a final submission.

Related papers

• AutoRG-Brain: Grounded Report Generation for Brain MRI [57.22149878985624]
  Radiologists are tasked with interpreting a large number of images in a daily base, with the responsibility of generating corresponding reports. This demanding workload elevates the risk of human error, potentially leading to treatment delays, increased healthcare costs, revenue loss, and operational inefficiencies. We initiate a series of work on grounded Automatic Report Generation (AutoRG) This system supports the delineation of brain structures, the localization of anomalies, and the generation of well-organized findings.
  arXiv  Detail & Related papers  (2024-07-23T17:50:00Z)
• SemEval-2023 Task 7: Multi-Evidence Natural Language Inference for Clinical Trial Data [1.6932802756478729]
  This paper describes the results of SemEval 2023 task 7 -- Multi-Evidence Natural Language Inference for Clinical Trial Data. The proposed challenges require multi-hop biomedical and numerical reasoning. We observe significantly better performance on the evidence selection task than on the entailment task.
  arXiv  Detail & Related papers  (2023-05-04T16:58:19Z)
• Task formulation for Extracting Social Determinants of Health from Clinical Narratives [0.0]
  The 2022 n2c2 NLP Challenge posed identification of social determinants of health in clinical narratives. We present three systems that we developed for the Challenge and discuss the distinctive task formulation used in each of the three systems.
  arXiv  Detail & Related papers  (2023-01-26T20:00:54Z)
• User-Driven Research of Medical Note Generation Software [49.85146209418244]
  We present three rounds of user studies carried out in the context of developing a medical note generation system. We discuss the participating clinicians' impressions and views of how the system ought to be adapted to be of value to them. We describe a three-week test run of the system in a live telehealth clinical practice.
  arXiv  Detail & Related papers  (2022-05-05T10:18:06Z)
• A Benchmark for Automatic Medical Consultation System: Frameworks, Tasks and Datasets [70.32630628211803]
  We propose two frameworks to support automatic medical consultation, namely doctor-patient dialogue understanding and task-oriented interaction. A new large medical dialogue dataset with multi-level fine-grained annotations is introduced. We report a set of benchmark results for each task, which shows the usability of the dataset and sets a baseline for future studies.
  arXiv  Detail & Related papers  (2022-04-19T16:43:21Z)
• Automated tabulation of clinical trial results: A joint entity and relation extraction approach with transformer-based language representations [5.825190876052148]
  This paper investigates automating evidence table generation by decomposing the problem across two language processing tasks. We focus on the automatic tabulation of sentences from published RCT abstracts that report the practice outcomes. To train and test these models, a new gold-standard corpus was developed, comprising almost 600 result sentences from six disease areas.
  arXiv  Detail & Related papers  (2021-12-10T15:26:43Z)
• MIMO: Mutual Integration of Patient Journey and Medical Ontology for Healthcare Representation Learning [49.57261599776167]
  We propose an end-to-end robust Transformer-based solution, Mutual Integration of patient journey and Medical Ontology (MIMO) for healthcare representation learning and predictive analytics.
  arXiv  Detail & Related papers  (2021-07-20T07:04:52Z)
• CAiRE in DialDoc21: Data Augmentation for Information-Seeking Dialogue System [55.43871578056878]
  In DialDoc21 competition, our system achieved 74.95 F1 score and 60.74 Exact Match score in subtask 1, and 37.72 SacreBLEU score in subtask 2.
  arXiv  Detail & Related papers  (2021-06-07T11:40:55Z)
• MS2: Multi-Document Summarization of Medical Studies [11.38740406132287]
  We release MS2 (Multi-Document Summarization of Medical Studies), a dataset of over 470k documents and 20k summaries derived from the scientific literature. This dataset facilitates the development of systems that can assess and aggregate contradictory evidence across multiple studies. We experiment with a summarization system based on BART, with promising early results.
  arXiv  Detail & Related papers  (2021-04-13T19:59:34Z)
• BiteNet: Bidirectional Temporal Encoder Network to Predict Medical Outcomes [53.163089893876645]
  We propose a novel self-attention mechanism that captures the contextual dependency and temporal relationships within a patient's healthcare journey. An end-to-end bidirectional temporal encoder network (BiteNet) then learns representations of the patient's journeys. We have evaluated the effectiveness of our methods on two supervised prediction and two unsupervised clustering tasks with a real-world EHR dataset.
  arXiv  Detail & Related papers  (2020-09-24T00:42:36Z)
• A comparative study of machine learning techniques used in non-clinical systems for continuous healthcare of independent livings [0.0]
  This study analyzes usages of machine learning techniques in healthcare systems for independent livings. It is observed that most of the systems created are for single purpose. There is need to create more generic systems that can be used for patients with multiple diseases.
  arXiv  Detail & Related papers  (2020-05-19T14:59:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.