BCH-NLP at BioCreative VII Track 3: medications detection in tweets using transformer networks and multi-task learning

• URL: http://arxiv.org/abs/2111.13726v1
• Date: Fri, 26 Nov 2021 19:22:51 GMT
• Title: BCH-NLP at BioCreative VII Track 3: medications detection in tweets using transformer networks and multi-task learning
• Authors: Dongfang Xu, Shan Chen, Timothy Miller
• Abstract summary: We implement a multi-task learning model that is jointly trained on text classification and sequence labelling. Our best system run achieved a strict F1 of 80.4, ranking first and more than 10 points higher than the average score of all participants.
• Score: 9.176393163624002
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: In this paper, we present our work participating in the BioCreative VII Track 3 - automatic extraction of medication names in tweets, where we implemented a multi-task learning model that is jointly trained on text classification and sequence labelling. Our best system run achieved a strict F1 of 80.4, ranking first and more than 10 points higher than the average score of all participants. Our analyses show that the ensemble technique, multi-task learning, and data augmentation are all beneficial for medication detection in tweets.

Related papers

• ThangDLU at #SMM4H 2024: Encoder-decoder models for classifying text data on social disorders in children and adolescents [49.00494558898933]
  This paper describes our participation in Task 3 and Task 5 of the #SMM4H (Social Media Mining for Health) 2024 Workshop. Task 3 is a multi-class classification task centered on tweets discussing the impact of outdoor environments on symptoms of social anxiety. Task 5 involves a binary classification task focusing on tweets reporting medical disorders in children. We applied transfer learning from pre-trained encoder-decoder models such as BART-base and T5-small to identify the labels of a set of given tweets.
  arXiv  Detail & Related papers  (2024-04-30T17:06:20Z)
• Automatic Extraction of Medication Names in Tweets as Named Entity Recognition [3.7462395049372894]
  Biocreative VII Task 3 focuses on mining this information by recognizing mentions of medications and dietary supplements in tweets. We approach this task by fine tuning multiple BERT-style language models to perform token-level classification. Our best system consists of five Megatron-BERT-345M models and achieves a strict F1 score of 0.764 on unseen test data.
  arXiv  Detail & Related papers  (2021-11-30T18:25:32Z)
• Extraction of Medication Names from Twitter Using Augmentation and an Ensemble of Language Models [55.44979919361194]
  The BioCreative VII Track 3 challenge focused on the identification of medication names in Twitter user timelines. For our submission to this challenge, we expanded the available training data by using several data augmentation techniques. The augmented data was then used to fine-tune an ensemble of language models that had been pre-trained on general-domain Twitter content.
  arXiv  Detail & Related papers  (2021-11-12T11:18:46Z)
• A PubMedBERT-based Classifier with Data Augmentation Strategy for Detecting Medication Mentions in Tweets [2.539568419434224]
  Twitter publishes a large number of user-generated text (tweets) on a daily basis. entity recognition (NER) presents some special challenges for tweet data. In this paper, we explore a PubMedBERT-based classifier trained with a combination of multiple data augmentation approaches. Our method achieved an F1 score of 0.762, which is substantially higher than the mean of all submissions.
  arXiv  Detail & Related papers  (2021-11-03T14:29:24Z)
• R-BERT-CNN: Drug-target interactions extraction from biomedical literature [1.8814209805277506]
  We present our participation for the DrugProt task BioCreative VII challenge. Drug-target interactions (DTIs) are critical for drug discovery and repurposing. There are >32M biomedical articles on PubMed and manually extracting DTIs from such a huge knowledge base is challenging.
  arXiv  Detail & Related papers  (2021-10-31T22:50:33Z)
• Training ELECTRA Augmented with Multi-word Selection [53.77046731238381]
  We present a new text encoder pre-training method that improves ELECTRA based on multi-task learning. Specifically, we train the discriminator to simultaneously detect replaced tokens and select original tokens from candidate sets.
  arXiv  Detail & Related papers  (2021-05-31T23:19:00Z)
• Neural networks for Anatomical Therapeutic Chemical (ATC) [83.73971067918333]
  We propose combining multiple multi-label classifiers trained on distinct sets of features, including sets extracted from a Bidirectional Long Short-Term Memory Network (BiLSTM) Experiments demonstrate the power of this approach, which is shown to outperform the best methods reported in the literature.
  arXiv  Detail & Related papers  (2021-01-22T19:49:47Z)
• Phonemer at WNUT-2020 Task 2: Sequence Classification Using COVID Twitter BERT and Bagging Ensemble Technique based on Plurality Voting [0.0]
  We develop a system that automatically identifies whether an English Tweet related to the novel coronavirus (COVID-19) is informative or not. Our final approach achieved an F1-score of 0.9037 and we were ranked sixth overall with F1-score as the evaluation criteria.
  arXiv  Detail & Related papers  (2020-10-01T10:54:54Z)
• Device-Robust Acoustic Scene Classification Based on Two-Stage Categorization and Data Augmentation [63.98724740606457]
  We present a joint effort of four groups, namely GT, USTC, Tencent, and UKE, to tackle Task 1 - Acoustic Scene Classification (ASC) in the DCASE 2020 Challenge. Task 1a focuses on ASC of audio signals recorded with multiple (real and simulated) devices into ten different fine-grained classes. Task 1b concerns with classification of data into three higher-level classes using low-complexity solutions.
  arXiv  Detail & Related papers  (2020-07-16T15:07:14Z)
• 3D medical image segmentation with labeled and unlabeled data using autoencoders at the example of liver segmentation in CT images [58.720142291102135]
  This work investigates the potential of autoencoder-extracted features to improve segmentation with a convolutional neural network. A convolutional autoencoder was used to extract features from unlabeled data and a multi-scale, fully convolutional CNN was used to perform the target task of 3D liver segmentation in CT images.
  arXiv  Detail & Related papers  (2020-03-17T20:20:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.