Extraction of Medication Names from Twitter Using Augmentation and an
Ensemble of Language Models
- URL: http://arxiv.org/abs/2111.06664v1
- Date: Fri, 12 Nov 2021 11:18:46 GMT
- Title: Extraction of Medication Names from Twitter Using Augmentation and an
Ensemble of Language Models
- Authors: Igor Kulev, Berkay K\"opr\"u, Raul Rodriguez-Esteban, Diego Saldana,
Yi Huang, Alessandro La Torraca, Elif Ozkirimli
- Abstract summary: The BioCreative VII Track 3 challenge focused on the identification of medication names in Twitter user timelines.
For our submission to this challenge, we expanded the available training data by using several data augmentation techniques.
The augmented data was then used to fine-tune an ensemble of language models that had been pre-trained on general-domain Twitter content.
- Score: 55.44979919361194
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The BioCreative VII Track 3 challenge focused on the identification of
medication names in Twitter user timelines. For our submission to this
challenge, we expanded the available training data by using several data
augmentation techniques. The augmented data was then used to fine-tune an
ensemble of language models that had been pre-trained on general-domain Twitter
content. The proposed approach outperformed the prior state-of-the-art
algorithm Kusuri and ranked high in the competition for our selected objective
function, overlapping F1 score.
Related papers
- LoGra-Med: Long Context Multi-Graph Alignment for Medical Vision-Language Model [55.80651780294357]
State-of-the-art medical multi-modal large language models (med-MLLM) leverage instruction-following data in pre-training.
LoGra-Med is a new multi-graph alignment algorithm that enforces triplet correlations across image modalities, conversation-based descriptions, and extended captions.
Our results show LoGra-Med matches LLAVA-Med performance on 600K image-text pairs for Medical VQA and significantly outperforms it when trained on 10% of the data.
arXiv Detail & Related papers (2024-10-03T15:52:03Z) - VIBE: Topic-Driven Temporal Adaptation for Twitter Classification [9.476760540618903]
We study temporal adaptation, where models trained on past data are tested in the future.
Our model, with only 3% of data, significantly outperforms previous state-of-the-art continued-pretraining methods.
arXiv Detail & Related papers (2023-10-16T08:53:57Z) - Time Series Contrastive Learning with Information-Aware Augmentations [57.45139904366001]
A key component of contrastive learning is to select appropriate augmentations imposing some priors to construct feasible positive samples.
How to find the desired augmentations of time series data that are meaningful for given contrastive learning tasks and datasets remains an open question.
We propose a new contrastive learning approach with information-aware augmentations, InfoTS, that adaptively selects optimal augmentations for time series representation learning.
arXiv Detail & Related papers (2023-03-21T15:02:50Z) - Ensemble Transfer Learning for Multilingual Coreference Resolution [60.409789753164944]
A problem that frequently occurs when working with a non-English language is the scarcity of annotated training data.
We design a simple but effective ensemble-based framework that combines various transfer learning techniques.
We also propose a low-cost TL method that bootstraps coreference resolution models by utilizing Wikipedia anchor texts.
arXiv Detail & Related papers (2023-01-22T18:22:55Z) - Rumour detection using graph neural network and oversampling in
benchmark Twitter dataset [0.30079490585515345]
We propose a novel method for building automatic rumour detection system by focusing on oversampling.
Our oversampling method relies on contextualised data augmentation to generate synthetic samples for underrepresented classes in the dataset.
Two graph neural networks(GNN) are proposed to model non-linear conversations on a thread.
arXiv Detail & Related papers (2022-12-20T08:43:10Z) - Learning Phone Recognition from Unpaired Audio and Phone Sequences Based
on Generative Adversarial Network [58.82343017711883]
This paper investigates how to learn directly from unpaired phone sequences and speech utterances.
GAN training is adopted in the first stage to find the mapping relationship between unpaired speech and phone sequence.
In the second stage, another HMM model is introduced to train from the generator's output, which boosts the performance.
arXiv Detail & Related papers (2022-07-29T09:29:28Z) - BCH-NLP at BioCreative VII Track 3: medications detection in tweets
using transformer networks and multi-task learning [9.176393163624002]
We implement a multi-task learning model that is jointly trained on text classification and sequence labelling.
Our best system run achieved a strict F1 of 80.4, ranking first and more than 10 points higher than the average score of all participants.
arXiv Detail & Related papers (2021-11-26T19:22:51Z) - A PubMedBERT-based Classifier with Data Augmentation Strategy for
Detecting Medication Mentions in Tweets [2.539568419434224]
Twitter publishes a large number of user-generated text (tweets) on a daily basis.
entity recognition (NER) presents some special challenges for tweet data.
In this paper, we explore a PubMedBERT-based classifier trained with a combination of multiple data augmentation approaches.
Our method achieved an F1 score of 0.762, which is substantially higher than the mean of all submissions.
arXiv Detail & Related papers (2021-11-03T14:29:24Z) - Deep learning based registration using spatial gradients and noisy
segmentation labels [52.78503776563559]
deep learning based approaches became quite popular, providing fast and performing registration strategies.
Our work relies on (i) a symmetric formulation, predicting the transformations from source to target and from target to source simultaneously, enforcing the trained representations to be similar.
Our method reports a mean dice of $0.64$ for task 3 and $0.85$ for task 4 on the test sets, taking third place on the challenge.
arXiv Detail & Related papers (2020-10-21T11:08:45Z) - Phonemer at WNUT-2020 Task 2: Sequence Classification Using COVID
Twitter BERT and Bagging Ensemble Technique based on Plurality Voting [0.0]
We develop a system that automatically identifies whether an English Tweet related to the novel coronavirus (COVID-19) is informative or not.
Our final approach achieved an F1-score of 0.9037 and we were ranked sixth overall with F1-score as the evaluation criteria.
arXiv Detail & Related papers (2020-10-01T10:54:54Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.