KESDT: knowledge enhanced shallow and deep Transformer for detecting adverse drug reactions

• URL: http://arxiv.org/abs/2308.09329v1
• Date: Fri, 18 Aug 2023 06:10:11 GMT
• Title: KESDT: knowledge enhanced shallow and deep Transformer for detecting adverse drug reactions
• Authors: Yunzhi Qiu, Xiaokun Zhang, Weiwei Wang, Tongxuan Zhang, Bo Xu, Hongfei Lin
• Abstract summary: We propose the Knowledge Enhanced Shallow and Deep Transformer(KESDT) model for ADR detection. To cope with the first issue, we incorporate the domain keywords into the Transformer model through a shallow fusion manner. To overcome the low annotated data, we integrate the synonym sets into the Transformer model through a deep fusion manner.
• Score: 14.095117843726511
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Adverse drug reaction (ADR) detection is an essential task in the medical field, as ADRs have a gravely detrimental impact on patients' health and the healthcare system. Due to a large number of people sharing information on social media platforms, an increasing number of efforts focus on social media data to carry out effective ADR detection. Despite having achieved impressive performance, the existing methods of ADR detection still suffer from three main challenges. Firstly, researchers have consistently ignored the interaction between domain keywords and other words in the sentence. Secondly, social media datasets suffer from the challenges of low annotated data. Thirdly, the issue of sample imbalance is commonly observed in social media datasets. To solve these challenges, we propose the Knowledge Enhanced Shallow and Deep Transformer(KESDT) model for ADR detection. Specifically, to cope with the first issue, we incorporate the domain keywords into the Transformer model through a shallow fusion manner, which enables the model to fully exploit the interactive relationships between domain keywords and other words in the sentence. To overcome the low annotated data, we integrate the synonym sets into the Transformer model through a deep fusion manner, which expands the size of the samples. To mitigate the impact of sample imbalance, we replace the standard cross entropy loss function with the focal loss function for effective model training. We conduct extensive experiments on three public datasets including TwiMed, Twitter, and CADEC. The proposed KESDT outperforms state-of-the-art baselines on F1 values, with relative improvements of 4.87%, 47.83%, and 5.73% respectively, which demonstrates the effectiveness of our proposed KESDT.

Related papers

• Unraveling the "Anomaly" in Time Series Anomaly Detection: A Self-supervised Tri-domain Solution [89.16750999704969]
  Anomaly labels hinder traditional supervised models in time series anomaly detection. Various SOTA deep learning techniques, such as self-supervised learning, have been introduced to tackle this issue. We propose a novel self-supervised learning based Tri-domain Anomaly Detector (TriAD)
  arXiv  Detail & Related papers  (2023-11-19T05:37:18Z)
• Automatic diagnosis of knee osteoarthritis severity using Swin transformer [55.01037422579516]
  Knee osteoarthritis (KOA) is a widespread condition that can cause chronic pain and stiffness in the knee joint. We propose an automated approach that employs the Swin Transformer to predict the severity of KOA.
  arXiv  Detail & Related papers  (2023-07-10T09:49:30Z)
• Improving the Robustness of Summarization Systems with Dual Augmentation [68.53139002203118]
  A robust summarization system should be able to capture the gist of the document, regardless of the specific word choices or noise in the input. We first explore the summarization models' robustness against perturbations including word-level synonym substitution and noise. We propose a SummAttacker, which is an efficient approach to generating adversarial samples based on language models.
  arXiv  Detail & Related papers  (2023-06-01T19:04:17Z)
• Accurate and Robust Lesion RECIST Diameter Prediction and Segmentation with Transformers [22.528235432455524]
  This paper proposes a transformer-based network for lesion RECIST diameter prediction and segmentation (LRDPS) It is formulated as three correlative and complementary tasks: lesion segmentation, heatmap prediction, and keypoint regression. MeaFormer achieves the state-of-the-art performance of LRDPS on the large-scale DeepLesion dataset.
  arXiv  Detail & Related papers  (2022-08-28T01:43:21Z)
• SFF-DA: Sptialtemporal Feature Fusion for Detecting Anxiety Nonintrusively [16.170315080992182]
  We propose a framework based on "3CNND+LSTM" and fused similarity features of facial behavior and noncontact physiology. Our framework was validated with dataset from the real world and two public datasets.
  arXiv  Detail & Related papers  (2022-08-12T01:20:51Z)
• Mining Adverse Drug Reactions from Unstructured Mediums at Scale [0.0]
  Adverse drug reactions / events (ADR/ADE) have a major impact on patient health and health care costs. Most ADR's are not reported via formal channels, but they are often documented in unstructured conversations. We propose a natural language processing (NLP) solution that detects ADR's in such unstructured free-text conversations.
  arXiv  Detail & Related papers  (2022-01-05T01:52:42Z)
• Data Augmentation for Mental Health Classification on Social Media [0.0]
  The mental disorder of online users is determined using social media posts. The major challenge in this domain is to avail the ethical clearance for using the user generated text on social media platforms. We have studied the effect of data augmentation techniques on domain specific user generated text for mental health classification.
  arXiv  Detail & Related papers  (2021-12-19T05:09:01Z)
• Towards Unbiased Visual Emotion Recognition via Causal Intervention [63.74095927462]
  We propose a novel Emotion Recognition Network (IERN) to alleviate the negative effects brought by the dataset bias. A series of designed tests validate the effectiveness of IERN, and experiments on three emotion benchmarks demonstrate that IERN outperforms other state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-07-26T10:40:59Z)
• Generic Semi-Supervised Adversarial Subject Translation for Sensor-Based Human Activity Recognition [6.2997667081978825]
  This paper presents a novel generic and robust approach for semi-supervised domain adaptation in Human Activity Recognition. It capitalizes on the advantages of the adversarial framework to tackle the shortcomings, by leveraging knowledge from annotated samples exclusively from the source subject and unlabeled ones of the target subject. The results demonstrate the effectiveness of our proposed algorithms over state-of-the-art methods, which led in up to 13%, 4%, and 13% improvement of our high-level activities recognition metrics for Opportunity, LISSI, and PAMAP2 datasets.
  arXiv  Detail & Related papers  (2020-11-11T12:16:23Z)
• Fader Networks for domain adaptation on fMRI: ABIDE-II study [68.5481471934606]
  We use 3D convolutional autoencoders to build the domain irrelevant latent space image representation and demonstrate this method to outperform existing approaches on ABIDE data.
  arXiv  Detail & Related papers  (2020-10-14T16:50:50Z)
• DecAug: Augmenting HOI Detection via Decomposition [54.65572599920679]
  Current algorithms suffer from insufficient training samples and category imbalance within datasets. We propose an efficient and effective data augmentation method called DecAug for HOI detection. Experiments show that our method brings up to 3.3 mAP and 1.6 mAP improvements on V-COCO and HICODET dataset.
  arXiv  Detail & Related papers  (2020-10-02T13:59:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.