NADE: A Benchmark for Robust Adverse Drug Events Extraction in Face of Negations

• URL: http://arxiv.org/abs/2109.10080v1
• Date: Tue, 21 Sep 2021 10:33:29 GMT
• Title: NADE: A Benchmark for Robust Adverse Drug Events Extraction in Face of Negations
• Authors: Simone Scaboro, Beatrice Portelli, Emmanuele Chersoni, Enrico Santus, Giuseppe Serra
• Abstract summary: Adverse Drug Event (ADE) extraction mod-els can rapidly examine large collections of so-cial media texts, detecting mentions of drug-related adverse reactions and trigger medicalinvestigations. Despite the recent ad-vances in NLP, it is currently unknown if suchmodels are robust in face ofnegation, which ispervasive across language varieties. In this paper we evaluate three state-of-the-art systems, showing their fragility against nega-tion, and then we introduce two possible strate-gies to increase the robustness of these mod-els.
• Score: 8.380439657099906
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Adverse Drug Event (ADE) extraction mod-els can rapidly examine large collections of so-cial media texts, detecting mentions of drug-related adverse reactions and trigger medicalinvestigations. However, despite the recent ad-vances in NLP, it is currently unknown if suchmodels are robust in face ofnegation, which ispervasive across language varieties.In this paper we evaluate three state-of-the-artsystems, showing their fragility against nega-tion, and then we introduce two possible strate-gies to increase the robustness of these mod-els: a pipeline approach, relying on a specificcomponent for negation detection; an augmen-tation of an ADE extraction dataset to artifi-cially create negated samples and further trainthe models.We show that both strategies bring significantincreases in performance, lowering the num-ber of spurious entities predicted by the mod-els. Our dataset and code will be publicly re-leased to encourage research on the topic.

Related papers

• PEEL the Layers and Find Yourself: Revisiting Inference-time Data Leakage for Residual Neural Networks [64.90981115460937]
  This paper explores inference-time data leakage risks of deep neural networks (NNs) We propose a novel backward feature inversion method, textbfPEEL, which can effectively recover block-wise input features from the intermediate output of residual NNs. Our results show that PEEL outperforms the state-of-the-art recovery methods by an order of magnitude when evaluated by mean squared error (MSE)
  arXiv  Detail & Related papers  (2025-04-08T20:11:05Z)
• Reproducing NevIR: Negation in Neural Information Retrieval [5.950812862331131]
  Negation is a fundamental aspect of human communication, yet it remains a challenge for Language Models in Information Retrieval (IR) We reproduce and extend the findings of NevIR, a benchmark study that revealed most IR models perform at or below the level of random ranking when dealing with negation. Our findings show that a recently emerging category-listwise Large Language Model (LLM) re-rankers-outperforms other models but still underperforms human performance.
  arXiv  Detail & Related papers  (2025-02-19T07:50:59Z)
• Generating on Generated: An Approach Towards Self-Evolving Diffusion Models [58.05857658085845]
  Recursive Self-Improvement (RSI) enables intelligence systems to autonomously refine their capabilities. This paper explores the application of RSI in text-to-image diffusion models, addressing the challenge of training collapse caused by synthetic data.
  arXiv  Detail & Related papers  (2025-02-14T07:41:47Z)
• The Surprising Harmfulness of Benign Overfitting for Adversarial Robustness [13.120373493503772]
  We prove a surprising result that even if the ground truth itself is robust to adversarial examples, the benignly overfitted model is benign in terms of the standard'' out-of-sample risk objective. Our finding provides theoretical insights into the puzzling phenomenon observed in practice, where the true target function (e.g., human) is robust against adverasrial attack, while beginly overfitted neural networks lead to models that are not robust.
  arXiv  Detail & Related papers  (2024-01-19T15:40:46Z)
• Increasing Adverse Drug Events extraction robustness on social media: case study on negation and speculation [7.052238842788185]
  In the last decade, an increasing number of users have started reporting Adverse Drug Events (ADE) on social media platforms. This paper takes into consideration four state-of-the-art systems for ADE detection on social media texts. We introduce SNAX, a benchmark to test their performance against samples containing negated and speculated ADEs.
  arXiv  Detail & Related papers  (2022-09-06T20:38:42Z)
• Improving negation detection with negation-focused pre-training [58.32362243122714]
  Negation is a common linguistic feature that is crucial in many language understanding tasks. Recent work has shown that state-of-the-art NLP models underperform on samples containing negation. We propose a new negation-focused pre-training strategy, involving targeted data augmentation and negation masking.
  arXiv  Detail & Related papers  (2022-05-09T02:41:11Z)
• SAIS: Supervising and Augmenting Intermediate Steps for Document-Level Relation Extraction [51.27558374091491]
  We propose to explicitly teach the model to capture relevant contexts and entity types by supervising and augmenting intermediate steps (SAIS) for relation extraction. Based on a broad spectrum of carefully designed tasks, our proposed SAIS method not only extracts relations of better quality due to more effective supervision, but also retrieves the corresponding supporting evidence more accurately.
  arXiv  Detail & Related papers  (2021-09-24T17:37:35Z)
• Artificial Text Detection via Examining the Topology of Attention Maps [58.46367297712477]
  We propose three novel types of interpretable topological features for this task based on Topological Data Analysis (TDA) We empirically show that the features derived from the BERT model outperform count- and neural-based baselines up to 10% on three common datasets. The probing analysis of the features reveals their sensitivity to the surface and syntactic properties.
  arXiv  Detail & Related papers  (2021-09-10T12:13:45Z)
• Avoiding Inference Heuristics in Few-shot Prompt-based Finetuning [57.4036085386653]
  We show that prompt-based models for sentence pair classification tasks still suffer from a common pitfall of adopting inferences based on lexical overlap. We then show that adding a regularization that preserves pretraining weights is effective in mitigating this destructive tendency of few-shot finetuning.
  arXiv  Detail & Related papers  (2021-09-09T10:10:29Z)
• Exploring Robustness of Unsupervised Domain Adaptation in Semantic Segmentation [74.05906222376608]
  We propose adversarial self-supervision UDA (or ASSUDA) that maximizes the agreement between clean images and their adversarial examples by a contrastive loss in the output space. This paper is rooted in two observations: (i) the robustness of UDA methods in semantic segmentation remains unexplored, which pose a security concern in this field; and (ii) although commonly used self-supervision (e.g., rotation and jigsaw) benefits image tasks such as classification and recognition, they fail to provide the critical supervision signals that could learn discriminative representation for segmentation tasks.
  arXiv  Detail & Related papers  (2021-05-23T01:50:44Z)
• Closeness and Uncertainty Aware Adversarial Examples Detection in Adversarial Machine Learning [0.7734726150561088]
  We explore and assess the usage of 2 different groups of metrics in detecting adversarial samples. We introduce a new feature for adversarial detection, and we show that the performances of all these metrics heavily depend on the strength of the attack being used.
  arXiv  Detail & Related papers  (2020-12-11T14:44:59Z)
• Proactive Pseudo-Intervention: Causally Informed Contrastive Learning For Interpretable Vision Models [103.64435911083432]
  We present a novel contrastive learning strategy called it Proactive Pseudo-Intervention (PPI) PPI leverages proactive interventions to guard against image features with no causal relevance. We also devise a novel causally informed salience mapping module to identify key image pixels to intervene, and show it greatly facilitates model interpretability.
  arXiv  Detail & Related papers  (2020-12-06T20:30:26Z)
• Retrofitting Vector Representations of Adverse Event Reporting Data to Structured Knowledge to Improve Pharmacovigilance Signal Detection [6.644784804652259]
  Adverse drug events (ADEs) are prevalent and costly. Clinical trials are constrained in their ability to identify potential ADEs. Statistical methods provide a convenient way to detect signals from these reports but have limitations in leveraging relationships between drugs and ADEs. Aer2vec generates distributed vector representations of ADE report entities that capture patterns of similarity but cannot utilize lexical knowledge. We address this limitation by retrofitting aer2vec drug embeddings to knowledge from RxNorm and developing a novel retrofitting variant using vector rescaling to preserve magnitude.
  arXiv  Detail & Related papers  (2020-08-07T19:11:51Z)
• Drug-Drug Interaction Prediction with Wasserstein Adversarial Autoencoder-based Knowledge Graph Embeddings [22.562175708415392]
  We propose a new knowledge graph embedding framework for drug-drug interactions. In our framework, the autoencoder is employed to generate high-quality negative samples. The discriminator learns the embeddings of drugs and interactions based on both positive and negative triplets.
  arXiv  Detail & Related papers  (2020-04-15T21:03:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.