Improving Continual Relation Extraction by Distinguishing Analogous Semantics

• URL: http://arxiv.org/abs/2305.06620v1
• Date: Thu, 11 May 2023 07:32:20 GMT
• Title: Improving Continual Relation Extraction by Distinguishing Analogous Semantics
• Authors: Wenzheng Zhao and Yuanning Cui and Wei Hu
• Abstract summary: Continual relation extraction aims to learn constantly emerging relations while avoiding forgetting the learned relations. Existing works store a small number of typical samples to re-train the model for alleviating forgetting. We conduct an empirical study on existing works and observe that their performance is severely affected by analogous relations.
• Score: 11.420578494453343
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Continual relation extraction (RE) aims to learn constantly emerging relations while avoiding forgetting the learned relations. Existing works store a small number of typical samples to re-train the model for alleviating forgetting. However, repeatedly replaying these samples may cause the overfitting problem. We conduct an empirical study on existing works and observe that their performance is severely affected by analogous relations. To address this issue, we propose a novel continual extraction model for analogous relations. Specifically, we design memory-insensitive relation prototypes and memory augmentation to overcome the overfitting problem. We also introduce integrated training and focal knowledge distillation to enhance the performance on analogous relations. Experimental results show the superiority of our model and demonstrate its effectiveness in distinguishing analogous relations and overcoming overfitting.

Related papers

• Rationale-Enhanced Language Models are Better Continual Relation Learners [29.311298089285753]
  Continual relation extraction (CRE) aims to solve the problem of catastrophic forgetting when learning a sequence of newly emerging relations. Recent CRE studies have found that catastrophic forgetting arises from the model's lack of robustness against future analogous relations.
  arXiv  Detail & Related papers  (2023-10-10T11:50:27Z)
• Open Set Relation Extraction via Unknown-Aware Training [72.10462476890784]
  We propose an unknown-aware training method, regularizing the model by dynamically synthesizing negative instances. Inspired by text adversarial attacks, we adaptively apply small but critical perturbations to original training instances. Experimental results show that this method achieves SOTA unknown relation detection without compromising the classification of known relations.
  arXiv  Detail & Related papers  (2023-06-08T05:45:25Z)
• Serial Contrastive Knowledge Distillation for Continual Few-shot Relation Extraction [35.79570854392989]
  We propose a new model, namely SCKD, to accomplish the continual few-shot RE task. Specifically, we design serial knowledge distillation to preserve the prior knowledge from previous models. Our experiments on two benchmark datasets validate the effectiveness of SCKD for continual few-shot RE.
  arXiv  Detail & Related papers  (2023-05-11T07:25:47Z)
• Less is More: Mitigate Spurious Correlations for Open-Domain Dialogue Response Generation Models by Causal Discovery [52.95935278819512]
  We conduct the first study on spurious correlations for open-domain response generation models based on a corpus CGDIALOG curated in our work. Inspired by causal discovery algorithms, we propose a novel model-agnostic method for training and inference of response generation model.
  arXiv  Detail & Related papers  (2023-03-02T06:33:48Z)
• Improving Narrative Relationship Embeddings by Training with Additional Inverse-Relationship Constraints [0.0]
  We consider the problem of embedding character-entity relationships from the reduced semantic space of narratives. We analyze this assumption and compare the approach to a baseline state-of-the-art model with a unique evaluation that simulates efficacy on a downstream clustering task with human-created labels.
  arXiv  Detail & Related papers  (2022-12-21T17:59:11Z)
• Learning Robust Representations for Continual Relation Extraction via Adversarial Class Augmentation [45.87125587600661]
  Continual relation extraction (CRE) aims to continually learn new relations from a class-incremental data stream. CRE model usually suffers from catastrophic forgetting problem, i.e., the performance of old relations seriously degrades when the model learns new relations. To address this issue, we encourage the model to learn more precise and robust representations through a simple yet effective adversarial class augmentation mechanism.
  arXiv  Detail & Related papers  (2022-10-10T08:50:48Z)
• Consistent Representation Learning for Continual Relation Extraction [18.694012937149495]
  A consistent representation learning method is proposed, which maintains the stability of the relation embedding. Our method significantly outperforms state-of-the-art baselines and yield strong robustness on the imbalanced dataset.
  arXiv  Detail & Related papers  (2022-03-05T12:16:34Z)
• Learning to Decouple Relations: Few-Shot Relation Classification with Entity-Guided Attention and Confusion-Aware Training [49.9995628166064]
  We propose CTEG, a model equipped with two mechanisms to learn to decouple easily-confused relations. On the one hand, an EGA mechanism is introduced to guide the attention to filter out information causing confusion. On the other hand, a Confusion-Aware Training (CAT) method is proposed to explicitly learn to distinguish relations.
  arXiv  Detail & Related papers  (2020-10-21T11:07:53Z)
• Few-shot Visual Reasoning with Meta-analogical Contrastive Learning [141.2562447971]
  We propose to solve a few-shot (or low-shot) visual reasoning problem, by resorting to analogical reasoning. We extract structural relationships between elements in both domains, and enforce them to be as similar as possible with analogical learning. We validate our method on RAVEN dataset, on which it outperforms state-of-the-art method, with larger gains when the training data is scarce.
  arXiv  Detail & Related papers  (2020-07-23T14:00:34Z)
• Amortized Causal Discovery: Learning to Infer Causal Graphs from Time-Series Data [63.15776078733762]
  We propose Amortized Causal Discovery, a novel framework to learn to infer causal relations from time-series data. We demonstrate experimentally that this approach, implemented as a variational model, leads to significant improvements in causal discovery performance.
  arXiv  Detail & Related papers  (2020-06-18T19:59:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.