Learning Robust Representations for Continual Relation Extraction via Adversarial Class Augmentation

• URL: http://arxiv.org/abs/2210.04497v1
• Date: Mon, 10 Oct 2022 08:50:48 GMT
• Title: Learning Robust Representations for Continual Relation Extraction via Adversarial Class Augmentation
• Authors: Peiyi Wang, Yifan Song, Tianyu Liu, Binghuai Lin, Yunbo Cao, Sujian Li, Zhifang Sui
• Abstract summary: Continual relation extraction (CRE) aims to continually learn new relations from a class-incremental data stream. CRE model usually suffers from catastrophic forgetting problem, i.e., the performance of old relations seriously degrades when the model learns new relations. To address this issue, we encourage the model to learn more precise and robust representations through a simple yet effective adversarial class augmentation mechanism.
• Score: 45.87125587600661
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Continual relation extraction (CRE) aims to continually learn new relations from a class-incremental data stream. CRE model usually suffers from catastrophic forgetting problem, i.e., the performance of old relations seriously degrades when the model learns new relations. Most previous work attributes catastrophic forgetting to the corruption of the learned representations as new relations come, with an implicit assumption that the CRE models have adequately learned the old relations. In this paper, through empirical studies we argue that this assumption may not hold, and an important reason for catastrophic forgetting is that the learned representations do not have good robustness against the appearance of analogous relations in the subsequent learning process. To address this issue, we encourage the model to learn more precise and robust representations through a simple yet effective adversarial class augmentation mechanism (ACA), which is easy to implement and model-agnostic. Experimental results show that ACA can consistently improve the performance of state-of-the-art CRE models on two popular benchmarks.

Related papers

• Rationale-Enhanced Language Models are Better Continual Relation Learners [29.311298089285753]
  Continual relation extraction (CRE) aims to solve the problem of catastrophic forgetting when learning a sequence of newly emerging relations. Recent CRE studies have found that catastrophic forgetting arises from the model's lack of robustness against future analogous relations.
  arXiv  Detail & Related papers  (2023-10-10T11:50:27Z)
• Improving Continual Relation Extraction by Distinguishing Analogous Semantics [11.420578494453343]
  Continual relation extraction aims to learn constantly emerging relations while avoiding forgetting the learned relations. Existing works store a small number of typical samples to re-train the model for alleviating forgetting. We conduct an empirical study on existing works and observe that their performance is severely affected by analogous relations.
  arXiv  Detail & Related papers  (2023-05-11T07:32:20Z)
• Enhancing Continual Relation Extraction via Classifier Decomposition [30.88081408988638]
  Continual relation extraction models aim at handling emerging new relations while avoiding forgetting old ones in the streaming data. Most models only adopt a vanilla strategy when models first learn representations of new relations. We propose a simple yet effective classifier decomposition framework that splits the last FFN layer into separated previous and current classifiers.
  arXiv  Detail & Related papers  (2023-05-08T11:29:33Z)
• Less is More: Rethinking State-of-the-art Continual Relation Extraction Models with a Frustratingly Easy but Effective Approach [35.377756110634515]
  Continual relation extraction (CRE) requires the model to continually learn new relations from class-incremental data streams. We propose a Frustratingly easy but Effective Approach (FEA) method with two learning stages for CRE.
  arXiv  Detail & Related papers  (2022-09-01T06:08:07Z)
• Consistent Representation Learning for Continual Relation Extraction [18.694012937149495]
  A consistent representation learning method is proposed, which maintains the stability of the relation embedding. Our method significantly outperforms state-of-the-art baselines and yield strong robustness on the imbalanced dataset.
  arXiv  Detail & Related papers  (2022-03-05T12:16:34Z)
• Learning of Visual Relations: The Devil is in the Tails [59.737494875502215]
  Visual relation learning is a long-tailed problem, due to the nature of joint reasoning about groups of objects. In this paper, we explore an alternative hypothesis, denoted the Devil is in the Tails. Under this hypothesis, better performance is achieved by keeping the model simple but improving its ability to cope with long-tailed distributions.
  arXiv  Detail & Related papers  (2021-08-22T08:59:35Z)
• Learning Relation Prototype from Unlabeled Texts for Long-tail Relation Extraction [84.64435075778988]
  We propose a general approach to learn relation prototypes from unlabeled texts. We learn relation prototypes as an implicit factor between entities. We conduct experiments on two publicly available datasets: New York Times and Google Distant Supervision.
  arXiv  Detail & Related papers  (2020-11-27T06:21:12Z)
• Learning to Decouple Relations: Few-Shot Relation Classification with Entity-Guided Attention and Confusion-Aware Training [49.9995628166064]
  We propose CTEG, a model equipped with two mechanisms to learn to decouple easily-confused relations. On the one hand, an EGA mechanism is introduced to guide the attention to filter out information causing confusion. On the other hand, a Confusion-Aware Training (CAT) method is proposed to explicitly learn to distinguish relations.
  arXiv  Detail & Related papers  (2020-10-21T11:07:53Z)
• High-order Semantic Role Labeling [86.29371274587146]
  This paper introduces a high-order graph structure for the neural semantic role labeling model. It enables the model to explicitly consider not only the isolated predicate-argument pairs but also the interaction between the predicate-argument pairs. Experimental results on 7 languages of the CoNLL-2009 benchmark show that the high-order structural learning techniques are beneficial to the strong performing SRL models.
  arXiv  Detail & Related papers  (2020-10-09T15:33:54Z)
• Relation-Guided Representation Learning [53.60351496449232]
  We propose a new representation learning method that explicitly models and leverages sample relations. Our framework well preserves the relations between samples. By seeking to embed samples into subspace, we show that our method can address the large-scale and out-of-sample problem.
  arXiv  Detail & Related papers  (2020-07-11T10:57:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.