Efficient Information Extraction in Few-Shot Relation Classification through Contrastive Representation Learning

• URL: http://arxiv.org/abs/2403.16543v1
• Date: Mon, 25 Mar 2024 08:36:06 GMT
• Title: Efficient Information Extraction in Few-Shot Relation Classification through Contrastive Representation Learning
• Authors: Philipp Borchert, Jochen De Weerdt, Marie-Francine Moens,
• Abstract summary: We introduce a novel approach to enhance information extraction combining multiple sentence representations and contrastive learning. Our method employs contrastive learning to extract complementary discriminative information from these individual representations.
• Score: 23.992247765851204
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Differentiating relationships between entity pairs with limited labeled instances poses a significant challenge in few-shot relation classification. Representations of textual data extract rich information spanning the domain, entities, and relations. In this paper, we introduce a novel approach to enhance information extraction combining multiple sentence representations and contrastive learning. While representations in relation classification are commonly extracted using entity marker tokens, we argue that substantial information within the internal model representations remains untapped. To address this, we propose aligning multiple sentence representations, such as the [CLS] token, the [MASK] token used in prompting, and entity marker tokens. Our method employs contrastive learning to extract complementary discriminative information from these individual representations. This is particularly relevant in low-resource settings where information is scarce. Leveraging multiple sentence representations is especially effective in distilling discriminative information for relation classification when additional information, like relation descriptions, are not available. We validate the adaptability of our approach, maintaining robust performance in scenarios that include relation descriptions, and showcasing its flexibility to adapt to different resource constraints.

Related papers

• Learning Representations without Compositional Assumptions [79.12273403390311]
  We propose a data-driven approach that learns feature set dependencies by representing feature sets as graph nodes and their relationships as learnable edges. We also introduce LEGATO, a novel hierarchical graph autoencoder that learns a smaller, latent graph to aggregate information from multiple views dynamically.
  arXiv  Detail & Related papers  (2023-05-31T10:36:10Z)
• Sentence-Level Relation Extraction via Contrastive Learning with Descriptive Relation Prompts [1.5736899098702974]
  We propose a new paradigm, Contrastive Learning with Descriptive Relation Prompts(CTL-), to jointly consider entity information, relational knowledge and entity type restrictions. The CTL- obtains a competitive F1-score of 76.7% on TACRED. The new presented paradigm achieves F1-scores of 85.8% and 91.6% on TACREV and Re-TACRED respectively, which are both the state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-04-11T02:15:13Z)
• Variational Distillation for Multi-View Learning [104.17551354374821]
  We design several variational information bottlenecks to exploit two key characteristics for multi-view representation learning. Under rigorously theoretical guarantee, our approach enables IB to grasp the intrinsic correlation between observations and semantic labels.
  arXiv  Detail & Related papers  (2022-06-20T03:09:46Z)
• VLCDoC: Vision-Language Contrastive Pre-Training Model for Cross-Modal Document Classification [3.7798600249187295]
  Multimodal learning from document data has achieved great success lately as it allows to pre-train semantically meaningful features as a prior into a learnable downstream task. In this paper, we approach the document classification problem by learning cross-modal representations through language and vision cues. The proposed method exploits high-level interactions and learns relevant semantic information from effective attention flows within and across modalities.
  arXiv  Detail & Related papers  (2022-05-24T12:28:12Z)
• Learning Debiased and Disentangled Representations for Semantic Segmentation [52.35766945827972]
  We propose a model-agnostic and training scheme for semantic segmentation. By randomly eliminating certain class information in each training iteration, we effectively reduce feature dependencies among classes. Models trained with our approach demonstrate strong results on multiple semantic segmentation benchmarks.
  arXiv  Detail & Related papers  (2021-10-31T16:15:09Z)
• Conditional Contrastive Learning: Removing Undesirable Information in Self-Supervised Representations [108.29288034509305]
  We develop conditional contrastive learning to remove undesirable information in self-supervised representations. We demonstrate empirically that our methods can successfully learn self-supervised representations for downstream tasks.
  arXiv  Detail & Related papers  (2021-06-05T10:51:26Z)
• Adaptive Attentional Network for Few-Shot Knowledge Graph Completion [16.722373937828117]
  Few-shot Knowledge Graph (KG) completion is a focus of current research, where each task aims at querying unseen facts of a relation given its few-shot reference entity pairs. Recent attempts solve this problem by learning static representations of entities and references, ignoring their dynamic properties. This work proposes an adaptive attentional network for few-shot KG completion by learning adaptive entity and reference representations.
  arXiv  Detail & Related papers  (2020-10-19T16:27:48Z)
• Null-sampling for Interpretable and Fair Representations [8.654168514863649]
  We learn invariant representations, in the data domain, to achieve interpretability in algorithmic fairness. By placing the representations into the data domain, the changes made by the model are easily examinable by human auditors.
  arXiv  Detail & Related papers  (2020-08-12T11:49:01Z)
• Active Learning for Coreference Resolution using Discrete Annotation [76.36423696634584]
  We improve upon pairwise annotation for active learning in coreference resolution. We ask annotators to identify mention antecedents if a presented mention pair is deemed not coreferent. In experiments with existing benchmark coreference datasets, we show that the signal from this additional question leads to significant performance gains per human-annotation hour.
  arXiv  Detail & Related papers  (2020-04-28T17:17:11Z)
• Fairness by Learning Orthogonal Disentangled Representations [50.82638766862974]
  We propose a novel disentanglement approach to invariant representation problem. We enforce the meaningful representation to be agnostic to sensitive information by entropy. The proposed approach is evaluated on five publicly available datasets.
  arXiv  Detail & Related papers  (2020-03-12T11:09:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.