Contrastive Triple Extraction with Generative Transformer

• URL: http://arxiv.org/abs/2009.06207v8
• Date: Thu, 21 Jul 2022 02:36:18 GMT
• Title: Contrastive Triple Extraction with Generative Transformer
• Authors: Hongbin Ye, Ningyu Zhang, Shumin Deng, Mosha Chen, Chuanqi Tan, Fei Huang, Huajun Chen
• Abstract summary: We introduce a novel model, contrastive triple extraction with a generative transformer. Specifically, we introduce a single shared transformer module for encoder-decoder-based generation. To generate faithful results, we propose a novel triplet contrastive training object.
• Score: 72.21467482853232
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Triple extraction is an essential task in information extraction for natural language processing and knowledge graph construction. In this paper, we revisit the end-to-end triple extraction task for sequence generation. Since generative triple extraction may struggle to capture long-term dependencies and generate unfaithful triples, we introduce a novel model, contrastive triple extraction with a generative transformer. Specifically, we introduce a single shared transformer module for encoder-decoder-based generation. To generate faithful results, we propose a novel triplet contrastive training object. Moreover, we introduce two mechanisms to further improve model performance (i.e., batch-wise dynamic attention-masking and triple-wise calibration). Experimental results on three datasets (i.e., NYT, WebNLG, and MIE) show that our approach achieves better performance than that of baselines.

Related papers

• Improving Recall of Large Language Models: A Model Collaboration Approach for Relational Triple Extraction [44.716502690026026]
  Relation triple extraction, which outputs a set of triples from long sentences, plays a vital role in knowledge acquisition. Large language models can accurately extract triples from simple sentences through few-shot learning or fine-tuning when given appropriate instructions. In this paper, we design an evaluation-filtering framework that integrates large language models with small models for relational triple extraction tasks.
  arXiv  Detail & Related papers  (2024-04-15T09:03:05Z)
• RAVEN: Rethinking Adversarial Video Generation with Efficient Tri-plane Networks [93.18404922542702]
  We present a novel video generative model designed to address long-term spatial and temporal dependencies. Our approach incorporates a hybrid explicit-implicit tri-plane representation inspired by 3D-aware generative frameworks. Our model synthesizes high-fidelity video clips at a resolution of $256times256$ pixels, with durations extending to more than $5$ seconds at a frame rate of 30 fps.
  arXiv  Detail & Related papers  (2024-01-11T16:48:44Z)
• Prompt Based Tri-Channel Graph Convolution Neural Network for Aspect Sentiment Triplet Extraction [63.0205418944714]
  Aspect Sentiment Triplet Extraction (ASTE) is an emerging task to extract a given sentence's triplets, which consist of aspects, opinions, and sentiments. Recent studies tend to address this task with a table-filling paradigm, wherein word relations are encoded in a two-dimensional table. We propose a novel model for the ASTE task, called Prompt-based Tri-Channel Graph Convolution Neural Network (PT-GCN), which converts the relation table into a graph to explore more comprehensive relational information.
  arXiv  Detail & Related papers  (2023-12-18T12:46:09Z)
• Generative Meta-Learning for Zero-Shot Relation Triplet Extraction [12.837901211741443]
  We propose a novel generative meta-learning framework to boost the generalization capability of generative models. Specifically, we first design a task-aware generative model which can learn the general knowledge by forcing the optimization process to be conducted across multiple tasks. Based on it, we then present three generative meta-learning approaches designated for three typical meta-learning categories.
  arXiv  Detail & Related papers  (2023-05-03T06:34:39Z)
• Towards Long-Term Time-Series Forecasting: Feature, Pattern, and Distribution [57.71199089609161]
  Long-term time-series forecasting (LTTF) has become a pressing demand in many applications, such as wind power supply planning. Transformer models have been adopted to deliver high prediction capacity because of the high computational self-attention mechanism. We propose an efficient Transformerbased model, named Conformer, which differentiates itself from existing methods for LTTF in three aspects.
  arXiv  Detail & Related papers  (2023-01-05T13:59:29Z)
• DORE: Document Ordered Relation Extraction based on Generative Framework [56.537386636819626]
  This paper investigates the root cause of the underwhelming performance of the existing generative DocRE models. We propose to generate a symbolic and ordered sequence from the relation matrix which is deterministic and easier for model to learn. Experimental results on four datasets show that our proposed method can improve the performance of the generative DocRE models.
  arXiv  Detail & Related papers  (2022-10-28T11:18:10Z)
• A Generative Model for Relation Extraction and Classification [23.1277041729626]
  We present a novel generative model for relation extraction and classification (which we call GREC) We explore various encoding representations for the source and target sequences, and design effective schemes that enable GREC to achieve state-of-the-art performance on three benchmark RE datasets. Our approach can be extended to extract all relation triples from a sentence in one pass.
  arXiv  Detail & Related papers  (2022-02-26T21:17:18Z)
• Construct Informative Triplet with Two-stage Hard-sample Generation [6.361348748202731]
  We propose a two-stage synthesis framework that produces hard samples through effective positive and negative sample generators. Our method achieves superior performance than the existing hard-sample generation algorithms. We also find that our proposed hard sample generation method combining the existing triplet mining strategies can further boost the deep metric learning performance.
  arXiv  Detail & Related papers  (2021-12-04T06:28:25Z)
• Transformers Solve the Limited Receptive Field for Monocular Depth Prediction [82.90445525977904]
  We propose TransDepth, an architecture which benefits from both convolutional neural networks and transformers. This is the first paper which applies transformers into pixel-wise prediction problems involving continuous labels.
  arXiv  Detail & Related papers  (2021-03-22T18:00:13Z)
• Joint Entity and Relation Extraction with Set Prediction Networks [24.01964730210045]
  We treat joint entity and relation extraction as a direct set prediction problem. Unlike autoregressive approaches that generate triples one by one in a certain order, the proposed networks directly output the final set of triples in one shot. Experiments on two benchmark datasets show that our proposed model significantly outperforms current state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-11-03T13:04:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.