Extraction multi-étiquettes de relations en utilisant des couches de Transformer

• URL: http://arxiv.org/abs/2502.15619v1
• Date: Fri, 21 Feb 2025 17:42:51 GMT
• Title: Extraction multi-étiquettes de relations en utilisant des couches de Transformer
• Authors: Ngoc Luyen Le, Gildas Tagny Ngompé,
• Abstract summary: We present the BTransformer18 model, a deep learning architecture designed for multi-label relation extraction in French texts.<n>Our approach combines the contextual representation capabilities of pre-trained language models with the power of Transformer encoders.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this article, we present the BTransformer18 model, a deep learning architecture designed for multi-label relation extraction in French texts. Our approach combines the contextual representation capabilities of pre-trained language models from the BERT family - such as BERT, RoBERTa, and their French counterparts CamemBERT and FlauBERT - with the power of Transformer encoders to capture long-term dependencies between tokens. Experiments conducted on the dataset from the TextMine'25 challenge show that our model achieves superior performance, particularly when using CamemBERT-Large, with a macro F1 score of 0.654, surpassing the results obtained with FlauBERT-Large. These results demonstrate the effectiveness of our approach for the automatic extraction of complex relations in intelligence reports.

Related papers

• TWSSenti: A Novel Hybrid Framework for Topic-Wise Sentiment Analysis on Social Media Using Transformer Models [0.0]
  This study explores a hybrid framework combining transformer-based models to improve sentiment classification accuracy and robustness. The framework addresses challenges such as noisy data, contextual ambiguity, and generalization across diverse datasets. This research highlights its applicability to real-world tasks such as social media monitoring, customer sentiment analysis, and public opinion tracking.
  arXiv  Detail & Related papers  (2025-04-14T05:44:11Z)
• Evaluating Named Entity Recognition: A comparative analysis of mono- and multilingual transformer models on a novel Brazilian corporate earnings call transcripts dataset [3.809702129519642]
  We identify two models pre-trained in Brazilian Portuguese (BERTimbau and PTT5) and two multilingual models (mBERT and mT5) Our study aimed to evaluate their performance on a financial Named Entity Recognition (NER) task and determine the computational requirements for fine-tuning and inference.
  arXiv  Detail & Related papers  (2024-03-18T19:53:56Z)
• Adapted Multimodal BERT with Layer-wise Fusion for Sentiment Analysis [84.12658971655253]
  We propose Adapted Multimodal BERT, a BERT-based architecture for multimodal tasks. adapter adjusts the pretrained language model for the task at hand, while the fusion layers perform task-specific, layer-wise fusion of audio-visual information with textual BERT representations. In our ablations we see that this approach leads to efficient models, that can outperform their fine-tuned counterparts and are robust to input noise.
  arXiv  Detail & Related papers  (2022-12-01T17:31:42Z)
• BERT for Sentiment Analysis: Pre-trained and Fine-Tuned Alternatives [0.0]
  BERT has revolutionized the NLP field by enabling transfer learning with large language models. This article studies how to better cope with the different embeddings provided by the BERT output layer and the usage of language-specific instead of multilingual models.
  arXiv  Detail & Related papers  (2022-01-10T15:05:05Z)
• HETFORMER: Heterogeneous Transformer with Sparse Attention for Long-Text Extractive Summarization [57.798070356553936]
  HETFORMER is a Transformer-based pre-trained model with multi-granularity sparse attentions for extractive summarization. Experiments on both single- and multi-document summarization tasks show that HETFORMER achieves state-of-the-art performance in Rouge F1.
  arXiv  Detail & Related papers  (2021-10-12T22:42:31Z)
• Learning Compact Metrics for MT [21.408684470261342]
  We investigate the trade-off between multilinguality and model capacity with RemBERT, a state-of-the-art multilingual language model. We show that model size is indeed a bottleneck for cross-lingual transfer, then demonstrate how distillation can help addressing this bottleneck. Our method yields up to 10.5% improvement over vanilla fine-tuning and reaches 92.6% of RemBERT's performance using only a third of its parameters.
  arXiv  Detail & Related papers  (2021-10-12T20:39:35Z)
• FBERT: A Neural Transformer for Identifying Offensive Content [67.12838911384024]
  fBERT is a BERT model retrained on SOLID, the largest English offensive language identification corpus available with over $1.4$ million offensive instances. We evaluate fBERT's performance on identifying offensive content on multiple English datasets and we test several thresholds for selecting instances from SOLID. The fBERT model will be made freely available to the community.
  arXiv  Detail & Related papers  (2021-09-10T19:19:26Z)
• Learning to Synthesize Data for Semantic Parsing [57.190817162674875]
  We propose a generative model which models the composition of programs and maps a program to an utterance. Due to the simplicity of PCFG and pre-trained BART, our generative model can be efficiently learned from existing data at hand. We evaluate our method in both in-domain and out-of-domain settings of text-to-Query parsing on the standard benchmarks of GeoQuery and Spider.
  arXiv  Detail & Related papers  (2021-04-12T21:24:02Z)
• MMFT-BERT: Multimodal Fusion Transformer with BERT Encodings for Visual Question Answering [68.40719618351429]
  We present MMFT-BERT(MultiModal Fusion Transformer with BERT encodings) to solve Visual Question Answering (VQA) Our approach benefits from processing multimodal data adopting the BERT encodings individually and using a novel transformer-based fusion method to fuse them together.
  arXiv  Detail & Related papers  (2020-10-27T06:34:14Z)
• BET: A Backtranslation Approach for Easy Data Augmentation in Transformer-based Paraphrase Identification Context [0.0]
  We call this approach BET by which we analyze the backtranslation data augmentation on the transformer-based architectures. Our findings suggest that BET improves the paraphrase identification performance on the Microsoft Research Paraphrase Corpus to more than 3% on both accuracy and F1 score.
  arXiv  Detail & Related papers  (2020-09-25T22:06:06Z)
• Conversational Question Reformulation via Sequence-to-Sequence Architectures and Pretrained Language Models [56.268862325167575]
  This paper presents an empirical study of conversational question reformulation (CQR) with sequence-to-sequence architectures and pretrained language models (PLMs) We leverage PLMs to address the strong token-to-token independence assumption made in the common objective, maximum likelihood estimation, for the CQR task. We evaluate fine-tuned PLMs on the recently-introduced CANARD dataset as an in-domain task and validate the models using data from the TREC 2019 CAsT Track as an out-domain task.
  arXiv  Detail & Related papers  (2020-04-04T11:07:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.