Enhancing Language Models for Financial Relation Extraction with Named Entities and Part-of-Speech

• URL: http://arxiv.org/abs/2405.06665v1
• Date: Thu, 2 May 2024 14:33:05 GMT
• Title: Enhancing Language Models for Financial Relation Extraction with Named Entities and Part-of-Speech
• Authors: Menglin Li, Kwan Hui Lim,
• Abstract summary: FinRE task involves identifying the entities and their relation, given a piece of financial statement/text. We propose a strategy that improves the performance of pre-trained language models by augmenting them with Named Entity Recognition (NER) and Part-Of-Speech (POS) Experiments on a financial relations dataset show promising results and highlights the benefits of incorporating NER and POS in existing models.
• Score: 5.104305392215512
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The Financial Relation Extraction (FinRE) task involves identifying the entities and their relation, given a piece of financial statement/text. To solve this FinRE problem, we propose a simple but effective strategy that improves the performance of pre-trained language models by augmenting them with Named Entity Recognition (NER) and Part-Of-Speech (POS), as well as different approaches to combine these information. Experiments on a financial relations dataset show promising results and highlights the benefits of incorporating NER and POS in existing models. Our dataset and codes are available at https://github.com/kwanhui/FinRelExtract.

Related papers

• Learning to Extract Structured Entities Using Language Models [52.281701191329]
  Recent advances in machine learning have significantly impacted the field of information extraction. We reformulate the task to be entity-centric, enabling the use of diverse metrics. We contribute to the field by introducing Structured Entity Extraction and proposing the Approximate Entity Set OverlaP metric.
  arXiv  Detail & Related papers  (2024-02-06T22:15:09Z)
• FinGPT: Instruction Tuning Benchmark for Open-Source Large Language Models in Financial Datasets [9.714447724811842]
  This paper introduces a distinctive approach anchored in the Instruction Tuning paradigm for open-source large language models. We capitalize on the interoperability of open-source models, ensuring a seamless and transparent integration. The paper presents a benchmarking scheme designed for end-to-end training and testing, employing a cost-effective progression.
  arXiv  Detail & Related papers  (2023-10-07T12:52:58Z)
• GPT-FinRE: In-context Learning for Financial Relation Extraction using Large Language Models [1.9559144041082446]
  This paper describes our solution to relation extraction on one such dataset REFinD. In this paper, we employed OpenAI models under the framework of in-context learning (ICL) We were able to achieve 3rd rank overall. Our best F1-score is 0.718.
  arXiv  Detail & Related papers  (2023-06-30T10:12:30Z)
• FinRED: A Dataset for Relation Extraction in Financial Domain [23.700539609170015]
  FinRED is a relation extraction dataset curated from financial news and earning call transcripts containing relations from the finance domain. We see a significant drop in their performance on FinRED compared to the general relation extraction datasets.
  arXiv  Detail & Related papers  (2023-06-06T14:52:47Z)
• Enriching Relation Extraction with OpenIE [70.52564277675056]
  Relation extraction (RE) is a sub-discipline of information extraction (IE) In this work, we explore how recent approaches for open information extraction (OpenIE) may help to improve the task of RE. Our experiments over two annotated corpora, KnowledgeNet and FewRel, demonstrate the improved accuracy of our enriched models.
  arXiv  Detail & Related papers  (2022-12-19T11:26:23Z)
• You can't pick your neighbors, or can you? When and how to rely on retrieval in the $k$NN-LM [65.74934004876914]
  Retrieval-enhanced language models (LMs) condition their predictions on text retrieved from large external datastores. One such approach, the $k$NN-LM, interpolates any existing LM's predictions with the output of a $k$-nearest neighbors model. We empirically measure the effectiveness of our approach on two English language modeling datasets.
  arXiv  Detail & Related papers  (2022-10-28T02:57:40Z)
• Visualizing the Relationship Between Encoded Linguistic Information and Task Performance [53.223789395577796]
  We study the dynamic relationship between the encoded linguistic information and task performance from the viewpoint of Pareto Optimality. We conduct experiments on two popular NLP tasks, i.e., machine translation and language modeling, and investigate the relationship between several kinds of linguistic information and task performances. Our empirical findings suggest that some syntactic information is helpful for NLP tasks whereas encoding more syntactic information does not necessarily lead to better performance.
  arXiv  Detail & Related papers  (2022-03-29T19:03:10Z)
• Exploiting Network Structures to Improve Semantic Representation for the Financial Domain [9.13755431537592]
  This paper presents the MiniTrue team in the FinSim-3 shared task on learning semantic similarities for the financial domain in English language. Our approach combines contextual embeddings learned by transformer-based language models with network structures embeddings extracted on external knowledge sources. Experimental results show that the model with the knowledge graph embeddings has achieved a superior result than these models with only contextual embeddings.
  arXiv  Detail & Related papers  (2021-07-13T07:32:18Z)
• Named Entity Recognition and Relation Extraction using Enhanced Table Filling by Contextualized Representations [14.614028420899409]
  The proposed method computes representations for entity mentions and long-range dependencies without complicated hand-crafted features or neural-network architectures. We also adapt a tensor dot-product to predict relation labels all at once without resorting to history-based predictions or search strategies. Despite its simplicity, the experimental results demonstrate that the proposed method outperforms the state-of-the-art methods on the CoNLL04 and ACE05 English datasets.
  arXiv  Detail & Related papers  (2020-10-15T04:58:23Z)
• Probing Linguistic Features of Sentence-Level Representations in Neural Relation Extraction [80.38130122127882]
  We introduce 14 probing tasks targeting linguistic properties relevant to neural relation extraction (RE) We use them to study representations learned by more than 40 different encoder architecture and linguistic feature combinations trained on two datasets. We find that the bias induced by the architecture and the inclusion of linguistic features are clearly expressed in the probing task performance.
  arXiv  Detail & Related papers  (2020-04-17T09:17:40Z)
• Coreferential Reasoning Learning for Language Representation [88.14248323659267]
  We present CorefBERT, a novel language representation model that can capture the coreferential relations in context. The experimental results show that, compared with existing baseline models, CorefBERT can achieve significant improvements consistently on various downstream NLP tasks.
  arXiv  Detail & Related papers  (2020-04-15T03:57:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.