Context-NER : Contextual Phrase Generation at Scale

• URL: http://arxiv.org/abs/2109.08079v4
• Date: Thu, 8 Jun 2023 18:33:01 GMT
• Title: Context-NER : Contextual Phrase Generation at Scale
• Authors: Himanshu Gupta, Shreyas Verma, Santosh Mashetty, Swaroop Mishra
• Abstract summary: We introduce CONTEXT-NER, a task that aims to generate relevant context for entities in a sentence. We present the EDGAR10-Q dataset, containing 1M sentences, 2.8M entities, and an average of 35 tokens per sentence. We find that T5-large, when pre-finetuned on EDGAR10-Q, achieve SOTA results on downstream finance tasks such as Headline, FPB, and FiQA SA, outperforming vanilla version by 10.81 points.
• Score: 4.7947627446578025
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Named Entity Recognition (NER) has seen significant progress in recent years, with numerous state-of-the-art (SOTA) models achieving high performance. However, very few studies have focused on the generation of entities' context. In this paper, we introduce CONTEXT-NER, a task that aims to generate the relevant context for entities in a sentence, where the context is a phrase describing the entity but not necessarily present in the sentence. To facilitate research in this task, we also present the EDGAR10-Q dataset, which consists of annual and quarterly reports from the top 1500 publicly traded companies. The dataset is the largest of its kind, containing 1M sentences, 2.8M entities, and an average of 35 tokens per sentence, making it a challenging dataset. We propose a baseline approach that combines a phrase generation algorithm with inferencing using a 220M language model, achieving a ROUGE-L score of 27% on the test split. Additionally, we perform a one-shot inference with ChatGPT, which obtains a 30% ROUGE-L, highlighting the difficulty of the dataset. We also evaluate models such as T5 and BART, which achieve a maximum ROUGE-L of 49% after supervised finetuning on EDGAR10-Q. We also find that T5-large, when pre-finetuned on EDGAR10-Q, achieve SOTA results on downstream finance tasks such as Headline, FPB, and FiQA SA, outperforming vanilla version by 10.81 points. To our surprise, this 66x smaller pre-finetuned model also surpasses the finance-specific LLM BloombergGPT-50B by 15 points. We hope that our dataset and generated artifacts will encourage further research in this direction, leading to the development of more sophisticated language models for financial text analysis

Related papers

• Information Extraction: An application to the domain of hyper-local financial data on developing countries [0.0]
  We develop and evaluate two Natural Language Processing (NLP) based techniques to address this issue. First, we curate a custom dataset specific to the domain of financial text data on developing countries. We then explore a text-to-text approach with the transformer-based T5 model with the goal of undertaking simultaneous NER and relation extraction.
  arXiv  Detail & Related papers  (2024-03-14T03:49:36Z)
• Text2Analysis: A Benchmark of Table Question Answering with Advanced Data Analysis and Unclear Queries [67.0083902913112]
  We develop the Text2Analysis benchmark, incorporating advanced analysis tasks. We also develop five innovative and effective annotation methods. We evaluate five state-of-the-art models using three different metrics.
  arXiv  Detail & Related papers  (2023-12-21T08:50:41Z)
• Evaluating Generative Models for Graph-to-Text Generation [0.0]
  We explore the capability of generative models to generate descriptive text from graph data in a zero-shot setting. Our results demonstrate that generative models are capable of generating fluent and coherent text. However, our error analysis reveals that generative models still struggle with understanding the semantic relations between entities.
  arXiv  Detail & Related papers  (2023-07-27T09:03:05Z)
• SEntFiN 1.0: Entity-Aware Sentiment Analysis for Financial News [0.03018439717785794]
  We make publicly available SEntFiN 1.0, a human-annotated dataset of 10,753 news headlines with entity-sentiment annotations. We propose a framework that enables the extraction of entity-relevant sentiments using a feature-based approach rather than an expression-based approach.
  arXiv  Detail & Related papers  (2023-05-20T18:20:39Z)
• Larger Probes Tell a Different Story: Extending Psycholinguistic Datasets Via In-Context Learning [14.606961537327345]
  We introduce new, larger datasets for negation and role reversal inspired by psycholinguistic studies. We dramatically extend existing NEG-136 and ROLE-88 benchmarks using GPT3, increasing their size from 18 and 44 sentence pairs to 750 each. We evaluate 22 models on the extended datasets, seeing model performance dip 20-57% compared to the original smaller benchmarks.
  arXiv  Detail & Related papers  (2023-03-29T04:00:53Z)
• Does Synthetic Data Generation of LLMs Help Clinical Text Mining? [51.205078179427645]
  We investigate the potential of OpenAI's ChatGPT to aid in clinical text mining. We propose a new training paradigm that involves generating a vast quantity of high-quality synthetic data. Our method has resulted in significant improvements in the performance of downstream tasks.
  arXiv  Detail & Related papers  (2023-03-08T03:56:31Z)
• You can't pick your neighbors, or can you? When and how to rely on retrieval in the $k$NN-LM [65.74934004876914]
  Retrieval-enhanced language models (LMs) condition their predictions on text retrieved from large external datastores. One such approach, the $k$NN-LM, interpolates any existing LM's predictions with the output of a $k$-nearest neighbors model. We empirically measure the effectiveness of our approach on two English language modeling datasets.
  arXiv  Detail & Related papers  (2022-10-28T02:57:40Z)
• Query Expansion Using Contextual Clue Sampling with Language Models [69.51976926838232]
  We propose a combination of an effective filtering strategy and fusion of the retrieved documents based on the generation probability of each context. Our lexical matching based approach achieves a similar top-5/top-20 retrieval accuracy and higher top-100 accuracy compared with the well-established dense retrieval model DPR. For end-to-end QA, the reader model also benefits from our method and achieves the highest Exact-Match score against several competitive baselines.
  arXiv  Detail & Related papers  (2022-10-13T15:18:04Z)
• Towards Complex Document Understanding By Discrete Reasoning [77.91722463958743]
  Document Visual Question Answering (VQA) aims to understand visually-rich documents to answer questions in natural language. We introduce a new Document VQA dataset, named TAT-DQA, which consists of 3,067 document pages and 16,558 question-answer pairs. We develop a novel model named MHST that takes into account the information in multi-modalities, including text, layout and visual image, to intelligently address different types of questions.
  arXiv  Detail & Related papers  (2022-07-25T01:43:19Z)
• HaT5: Hate Language Identification using Text-to-Text Transfer Transformer [1.2532400738980594]
  We investigate the performance of a state-of-the art (SoTA) architecture T5 across 5 different tasks from 2 relatively diverse datasets. To improve performance, we augment the training data by using an autoregressive model. It reveals the difficulties of poor data annotation by using a small set of examples.
  arXiv  Detail & Related papers  (2022-02-11T15:21:27Z)
• TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance [71.76018597965378]
  We build a new large-scale Question Answering dataset containing both Tabular And Textual data, named TAT-QA. We propose a novel QA model termed TAGOP, which is capable of reasoning over both tables and text.
  arXiv  Detail & Related papers  (2021-05-17T06:12:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.