Yes but.. Can ChatGPT Identify Entities in Historical Documents?

• URL: http://arxiv.org/abs/2303.17322v1
• Date: Thu, 30 Mar 2023 12:23:39 GMT
• Title: Yes but.. Can ChatGPT Identify Entities in Historical Documents?
• Abstract summary: Large language models (LLMs) have been leveraged for several years now, obtaining state-of-the-art performance in recognizing entities from modern documents. ChatGPT has "prompted" a lot of interest in the scientific community and public due to its capacity of generating plausible-sounding answers.
• Score: 8.591605318423321
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Large language models (LLMs) have been leveraged for several years now, obtaining state-of-the-art performance in recognizing entities from modern documents. For the last few months, the conversational agent ChatGPT has "prompted" a lot of interest in the scientific community and public due to its capacity of generating plausible-sounding answers. In this paper, we explore this ability by probing it in the named entity recognition and classification (NERC) task in primary sources (e.g., historical newspapers and classical commentaries) in a zero-shot manner and by comparing it with state-of-the-art LM-based systems. Our findings indicate several shortcomings in identifying entities in historical text that range from the consistency of entity annotation guidelines, entity complexity, and code-switching, to the specificity of prompting. Moreover, as expected, the inaccessibility of historical archives to the public (and thus on the Internet) also impacts its performance.

Related papers

• Robust Interpretation of Historical Documents in Knowledge Graphs Through Query Inference and Execution [5.02509168950652]
  We present an agentic retrieval system designed to deliver more accurate and verifiable access to historical data.<n>We introduce a semi-symbolic framework that integrates word-spotting techniques for post-OCR correction with a knowledge graph representation.
  arXiv  Detail & Related papers  (2026-07-27T14:11:20Z)
• Conversational Retrieval and On-the-Fly Knowledge Modeling of Historical Penitentiary Repression Records [1.4804887845421213]
  We present a document analysis system designed for the management of historical digital libraries.<n>The system is equipped with the capability to store facts produced by expert archivists or derived from document retrieval processes.
  arXiv  Detail & Related papers  (2026-07-09T13:18:52Z)
• Inductive Reasoning for Temporal Knowledge Graphs with Emerging Entities [54.48612448535698]
  Reasoning on Temporal Knowledge Graphs (TKGs) is essential for predicting future events and time-aware facts.<n>We propose TransFIR, a novel framework that leverages historical interaction sequences from semantically similar known entities to support inductive reasoning.
  arXiv  Detail & Related papers  (2026-04-11T11:23:05Z)
• DELICATE: Diachronic Entity LInking using Classes And Temporal Evidence [9.341457688757073]
  The goal of this paper is to address issues with two main contributions in the field of humanities.<n>The first contribution is DELICATE, a novel neuro-symbolic method for EL on historical Italian.<n>The second contribution is ENEIDE, a multi-domain EL corpus in historical Italian semi-automatically extracted from two annotated editions spanning from the 19th to the 20th century.
  arXiv  Detail & Related papers  (2025-11-13T15:24:27Z)
• Named Entity Recognition of Historical Text via Large Language Model [1.5469452301122175]
  Named Entity Recognition (NER) plays a crucial role in extracting information from unstructured text.<n>Traditionally, NER is addressed using supervised machine learning approaches, which require large amounts of annotated training data.<n>In this study, we explore the feasibility of applying LLMs to NER in historical documents using zero-shot and few-shot prompting strategies.<n>Our experiments, conducted on the HIPE-2022 dataset, show that LLMs can achieve reasonably strong performance on NER tasks in this setting.
  arXiv  Detail & Related papers  (2025-08-25T14:52:11Z)
• Musical Heritage Historical Entity Linking [2.848644509520432]
  We introduce Musical Heritage named Entities Recognition, Classification and Linking (MHERCL) MHERCL is a novel benchmark consisting of manually annotated sentences extrapolated from historical periodicals of the music domain. We propose a novel unsupervised Entity Linking (EL) model and a method to extend supervised entity linkers by using Knowledge Graphs (KGs)
  arXiv  Detail & Related papers  (2025-02-13T10:51:40Z)
• Hypergraph based Understanding for Document Semantic Entity Recognition [65.84258776834524]
  We build a novel hypergraph attention document semantic entity recognition framework, HGA, which uses hypergraph attention to focus on entity boundaries and entity categories at the same time. Our results on FUNSD, CORD, XFUNDIE show that our method can effectively improve the performance of semantic entity recognition tasks.
  arXiv  Detail & Related papers  (2024-07-09T14:35:49Z)
• Contrastive Entity Coreference and Disambiguation for Historical Texts [2.446672595462589]
  Existing entity disambiguation methods often fall short in accuracy for historical documents, which are replete with individuals not remembered in contemporary knowledgebases. This study makes three key contributions to improve cross-document coreference resolution and disambiguation in historical texts.
  arXiv  Detail & Related papers  (2024-06-21T18:22:14Z)
• Fetch-A-Set: A Large-Scale OCR-Free Benchmark for Historical Document Retrieval [2.7471068141502]
  The benchmark comprises a vast repository of documents dating back to the XVII century. It fills a critical gap in the literature by focusing on complex extractive tasks within the domain of cultural heritage.
  arXiv  Detail & Related papers  (2024-06-11T14:45:00Z)
• DIVKNOWQA: Assessing the Reasoning Ability of LLMs via Open-Domain Question Answering over Knowledge Base and Text [73.68051228972024]
  Large Language Models (LLMs) have exhibited impressive generation capabilities, but they suffer from hallucinations when relying on their internal knowledge. Retrieval-augmented LLMs have emerged as a potential solution to ground LLMs in external knowledge.
  arXiv  Detail & Related papers  (2023-10-31T04:37:57Z)
• Entity Disambiguation with Entity Definitions [50.01142092276296]
  Local models have recently attained astounding performances in Entity Disambiguation (ED) Previous works limited their studies to using, as the textual representation of each candidate, only its Wikipedia title. In this paper, we address this limitation and investigate to what extent more expressive textual representations can mitigate it. We report a new state of the art on 2 out of 6 benchmarks we consider and strongly improve the generalization capability over unseen patterns.
  arXiv  Detail & Related papers  (2022-10-11T17:46:28Z)
• Generate rather than Retrieve: Large Language Models are Strong Context Generators [74.87021992611672]
  We present a novel perspective for solving knowledge-intensive tasks by replacing document retrievers with large language model generators. We call our method generate-then-read (GenRead), which first prompts a large language model to generate contextutal documents based on a given question, and then reads the generated documents to produce the final answer.
  arXiv  Detail & Related papers  (2022-09-21T01:30:59Z)
• The Fellowship of the Authors: Disambiguating Names from Social Network Context [2.3605348648054454]
  Authority lists with extensive textual descriptions for each entity are lacking and ambiguous named entities. We combine BERT-based mention representations with a variety of graph induction strategies and experiment with supervised and unsupervised cluster inference methods. We find that in-domain language model pretraining can significantly improve mention representations, especially for larger corpora.
  arXiv  Detail & Related papers  (2022-08-31T21:51:55Z)
• Entity Cloze By Date: What LMs Know About Unseen Entities [79.34707800653597]
  Language models (LMs) are typically trained once on a large-scale corpus and used for years without being updated. We propose a framework to analyze what LMs can infer about new entities that did not exist when the LMs were pretrained. We derive a dataset of entities indexed by their origination date and paired with their English Wikipedia articles, from which we can find sentences about each entity.
  arXiv  Detail & Related papers  (2022-05-05T17:59:31Z)
• Knowledge-Rich Self-Supervised Entity Linking [58.838404666183656]
  Knowledge-RIch Self-Supervision ($tt KRISSBERT$) is a universal entity linker for four million UMLS entities. Our approach subsumes zero-shot and few-shot methods, and can easily incorporate entity descriptions and gold mention labels if available. Without using any labeled information, our method produces $tt KRISSBERT$, a universal entity linker for four million UMLS entities.
  arXiv  Detail & Related papers  (2021-12-15T05:05:12Z)
• Named Entity Recognition and Classification on Historical Documents: A Survey [0.9039665244779184]
  We present the array of challenges posed by historical documents to named entity recognition systems. We describe the main approaches deployed so far, and identify key priorities for future developments.
  arXiv  Detail & Related papers  (2021-09-23T14:37:40Z)
• Combining Visual and Textual Features for Semantic Segmentation of Historical Newspapers [2.5899040911480187]
  We introduce a multimodal approach for the semantic segmentation of historical newspapers. Based on experiments on diachronic Swiss and Luxembourgish newspapers, we investigate the predictive power of visual and textual features. Results show consistent improvement of multimodal models in comparison to a strong visual baseline.
  arXiv  Detail & Related papers  (2020-02-14T17:56:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.