BOOKCOREF: Coreference Resolution at Book Scale

• URL: http://arxiv.org/abs/2507.12075v1
• Date: Wed, 16 Jul 2025 09:35:38 GMT
• Title: BOOKCOREF: Coreference Resolution at Book Scale
• Authors: Giuliano Martinelli, Tommaso Bonomo, Pere-Lluís Huguet Cabot, Roberto Navigli,
• Abstract summary: We create the first book-scale coreference benchmark, BOOKCOREF, with an average document length of more than 200,000 tokens.<n>We report on the new challenges introduced by this unprecedented book-scale setting, highlighting that current models fail to deliver the same performance.<n>We release our data and code to encourage research and development of new book-scale Coreference Resolution systems.
• Score: 44.08932883054499
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Coreference Resolution systems are typically evaluated on benchmarks containing small- to medium-scale documents. When it comes to evaluating long texts, however, existing benchmarks, such as LitBank, remain limited in length and do not adequately assess system capabilities at the book scale, i.e., when co-referring mentions span hundreds of thousands of tokens. To fill this gap, we first put forward a novel automatic pipeline that produces high-quality Coreference Resolution annotations on full narrative texts. Then, we adopt this pipeline to create the first book-scale coreference benchmark, BOOKCOREF, with an average document length of more than 200,000 tokens. We carry out a series of experiments showing the robustness of our automatic procedure and demonstrating the value of our resource, which enables current long-document coreference systems to gain up to +20 CoNLL-F1 points when evaluated on full books. Moreover, we report on the new challenges introduced by this unprecedented book-scale setting, highlighting that current models fail to deliver the same performance they achieve on smaller documents. We release our data and code to encourage research and development of new book-scale Coreference Resolution systems at https://github.com/sapienzanlp/bookcoref.

Related papers

• The Elephant in the Coreference Room: Resolving Coreference in Full-Length French Fiction Works [2.6547708221528987]
  We introduce a new annotated corpus of three full-length French novels, totaling over 285,000 tokens.<n>Unlike previous datasets focused on shorter texts, our corpus addresses the challenges posed by long, complex literary works.<n>We show that our approach is competitive and scales effectively to long documents.
  arXiv  Detail & Related papers  (2025-10-17T12:40:33Z)
• SitEmb-v1.5: Improved Context-Aware Dense Retrieval for Semantic Association and Long Story Comprehension [77.93156509994994]
  We show how to represent short chunks in a way that is conditioned on a broader context window to enhance retrieval performance.<n>Existing embedding models are not well-equipped to encode such situated context effectively.<n>Our method substantially outperforms state-of-the-art embedding models.
  arXiv  Detail & Related papers  (2025-08-03T23:59:31Z)
• Enhanced Retrieval of Long Documents: Leveraging Fine-Grained Block Representations with Large Language Models [24.02950598944251]
  We introduce a novel, fine-grained approach aimed at enhancing the accuracy of relevance scoring for long documents.<n>Our methodology firstly segments a long document into blocks, each of which is embedded using an LLM.<n>We aggregate the query-block relevance scores through a weighted sum method, yielding a comprehensive score for the query with the entire document.
  arXiv  Detail & Related papers  (2025-01-28T16:03:52Z)
• Long-Span Question-Answering: Automatic Question Generation and QA-System Ranking via Side-by-Side Evaluation [65.16137964758612]
  We explore the use of long-context capabilities in large language models to create synthetic reading comprehension data from entire books. Our objective is to test the capabilities of LLMs to analyze, understand, and reason over problems that require a detailed comprehension of long spans of text.
  arXiv  Detail & Related papers  (2024-05-31T20:15:10Z)
• TextMonkey: An OCR-Free Large Multimodal Model for Understanding Document [60.01330653769726]
  We present TextMonkey, a large multimodal model (LMM) tailored for text-centric tasks. By adopting Shifted Window Attention with zero-initialization, we achieve cross-window connectivity at higher input resolutions. By expanding its capabilities to encompass text spotting and grounding, and incorporating positional information into responses, we enhance interpretability.
  arXiv  Detail & Related papers  (2024-03-07T13:16:24Z)
• Evaluating Code Summarization Techniques: A New Metric and an Empirical Characterization [16.127739014966487]
  We investigate the complementarity of different types of metrics in capturing the quality of a generated summary. We present a new metric based on contrastive learning to capture said aspect.
  arXiv  Detail & Related papers  (2023-12-24T13:12:39Z)
• LongDocFACTScore: Evaluating the Factuality of Long Document Abstractive Summarisation [28.438103177230477]
  We evaluate the efficacy of automatic metrics for assessing the factual consistency of long document text summarisation. We propose a new evaluation framework, LongDocFACTScore, which is suitable for evaluating long document summarisation data sets.
  arXiv  Detail & Related papers  (2023-09-21T19:54:54Z)
• How Does Generative Retrieval Scale to Millions of Passages? [68.98628807288972]
  We conduct the first empirical study of generative retrieval techniques across various corpus scales. We scale generative retrieval to millions of passages with a corpus of 8.8M passages and evaluating model sizes up to 11B parameters. While generative retrieval is competitive with state-of-the-art dual encoders on small corpora, scaling to millions of passages remains an important and unsolved challenge.
  arXiv  Detail & Related papers  (2023-05-19T17:33:38Z)
• Autoregressive Search Engines: Generating Substrings as Document Identifiers [53.0729058170278]
  Autoregressive language models are emerging as the de-facto standard for generating answers. Previous work has explored ways to partition the search space into hierarchical structures. In this work we propose an alternative that doesn't force any structure in the search space: using all ngrams in a passage as its possible identifiers.
  arXiv  Detail & Related papers  (2022-04-22T10:45:01Z)
• Long Document Summarization with Top-down and Bottom-up Inference [113.29319668246407]
  We propose a principled inference framework to improve summarization models on two aspects. Our framework assumes a hierarchical latent structure of a document where the top-level captures the long range dependency. We demonstrate the effectiveness of the proposed framework on a diverse set of summarization datasets.
  arXiv  Detail & Related papers  (2022-03-15T01:24:51Z)
• Efficient Attentions for Long Document Summarization [25.234852272297598]
  Hepos is a novel efficient encoder-decoder attention with head-wise positional strides. We are able to process ten times more tokens than existing models that use full attentions.
  arXiv  Detail & Related papers  (2021-04-05T18:45:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.